Commit b78c4e6a authored by Pierre Smeyers's avatar Pierre Smeyers
Browse files

feat(publish): support extra tags

parent 79d7e0cd
Loading
Loading
Loading
Loading
+47 −1
Original line number Diff line number Diff line
@@ -425,6 +425,8 @@ This job pushes (_promotes_) the built image as the _release_ image [skopeo](htt
| `DOCKER_SKOPEO_IMAGE` | The Docker image used to run [skopeo](https://github.com/containers/skopeo) | `quay.io/skopeo/stable:latest` |
| `DOCKER_PUBLISH_ARGS` | Additional [`skopeo copy` arguments](https://github.com/containers/skopeo/blob/master/docs/skopeo-copy.1.md#options) | _(none)_          |
| `DOCKER_PROD_PUBLISH_STRATEGY`| Defines the publish to production strategy. One of `manual` (i.e. _one-click_), `auto` or `none` (disabled). | `manual` |
| `DOCKER_RELEASE_EXTRA_TAGS_PATTERN` | Defines the image tag pattern that `$DOCKER_RELEASE_IMAGE` should match to push extra tags (supports capturing groups - [see below](#using-extra-tags)) | `^v?(?P<major>[0-9]+)\\.(?P<minor>[0-9]+)\\.(?P<patch>[0-9]+)(?P<suffix>(?P<prerelease>-[0-9A-Za-z-\\.]+)?(?P<build>\\+[0-9A-Za-z-\\.]+)?)$` _(SemVer pattern)_ |
| `DOCKER_RELEASE_EXTRA_TAGS`   | Defines extra tags to publish the _release_ image (supports capturing group references from `$DOCKER_RELEASE_EXTRA_TAGS_PATTERN` - [see below](#using-extra-tags))       | _(none)_          |
| `DOCKER_SEMREL_RELEASE_DISABLED` | Set to `true` to disable [semantic-release integration](#semantic-release-integration)   | _none_ (enabled) |

This job produces _output variables_ that are propagated to downstream jobs (using [dotenv artifacts](https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html#artifactsreportsdotenv)):
@@ -439,7 +441,51 @@ This job produces _output variables_ that are propagated to downstream jobs (usi

They may be freely used in downstream jobs (for instance to deploy the upstream built Docker image, whatever the branch or tag).

### `semantic-release` integration
#### Using extra tags

When publishing the _release_ image, the Docker template might publish it again with additional tags (aliases):

* the original published image tag (extracted from `$DOCKER_RELEASE_IMAGE`) must match `$DOCKER_RELEASE_EXTRA_TAGS_PATTERN` ([semantic versioning](https://semver.org/) pattern by default),
* extra tags to publish can be defined in the `$DOCKER_RELEASE_EXTRA_TAGS` variable, each separated with a whitespace.

:information_source: the Docker template supports [group references substitution](https://learnbyexample.github.io/py_regular_expressions/groupings-and-backreferences.html) to evaluate extra tags:

* `$DOCKER_RELEASE_EXTRA_TAGS_PATTERN` supports capturing groups:
    * `v([0-9]+)\.([0-9]+)\.([0-9]+)` has 3 (unnamed) capturing groups, each capturing any number of digits
    * `v(P<major>[0-9]+)\.(P<minor>[0-9]+)\.(P<patch>[0-9]+)` has 3 **named** capturing groups (_major_, _minor_ and _patch_), each capturing any number of digits
* `$DOCKER_RELEASE_EXTRA_TAGS` supports capturing group references from `$DOCKER_RELEASE_EXTRA_TAGS_PATTERN`: 
    * `\g1` is a reference to capturing group number 1
    * `\g<major>` is a reference to capturing group named _major_

:information_source: the default value of `$DOCKER_RELEASE_EXTRA_TAGS_PATTERN` matches and captures all parts of a standard [semantic versioning](https://semver.org/)-compliant tag:

* the **major** group captures the major version
* the **minor** group captures the minor version
* the **patch** group captures the patch version
* the **prerelease** group captures the (optional) pre-release version (including the leading `-`)
* the **build** group captures the (optional) build version (including the leading `+`)
* the **suffix** group captures the (optional) entire suffix (including pre-release and/or build)

Example: publish latest, major.minor and major aliases for a SemVer release:

```yaml
variables:
  # ⚠ don't forget to escape backslash character in yaml
  DOCKER_RELEASE_EXTRA_TAGS: "latest \\g<major>.\\g<minor>\\g<build> \\g<major>\\g<build>"
```

With this contiguration, the following extra tags would be published:

| original tag          | extra tags |
| --------------------- | ---------- |
| `main`                | _none_ (doesn't match `$DOCKER_RELEASE_EXTRA_TAGS_PATTERN`) |
| `some-manual-tag`     | _none_ (doesn't match `$DOCKER_RELEASE_EXTRA_TAGS_PATTERN`) |
| `1.2.3`               | `latest`, `1.2`, `1` |
| `1.2.3-alpha.12`      | `latest`, `1.2`, `1` |
| `1.2.3+linux`         | `latest`, `1.2+linux`, `1+linux` |
| `1.2.3-alpha.12+linux`| `latest`, `1.2+linux`, `1+linux` |

#### `semantic-release` integration

If you activate the [`semantic-release-info` job from the `semantic-release` template](https://gitlab.com/to-be-continuous/semantic-release/#semantic-release-info-job), the `docker-publish` job will rely on the generated next version info.

+11 −0
Original line number Diff line number Diff line
@@ -62,6 +62,17 @@
      "description": "Docker release image",
      "default": "$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME"
    },
    {
      "name": "DOCKER_RELEASE_EXTRA_TAGS_PATTERN",
      "description": "Defines the image tag pattern that `$DOCKER_RELEASE_IMAGE` should match to push extra tags (supports capturing groups)\n\nDefaults to [SemVer](https://semver.org/) pattern.",
      "default": "^v?(?P<major>[0-9]+)\\.(?P<minor>[0-9]+)\\.(?P<patch>[0-9]+)(?P<suffix>(?P<prerelease>-[0-9A-Za-z-\\.]+)?(?P<build>\\+[0-9A-Za-z-\\.]+)?)$",
      "advanced": true
    },
    {
      "name": "DOCKER_RELEASE_EXTRA_TAGS",
      "description": "Defines extra tags to publish the _release_ image\n\nSupports capturing group references from `$DOCKER_RELEASE_EXTRA_TAGS_PATTERN` (ex: `latest \\g<major>.\\g<minor> \\g<major>`)",
      "advanced": true
    },
    {
      "name": "DOCKER_BUILD_ARGS",
      "description": "Additional docker/kaniko/buildah build arguments"
+65 −24
Original line number Diff line number Diff line
@@ -91,6 +91,7 @@ variables:

  # default: one-click publish
  DOCKER_PROD_PUBLISH_STRATEGY: manual
  DOCKER_RELEASE_EXTRA_TAGS_PATTERN: "^v?(?P<major>[0-9]+)\\.(?P<minor>[0-9]+)\\.(?P<patch>[0-9]+)(?P<suffix>(?P<prerelease>-[0-9A-Za-z-\\.]+)?(?P<build>\\+[0-9A-Za-z-\\.]+)?)$"

  # default production ref name (pattern)
  PROD_REF: '/^(master|main)$/'
@@ -443,6 +444,30 @@ stages:
    fi
  }

  function publish_extra_tags() {
    if [[ -z "$DOCKER_RELEASE_EXTRA_TAGS" ]]
    then
      return
    fi
    # check if tag matches pattern
    # shellcheck disable=SC2154
    matches=$(python3 -c "import re;print('match' if re.match(r'$DOCKER_RELEASE_EXTRA_TAGS_PATTERN', '$docker_tag') else '')")
    if [[ "$matches" ]]
    then
      # apply extra tags substitution
      extra_tags=$(python3 -c "import re;print(re.sub(r'$DOCKER_RELEASE_EXTRA_TAGS_PATTERN', r'$DOCKER_RELEASE_EXTRA_TAGS', '$docker_tag'))")
      log_info "Pushing extra tags (evaluated from original tag \\e[33;1m${docker_tag}\\e[0m)..."
      for extra_tag in $extra_tags
      do
        log_info "... pushing extra tag: \\e[33;1m${extra_tag}\\e[0m..."
        # shellcheck disable=SC2086,SC2154
        skopeo copy --src-authfile "$BUILDTOOL_HOME/skopeo/.docker/dest-config.json" --dest-authfile "$BUILDTOOL_HOME/skopeo/.docker/dest-config.json" ${DOCKER_PUBLISH_ARGS} "docker://$DOCKER_RELEASE_IMAGE" "docker://$docker_repository:$extra_tag"
      done
    else
      log_info "Extra tags configured, but the released tag (\\e[33;1m${docker_tag}\\e[0m) doesn't match \$DOCKER_RELEASE_EXTRA_TAGS_PATTERN..."
    fi
  }

  init_workspace

  # ENDSCRIPT
@@ -562,11 +587,14 @@ docker-kaniko-build:
    - docker_digest=$(cat .img-digest.txt)
    - docker_repository=${DOCKER_SNAPSHOT_IMAGE%:*}
    - docker_tag=${DOCKER_SNAPSHOT_IMAGE##*:}
    - echo "docker_image=$DOCKER_SNAPSHOT_IMAGE" > docker.env
    - echo "docker_image_digest=$docker_repository@$docker_digest" >> docker.env
    - echo "docker_repository=$docker_repository" >> docker.env
    - echo "docker_tag=$docker_tag" >> docker.env
    - echo "docker_digest=$docker_digest" >> docker.env
    - |
      {
        echo "docker_image=$DOCKER_SNAPSHOT_IMAGE"
        echo "docker_image_digest=$docker_repository@$docker_digest"
        echo "docker_repository=$docker_repository"
        echo "docker_tag=$docker_tag"
        echo "docker_digest=$docker_digest"
      } > docker.env
  artifacts:
    reports:
      dotenv:
@@ -591,11 +619,14 @@ docker-dind-build:
    - docker_digest=${image_with_digest##*@}
    - docker_repository=${DOCKER_SNAPSHOT_IMAGE%:*}
    - docker_tag=${DOCKER_SNAPSHOT_IMAGE##*:}
    - echo "docker_image=$DOCKER_SNAPSHOT_IMAGE" > docker.env
    - echo "docker_image_digest=$docker_repository@$docker_digest" >> docker.env
    - echo "docker_repository=$docker_repository" >> docker.env
    - echo "docker_tag=$docker_tag" >> docker.env
    - echo "docker_digest=$docker_digest" >> docker.env
    - |
      {
        echo "docker_image=$DOCKER_SNAPSHOT_IMAGE"
        echo "docker_image_digest=$docker_repository@$docker_digest"
        echo "docker_repository=$docker_repository"
        echo "docker_tag=$docker_tag"
        echo "docker_digest=$docker_digest"
      } > docker.env
  artifacts:
    reports:
      dotenv:
@@ -621,11 +652,14 @@ docker-buildah-build:
    - docker_digest=$(cat .img-digest.txt)
    - docker_repository=${DOCKER_SNAPSHOT_IMAGE%:*}
    - docker_tag=${DOCKER_SNAPSHOT_IMAGE##*:}
    - echo "docker_image=$DOCKER_SNAPSHOT_IMAGE" > docker.env
    - echo "docker_image_digest=$docker_repository@$docker_digest" >> docker.env
    - echo "docker_repository=$docker_repository" >> docker.env
    - echo "docker_tag=$docker_tag" >> docker.env
    - echo "docker_digest=$docker_digest" >> docker.env
    - |
      {
        echo "docker_image=$DOCKER_SNAPSHOT_IMAGE"
        echo "docker_image_digest=$docker_repository@$docker_digest"
        echo "docker_repository=$docker_repository"
        echo "docker_tag=$docker_tag"
        echo "docker_digest=$docker_digest"
      } > docker.env
  artifacts:
    reports:
      dotenv:
@@ -813,17 +847,24 @@ docker-publish:
        log_warn "\\e[93mYou should consider distinguishing snapshot and release images as they do not differ. Skipping publish phase as image has already been created by previous job.\\e[0m"
        exit 0
      fi
      BUILDTOOL_HOME=${BUILDTOOL_HOME:-$HOME}
      skopeo copy --src-authfile $BUILDTOOL_HOME/skopeo/.docker/src-config.json --dest-authfile $BUILDTOOL_HOME/skopeo/.docker/dest-config.json ${DOCKER_PUBLISH_ARGS} docker://$DOCKER_SNAPSHOT_IMAGE docker://$DOCKER_RELEASE_IMAGE
      log_info "Well done your image is published and can be downloaded by doing: docker pull $DOCKER_RELEASE_IMAGE"
    - docker_digest=$(skopeo inspect --authfile $BUILDTOOL_HOME/skopeo/.docker/dest-config.json --format='{{ .Digest }}' "docker://$DOCKER_RELEASE_IMAGE")
    - BUILDTOOL_HOME=${BUILDTOOL_HOME:-$HOME}
    # 1: push main image
    - skopeo copy --src-authfile "$BUILDTOOL_HOME/skopeo/.docker/src-config.json" --dest-authfile "$BUILDTOOL_HOME/skopeo/.docker/dest-config.json" ${DOCKER_PUBLISH_ARGS} "docker://$DOCKER_SNAPSHOT_IMAGE" "docker://$DOCKER_RELEASE_IMAGE"
    - |
      log_info "Well done your image is pushed and can be pulled with: docker pull $DOCKER_RELEASE_IMAGE"
    # 2: extract info and generate output dotenv
    - docker_digest=$(skopeo inspect --authfile "$BUILDTOOL_HOME/skopeo/.docker/dest-config.json" --format='{{ .Digest }}' "docker://$DOCKER_RELEASE_IMAGE")
    - docker_repository=${DOCKER_RELEASE_IMAGE%:*}
    - docker_tag=${DOCKER_RELEASE_IMAGE##*:}
    - echo "docker_image=$DOCKER_RELEASE_IMAGE" > docker.env
    - echo "docker_image_digest=$docker_repository@$docker_digest" >> docker.env
    - echo "docker_repository=$docker_repository" >> docker.env
    - echo "docker_tag=$docker_tag" >> docker.env
    - echo "docker_digest=$docker_digest" >> docker.env
    - |
      {
        echo "docker_image=$DOCKER_RELEASE_IMAGE"
        echo "docker_image_digest=$docker_repository@$docker_digest"
        echo "docker_repository=$docker_repository"
        echo "docker_tag=$docker_tag"
        echo "docker_digest=$docker_digest"
      } > docker.env
    - publish_extra_tags
  artifacts:
    reports:
      dotenv: