Commit 44777450 authored by Pierre Smeyers's avatar Pierre Smeyers
Browse files

Merge branch 'feat/add-extra-tags-on-publish' into 'master'

Add (optional) extra release tags on publish

See merge request to-be-continuous/cnb!49
parents ee8ca757 4cf5ade4
Loading
Loading
Loading
Loading
+46 −0
Original line number Diff line number Diff line
@@ -252,6 +252,8 @@ This job pushes (_promotes_) the built image as the _release_ image using [skope
| `skopeo-image` / `CNB_SKOPEO_IMAGE` | The Docker image used to run [skopeo](https://github.com/containers/skopeo) | `quay.io/containers/skopeo:latest` <br/>[![Trivy Badge](https://to-be-continuous.gitlab.io/doc/secu/trivy-badge-CNB_SKOPEO_IMAGE.svg)](https://to-be-continuous.gitlab.io/doc/secu/trivy-CNB_SKOPEO_IMAGE) |
| `publish-args` / `CNB_PUBLISH_ARGS` | Additional [`skopeo copy` arguments](https://github.com/containers/skopeo/blob/master/docs/skopeo-copy.1.md#options) | _(none)_          |
| `prod-publish-strategy` / `CNB_PROD_PUBLISH_STRATEGY` | Defines the publish to production strategy. One of `manual` (i.e. _one-click_), `auto` or `none` (disabled). | `manual` |
| `release-extra-tags-pattern` / `CNB_RELEASE_EXTRA_TAGS_PATTERN` | Defines the image tag pattern that `$CNB_RELEASE_IMAGE` should match to push extra tags (supports capturing groups - [see below](#using-extra-tags)) | `^v?(?P<major>[0-9]+)\\.(?P<minor>[0-9]+)\\.(?P<patch>[0-9]+)(?P<suffix>(?P<prerelease>-[0-9A-Za-z-\\.]+)?(?P<build>\\+[0-9A-Za-z-\\.]+)?)$` _(SemVer pattern)_ |
| `release-extra-tags` / `CNB_RELEASE_EXTRA_TAGS`                 | Defines extra tags to publish the _release_ image (supports capturing group references from `$CNB_RELEASE_EXTRA_TAGS_PATTERN` - [see below](#using-extra-tags)) | _(none)_ |

This job produces _output variables_ that are propagated to downstream jobs (using [dotenv artifacts](https://docs.gitlab.com/ci/yaml/artifacts_reports/#artifactsreportsdotenv)):

@@ -264,3 +266,47 @@ This job produces _output variables_ that are propagated to downstream jobs (usi
| `cnb_digest`       | release image digest                                  | `sha256:b7914a91...`                    |

They may be freely used in downstream jobs (for instance to deploy the upstream built image, whatever the branch or tag).

#### Using extra tags

When publishing the _release_ image, the Docker template might publish it again with additional tags (aliases):

- the original published image tag (extracted from `$CNB_RELEASE_IMAGE`) must match `$CNB_RELEASE_EXTRA_TAGS_PATTERN` ([semantic versioning](https://semver.org/) pattern by default),
- extra tags to publish can be defined in the `$CNB_RELEASE_EXTRA_TAGS` variable, each separated with a whitespace.

:information_source: the Docker template supports [group references substitution](https://learnbyexample.github.io/py_regular_expressions/groupings-and-backreferences.html) to evaluate extra tags:

- `$CNB_RELEASE_EXTRA_TAGS_PATTERN` supports capturing groups:
  - `v([0-9]+)\.([0-9]+)\.([0-9]+)` has 3 (unnamed) capturing groups, each capturing any number of digits
  - `v(?<major>[0-9]+)\.(?<minor>[0-9]+)\.(?<patch>[0-9]+)` has 3 **named** capturing groups (_major_, _minor_ and _patch_), each capturing any number of digits
- `$CNB_RELEASE_EXTRA_TAGS` supports capturing group references from `$CNB_RELEASE_EXTRA_TAGS_PATTERN`:
  - `\g1` is a reference to capturing group number 1
  - `\g<major>` is a reference to capturing group named _major_

:information_source: the default value of `$CNB_RELEASE_EXTRA_TAGS_PATTERN` matches and captures all parts of a standard [semantic versioning](https://semver.org/)-compliant tag:

- the **major** group captures the major version
- the **minor** group captures the minor version
- the **patch** group captures the patch version
- the **prerelease** group captures the (optional) pre-release version (including the leading `-`)
- the **build** group captures the (optional) build version (including the leading `+`)
- the **suffix** group captures the (optional) entire suffix (including pre-release and/or build)

Example: publish latest, major.minor and major aliases for a SemVer release:

```yaml
variables:
  # ⚠ don't forget to escape backslash character in yaml
  CNB_RELEASE_EXTRA_TAGS: "latest \\g<major>.\\g<minor>\\g<build> \\g<major>\\g<build>"
```

With this contiguration, the following extra tags would be published:

| original tag           | extra tags                                                  |
| ---------------------- | ----------------------------------------------------------- |
| `main`                 | _none_ (doesn't match `$CNB_RELEASE_EXTRA_TAGS_PATTERN`)    |
| `some-manual-tag`      | _none_ (doesn't match `$CNB_RELEASE_EXTRA_TAGS_PATTERN`)    |
| `1.2.3`                | `latest`, `1.2`, `1`                                        |
| `1.2.3-alpha.12`       | `latest`, `1.2`, `1`                                        |
| `1.2.3+linux`          | `latest`, `1.2+linux`, `1+linux`                            |
| `1.2.3-alpha.12+linux` | `latest`, `1.2+linux`, `1+linux`                            |
+11 −0
Original line number Diff line number Diff line
@@ -43,6 +43,17 @@
      "name": "CNB_RELEASE_IMAGE",
      "description": "CNB release image",
      "default": "$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME"
    },
    {
      "name": "CNB_RELEASE_EXTRA_TAGS_PATTERN",
      "description": "Defines the image tag pattern that `$CNB_RELEASE_IMAGE` should match to push extra tags (supports capturing groups)\n\nDefaults to [SemVer](https://semver.org/) pattern.",
      "default": "^v?(?P<major>[0-9]+)\\.(?P<minor>[0-9]+)\\.(?P<patch>[0-9]+)(?P<suffix>(?P<prerelease>-[0-9A-Za-z-\\.]+)?(?P<build>\\+[0-9A-Za-z-\\.]+)?)$",
      "advanced": true
    },
    {
      "name": "CNB_RELEASE_EXTRA_TAGS",
      "description": "Defines extra tags to publish the _release_ image\n\nSupports capturing group references from `$CNB_RELEASE_EXTRA_TAGS_PATTERN` (ex: `latest \\g<major>.\\g<minor> \\g<major>`)",
      "advanced": true
    }
  ],
  "features": [
+63 −0
Original line number Diff line number Diff line
@@ -41,6 +41,18 @@ spec:
    release-image:
      description: CNB release image
      default: $CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME
    release-extra-tags-pattern:
      description: |-
        Defines the image tag pattern that `$CNB_RELEASE_IMAGE` should match to push extra tags (supports capturing groups)

        Defaults to [SemVer](https://semver.org/) pattern.
      default: ^v?(?P<major>[0-9]+)\.(?P<minor>[0-9]+)\.(?P<patch>[0-9]+)(?P<suffix>(?P<prerelease>-[0-9A-Za-z-\.]+)?(?P<build>\+[0-9A-Za-z-\.]+)?)$
    release-extra-tags:
      description: |-
        Defines extra tags to publish the _release_ image

        Supports capturing group references from `$CNB_RELEASE_EXTRA_TAGS_PATTERN` (ex: `latest \g<major>.\g<minor> \g<major>`)
      default: ''
    skopeo-image:
      description: The docker image used to publish docker image with Skopeo
      default: quay.io/containers/skopeo:latest
@@ -148,6 +160,8 @@ variables:

  # default: one-click publish
  CNB_PROD_PUBLISH_STRATEGY: $[[ inputs.prod-publish-strategy ]]
  CNB_RELEASE_EXTRA_TAGS_PATTERN: $[[ inputs.release-extra-tags-pattern ]]
  CNB_RELEASE_EXTRA_TAGS: $[[ inputs.release-extra-tags ]]
  CNB_PUBLISH_ARGS: $[[ inputs.publish-args ]]

  # default trivy
@@ -266,6 +280,29 @@ stages:
    fi
  }

  function maybe_install_python3() {
    if ! command -v python3 > /dev/null
    then
      if command -v apt-get > /dev/null
      then
        # Debian
        apt-get update
        apt-get install --no-install-recommends --yes --quiet python3
      elif command -v apk > /dev/null
      then
        # Alpine
        apk add --no-cache python3
      elif command -v dnf > /dev/null
      then 
        # Fedora
        dnf install -y -q python3
      else
        log_error "... didn't find any supported package manager to install python3"
        exit 1
      fi
    fi
  }

  function unscope_variables() {
    _scoped_vars=$(env | awk -F '=' "/^scoped__[a-zA-Z0-9_]+=/ {print \$1}" | sort)
    if [[ -z "$_scoped_vars" ]]; then return; fi
@@ -659,6 +696,31 @@ stages:
    rm -f "${FIND_PATTERNS_FILE}"
  }

  function publish_extra_tags() {
    if [[ -z "$CNB_RELEASE_EXTRA_TAGS" ]]
    then
      return
    fi
    # check if tag matches pattern
    maybe_install_python3
    # shellcheck disable=SC2154
    matches=$(python3 -c "import re;print('match' if re.match(r'$CNB_RELEASE_EXTRA_TAGS_PATTERN', '$cnb_tag') else '')")
    if [[ "$matches" ]]
    then
      # apply extra tags substitution
      extra_tags=$(python3 -c "import re;print(re.sub(r'$CNB_RELEASE_EXTRA_TAGS_PATTERN', r'$CNB_RELEASE_EXTRA_TAGS', '$cnb_tag'))")
      log_info "Pushing extra tags (evaluated from original tag \\e[33;1m${cnb_tag}\\e[0m)..."
      for extra_tag in $extra_tags
      do
        log_info "... pushing extra tag: \\e[33;1m${extra_tag}\\e[0m..."
        # shellcheck disable=SC2086,SC2154
        skopeo copy ${TRACE+--debug} --all --src-authfile ~/.docker/config.json --dest-authfile ~/.docker/release-config.json ${CNB_PUBLISH_ARGS} "docker://$CNB_RELEASE_IMAGE" "docker://$cnb_repository:$extra_tag"
      done
    else
      log_info "Extra tags configured, but the released tag (\\e[33;1m${cnb_tag}\\e[0m) doesn't match \$CNB_RELEASE_EXTRA_TAGS_PATTERN..."
    fi
  }

  # ENDSCRIPT

# job prototype
@@ -798,6 +860,7 @@ cnb-publish:
        echo "cnb_digest=$cnb_digest"
      } > cnb.env
      chmod 644 cnb.env
    - publish_extra_tags
  artifacts:
    reports:
      dotenv: