Commit 676c41cb authored by Pierre Smeyers's avatar Pierre Smeyers
Browse files

feat: support various publish methods (push, post, put, custom) with auto detection 

BREAKING CHANGE: default publish URL uses GitLab registry
parent 6d7bc432

README.md

+39 −4
Original line number Diff line number Diff line
@@ -311,14 +311,49 @@ Note: You can disable the `semantic-release` integration described herebefore th 


### `helm-publish` job



This job publishes the packaged chart to a release repository or registry. It uses the following variables:

This job publishes the packaged chart to a [chart repository](https://helm.sh/docs/topics/chart_repository/) or [OCI-based registry](https://helm.sh/docs/topics/registries/). It uses the following variables:



| Name                                | description                                   | default value                     |

| ----------------------------------- | --------------------------------------------- | --------------------------------- |

| `HELM_PUBLISH_METHOD`               | HTTP method to use to push the package       | `POST`                  |

| `HELM_PUBLISH_METHOD`               | Method to use to publish the packaged chart (one of `auto`, `push`, `post`, `put`, `custom`, `disabled`) | `auto`                  |

| :lock: `HELM_PUBLISH_USER`          | Helm registry username                       | `$CI_REGISTRY_USER`     |

| :lock: `HELM_PUBLISH_PASSWORD`      | Helm registry password                       | `$CI_REGISTRY_PASSWORD` |

| `HELM_PUBLISH_URL`                  | The URL of the Helm repository to publish your Helm package | `${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/helm/api/release/charts` ([Helm chart registry for GitLab](https://docs.gitlab.com/ee/user/packages/helm_repository/#publish-a-package) on _release_ channel) |

| `HELM_PUBLISH_URL`                  | The URL of the Helm repository to publish your Helm package.<br/>Supports both [chart repository](https://helm.sh/docs/topics/chart_repository/) or [OCI-based registry](https://helm.sh/docs/topics/registries/) (url must be prefixed with `oci://`) | `oci://$CI_REGISTRY/$CI_PROJECT_PATH/charts` ([GitLab's container registry](https://docs.gitlab.com/ee/user/packages/container_registry/)) |

| `HELM_CM_PUSH_PLUGIN_VERSION`       | cm-push plugin version to install (only when using `push` method with a regular chart [repository](https://helm.sh/docs/topics/chart_repository/)) | _none_ (latest) |



#### Supported publish methods



The Helm publish supports several methods, configurable with the `$HELM_PUBLISH_URL` variable:



| Value           | description                                   |

| --------------- | --------------------------------------------- |

|`auto` (default) | tries to auto-detect the most appropriate method | 

|`disabled`       | disables the `helm-publish` job |

|`push`           | if publishing to an [OCI-based registry](https://helm.sh/docs/topics/registries/), publishes with [helm push](https://helm.sh/docs/helm/helm_push/) command; else uses the [cm-push plugin](https://github.com/chartmuseum/helm-push) | 

|`post`           | publishes the package using http `POST` method (compatible with [GitLab packages repository](https://docs.gitlab.com/ee/user/packages/helm_repository/)) | 

|`put`            | publishes the package using http `PUT` method | 

|`custom`         | forces the use of a [custom publish script](#custom-publish-script) | 



> :information_source: The default configuration will use [GitLab's container registry](https://docs.gitlab.com/ee/user/packages/container_registry/) to publish your charts

>

> If you wish to use [GitLab's Helm package repository](https://docs.gitlab.com/ee/user/packages/helm_repository/) instead, simply override:

>

> ```yaml

> variables:

>   # use channel 'release' (can be changed)

>   HELM_PUBLISH_URL: "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/helm/release"

> ```

>

> and leave default `$HELM_PUBLISH_USER`/`$HELM_PUBLISH_PASSWORD` values.



#### Custom publish script



If supported methods don't fit your needs, you may provide a `helm-publish.sh` script (with execution permissions) in your `$HELM_SCRIPTS_DIR` directory to implement the required publish method.



This script may use the following variables:



* `$helm_package`: the packaged chart to publish,

* `$HELM_PUBLISH_USER`, `$HELM_PUBLISH_PASSWORD` and `$HELM_PUBLISH_URL` (see above).



### `helm-test` job

kicker.json

+11 −5
Original line number Diff line number Diff line
@@ -178,12 +178,19 @@ 
    {

      "id": "publish",

      "name": "Publish your chart",

      "description": "Publish your Helm chart",

      "description": "Publishes the chart to a [Helm repository](https://helm.sh/docs/topics/chart_repository/) or [OCI-based registry](https://helm.sh/docs/topics/registries/)",

      "variables": [

        {

          "name": "HELM_PUBLISH_URL",

          "description": "The URL of the Helm repository to publish your Helm package",

          "default": "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/helm/api/release/charts",

          "default": "oci://$CI_REGISTRY/$CI_PROJECT_PATH/charts"

        },

        {

          "name": "HELM_PUBLISH_METHOD",

          "description": "HTTP method to use to push the package",

          "default": "auto",

          "type": "enum",

          "values": ["auto", "push", "post", "put", "custom", "disabled"],

          "advanced": true

        },

        {
@@ -199,9 +206,8 @@ 
          "secret": true

        },

        {

          "name": "HELM_PUBLISH_METHOD",

          "description": "HTTP method to use to push the package",

          "default": "POST",

          "name": "HELM_CM_PUSH_PLUGIN_VERSION",

          "description": "cm-push plugin version to install (only when using `push` method with a regular chart [repository](https://helm.sh/docs/topics/chart_repository/)",

          "advanced": true

        }

      ]

templates/gitlab-ci-helm.yml

+103 −32
Original line number Diff line number Diff line
@@ -58,8 +58,12 @@ variables: 
  HELM_CHART_DIR: "."

  HELM_SCRIPTS_DIR: "."

  HELM_PACKAGE_ARGS: "package --dependency-update"

  HELM_PUBLISH_URL: "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/helm/api/release/charts"

  HELM_PUBLISH_METHOD: "POST"

  # to GitLab's container registry (OCI-compliant)

  HELM_PUBLISH_URL: "oci://$CI_REGISTRY/$CI_PROJECT_PATH/charts"

  # to GitLab's packages repository

  # HELM_PUBLISH_URL: "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/helm/release"

  # HELM_PUBLISH_URL: "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/helm/release"

  HELM_PUBLISH_METHOD: "auto"



  HELM_REPOS: "stable@https://charts.helm.sh/stable bitnami@https://charts.bitnami.com/bitnami"
@@ -542,12 +546,102 @@ stages: 
    helm ${TRACE+--debug} $helm_opts $HELM_TEST_ARGS $environment_name

  }



  function maybe_install_curl() {

  function helm_package() {

    # semantic-release integration

    if [[ "${SEMREL_INFO_ON}" && "${DOCKER_SEMREL_RELEASE_DISABLED}" != "true" ]]

    then

      if [[ -z "${SEMREL_INFO_NEXT_VERSION}" ]]

      then

        log_warn "[semantic-release] no new version to release: skip"

        exit 0

      else

        log_info "[semantic-release] use computed next version: \\e[1;94m${SEMREL_INFO_NEXT_VERSION}\\e[0m"

        helm_version_opts="--app-version ${SEMREL_INFO_NEXT_VERSION} --version ${SEMREL_INFO_NEXT_VERSION}"

      fi

    fi



    add_helm_repositories

    helm_opts=$(get_helm_config_opt)



    # helm package

    # shellcheck disable=SC2086

    helm ${TRACE+--debug} $helm_opts $HELM_PACKAGE_ARGS $helm_version_opts $HELM_CHART_DIR --destination helm_packages

  }



  function helm_publish() {

    helm_opts=$(get_helm_config_opt)



    helm_package=$(ls -1 ./helm_packages/*.tgz 2>/dev/null || echo "")

    if [[ -z "$helm_package" ]]; then

      log_error "No package found to deploy"

      exit 1

    fi

    helm_package_name=$(basename "$helm_package")

    log_info "--- Publishing Helm package ${helm_package_name} to: ${HELM_PUBLISH_URL}..."



    # method to lowercase

    HELM_PUBLISH_METHOD=$(echo "$HELM_PUBLISH_METHOD" | tr '[:upper:]' '[:lower:]')



    # auto-detect method

    if [[ "$HELM_PUBLISH_METHOD" == "auto" ]]

    then

        log_info "--- trying to auto detect publish method..."

        pubscript="$HELM_SCRIPTS_DIR/helm-publish.sh"

        if [[ -f "$pubscript" ]]

        then

          log_info "--- ... custom publish script (\\e[33;1m${postscript}\\e[0m) found: will use"

          HELM_PUBLISH_METHOD=custom

        elif [[ "$HELM_PUBLISH_URL" =~ oci://.* ]]

        then

          log_info "--- ... publish url looks like an OCI registry: will use helm push"

          HELM_PUBLISH_METHOD=push

        else

          log_info "--- ... publish url looks like a Chart repository: will use push method (uses cm-push plugin)"

          log_info "--- ... if auto-selected method is not suited, override with \$HELM_PUBLISH_METHOD or provide a custom publish script"

          HELM_PUBLISH_METHOD=push

        fi

    fi



    username="${HELM_PUBLISH_USER:-$CI_REGISTRY_USER}"

    password="${HELM_PUBLISH_PASSWORD:-$CI_REGISTRY_PASSWORD}"

    case "$HELM_PUBLISH_METHOD" in

    push)

      if [[ "$HELM_PUBLISH_URL" =~ oci://.* ]]

      then

        registry_host=$(echo "$HELM_PUBLISH_URL" | awk -F[/:] '{print $4}')

        echo "$password" | helm registry login "$registry_host" --username "$username" --password-stdin

        # enable OCI support prior to v3.8.0

        export HELM_EXPERIMENTAL_OCI=1

        helm ${TRACE+--debug} $helm_opts push "$helm_package" "$HELM_PUBLISH_URL"

      else

        log_info "Installing cm-push plugin (version ${HELM_CM_PUSH_PLUGIN_VERSION:-latest})..."

        # shellcheck disable=SC2086

        helm $helm_opts plugin install ${HELM_CM_PUSH_PLUGIN_VERSION:+--version "$HELM_CM_PUSH_PLUGIN_VERSION"} https://github.com/chartmuseum/helm-push || true

        # shellcheck disable=SC2086

        helm $helm_opts cm-push --username "$username" --password "$password" "$helm_package" "$HELM_PUBLISH_URL"

      fi

      ;;

    post)

      if ! command -v curl > /dev/null

      then

        log_info "--- installing curl (required to publish Helm charts)..."

        apk add --no-cache curl

      fi

      curl --fail --request POST --form "chart=@$helm_package" --user "$username:$password" "$HELM_PUBLISH_URL"

      ;;

    put)

      wget -v --method=PUT --user="$username" --password="$password" --body-file="$helm_package" "$HELM_PUBLISH_URL/$helm_package_name" -O -

      ;;

    custom)

      pubscript="$HELM_SCRIPTS_DIR/helm-publish.sh"

      log_info "--- run custom publish script (\\e[33;1m${pubscript}\\e[0m)"

      exec_hook "$pubscript"

      ;;

    *)

      log_error "Unsupported publish method: $HELM_PUBLISH_METHOD"

      exit 1

      ;;

    esac

  }



  unscope_variables
@@ -767,13 +861,7 @@ helm-package: 
    - install_ca_certs "${CUSTOM_CA_CERTS:-$DEFAULT_CA_CERTS}"

    - add_helm_repositories

  script:

    - |

      if [[ "$SEMREL_INFO_ON" ]] && [[ "$SEMREL_INFO_NEXT_VERSION" ]] && [[ "$HELM_SEMREL_RELEASE_DISABLED" != "true" ]]

      then

        log_info "semantic-release info is activated, using computed next version for release: \\e[1;94m${SEMREL_INFO_NEXT_VERSION}\\e[0m"

        helm_version_opts="--version ${SEMREL_INFO_NEXT_VERSION}"

      fi

    - helm $HELM_PACKAGE_ARGS ${TRACE+--debug} $helm_version_opts $HELM_CHART_DIR --destination helm_packages

    - helm_package

  rules:

    - exists:

        - "**/Chart.yaml"
@@ -790,26 +878,9 @@ helm-publish: 
  extends: .helm-base

  stage: publish

  script:

    - |

      package=$(ls -1 ./helm_packages/*.tgz 2>/dev/null || echo "")

      if [ -n "$HELM_PUBLISH_URL" ] && [ -n "${package}" ]

      then

        package_file=$(basename ${package})

        log_info "publishing helm chart ${package_file} to release url: ${HELM_PUBLISH_URL}"

        username="${HELM_PUBLISH_USER:-$CI_REGISTRY_USER}"

        password="${HELM_PUBLISH_PASSWORD:-$CI_REGISTRY_PASSWORD}"

        if [[ "$HELM_PUBLISH_METHOD" == "POST" ]]

        then

          maybe_install_curl

          curl --fail --request POST --form "chart=@${package}" --user "$username:$password" $HELM_PUBLISH_URL

        else

          wget -v --method=PUT --user="$username" --password="$password" --body-file="${package}" "$HELM_PUBLISH_URL/${package_file}" -O -

        fi

      else

        log_error "No Chart to deploy! url is: $HELM_PUBLISH_URL, and package found is: ${package}"

      fi

    - helm_publish

  rules:

    - if: $HELM_PUBLISH_URL == null || $CI_COMMIT_REF_NAME !~ $PROD_REF

    - if: '$HELM_PUBLISH_URL == null || $HELM_PUBLISH_URL == "" || $CI_COMMIT_REF_NAME !~ $PROD_REF || $HELM_PUBLISH_METHOD == "disabled"'

      when: never

    - if: '$AUTODEPLOY_TO_PROD == "true"'

    # else: manual + blocking