Commit 7ea406aa authored by Pierre Smeyers's avatar Pierre Smeyers
Browse files

feat: support OCI-based registry as pull repos & support user/password authentication

parent 676c41cb

README.md

+31 −1
Original line number Diff line number Diff line
@@ -155,6 +155,36 @@ Each deployment job produces _output variables_ that are propagated to downstrea 


Those variables may be freely used in downstream jobs (for instance to run acceptance tests against the latest deployed environment).



### Working with repositories & OCI-based registries



The Helm template supports indifferently the use of [chart repositories](https://helm.sh/docs/topics/chart_repository/) and [OCI-based registries](https://helm.sh/docs/topics/registries/) (requires Helm 3 or above).



Those can be used both for pulling and/or pushing charts.



#### Configuring pull repositories



The pulling repositories/registries can be configured with the `$HELM_REPOS`.

The value is expected as a (whitespace-separated) list of `repo_name@repo_url` (defaults to `stable@https://charts.helm.sh/stable bitnami@https://charts.bitnami.com/bitnami`).



:warning: When using OCI-based registries, simply prefix the url with `oci://`.



The Helm template also supports user/password authentication for each, simply by defining `HELM_REPO_<NAME>_USER` and `HELM_REPO_<NAME>_PASSWORD` (as project or group secret variables).



:warning: The `<NAME>` part is the `repo_name` transformed in [SCREAMING_SNAKE_CASE](https://en.wikipedia.org/wiki/Snake_case) (uppercase words separated by underscores).



Example: declare the GitLab chart repository from another GitLab project



```yml

variables:

  HELM_REPOS: "stable@https://charts.helm.sh/stable bitnami@https://charts.bitnami.com/bitnami other-proj@${CI_API_V4_URL}/projects/1234/packages/helm/release"

  HELM_REPO_OTHER_PROJ_USER: "gitlab-token"

  # HELM_REPO_OTHER_PROJ_PASSWORD set as a project secret variables

```



#### Configuring the push repository



All configuration parameters are extensively documented in the [`helm-publish` job](#helm-publish-job) chapter.



## Configuration reference



### Secrets management
@@ -188,7 +218,7 @@ The Helm template uses some global configuration used throughout all jobs. 
| `HELM_DEPLOY_ARGS`    | The Helm [command with options](https://helm.sh/docs/helm/helm_upgrade/) to deploy the application (_without dynamic arguments such as release name and chart_) | `upgrade --install --atomic --timeout 120s` |

| `HELM_DELETE_ARGS`    | The Helm [command with options](https://helm.sh/docs/helm/helm_uninstall/) to cleanup the application (_without dynamic arguments such as release name_) | `uninstall` |

| `HELM_DEPLOY_CHART`   | The Helm [chart](https://helm.sh/docs/topics/charts/) to deploy. _Only required if you want to deploy an **external** chart._  | _none_ |

| `HELM_REPOS`          | The Helm [chart repositories](https://helm.sh/docs/topics/chart_repository/) to use (formatted as `repo_name_1@:repo_url_1 repo_name_2@:repo_url_2 ...`) | `stable@https://charts.helm.sh/stable bitnami@https://charts.bitnami.com/bitnami` |

| `HELM_REPOS`          | The Helm [chart repositories](https://helm.sh/docs/topics/chart_repository/) to use (formatted as `repo_name_1@repo_url_1 repo_name_2@repo_url_2 ...`) | `stable@https://charts.helm.sh/stable bitnami@https://charts.bitnami.com/bitnami` |

| `HELM_BASE_APP_NAME`  | Base application name                  | `$CI_PROJECT_NAME` ([see GitLab doc](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html)) |

| `HELM_ENVIRONMENT_URL`    | Default environments url _(only define for static environment URLs declaration)_<br/>_supports late variable expansion (ex: `https://%{environment_name}.helm.acme.com`)_ | _none_ |

kicker.json

+1 −1
Original line number Diff line number Diff line
@@ -29,7 +29,7 @@ 
    },

    {

      "name": "HELM_REPOS",

      "description": "The Helm [chart repositories](https://helm.sh/docs/topics/chart_repository/) to use (formatted as `repo_name_1@:repo_url_1 repo_name_2@:repo_url_2 ...`)",

      "description": "The Helm [chart repositories](https://helm.sh/docs/topics/chart_repository/) to use (formatted as `repo_name_1@repo_url_1 repo_name_2@repo_url_2 ...`)",

      "default": "stable@https://charts.helm.sh/stable bitnami@https://charts.bitnami.com/bitnami"

    },

    {

templates/gitlab-ci-helm.yml

+44 −19
Original line number Diff line number Diff line
@@ -325,10 +325,8 @@ stages: 
    done

  }



  function get_helm_config_opt() {

    echo -n "--registry-config $CI_PROJECT_DIR/.config/helm/registry.json "

    echo -n "--repository-cache $CI_PROJECT_DIR/.cache/helm/repository "

    echo -n "--repository-config $CI_PROJECT_DIR/.config/helm/repositories.yaml "

  function get_helm_opts() {

    echo "${TRACE+--debug} --registry-config $CI_PROJECT_DIR/.config/helm/registry.json --repository-cache $CI_PROJECT_DIR/.cache/helm/repository --repository-config $CI_PROJECT_DIR/.config/helm/repositories.yaml"

  }



  function setup_kubeconfig() {
@@ -350,16 +348,41 @@ stages: 
    ln -s "$CI_PROJECT_DIR/.cache" ~/.cache

    ln -s "$CI_PROJECT_DIR/.config" ~/.config



    helm_opts=$(get_helm_config_opt)

    helm_opts=$(get_helm_opts)



    # Install helm repositories

    for repo in $HELM_REPOS

    do

      repo_name=$(echo "$repo" | cut -d@ -f 1)

      repo_url=$(echo "$repo" | cut -d@ -f 2)

      log_info "--- add repository \\e[32m${repo_name}\\e[0m: \\e[33;1m${repo_url}\\e[0m"

      repo_name_ssc=$(echo "$repo_name" | tr '[:lower:]' '[:upper:]' | tr '[:punct:]' '_')

      repo_user=$(eval echo "\$HELM_REPO_${repo_name_ssc}_USER")

      repo_password=$(eval echo "\$HELM_REPO_${repo_name_ssc}_PASSWORD")



      if [[ "$repo_url" =~ oci://.* ]]

      then

        if [[ "$repo_user" ]] && [[ "$repo_password" ]]

        then

          registry_host=$(echo "$repo_url" | awk -F[/:] '{print $4}')

          log_info "--- login to OCI-registry \\e[32m${repo_name}\\e[0m: \\e[33;1m${registry_host}\\e[0m"

          export HELM_EXPERIMENTAL_OCI=1

          # shellcheck disable=SC2086

          echo "$repo_password" | helm $helm_opts registry login "$registry_host" --username "$repo_user" --password-stdin

        else

          log_warn "--- OCI-registry \\e[32m${repo_name}\\e[0m (\\e[33;1m${repo_url}\\e[0m) defined, but no credentials found (\$HELM_REPO_${repo_name_ssc}_USER/\$HELM_REPO_${repo_name_ssc}_PASSWORD)"

        fi

      else

        if [[ "$repo_user" ]] && [[ "$repo_password" ]]

        then

          log_info "--- add repository \\e[32m${repo_name}\\e[0m: \\e[33;1m${repo_url}\\e[0m (with user/password auth)"

          # shellcheck disable=SC2086

          echo "$repo_password" | helm $helm_opts repo add "$repo_name" "$repo_url" --username "$repo_user" --password-stdin

        else

          log_info "--- add repository \\e[32m${repo_name}\\e[0m: \\e[33;1m${repo_url}\\e[0m (unauthenticated)"

          # shellcheck disable=SC2086

          helm $helm_opts repo add "$repo_name" "$repo_url"

        fi

      fi

    done



    # shellcheck disable=SC2086
@@ -415,7 +438,7 @@ stages: 
      log_info "--- \\e[32mpre-deploy hook\\e[0m (\\e[33;1m${prescript}\\e[0m) not found: skip"

    fi



    helm_opts=$(get_helm_config_opt)

    helm_opts=$(get_helm_opts)

    

    helm_opts="$helm_opts --set ${HELM_ENV_VALUE_NAME}=$environment_type"

    helm_opts="$helm_opts --set ${HELM_HOSTNAME_VALUE_NAME}=$hostname"
@@ -451,7 +474,7 @@ stages: 
    log_info "--- using \\e[32mchart\\e[0m: \\e[33;1m${chart}\\e[0m"



    # shellcheck disable=SC2086

    helm ${TRACE+--debug} $helm_opts $HELM_DEPLOY_ARGS $environment_name $chart

    helm $helm_opts $HELM_DEPLOY_ARGS $environment_name $chart



    # maybe execute post deploy script

    postscript="$HELM_SCRIPTS_DIR/helm-post-deploy.sh"
@@ -494,7 +517,7 @@ stages: 
      log_info "--- \\e[32mpre-delete hook\\e[0m (\\e[33;1m${prescript}\\e[0m) not found: skip"

    fi



    helm_opts=$(get_helm_config_opt)

    helm_opts=$(get_helm_opts)



    if [ -f "$CI_PROJECT_DIR/.kubeconfig" ]; then

      log_info "--- using \\e[32mkubeconfig\\e[0m: \\e[33;1m$CI_PROJECT_DIR/.kubeconfig\\e[0m"
@@ -507,7 +530,7 @@ stages: 
    fi



    # shellcheck disable=SC2086

    helm ${TRACE+--debug} $helm_opts $HELM_DELETE_ARGS $environment_name

    helm $helm_opts $HELM_DELETE_ARGS $environment_name



    # maybe execute post delete script

    postscript="$HELM_SCRIPTS_DIR/helm-post-delete.sh"
@@ -530,7 +553,7 @@ stages: 
    log_info "--- \$environment_name: \\e[33;1m${environment_name}\\e[0m"

    log_info "--- \$environment_type: \\e[33;1m${environment_type}\\e[0m"



    helm_opts=$(get_helm_config_opt)

    helm_opts=$(get_helm_opts)



    if [ -f "$CI_PROJECT_DIR/.kubeconfig" ]; then

      log_info "--- using \\e[32mkubeconfig\\e[0m: \\e[33;1m$CI_PROJECT_DIR/.kubeconfig\\e[0m"
@@ -543,7 +566,7 @@ stages: 
    fi



    # shellcheck disable=SC2086

    helm ${TRACE+--debug} $helm_opts $HELM_TEST_ARGS $environment_name

    helm $helm_opts $HELM_TEST_ARGS $environment_name

  }



  function helm_package() {
@@ -561,15 +584,15 @@ stages: 
    fi



    add_helm_repositories

    helm_opts=$(get_helm_config_opt)

    helm_opts=$(get_helm_opts)



    # helm package

    # shellcheck disable=SC2086

    helm ${TRACE+--debug} $helm_opts $HELM_PACKAGE_ARGS $helm_version_opts $HELM_CHART_DIR --destination helm_packages

    helm $helm_opts $HELM_PACKAGE_ARGS $helm_version_opts $HELM_CHART_DIR --destination helm_packages

  }



  function helm_publish() {

    helm_opts=$(get_helm_config_opt)

    helm_opts=$(get_helm_opts)



    helm_package=$(ls -1 ./helm_packages/*.tgz 2>/dev/null || echo "")

    if [[ -z "$helm_package" ]]; then
@@ -609,10 +632,12 @@ stages: 
      if [[ "$HELM_PUBLISH_URL" =~ oci://.* ]]

      then

        registry_host=$(echo "$HELM_PUBLISH_URL" | awk -F[/:] '{print $4}')

        echo "$password" | helm registry login "$registry_host" --username "$username" --password-stdin

        # shellcheck disable=SC2086

        echo "$password" | helm $helm_opts registry login "$registry_host" --username "$username" --password-stdin

        # enable OCI support prior to v3.8.0

        export HELM_EXPERIMENTAL_OCI=1

        helm ${TRACE+--debug} $helm_opts push "$helm_package" "$HELM_PUBLISH_URL"

        # shellcheck disable=SC2086

        helm $helm_opts push "$helm_package" "$HELM_PUBLISH_URL"

      else

        log_info "Installing cm-push plugin (version ${HELM_CM_PUSH_PLUGIN_VERSION:-latest})..."

        # shellcheck disable=SC2086