Commit e56d313c authored by Pierre Smeyers's avatar Pierre Smeyers
Browse files

Merge branch 'feat/refactoring' into 'master' 

redesign & breaking changes

Closes #29, #25, #28, and #23

See merge request to-be-continuous/helm!49
parents c5770bd5 ed861d9e

README.md

+106 −44
Original line number Diff line number Diff line
@@ -155,6 +155,36 @@ Each deployment job produces _output variables_ that are propagated to downstrea 


Those variables may be freely used in downstream jobs (for instance to run acceptance tests against the latest deployed environment).



### Working with repositories & OCI-based registries



The Helm template supports indifferently the use of [chart repositories](https://helm.sh/docs/topics/chart_repository/) and [OCI-based registries](https://helm.sh/docs/topics/registries/) (requires Helm 3 or above).



Those can be used both for pulling and/or pushing charts.



#### Configuring pull repositories



The pulling repositories/registries can be configured with the `$HELM_REPOS`.

The value is expected as a (whitespace-separated) list of `repo_name@repo_url` (defaults to `stable@https://charts.helm.sh/stable bitnami@https://charts.bitnami.com/bitnami`).



:warning: When using OCI-based registries, simply prefix the url with `oci://`.



The Helm template also supports user/password authentication for each, simply by defining `HELM_REPO_<NAME>_USER` and `HELM_REPO_<NAME>_PASSWORD` (as project or group secret variables).



:warning: The `<NAME>` part is the `repo_name` transformed in [SCREAMING_SNAKE_CASE](https://en.wikipedia.org/wiki/Snake_case) (uppercase words separated by underscores).



Example: declare the GitLab chart repository from another GitLab project



```yml

variables:

  HELM_REPOS: "stable@https://charts.helm.sh/stable bitnami@https://charts.bitnami.com/bitnami other-proj@${CI_API_V4_URL}/projects/1234/packages/helm/release"

  HELM_REPO_OTHER_PROJ_USER: "gitlab-token"

  # HELM_REPO_OTHER_PROJ_PASSWORD set as a project secret variables

```



#### Configuring the push repository



All configuration parameters are extensively documented in the [`helm-publish` job](#helm-publish-job) chapter.



## Configuration reference



### Secrets management
@@ -188,7 +218,7 @@ The Helm template uses some global configuration used throughout all jobs. 
| `HELM_DEPLOY_ARGS`    | The Helm [command with options](https://helm.sh/docs/helm/helm_upgrade/) to deploy the application (_without dynamic arguments such as release name and chart_) | `upgrade --install --atomic --timeout 120s` |

| `HELM_DELETE_ARGS`    | The Helm [command with options](https://helm.sh/docs/helm/helm_uninstall/) to cleanup the application (_without dynamic arguments such as release name_) | `uninstall` |

| `HELM_DEPLOY_CHART`   | The Helm [chart](https://helm.sh/docs/topics/charts/) to deploy. _Only required if you want to deploy an **external** chart._  | _none_ |

| `HELM_REPOS`          | The Helm [chart repositories](https://helm.sh/docs/topics/chart_repository/) to use (formatted as `repo_name_1@:repo_url_1 repo_name_2@:repo_url_2 ...`) | `stable@https://charts.helm.sh/stable bitnami@https://charts.bitnami.com/bitnami` |

| `HELM_REPOS`          | The Helm [chart repositories](https://helm.sh/docs/topics/chart_repository/) to use (formatted as `repo_name_1@repo_url_1 repo_name_2@repo_url_2 ...`) | `stable@https://charts.helm.sh/stable bitnami@https://charts.bitnami.com/bitnami` |

| `HELM_BASE_APP_NAME`  | Base application name                  | `$CI_PROJECT_NAME` ([see GitLab doc](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html)) |

| `HELM_ENVIRONMENT_URL`    | Default environments url _(only define for static environment URLs declaration)_<br/>_supports late variable expansion (ex: `https://%{environment_name}.helm.acme.com`)_ | _none_ |
@@ -196,13 +226,13 @@ The Helm template uses some global configuration used throughout all jobs. 


Review environments are dynamic and ephemeral environments to deploy your _ongoing developments_ (a.k.a. _feature_ or _topic_ branches).



They are **enabled by default** and can be disabled by setting the `HELM_REVIEW_DISABLED` variable (see below).

They are **disabled by default** and can be enabled by setting the `HELM_REVIEW_ENABLED` variable (see below).



Here are variables supported to configure review environments:



| Name                     | description                            | default value     |

| ------------------------ | -------------------------------------- | ----------------- |

| `HELM_REVIEW_DISABLED`   | Set to `true` to disable `review` env            | _none_ (enabled) |

| `HELM_REVIEW_ENABLED`    | Set to `true` to enable `review` env   | _none_ (enabled) |

| `HELM_REVIEW_APP_NAME`   | Application name for `review` env      | `"${HELM_BASE_APP_NAME}-${CI_ENVIRONMENT_SLUG}"` (ex: `myproject-review-fix-bug-12`) |

| `HELM_REVIEW_ENVIRONMENT_URL`| The review environments url _(only define for static environment URLs declaration and if different from default)_ | `$HELM_ENVIRONMENT_URL` |

| `HELM_REVIEW_NAMESPACE`  | The Kubernetes namespace to use for `review` env _(only define to override default)_ | `$KUBE_NAMESPACE` |
@@ -213,13 +243,13 @@ Here are variables supported to configure review environments: 


The integration environment is the environment associated to your integration branch (`develop` by default).



It is **enabled by default** and can be disabled by setting the `HELM_INTEG_DISABLED` variable (see below).

It is **disabled by default** and can be enabled by setting the `HELM_INTEG_ENABLED` variable (see below).



Here are variables supported to configure the integration environment:



| Name                     | description                            | default value     |

| ------------------------ | -------------------------------------- | ----------------- |

| `HELM_INTEG_DISABLED`    | Set to `true` to disable `integration` env       | _none_ (enabled) |

| `HELM_INTEG_ENABLED`     | Set to `true` to enable `integration` env | _none_ (enabled) |

| `HELM_INTEG_APP_NAME`    | Application name for `integration` env | `$HELM_BASE_APP_NAME-integration` |

| `HELM_INTEG_ENVIRONMENT_URL`| The integration environment url _(only define for static environment URLs declaration and if different from default)_ | `$HELM_ENVIRONMENT_URL` |

| `HELM_INTEG_NAMESPACE`   | The Kubernetes namespace to use for `integration` env _(only define to override default)_ | `$KUBE_NAMESPACE` |
@@ -230,13 +260,13 @@ Here are variables supported to configure the integration environment: 


The staging environment is an iso-prod environment meant for testing and validation purpose associated to your production branch (`master` by default).



It is **enabled by default** and can be disabled by setting the `HELM_STAGING_DISABLED` variable (see below).

It is **disabled by default** and can be enabled by setting the `HELM_STAGING_ENABLED` variable (see below).



Here are variables supported to configure the staging environment:



| Name                     | description                            | default value     |

| ------------------------ | -------------------------------------- | ----------------- |

| `HELM_STAGING_DISABLED`  | Set to `true` to disable `staging` env           | _none_ (enabled) |

| `HELM_STAGING_ENABLED`   | Set to `true` to enable `staging` env  | _none_ (enabled) |

| `HELM_STAGING_APP_NAME`  | Application name for `staging` env     | `$HELM_BASE_APP_NAME-staging` |

| `HELM_STAGING_ENVIRONMENT_URL`| The staging environment url _(only define for static environment URLs declaration and if different from default)_ | `$HELM_ENVIRONMENT_URL` |

| `HELM_STAGING_NAMESPACE` | The Kubernetes namespace to use for `staging` env _(only define to override default)_ | `$KUBE_NAMESPACE` |
@@ -247,13 +277,13 @@ Here are variables supported to configure the staging environment: 


The production environment is the final deployment environment associated with your production branch (`master` by default).



It is **enabled by default** and can be disabled by setting the `HELM_PROD_DISABLED` variable (see below).

It is **disabled by default** and can be enabled by setting the `HELM_PROD_ENABLED` variable (see below).



Here are variables supported to configure the production environment:



| Name                     | description                            | default value     |

| ------------------------ | -------------------------------------- | ----------------- |

| `HELM_PROD_DISABLED`     | Set to `true` to disable `production` env        | _none_ (enabled)  |

| `HELM_PROD_ENABLED`      | Set to `true` to enable `production` env | _none_ (enabled)  |

| `HELM_PROD_APP_NAME`     | Application name for `production` env  | `$HELM_BASE_APP_NAME` |

| `HELM_PROD_ENVIRONMENT_URL`| The production environment url _(only define for static environment URLs declaration and if different from default)_ | `$HELM_ENVIRONMENT_URL` |

| `HELM_PROD_NAMESPACE`    | The Kubernetes namespace to use for `production` env _(only define to override default)_ | `$KUBE_NAMESPACE` |
@@ -292,55 +322,87 @@ This job runs [Kube-Score](https://kube-score.com/) on the resources to be creat 
| `HELM_KUBE_SCORE_IMAGE`   | The Docker image used to run [Kube-Score](https://kube-score.com/)   | `zegl/kube-score:latest-helm3` |

| `HELM_KUBE_SCORE_ARGS`   | Arguments used by the helm-score job   | _none_ |



### Charts publishing



The template builds a chart package that may be pushed as two distinct packages, depending on a certain _workflow_:



1. **snapshot**: the chart is first packaged and then pushed to some registry as

  the **snapshot** image. It can be seen as the raw result of the build, but still **untested and unreliable**.

2. **release**: once the snapshot chart has been thoroughly tested (both by `package-test` stage jobs and/or `acceptance`

  stage jobs after being deployed to some server), then the chart is pushed one more time as the **release** chart.

  This second push can be seen as the **promotion** of the snapshot chart being now **tested and reliable**.



Common variables for `helm-package` and `helm-pusblish`:



| Name                              | description                                  | default value           |

| --------------------------------- | -------------------------------------------- | ----------------------- |

| `HELM_REPO_PUBLISH_METHOD`        | HTTP method to use to push the package       | `POST`                  |

| :lock: `HELM_REPO_USER`           | Helm registry username                       | `$CI_REGISTRY_USER`     |

| :lock: `HELM_REPO_PASSWORD`       | Helm registry password                       | `$CI_REGISTRY_PASSWORD` |



#### `helm-package` job

### `helm-package` job



This job [packages your chart into an archive](https://helm.sh/docs/helm/helm_package/), optionaly push it to a snapshot repository and uses the following variables:

This job [packages](https://helm.sh/docs/helm/helm_package/) the Helm chart. It uses the following variables:



| Name                                | description                                   | default value                     |

| ------------------------------------- | ---------------------------------------- | ----------------- |

| ----------------------------------- | --------------------------------------------- | --------------------------------- |

| `HELM_PACKAGE_ARGS`                 | The Helm [command with options](https://helm.sh/docs/helm/helm_package/) to perform the packaging (_without dynamic arguments such as the chart path_)   | `package --dependency-update` |

| `HELM_PUBLISH_SNAPSHOT_ENABLED`     | Set to `true` to enable publishing the snapshot (untested) chart during the packaging step | _none_ (disabled) |

| `HELM_SEMREL_RELEASE_DISABLED`      | Set to `true` to disable usage of `semantic-release` release info for helm package (see next chapter) | _none_ (enabled) |

| `HELM_PUBLISH_SNAPSHOT_URL`           | The URL of the Helm repository to publish your Helm package as a snapshot | _gitlab repository on snapshot channel_ `${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/helm/api/snapshot/charts` |

| `HELM_REPO_SNAPSHOT_PUBLISH_METHOD`   | HTTP method to use to push the package        | `$HELM_REPO_PUBLISH_METHOD`       |

| `HELM_REPO_SNAPSHOT_USER`             | Snapshot repository username | `$HELM_REPO_USER` |

| :lock: `HELM_REPO_SNAPSHOT_PASSWORD`  | Snapshot repository password | `$HELM_REPO_PASSWORD` |



#### `semantic-release` integration



If you activate the [`semantic-release-info` job from the `semantic-release` template](https://gitlab.com/to-be-continuous/semantic-release/#semantic-release-info-job), the `helm-publish` job will automatically use the generated next version info for both application version (`--app-version`) and chart version (`--version`).

If you activate the [`semantic-release-info` job from the `semantic-release` template](https://gitlab.com/to-be-continuous/semantic-release/#semantic-release-info-job), the `helm-package` job will automatically use the generated next version info for the chart version (`--version`).



If no next version info is generated by `semantic-release`, the package will be created, but without versioning info.

If no next version info is determined by `semantic-release`, the package will be created, but without versioning info.



Note: You can disable the `semantic-release` integration described herebefore the `HELM_SEMREL_RELEASE_DISABLED` variable.



#### `helm-publish` job

#### Chart version management



Depending on the branch and the step in the CI/CD pipeline, the chart will be packaged with a different version.



The general version format will be `<x.y.z>-<label>`:



* `<x.y.z>`: 

    * if [semantic-release integration](#semantic-release-integration) is enabled: uses the version determined by `semantic-release`, 

    * otherwise uses the version from the chart file

* `<label>`:

    * on the production branch (`main` or `master` by default), no trailing label is used

    * on any other branch, `$CI_COMMIT_REF_SLUG` is used as trailing label<br/>

    _(ex: `review-feature-12` on branch `review/feature-12`)_

    * :warning: when `HELM_PUBLISH_SNAPSHOT_ENABLED` is enabled, the chart is additionally packaged (and published) with a label suffixed with `snapshot`<br/>

    _(ex: `snapshot` on production branch and `review-feature-12-snapshot` on branch `review/feature-12`)_



### `helm-publish` job



This job push helm package to a release repository and uses the following variables:

This job publishes the packaged chart to a [chart repository](https://helm.sh/docs/topics/chart_repository/) or [OCI-based registry](https://helm.sh/docs/topics/registries/). It uses the following variables:



| Name                                | description                                   | default value                     |

| ----------------------------------- | --------------------------------------------- | --------------------------------- |

| `HELM_PUBLISH_URL`                  | The URL of the Helm repository to publish your Helm package | _gitlab repository on release channel_ `${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/helm/api/release/charts`  |

| `HELM_REPO_RELEASE_PUBLISH_METHOD`  | HTTP method to use to push the package        | `$HELM_REPO_PUBLISH_METHOD`       |

| `HELM_REPO_RELEASE_USER`            | Release repository username (for PUT request auth)  | `$HELM_REPO_USER`                 |

| :lock: `HELM_REPO_RELEASE_PASSWORD` | Release repository password (for PUT request auth)  | `$HELM_REPO_PASSWORD`             |

| `HELM_PUBLISH_METHOD`               | Method to use to publish the packaged chart (one of `auto`, `push`, `post`, `put`, `custom`, `disabled`) | `auto`                  |

| :lock: `HELM_PUBLISH_USER`          | Helm registry username                       | `$CI_REGISTRY_USER`     |

| :lock: `HELM_PUBLISH_PASSWORD`      | Helm registry password                       | `$CI_REGISTRY_PASSWORD` |

| `HELM_PUBLISH_URL`                  | The URL of the Helm repository to publish your Helm package.<br/>Supports both [chart repository](https://helm.sh/docs/topics/chart_repository/) or [OCI-based registry](https://helm.sh/docs/topics/registries/) (url must be prefixed with `oci://`) | `oci://$CI_REGISTRY/$CI_PROJECT_PATH/charts` ([GitLab's container registry](https://docs.gitlab.com/ee/user/packages/container_registry/)) |

| `HELM_PUBLISH_ON`                   | Defines on which branch(es) the publish job shall be enabled (`prod` to enable on production branch only, `protected` to enable on protected references and `all` to enable on all Git references) | `prod`                  |

| `HELM_CM_PUSH_PLUGIN_VERSION`       | cm-push plugin version to install (only when using `push` method with a regular chart [repository](https://helm.sh/docs/topics/chart_repository/)) | _none_ (latest) |





#### Supported publish methods



The Helm publish supports several methods, configurable with the `$HELM_PUBLISH_URL` variable:



| Value           | description                                   |

| --------------- | --------------------------------------------- |

|`auto` (default) | tries to auto-detect the most appropriate method | 

|`disabled`       | disables the `helm-publish` job |

|`push`           | if publishing to an [OCI-based registry](https://helm.sh/docs/topics/registries/), publishes with [helm push](https://helm.sh/docs/helm/helm_push/) command; else uses the [cm-push plugin](https://github.com/chartmuseum/helm-push) | 

|`post`           | publishes the package using http `POST` method (compatible with [GitLab packages repository](https://docs.gitlab.com/ee/user/packages/helm_repository/)) | 

|`put`            | publishes the package using http `PUT` method | 

|`custom`         | forces the use of a [custom publish script](#custom-publish-script) | 



> :information_source: The default configuration will use [GitLab's container registry](https://docs.gitlab.com/ee/user/packages/container_registry/) to publish your charts

>

> If you wish to use [GitLab's Helm package repository](https://docs.gitlab.com/ee/user/packages/helm_repository/) instead, simply override:

>

> ```yaml

> variables:

>   # use channel 'release' (can be changed)

>   HELM_PUBLISH_URL: "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/helm/release"

> ```

>

> and leave default `$HELM_PUBLISH_USER`/`$HELM_PUBLISH_PASSWORD` values.



#### Custom publish script



If supported methods don't fit your needs, you may provide a `helm-publish.sh` script (with execution permissions) in your `$HELM_SCRIPTS_DIR` directory to implement the required publish method.



This script may use the following variables:



* `$helm_package`: the packaged chart to publish,

* `$HELM_PUBLISH_USER`, `$HELM_PUBLISH_PASSWORD` and `$HELM_PUBLISH_URL` (see above).



### `helm-test` job

kicker.json

+36 −66
Original line number Diff line number Diff line
@@ -29,7 +29,7 @@ 
    },

    {

      "name": "HELM_REPOS",

      "description": "The Helm [chart repositories](https://helm.sh/docs/topics/chart_repository/) to use (formatted as `repo_name_1@:repo_url_1 repo_name_2@:repo_url_2 ...`)",

      "description": "The Helm [chart repositories](https://helm.sh/docs/topics/chart_repository/) to use (formatted as `repo_name_1@repo_url_1 repo_name_2@repo_url_2 ...`)",

      "default": "stable@https://charts.helm.sh/stable bitnami@https://charts.bitnami.com/bitnami"

    },

    {
@@ -82,24 +82,6 @@ 
      "description": "The name of the Helm [value](https://helm.sh/docs/chart_best_practices/values/) containing the _environment hostname_ (extracted from the environment URL)",

      "default": "hostname",

      "advanced": true

    },

    {

      "name": "HELM_REPO_USER",

      "description": "Helm repository username",

      "default": "$CI_REGISTRY_USER",

      "advanced": true

    },

    {

      "name": "HELM_REPO_PASSWORD",

      "description": "Helm repository password",

      "default": "$CI_REGISTRY_PASSWORD",

      "advanced": true

    },

    {

      "name": "HELM_REPO_PUBLISH_METHOD",

      "description": "HTTP method to use to push the package",

      "default": "POST",

      "advanced": true

    }

  ],

  "features": [
@@ -186,71 +168,59 @@ 
          "advanced": true

        },

        {

          "name": "HELM_SEMREL_RELEASE_DISABLED",

          "description": "Disable semantic-release integration",

          "name": "HELM_PUBLISH_SNAPSHOT_ENABLED",

          "description": "Set to `true` to enable publishing the snapshot (untested) chart during the packaging step",

          "type": "boolean",

          "advanced": true

        },

        {

          "name": "HELM_PUBLISH_SNAPSHOT_URL",

          "description": "The URL of the Helm repository to publish your Helm package as a snapshot",

          "default": "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/helm/api/snapshot/charts",

          "advanced": true

        },

        {

          "name": "HELM_REPO_SNAPSHOT_USER",

          "description": "Snapshot repository username",

          "default": "$HELM_REPO_USER",

          "secret": true,

          "advanced": true

        },

        {

          "name": "HELM_REPO_SNAPSHOT_PASSWORD",

          "description": "Snapshot repository password",

          "default": "$HELM_REPO_PASSWORD",

          "secret": true,

          "advanced": true

        },

        {

          "name": "HELM_REPO_SNAPSHOT_PUBLISH_METHOD",

          "description": "HTTP method to use to push the snapshot package",

          "default": "$HELM_REPO_PUBLISH_METHOD",

          "name": "HELM_SEMREL_RELEASE_DISABLED",

          "description": "Disable semantic-release integration",

          "type": "boolean",

          "advanced": true

        }

        

        

        

      ]

    },

    {

      "id": "publish",

      "name": "Publish your chart",

      "description": "Publish your Helm chart",

      "description": "Publishes the chart to a [Helm repository](https://helm.sh/docs/topics/chart_repository/) or [OCI-based registry](https://helm.sh/docs/topics/registries/)",

      "variables": [

        {

          "name": "HELM_PUBLISH_URL",

          "description": "The URL of the Helm repository to publish your Helm package",

          "default": "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/helm/api/release/charts",

          "advanced": true

          "default": "oci://$CI_REGISTRY/$CI_PROJECT_PATH/charts"

        },

        {

          "name": "HELM_REPO_RELEASE_USER",

          "description": "Release repository username",

          "default": "$HELM_REPO_USER",

          "secret": true,

          "name": "HELM_PUBLISH_METHOD",

          "description": "HTTP method to use to push the package",

          "default": "auto",

          "type": "enum",

          "values": ["auto", "push", "post", "put", "custom", "disabled"],

          "advanced": true

        },

        {

          "name": "HELM_REPO_RELEASE_PASSWORD",

          "description": "Release repository password",

          "default": "$HELM_REPO_PASSWORD",

          "secret": true,

          "advanced": true

          "name": "HELM_PUBLISH_USER",

          "description": "Helm repository username",

          "default": "$CI_REGISTRY_USER",

          "secret": true

        },

        {

          "name": "HELM_PUBLISH_PASSWORD",

          "description": "Helm repository password",

          "default": "$CI_REGISTRY_PASSWORD",

          "secret": true

        },

        {

          "name": "HELM_PUBLISH_ON",

          "description": "Defines on which branch(es) the publish job shall be enabled",

          "default": "prod",

          "type": "enum",

          "values": ["prod", "protected", "all"]

        },

        {

          "name": "HELM_REPO_RELEASE_PUBLISH_METHOD",

          "description": "HTTP method to use to push the release package",

          "default": "$HELM_REPO_PUBLISH_METHOD",

          "name": "HELM_CM_PUSH_PLUGIN_VERSION",

          "description": "cm-push plugin version to install (only when using `push` method with a regular chart [repository](https://helm.sh/docs/topics/chart_repository/)",

          "advanced": true

        }

      ]
@@ -259,7 +229,7 @@ 
      "id": "review",

      "name": "Review",

      "description": "Dynamic review environments for your topic branches (see GitLab [Review Apps](https://docs.gitlab.com/ee/ci/review_apps/))",

      "disable_with": "HELM_REVIEW_DISABLED",

      "enable_with": "HELM_REVIEW_ENABLED",

      "variables": [

        {

          "name": "HELM_REVIEW_APP_NAME",
@@ -296,7 +266,7 @@ 
      "id": "integration",

      "name": "Integration",

      "description": "A continuous-integration environment associated to your integration branch (`develop` by default)",

      "disable_with": "HELM_INTEG_DISABLED",

      "enable_with": "HELM_INTEG_ENABLED",

      "variables": [

        {

          "name": "HELM_INTEG_APP_NAME",
@@ -333,7 +303,7 @@ 
      "id": "staging",

      "name": "Staging",

      "description": "An iso-prod environment meant for testing and validation purpose on your production branch (`master` by default)",

      "disable_with": "HELM_STAGING_DISABLED",

      "enable_with": "HELM_STAGING_ENABLED",

      "variables": [

        {

          "name": "HELM_STAGING_APP_NAME",
@@ -370,7 +340,7 @@ 
      "id": "prod",

      "name": "Production",

      "description": "The production environment",

      "disable_with": "HELM_PROD_DISABLED",

      "enable_with": "HELM_PROD_ENABLED",

      "variables": [

        {

          "name": "AUTODEPLOY_TO_PROD",

templates/gitlab-ci-helm.yml

+197 −112

File changed.

Preview size limit exceeded, changes collapsed.