1. Create an issue describing the bug or enhancement you want to propose (select the right issue template).
2. Make sure the issue has been reviewed and agreed.
3. Create a Merge Request, from your **own** fork (see [forking workflow](https://docs.gitlab.com/ee/user/project/repository/forking_workflow.html) documentation).
3. Create a Merge Request, from your **own** fork (see [forking workflow](https://docs.gitlab.com/user/project/repository/forking_workflow/) documentation).
Don't hesitate to mark your MR as `Draft` as long as you think it's not ready to be reviewed.
This project implements a GitLab CI/CD template to run your automated tests with [GitOps](https://doc.acme-tech.com).
This project implements a template to integrate GitOps deployment strategy with your GitLab CI/CD pipelines.
More specifically, this template allows you to trigger a change to a JSON or YAML file in a remote Git repository from your project pipeline.
For example, it will allow you to automatically update an image tag in a [Helm Values file](https://helm.sh/docs/chart_template_guide/values_files/) as soon as a new version of the container image is produced.
> [!important]
> This technique is certainly not the only nor the best way to implement a GitOps deployment strategy but in some cases it might help.
>
> Whenever possible, we recommend using dedicated tools, such as:
>
> - [Auto image update](https://fluxcd.io/flux/guides/image-update/) with FluxCD,
> - [Argo CD Image Updater](https://argocd-image-updater.readthedocs.io/) with ArgoCD,
> - even [Renovate](https://docs.renovatebot.com/) can help you implement the same very smartly.
## Usage
**component: $CI_SERVER_FQDN**
This template can be used both as a [CI/CD component](https://docs.gitlab.com/ee/ci/components/#use-a-component)
or using the legacy [`include:project`](https://docs.gitlab.com/ee/ci/yaml/index.html#includeproject) syntax.
This template can be used both as a [CI/CD component](https://docs.gitlab.com/ci/components/#use-a-component)
or using the legacy [`include:project`](https://docs.gitlab.com/ci/yaml/#includeproject) syntax.
### Use as a CI/CD component
@@ -15,9 +28,8 @@ Add the following to your `.gitlab-ci.yml`:
| `image` / `GITOPS_IMAGE` | The Docker image used to run GitOps. | `docker.io/gitops:latest`<br/>[](https://to-be-continuous.gitlab.io/doc/secu/trivy-GITOPS_IMAGE) |
| `project-dir` / `GITOPS_PROJECT_DIR` | The GitOps project directory (containing test scripts) | `.` |
| `review-enabled` / `REVIEW_ENABLED` | Set to `true` to enable GitOps tests on review environments (dynamic environments instantiated on development branches) | _none_ (disabled) |
| `image` / `GITOPS_IMAGE` | The Docker image used to run GitOps (`git` + `yq`) | `docker.io/alpine/git:latest`<br/>[](https://to-be-continuous.gitlab.io/doc/secu/trivy-GITOPS_IMAGE) |
| :lock: `GITOPS_TRIGGER_ONPROD` | The GitOps changes to trigger on the production branch (JSON) - [see below](#triggers-definition-json) | _none_ (disabled) |
| :lock: `GITOPS_TRIGGER_ONINTEG` | The GitOps changes to trigger on the integration branch (JSON) - [see below](#triggers-definition-json) | _none_ (disabled) |
| :lock: `GITOPS_TRIGGER_ONDEV` | The GitOps changes to trigger on the development branches (JSON) - [see below](#triggers-definition-json) | _none_ (disabled) |
| :lock: `GITOPS_TRIGGER_ONRELEASE` | The GitOps changes to trigger on release tags (JSON) - [see below](#triggers-definition-json) | _none_ (disabled) |
### Triggers definition (JSON)
The GitOps changes to trigger must be defined as a JSON structure, defining:
- whether to trigger automatically or manually,
- the target Git repositories and branch,
- the files to modify,
- the [`yq eval` command](https://mikefarah.gitbook.io/yq/operators/eval) to apply (supports late variable expansion).
:warning: the triggers definition MUST be managed as a project CI/CD variable (outside of the project `.gitlab-ci.yaml` then) for two reasons:
- target repository URL bears their own authentication credentials,
- allows a loose coupling between the source project and the target repositories to trigger (you don't necessarilly want to update the source project `.gitlab-ci.yaml` every time you change the triggers).
The triggers definition is formatted as follows:
```json
{
"when":"<whether to trigger changes manually or automatically (one of 'manual' or 'on_success')>",
"commit_message":"<optional /The commit message template to use for GitOps trigger (supports late variable expansion)>",
"triggers":[
{
"url":"<The target repository URL to push changes, including credentials>",
"branch":"<The target repository branch to push changes to>",
"commit_message":"<optional / The repo specific commit message template to use (supports late variable expansion)>",
"dry_run":"<optional / true or false>",
"changes":[
{
"file":"<The target file to apply the 'yq eval' command>",
"yq_eval":"<The 'yq eval' command to apply (supports late variable expansion) - see https://mikefarah.gitbook.io/yq/operators/eval>"
},
...
]
},
...
]
}
```
:information_source: The triggers definition must respect [this JSON schema](./triggers-schema-1.json).
### Example
```json
{
"when":"manual",
"commit_message":"chore(gitops): update deployment values triggered from commit #$CI_COMMIT_SHA (see $CI_PIPELINE_URL)",
> The template supports a **late variable expansion mechanism** for the `commit_message` and `yq_eval` JSON fields.
>
> It allows you to inject dynamically evaluated variables (coming from the pipeline execution context), using the `%{somevar}` (or `$${somevar}`) syntax.
>
> The above example makes use of it to inject `$docker_tag` and `$docker_digest` variables, propagated by the Docker template.
>
> :bulb: on the other hand, the here-above `commit_message` field doesn't need to use this technique here as `$CI_COMMIT_SHA` and `$CI_PIPELINE_URL` are not _late evaluated_.
> This template doesn't support any fancy mechanism to provide authentication, but instead assumes the target Git repository URL bears its own authentication credentials.
> :warning: make sure to manage those credentials safely, as any other secret:
>
> - never store them directly in your source code,
> - manage them as [project or group CI/CD variables](https://docs.gitlab.com/ci/variables/#for-a-project):
> * [**masked**](https://docs.gitlab.com/ci/variables/#mask-a-cicd-variable) to prevent them from being inadvertently displayed in your job logs,
> * [**protected**](https://docs.gitlab.com/ci/variables/#protected-cicd-variables) if you want to secure some secrets you don't want everyone in the project to have access to (for instance production secrets).
In addition to a textual report in the console, this job produces the following reports, kept for one day:
| `TBC_VAULT_IMAGE` | The [Vault Secrets Provider](https://gitlab.com/to-be-continuous/tools/vault-secrets-provider) image to use (can be overridden) | `registry.gitlab.com/to-be-continuous/tools/vault-secrets-provider:master` |
| `vault-base-url` / `VAULT_BASE_URL` | The Vault server base API url | **must be defined** |
| `vault-oidc-aud` / `VAULT_OIDC_AUD` | The `aud` claim for the JWT | `$CI_SERVER_URL` |
| :lock: `VAULT_ROLE_ID` | The [AppRole](https://www.vaultproject.io/docs/auth/approle) RoleID | _none_ |
| :lock: `VAULT_SECRET_ID` | The [AppRole](https://www.vaultproject.io/docs/auth/approle) SecretID | _none_ |
### Hook scripts
By default, the variant will authentifacte using a [JWT ID token](https://docs.gitlab.com/ci/secrets/id_token_authentication/). To use [AppRole](https://www.vaultproject.io/docs/auth/approle) instead the `VAULT_ROLE_ID` and `VAULT_SECRET_ID` should be defined as secret project variables.
The GitOps template supports _optional_ **hook scripts** from your project, located in the `$GITOPS_PROJECT_DIR` directory to perform additional project-specific logic:
#### Usage
*`pre-gitops.sh` is executed **before** running GitOps,
*`post-gitops.sh` is executed **after** running GitOps (whichever the tests status).
Then you may retrieve any of your secret(s) from Vault using the following syntax:
