This project implements a generic GitLab CI template for [XXX](https://link.to.tool.com/).
This project implements a generic GitLab CI template for [dbt](https://www.getdbt.com/).
It provides several features, usable in different modes (by configuration).
### Review environments
The template supports **review** environments: those are dynamic and ephemeral environments to deploy your
_ongoing developments_ (a.k.a. _feature_ or _topic_ branches).
When enabled, it deploys the result from upstream build stages to a dedicated environment.
It is only active for non-production, non-integration branches.
It is a strict equivalent of GitLab's [Review Apps](https://docs.gitlab.com/ee/ci/review_apps/) feature.
### Integration environment
If you're using a Git Workflow with an integration branch (such as [Gitflow](https://www.atlassian.com/git/tutorials/comparing-workflows/gitflow-workflow)),
the template supports an **integration** environment.
When enabled, it deploys the result from upstream build stages to a dedicated environment.
It is only active for your integration branch (`develop` by default).
### Production environments
Lastly, the template supports 2 environments associated to your production branch (`master` by default):
* a **staging** environment (an iso-prod environment meant for testing and validation purpose),
* the **production** environment.
You're free to enable whichever or both, and you can also choose your deployment-to-production policy:
***continuous deployment**: automatic deployment to production (when the upstream pipeline is successful),
***continuous delivery**: deployment to production can be triggered manually (when the upstream pipeline is successful).
## Usage
In order to include this template in your project, add the following to your `gitlab-ci.yml`:
```yaml
include:
-project:'to-be-continuous/xxx'
-project:'to-be-continuous/dbt'
ref:'1.0.0'
file:'/templates/gitlab-ci-xxx.yml'
file:'/templates/gitlab-ci-dbt.yml'
```
You can find a sample of dbt project here : https://gitlab.com/to-be-continuous/samples/dbt-sample/
## Global configuration
The XXX template uses some global configuration used throughout all jobs.
The dbt template uses some global configuration used throughout all jobs.
| `XXX_IMAGE` | The Docker image used to run XXX | `xxx:latest` |
| `DBT_IMAGE` | The Docker image used to run dbt | `python:3.11` |
| `DBT_PROJECT_DIR` | The [dbt_project.yml](https://docs.getdbt.com/reference/dbt_project.yml) dir | `.` |
| `DBT_PROFILES_DIR` | The dbt [profile](https://docs.getdbt.com/dbt-cli/configure-your-profile) location | `.` |
| `DBT_ADAPTER` | The dbt [adapter](https://docs.getdbt.com/docs/available-adapters) to use | __none__ (required) |
| `DBT_TARGET` | The dbt [target](https://docs.getdbt.com/reference/dbt-jinja-functions/target) to use | __none__ (required) |
| `DBT_BUILD_ARGS` | Arguments used by [`dbt cli`](https://docs.getdbt.com/reference/global-configs#command-line-flags) | __none__ |
## Jobs
### Environments configuration
As seen above, the dbt template may support up to 4 environments (`review`, `integration`, `staging` and `production`).
Each deployment job produces _output variables_ that are propagated to downstream jobs (using [dotenv artifacts](https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html#artifactsreportsdotenv)):
### `xxx-build` job
*`environment_type`: set to the type of environment (`review`, `integration`, `staging` or `production`),
*`environment_name`: the application name (see below),
This job performs **build and tests** at once.
They may be freely used in downstream jobs (for instance to run acceptance tests against the latest deployed environment).
It uses the following variable:
Here are configuration details for each environment.
#### Review environments
Review environments are dynamic and ephemeral environments to deploy your _ongoing developments_ (a.k.a. _feature_ or _topic_ branches).
They are **disabled by default** and can be enabled by setting the `DBT_REVIEW_TARGET` variable (see below).
Here are variables supported to configure review environments:
The staging environment is an iso-prod environment meant for testing and validation purpose associated to your production branch (`master` by default).
This job performs a [lint](link-to-the-tool) analysis of your code, mapped to the `build` stage.
It is **disabled by default** and can be enabled by setting the `DBT_STAGING_TARGET` variable (see below).
It uses the following variables:
Here are variables supported to configure the staging environment:
| `DBT_DEPLOY_ENABLED` | set to `true` to enable deployment | _none_ (disabled) |
### Secrets management
@@ -110,3 +155,24 @@ Here are some advices about your **secrets** (variables marked with a :lock:):
simply define its value as the [Base64](https://en.wikipedia.org/wiki/Base64) encoded value prefixed with `@b64@`:
it will then be possible to mask it and the template will automatically decode it prior to using it.
3. Don't forget to escape special characters (ex: `$` -> `$$`).
## Variants
### GitLab Pages variant
Basically it copies the content of the dbt generated site folder (`target` by default) to the `public` folder which is published by [GitLab pages](https://docs.gitlab.com/ee/user/project/pages/#how-it-works).
If you wish to use it, add the following to your `gitlab-ci.yml`:
