Any questions, problems, suggestions? [Create an issue](issues/new)!
Any questions, problems, suggestions? [Create an issue](https://gitlab.com/just-ci/templates/-/issues/new)!
This repository contains a collection of modular gitlab ci jobs, pipelines and templates.
This repository contains a collection of modular GitLab CI jobs, pipelines and templates.
## Setup
## Structure
In order to use our templates your project will need:
***[Jobs](https://docs.gitlab.com/ee/ci/jobs/)** are individual tasks/tests performed over your repository. You can find them in the directory where they logically belong to, for example the job for pytest is in `python/pytest.yml`. Jobs have the same name of the file they are in (`/` are replaced by `:`) so pytest gitlab job name is `python:pytest`. We try to keep the jobs as unopinionated/universal as possible. Most of our jobs are for python projects, but there are also jobs for c projects and for other automation tasks (badges, semantic-versioning, documentation).
* a [GitLab runner](https://docs.gitlab.com/ee/ci/runners/) configured at project/group level.
* a `.gitlab-ci.yml` file in the root directory of your repository.
***[Pipelines](https://docs.gitlab.com/ee/ci/pipelines/)** are collections of jobs. Jobs can be grouped in [stages](https://docs.gitlab.com/ee/ci/pipelines/). Jobs define what to do, stages define when (and thus in which order) to run them. Our jobs are all configured to always use the default stages `.pre`, `build`, `test`, `deploy`, `.post`. We keep our pipelines in `pipelines`. Our pipelines files group logically connected jobs. For example `pipelines/python.yml` allows to easily import all our python jobs, and `pipelines/docker.yml` allows to easily import the Docker jobs and set some other global variables accordingly. Pipelines are opinionated, this is where we put workflow rules and make decisions on which jobs should run together.
and then for the repository automation jobs to work as intended you will need to:
***Templates** are collections of pipelines. Users should import templates rather than dealing with pipelines or jobs directly. They are in the directory `templates`. For example the `templates/python-docker.yml` provides the user with pipelines for python and docker jobs, our default workflow rules and project-automation pipeline.
* create a Project Access Token with 'api' scope access (settings -> Access Tokens), call it `GL_TOKEN`
* store the Project Access Token in a Gitlab CI/CD variable (settings -> CI/CD -> variables) called `GL_TOKEN`
* set the token to be `protected`
* enable push permission in your protected branches (probably `master` or `main`) of the fictitious user `GL_TOKEN`
* a `.releaserc` file for `semantic-release`. We use one here, you can copy it. If you don't have this file, semantic-release defaults will be used.
* a `tbump.toml` file for `tbump` or a `tbump` section in your `pyproject.toml`. If you don't have any of these files, the job will be skipped.
# How to use
## Version 4
In order to use our templates your project will need:
*[Jobs](https://docs.gitlab.com/ee/ci/jobs/): are individual tasks/tests performed over your repository. You can find them in the directory where they logically belong to, for example the job for pytest is in `python/pytest.yml`. Jobs have the same name of the file they are in (`/` are replaced by `:`) so pytest gitlab job name is `python:pytest`. We try to keep the jobs as unopinionated/universal as possible. Most of our jobs are for python projects, but there are also jobs for c projects and for other automation tasks (badges, semantic-versioning, documentation).
*[Pipelines](https://docs.gitlab.com/ee/ci/pipelines/): are collections of jobs. Jobs can be grouped in [stages](https://docs.gitlab.com/ee/ci/pipelines/). Jobs define what to do, stages define when (and thus in which order) to run them. Our jobs are all configured to always use the default stages `.pre`, `build`, `test`, `deploy`, `.post`. We keep our pipelines in `pipelines`. Our pipelines files group logically connected jobs. For example `pipelines/python.yml` allows to easily import all our python jobs, and `pipelines/docker.yml` allows to easily import the docker jobs and set some other global variables accordingly. Pipelines are opinionated, this is where we put workflow rules and make decisions on which jobs should run together.
* Templates: are collections of pipelines. Users should import templates rather than dealing with pipelines or jobs directly. They are in the directory `templates`. For example the `templates/python-docker.yml` provides the user with pipelines for python and docker jobs, our default workflow rules and project-automation pipeline.
* A `.gitlab-ci.yml` file in the root directory of your repository. This will contain references to the jobs, pipelines, and templates you wish to use.
* Optionally a [GitLab runner](https://docs.gitlab.com/ee/ci/runners/) configured at project/group level if you don't have shared runners available.
### Usage
and then for the repository automation jobs to work as intended you will need to:
You can import our templates in your projects using gitlab [include](https://docs.gitlab.com/ee/ci/yaml/includes.html) functionality.
* Create a [Project Access Token](https://docs.gitlab.com/ee/user/project/settings/project_access_tokens.html) or [Personal Access Token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) with **api** scope access. Call it `GL_TOKEN`.
* Store the Access Token in a [Gitlab CI/CD variable](https://docs.gitlab.com/ee/ci/variables/#add-a-cicd-variable-to-a-project) called `GL_TOKEN`.
* Set the token to be **Protected** and **Masked**.
* Enable push permission for your [protected branches](https://docs.gitlab.com/ee/user/project/protected_branches.html#configure-a-protected-branch) for the fictitious user `GL_TOKEN`.
* Create a `.releaserc` file in the root of your repository. If you don't have this file, we provide a [default](https://gitlab.com/just-ci/utils/-/raw/main/semantic-release/gitlab-default.json).
* Create a `tbump.toml` file or a `tbump` section in your `pyproject.toml`. See [here](https://pypi.org/project/tbump/).
## Choosing a template
You can import our templates in your projects using GitLab [include](https://docs.gitlab.com/ee/ci/yaml/includes.html) functionality.
Here is an example `.gitlab-ci.yml` importing the python template:
```yaml
---
include:
-project:'ci/templates'
- project: 'just-ci/templates'
file: 'templates/python.yml'
```
### Chosing a template
The above is a template for Python projects. Other templates can be found in `templates/`, for example:
* python project: `templates/python.yml`
* python project with custom docker image (to be used in the required tests): `templates/python-docker`
* c project: `templates/c.yml`
* Python projects with a `Dockerfile`: `templates/python-docker.yml`. This template will use your `Dockerfile` to create an image in which we run tests.
* C projects: `templates/c.yml`.
### Project Automation jobs
In order for the jobs in `project-automation/` to work as intended you will need to:
* create a Project Access Token with 'api' scope access
* store the Project Access Token in a Gitlab CI/CD variable (settings -> CI/CD -> variables) called `GL_TOKEN`
* set the token to be `protected`
### Disabling specific jobs
## Disabling specific jobs
Templates and pipelines may come with jobs you don't want/need to run. We have carefuly selected our defaults to provide the best code quality possible, but if you want you can always disable specific jobs with custom rules.
```yaml
---
include:
-project:'ci/templates'
-project:'just-ci/templates'
file:'templates/python.yml'
python:pytest:
@@ -69,29 +64,31 @@ python:pytest:
when:never
```
## Semantic versioning and breaking changes
# Pinning your template version
We tag changes automagically using [semantic-release](https://semantic-release.gitbook.io/semantic-release/). Every time a merge request is accepted, a semantic-release job produces arelease (and an associated tag).
You can import jobs and pipelines from a specific tag or branch if you desire, for example if you like the way things were before.
If you do not specify a tag or branch, you will import what is on `master`, i.e. the latest.
We tag changes in our templates automagically using [semantic-release](https://semantic-release.gitbook.io/semantic-release/). Every time a merge request is accepted, a semantic-release job produces arelease (and an associated tag).
You can import jobs and pipelines from a specific tag or branch if you desire.
If you do not specify a tag or branch, you will import what is on the default branch.
If your project does not work anymore because there has been a breaking change in `master`, use an older tag like this:
```yaml
---
include:
-project:'ci/templates'
- project: 'just-ci/templates'
file: 'python/pylint.yml'
ref: 'v3.19.2'
```
## Monorepos with multiple Docker images
# Repositories with multiple `Dockerfile`s
Use [child pipelines](https://docs.gitlab.com/ee/ci/pipelines/parent_child_pipelines.html) to work with multiple dockerfiles. An example is provided, based on a mockup project used within the tests of this repository.
> `.gitlab-ci.yml`
```yaml
# .gitlab-ci.yml
---
include:
-project:'ci/templates'
-project:'just-ci/templates'
templates:'templates/docker.yml'
# root image will build fine without any further change needed
