1. Fork this project, commit your changes there, and do a Merge Request against this project. Don't forget to assign either @falconierif or @hoverht to have a look at your code.
2. Ask us to become a Developer in this repo, so you can develop in a branch here.
1. Fork this project, commit your changes there, and do a Merge Request against
this project. Don't forget to assign either @falconierif or @hoverht to have
a look at your code.
2. Ask us to become a Developer in this repo, so you can develop in a branch
here.
### pre-commit hooks
We use pre-commit hooks in this project.
You don't have to use them, but if you want you can install them this way:
We use pre-commit hooks in this project. You don't have to use them, but if you
want you can install them this way:
```bash
pip3 install poetry
poetry install
@@ -16,9 +22,15 @@ poetry run pre-commit install --hook-type post-commit
```
### testing
We try to keep the code coverage as high as possible here, so if you contribute with a functionality please add a test for it.
How do we test things? We use Gitlab-CI to test our gitlab ci templates of jobs and pipelines! This way we know we don't break stuff in your downstream project.
`tests/jobs` is where we keep jobs tests, `tests/pipelines` is where we keep pipeline tests.
`tests/mockup_projects` is where we keep fake projects to test our jobs and pipelines on.
We use [child pipelines](https://docs.gitlab.com/ee/ci/pipelines/parent_child_pipelines.html) to keep things neat and separate logically separated jobs (different language/pipeline scenario).
For example if you contribute a python job, please add it to the `include` block in `tests/jobs/python.yml`.
We try to keep the code coverage as high as possible here, so if you contribute
with a functionality please add a test for it. How do we test things? We use
Gitlab-CI to test our gitlab ci templates of jobs and pipelines! This way we
know we don't break stuff in your downstream project. `tests/jobs` is where we
keep jobs tests, `tests/pipelines` is where we keep pipeline tests.
`tests/mockup_projects` is where we keep fake projects to test our jobs and
Any questions, problems, suggestions? [Create an issue](https://gitlab.com/just-ci/templates/-/issues/new)!
- Federico Falconieri: @falcorocks
- Ruben ten Hove: @rhtenhove
This repository contains a collection of modular GitLab CI jobs, pipelines and templates.
Any questions, problems, suggestions?
[Create an issue](https://gitlab.com/just-ci/templates/-/issues/new)!
## Structure
***[Jobs](https://docs.gitlab.com/ee/ci/jobs/)** are individual tasks/tests performed over your repository. You can find them in the directory where they logically belong to, for example the job for pytest is in `python/pytest.yml`. Jobs have the same name of the file they are in (`/` are replaced by `:`) so pytest gitlab job name is `python:pytest`. We try to keep the jobs as unopinionated/universal as possible. Most of our jobs are for python projects, but there are also jobs for c projects and for other automation tasks (badges, semantic-versioning, documentation).
This repository contains a collection of modular GitLab CI jobs, pipelines and
templates.
***[Pipelines](https://docs.gitlab.com/ee/ci/pipelines/)** are collections of jobs. Jobs can be grouped in [stages](https://docs.gitlab.com/ee/ci/pipelines/). Jobs define what to do, stages define when (and thus in which order) to run them. Our jobs are all configured to always use the default stages `.pre`, `build`, `test`, `deploy`, `.post`. We keep our pipelines in `pipelines`. Our pipelines files group logically connected jobs. For example `pipelines/python.yml` allows to easily import all our python jobs, and `pipelines/docker.yml` allows to easily import the Docker jobs and set some other global variables accordingly. Pipelines are opinionated, this is where we put workflow rules and make decisions on which jobs should run together.
## Structure
***Templates** are collections of pipelines. Users should import templates rather than dealing with pipelines or jobs directly. They are in the directory `templates`. For example the `templates/python-docker.yml` provides the user with pipelines for python and docker jobs, our default workflow rules and project-automation pipeline.
-**[Jobs](https://docs.gitlab.com/ee/ci/jobs/)** are individual tasks/tests
performed over your repository. You can find them in the directory where they
logically belong to, for example the job for pytest is in `python/pytest.yml`.
Jobs have the same name of the file they are in (`/` are replaced by `:`) so
pytest gitlab job name is `python:pytest`. We try to keep the jobs as
unopinionated/universal as possible. Most of our jobs are for python projects,
but there are also jobs for c projects and for other automation tasks (badges,
semantic-versioning, documentation).
-**[Pipelines](https://docs.gitlab.com/ee/ci/pipelines/)** are collections of
jobs. Jobs can be grouped in
[stages](https://docs.gitlab.com/ee/ci/pipelines/). Jobs define what to do,
stages define when (and thus in which order) to run them. Our jobs are all
configured to always use the default stages `.pre`, `build`, `test`, `deploy`,
`.post`. We keep our pipelines in `pipelines`. Our pipelines files group
logically connected jobs. For example `pipelines/python.yml` allows to easily
import all our python jobs, and `pipelines/docker.yml` allows to easily import
the Docker jobs and set some other global variables accordingly. Pipelines are
opinionated, this is where we put workflow rules and make decisions on which
jobs should run together.
-**Templates** are collections of pipelines. Users should import templates
rather than dealing with pipelines or jobs directly. They are in the directory
`templates`. For example the `templates/python-docker.yml` provides the user
with pipelines for python and docker jobs, our default workflow rules and
project-automation pipeline.
# How to use
In order to use our templates your project will need:
* A `.gitlab-ci.yml` file in the root directory of your repository. This will contain references to the jobs, pipelines, and templates you wish to use.
* Optionally a [GitLab runner](https://docs.gitlab.com/ee/ci/runners/) configured at project/group level if you don't have shared runners available.
and then for the repository automation jobs to work as intended you will need to:
* Create a [Project Access Token](https://docs.gitlab.com/ee/user/project/settings/project_access_tokens.html) or [Personal Access Token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) with **api** scope access. Call it `GL_TOKEN`.
* Store the Access Token in a [Gitlab CI/CD variable](https://docs.gitlab.com/ee/ci/variables/#add-a-cicd-variable-to-a-project) called `GL_TOKEN`.
* Set the token to be **Protected** and **Masked**.
* Enable push permission for your [protected branches](https://docs.gitlab.com/ee/user/project/protected_branches.html#configure-a-protected-branch) for the fictitious user `GL_TOKEN`.
* Create a `.releaserc` file in the root of your repository. If you don't have this file, we provide a [default](https://gitlab.com/just-ci/utils/-/raw/main/semantic-release/gitlab-default.json).
* Create a `tbump.toml` file or a `tbump` section in your `pyproject.toml`. See [here](https://pypi.org/project/tbump/).
- A `.gitlab-ci.yml` file in the root directory of your repository. This will
contain references to the jobs, pipelines, and templates you wish to use.
- Optionally a [GitLab runner](https://docs.gitlab.com/ee/ci/runners/)
configured at project/group level if you don't have shared runners available.
and then for the repository automation jobs to work as intended you will need
- Create a `tbump.toml` file or a `tbump` section in your `pyproject.toml`. See
[here](https://pypi.org/project/tbump/).
## Choosing a template
You can import our templates in your projects using GitLab [include](https://docs.gitlab.com/ee/ci/yaml/includes.html) functionality.
Here is an example `.gitlab-ci.yml` importing the python template:
You can import our templates in your projects using GitLab
[include](https://docs.gitlab.com/ee/ci/yaml/includes.html) functionality. Here
is an example `.gitlab-ci.yml` importing the python template:
```yaml
---
include:
- project: 'just-ci/templates'
file: 'templates/python.yml'
ref: 'v5.2.1'
- project: "just-ci/templates"
file: "templates/python.yml"
ref: "v5.2.1"
```
The above is a template for Python projects. Other templates can be found in `templates/`, for example:
The above is a template for Python projects. Other templates can be found in
`templates/`, for example:
* Python projects with a `Dockerfile`: `templates/container/python.yml`. This template will use your `Dockerfile` to create an image in which we run tests.
* C projects: `templates/c.yml`.
- Python projects with a `Dockerfile`: `templates/container/python.yml`. This
template will use your `Dockerfile` to create an image in which we run tests.
- C projects: `templates/c.yml`.
## Disabling specific jobs
Templates and pipelines may come with jobs you don't want/need to run. We have carefuly selected our defaults to provide the best code quality possible, but if you want you can always disable specific jobs with custom rules.
Templates and pipelines may come with jobs you don't want/need to run. We have
carefuly selected our defaults to provide the best code quality possible, but if
you want you can always disable specific jobs with custom rules.
```yaml
---
include:
-project:'just-ci/templates'
file:'templates/python.yml'
ref:'v5.2.1'
-project:"just-ci/templates"
file:"templates/python.yml"
ref:"v5.2.1"
python:pytest:
rules:
@@ -68,22 +111,26 @@ python:pytest:
# Pinning your template version
We tag changes in our templates automagically using [semantic-release](https://semantic-release.gitbook.io/semantic-release/). Every time a merge request is accepted, a semantic-release job produces a release (and an associated tag).
You can import jobs and pipelines from a specific tag or branch if you desire.
If you do not specify a tag or branch, you will import what is on the default branch.
If your project does not work anymore because there has been a breaking change in `master`, use an older tag like this:
We tag changes in our templates automagically using
[semantic-release](https://semantic-release.gitbook.io/semantic-release/). Every
time a merge request is accepted, a semantic-release job produces a release (and
an associated tag). You can import jobs and pipelines from a specific tag or
branch if you desire. If you do not specify a tag or branch, you will import
what is on the default branch. If your project does not work anymore because
there has been a breaking change in `master`, use an older tag like this:
```yaml
---
include:
- project: 'just-ci/templates'
file: 'python/pylint.yml'
ref: 'v3.19.2'
- project: "just-ci/templates"
file: "python/pylint.yml"
ref: "v3.19.2"
```
# monorepos vs polirepos
We support both! Read [this](pipelines/container/readme.md) for more information on how to configure pipelines for monorepos and polirepos.
We support both! Read [this](pipelines/container/readme.md) for more information
on how to configure pipelines for monorepos and polirepos.
### example python polyrepo
@@ -92,24 +139,25 @@ Polirepos will likely work out of the box without extra configuration needed.
```yaml
---
include:
-project:'just-ci/templates'
file:'templates/container/python.yml'
-project:"just-ci/templates"
file:"templates/container/python.yml"
ref:v5.2.1-beta.1
```
### example monorepo
In monorepos you can start by including the following, but you will need to write your own kaniko and grype job. Again, read [this](pipelines/container/readme.md)
In monorepos you can start by including the following, but you will need to
