In order to contribute efficiently, we recommend you to know following topics:
Guide to add efficiently and easily new jobs
*[R2Devops hub job structure](#job-structure)
*[Gitlab CI/CD](https://docs.gitlab.com/ee/ci/)
## hub structure
TODO: define whole structyre
## vision and mission
TODO: define vision and mission but in concept page, not here
## Create an issue
Describe how to write an issue
## Job structure
TODO: move this section in dedicated page ?
All jobs are based on a docker image
Add information about the structure of a job directory and what need to be here.
```tree
.
└── jobs # Folder containing jobs sources
└── docker_build
├── docker_build.yml # Job content
├── job.yml # Job metadata
├── README.md # Job documentation
└── versions # Jobs changelogs
├── 0.1.0.md
└── ...
```
## Workflow
## Contributing workflow
!!! note
To leverage the R2Devops validaty and security checks on your job, do not
@@ -45,47 +16,52 @@ Add information about the structure of a job directory and what need to be here.
If you alter it, we will not be able to merge your job in `r2devops/hub`
repository.
* Fork the [`r2devops/hub`](https://gitlab.com/r2devops/hub/-/forks/new) project
<!-- TODO: * Do we need to specify rules about fork visibility ? -->
* Clone the repository locally
* Create a new directory dedicated to your job in `jobs/` folder
* Ensure to respect rules in this guide
* Do not update the CI/CD configuration
### Step 1: Fork !
The first step is to create your own copy of the
[`r2devops/hub`](https://gitlab.com/r2devops/hub/) repository to be
able to work on it before merging your update in the real project.
1. Go on the fork page creation: [`r2devops/hub`](https://gitlab.com/r2devops/hub/-/forks/new)
2. Select the group in which you want to create the fork
### Step 2: Work in your fork
1. If you want to add a new job, create a new directory dedicated to your job in `jobs/` folder
1. Ensure to respect rules in this guide
1. Do not update the CI/CD configuration (file `.gitlab-ci.yml`)
1. Merge your update to [`r2devops/hub`](https://gitlab.com/r2devops/hub/):
1. TODO: test your job
### Step 3: Merge request
1. Create a new merge request in your fork (`Merge Requests > New merge request`)
1. Select branches
* As `Source branch`, select the branch in which you have worked in
your fork (usually `latest`)
* As `Target branch`, select latest in `r2devops/hub` project
* Click on `Compare branches and continue`
1. In `Title`: short description of your contribution
1. In `Description`:
* Do not remove the default content, this is the Definition of Done
(DoD)
* Add a description of your contribution. If your contribution is related to an existing issue, refers it in Example:
```md
## Contribution
Addition of a new job permitting to build go binaries. Issue
related: r2devops/hub#945
## Definition of DOne
[...]
```
* Ensure that the last pipeline in your fork passed
* Ensure to fulfill the [job Definition of Done](#job-definition-of-done)
* Submit Merge request from your fork to source project
* Go2Scale team check and validate your job using automated tests
<!-- TODO: * How to manage CI/CD pipeline from their project ? We have to manually ensure that they don't alter it? -->
## Create a new job
* What about code-owners
### How to choose the best docker to implement new job ?
Check the job is maintained
Check the size (smallest is the best)
Check the job has versioning (not only latest)
If criteria above are not met --> ##Create my docker from Alpine image
Choose the officiel Alpine image in latest (think to fix version after)
Install doxygen in the frist line of the script 'apk add --no-cache doxygen==1.8.18-r0'
The installed version of Doxygen must be specified as variable with the default value on the current latest version 1.8.18-r0
A fixed tag must be specified for the alpine image used
### How can I test my job locally/easily ?
Locally ? using the doxygen binary in my own operating system ? (brew install...)
Using a blank repo in gitlab ? ex: https://gitlab.com/coconux/doxygen and link with the job in WIP Ex: - remote: 'https://gitlab.com/go2scale/hub/-/raw/111-add-doxygen-job/jobs/doxygen/doxygen.yml'
How can I check the hub documentation went well ?
...
### About versions:
The installed version of Doxygen must be specified as variable with the default value on the current latest version 1.8.18-r0
A fixed tag must be specified for the alpine image used
### About process:
The configuration file path must be a variable in the job with default value Doxygen
@@ -104,7 +80,6 @@ Add information about the structure of a job directory and what need to be here.
### About artifact:
Use expose_as like in mkdocs job to expose the artifact in the MR
### About compliance with pages job:
@@ -136,7 +111,6 @@ Add information about the structure of a job directory and what need to be here.
Test it (How ?)
Create the documentation page following template
Add the job in the index list
Creation of a new tag
## Modify existing job
@@ -144,5 +118,34 @@ Add information about the structure of a job directory and what need to be here.
Test the job (How ?)
Update the doc
Creation of a new tag
Creation of a new version
**TODO:**
* Must be compliant with our job definition (see #39) (ex: a job mustn't be a daemon)
* Ensure that every resource used by the job has a license permitting to anyone to use it without a strong copyleft
* All mandatory files of the [job structure](/structure#job-structure) must be provided
* Provide evidences of job working (url, screenshots, output, ...)
* Job file must be validated by gitlab (CI lint)
* Must be compliant with our security jobs (#230, #229)
* Ensure that every resource used by the job has a license permitting to anyone to use it without a strong copyleft
* Docker image version and version of all tools retrieved must be fixed
* Tag of the image must be a fixed version (not latest)
* Any tools used in the image must Docker image version and version of all tools retrieved must be fixed
* Variables naming convention (to be defined, should we require the job name uppercase as prefix for all variables ? Think about a way to verify variables in hub pipeline and permit various configuration for on-premise #231)
* Use expose_as like in mkdocs job to expose the artifact in the MR
* Explain verification pipeline
* Explain in how to use the hub the fact that jobs from the hub run in docker
Jobs keep their changelogs in one folder named `versions`. This folder contains
`markdown` files, each of them representing a version and containing list of
@@ -67,7 +105,7 @@ changes provided by this version.
format (`MAJOR.MINOR.PATCH`)
* The first version for a job must be `0.1.0`
Example of a `versions` folder for a job:
**Example of a `versions` folder for a job:**
```shell
.
@@ -77,18 +115,15 @@ Example of a `versions` folder for a job:
└── 0.3.0.md
```
**📃 `0.1.0.md`**
***📃 `0.1.0.md`**
```md
* Initial version
```
**📃 `0.2.0.md`**
***📃 `0.2.0.md`**
```md
* Add variable `DOCKERFILE_PATH` which permits specifying custom path to Dockerfile
```
**📃 `0.3.0.md`**
***📃 `0.3.0.md`**
```md
* New variable `DOCKER_USE_CACHE` to be able to cache layers of build
* New variable `DOCKER_CACHE_TTL` to define time to live of cache
@@ -96,7 +131,7 @@ Example of a `versions` folder for a job:
* New variable `DOCKER_OPTIONS` to be able to add additional options%
```
### 🤖 Job definition
## 🤖 Job definition
This file must have the same name that the job with the `yml` extension:
`<job_name>.yml`. It contains the Gitlab job configuration in `yaml` format.
@@ -106,127 +141,12 @@ This file must have the same name that the job with the `yml` extension:
* Jobs of the hub must specify a Docker image to be run in a container
* See [GitLab CI/CD pipeline configuration reference](https://docs.gitlab.com/ee/ci/yaml/)
Job definition usually contains the following fields:
* What about `stages` global list ?
Your job definition file must at least contains the following fields:
***[`image`](https://docs.gitlab.com/ee/ci/yaml/#image)**: docker image used to run the job (TODO: link on how to chose my docker image)
***`stage`**: default stage for the job, must be in our [default stage list](/use-the-hub/#stages)(TODO: link on how to chose a stage)
***[`script`](https://docs.gitlab.com/ee/ci/yaml/#script)**: this is the heart of the job. It contains a list of shell commands to run the job (TODO: link on how to create my script and talk about before and after sript)
***[`variables`](https://docs.gitlab.com/ee/ci/yaml/#variables)**: variables used by the `script` part of the job to customize its behaviour (TODO: link on how to define variables)
***[`artifacts`](https://docs.gitlab.com/ee/ci/yaml/#artifacts)**: specify the result of the job that should be exposed to the user trough classic artifact or Gitlab reports(TODO: link on how to define variables)
See our Best Practices: TODO LINK
* What about code-owners
### About process:
The configuration file path must be a variable in the job with default value Doxygen
If the configuration file already exists: just run Doxygen with variable specifying path to conf file
Else, as you have started to do:
Select relevant configuration options in the doxygen doc
Add them in variables section of the job with the same default value that Doxygen
DO NOT Generate a default configuration file.
Create an empty the configuration file (using path specified in variable)
For all input variables, add them in the file, like:
Then, run doxygen like : - doxygen $DOXYGEN_CONFIGURATION
### About artifact:
### About compliance with pages job:
Add this job to the list of documentation builder jobs in pages documentation
The documentation output must be specified in a variable with the default value to documentation_build/ in order to be retrieved by pages job by default
### Documentation of your job `README.md`
<!-- TODO: est ce qu'on met les variables dans le fichier job.yml ? -->
### Job definition of done
<!-- TODO -->
### Advises to keep your job generic
<!-- TODO: * Advises or rules ? -->
* Try to avoid using `rules`: in majority of cases, `rules` of a job are specific to the project on which they will be used. If you use it, your job will loose its genericity. Also, some feature requiring specific workflows, as [Gitlab Merge Trains](https://docs.gitlab.com/ee/ci/merge_request_pipelines/pipelines_for_merged_results/merge_trains/) are more easily implemented by users if you don't specify `rules` in your job.
<!-- TODO: * What if the user push the job only for him ? should we refuse ? What about private jobs ? -->
### Merge to `r2devops/hub` latest
<!-- TODO: Guide must include commit squash guidelines: see @Protocole presentation -->
### ABCD
Create the job
Test it (How ?)
Create the documentation page following template
Add the job in the index list
## Modify existing job
* What about code-owners
Test the job (How ?)
Update the doc
Creation of a new version
## Workflow
!!! note
To leverage the R2Devops validaty and security checks on your job, do not
update the CI/CD configuration file in your fork (`.gitlab-ci.yml` file).
If you alter it, we will not be able to merge your job in `r2devops/hub`
repository.
* Fork the [`r2devops/hub`](https://gitlab.com/r2devops/hub/-/forks/new) project
<!-- TODO: * Do we need to specify rules about fork visibility ? -->
* Clone the repository locally
* Create a new directory dedicated to your job in `jobs/` folder
* Ensure to respect rules in this guide
* Do not update the CI/CD configuration
* Ensure that the last pipeline in your fork passed
* Ensure to fulfill the [job Definition of Done](#job-definition-of-done)
* Submit Merge request from your fork to source project
* Go2Scale team check and validate your job using automated tests
<!-- TODO: * How to manage CI/CD pipeline from their project ? We have to manually ensure that they don't alter it? -->
***[`image`](https://docs.gitlab.com/ee/ci/yaml/#image)**: docker image used to run the job
***`stage`**: default stage for the job, must be in our [default stage list](/use-the-hub/#stages)
***[`script`](https://docs.gitlab.com/ee/ci/yaml/#script)**: this is the heart of the job. It contains a list of shell commands to run the job
***[`variables`](https://docs.gitlab.com/ee/ci/yaml/#variables)**: variables used by the `script` part of the job to customize its behaviour
***[`artifacts`](https://docs.gitlab.com/ee/ci/yaml/#artifacts)**: specify the result of the job that should be exposed to the user trough classic artifact or Gitlab reports
**TODO:**
* Must be compliant with our job definition (see #39) (ex: a job mustn't be a daemon)
* Ensure that every resource used by the job has a license permitting to anyone to use it without a strong copyleft
* All mandatory files of the [job structure](/structure#job-structure) must be provided
* Provide evidences of job working (url, screenshots, output, ...)
* Job file must be validated by gitlab (CI lint)
* Must be compliant with our security jobs (#230, #229)
* Ensure that every resource used by the job has a license permitting to anyone to use it without a strong copyleft
* Docker image version and version of all tools retrieved must be fixed
* Tag of the image must be a fixed version (not latest)
* Any tools used in the image must Docker image version and version of all tools retrieved must be fixed
* Variables naming convention (to be defined, should we require the job name uppercase as prefix for all variables ? Think about a way to verify variables in hub pipeline and permit various configuration for on-premise #231)
* Use expose_as like in mkdocs job to expose the artifact in the MR
