Commit 2b040bf6 authored by Thomas Boni's avatar Thomas Boni
Browse files

Merge branch '560-group-all-tests-jobs' into 'latest' 

Resolve "Group all tests jobs"

Closes #560

See merge request r2devops/hub!342
parents 24bdbfce 1a540dc2

.gitlab-ci.yml

+5 −5
Original line number Diff line number Diff line
@@ -7,10 +7,10 @@ stages: 
  - deploy



include:

  - remote: 'https://jobs.r2devops.io/0.2.0/links_checker.yml?scope=0.yml'

  - remote: 'https://jobs.r2devops.io/0.2.1/spell_check.yml?scope=0.yml'

  - remote: 'https://jobs.r2devops.io/1.4.0/mkdocs.yml?scope=0.yml'

  - remote: 'https://jobs.r2devops.io/0.3.0/pages.yml?scope=0.yml'

  - remote: 'https://api.r2devops.io/job/r/r2devops-bot/links_checker/0.2.0.yml?ignore=true.yml'

  - remote: 'https://api.r2devops.io/job/r/r2devops-bot/spell_check/0.2.1.yml?ignore=true.yml'

  - remote: 'https://api.r2devops.io/job/r/r2devops-bot/mkdocs/1.4.0.yml?ignore=true.yml'

  - remote: 'https://api.r2devops.io/job/r/r2devops-bot/pages/0.3.0.yml?ignore=true.yml'



workflow:

  rules:
@@ -165,7 +165,7 @@ links_checker: 
  variables:

    # The following hard excluded link is due that this

    # job will be migrated to a new tool soon (see #314)

    LICHE_EXCLUDE: ^([^http]|https://gitlab.com/r2devops/hub/-/forks/new)|^([^http]|https://r2devops.io/.+)

    LICHE_EXCLUDE: ^([^http]|https://gitlab.com/r2devops/hub/-/forks/new)|^([^http]|https://r2devops.io/.+)|^([^http]|https://api.r2devops.io/.+)

    FAIL_ON_BROKEN: "true"

    LICHE_DIRECTORY: "docs/"

  allow_failure: true

cookiecutter.config.js

+1 −1
Original line number Diff line number Diff line
@@ -15,7 +15,7 @@ module.exports = [ 
        {

          templateVariable: "r2_stage",

          question: "What is the default stage of your job?",

          choices: ["static_tests", "build", "dynamic_tests", "provision", "review", "release", "deploy", "others"]

          choices: ["build", "tests", "provision", "review", "release", "deploy", "others"]

        },

        {

          templateVariable: "r2_maintainer",

docs/jobs/static_tests/.gitkeep

deleted100644 → 0
+0 −0

Empty file deleted.

docs/jobs/dynamic_tests/.gitkeep→docs/jobs/tests/.gitkeep

+0 −0

File moved.

docs/use-the-hub.md

+47 −52
Original line number Diff line number Diff line 
---

title: How to use the hub | Prerequisites and quick setup

description: Discover how to use the hub! You’ll find here the prerequisites, a quick setup and how to customize a job. Those topics will have no secret for you!

description: Discover how to use the hub. You’ll find here the prerequisites, a quick setup and how to customize a job. Those topics will have no secret for you!

---



# Use the hub



## 📝 Prerequisites



* 🦊  Manage your project in Gitlab and understand what is [CI/CD with Gitlab](https://docs.gitlab.com/ee/ci/){:target="_blank"}

* 🔫  Be aware each file modification in your project can trigger a [Pipeline](/r2bulary/#pipeline)

* 🗝  Have access to the pipelines page in your Gitlab project and write access to your project `.gitlab-ci.yml` file

* 🦊  Manage your project in GitLab and understand what is [CI/CD with GitLab](https://docs.gitlab.com/ee/ci/){:target="_blank"}

* 🔫  Be aware: each file modification in your project can trigger a [Pipeline](/r2bulary/#pipeline)

* 🗝  Have access to the pipelines page in your GitLab project and write access to your project `.gitlab-ci.yml` file



## ⏳ Quick setup



Follows these steps to setup your CI/CD pipeline in less than 5 minutes!



1. If you haven't yet a `.gitlab-ci.yml` file in the root on your repository:

   create it with the list of stages:

   create it with the following list of stages:



    ```yaml

    stages:

      - static_tests

      - build

      - dynamic_tests

      - tests

      - provision

      - review

      - release
@@ -31,41 +30,41 @@ Follows these steps to setup your CI/CD pipeline in less than 5 minutes ! 
    ```



    !!! info

        Check [stages](#stages) section to get more information about this list

        You can check the [stages](#stages) section to get more information about this list

        or if you already have a configuration with different stages.



2. Select Jobs you want in [jobs section](/jobs/) and append their URL in the

2. Select the Jobs you want in the [jobs section](https://r2devops.io/jobs) and append their URL in the

   `include` list of your `.gitlab-ci.yml` file:



    ```yaml

    include:

      - remote: 'https://jobs.r2devops.io/<job_name>.yml'

      - remote: 'https://jobs.r2devops.io/<job_name>.yml'

       - remote: 'https://api.r2devops.io/job/r/<owner_name>/<job_name>/<version_number>.yml'

       - remote: 'https://api.r2devops.io/job/r/<owner_name>/<job_name>/<version_number>.yml'

      - ...

    ```



    !!! note



        By default, the `latest` version of a job is used. You can choose to

        use a specific version using a `tag`. Available tags are described for

        each job in [jobs section](/jobs/). Description of `tag` format is

        available in [Versioning page](/versioning/).

        use a specific version thank to a `tag`. Available tags are described for

        each job in the [labels section](/labels). The description of the `tag` format is

        available in the [versioning page](/versioning/).



        Once your pipeline is functional, we recommend using a specific version

        for jobs in order to ensure that your pipeline will not be broken by a

        for the jobs in order to ensure that your pipeline will not be broken by a

        job update.



3. Jobs can be customized 👉 check the [jobs

3. The jobs can be customized 👉 check the [jobs

   customization](#jobs-customization) section.



4. Everything is ready! You can now benefit the full power of a CI/CD

   pipeline 🎉🚀



    !!! tip

        You can also combine jobs templates and your own jobs in

        You can also combine job's templates and your own jobs in your

        `.gitlab-ci.yml` configuration file.



<a alt="See all jobs" href="/jobs">

<a alt="See all jobs" href="https://r2devops.io/_/jobs">

    <button class="md-button border-radius-10 md-button-center" >

        See all jobs available <img alt="" class="heart" src="../images/rocket.png">

    </button>
@@ -73,7 +72,7 @@ Follows these steps to setup your CI/CD pipeline in less than 5 minutes ! 


## 🤓 Pipeline examples



* Several examples of projects using the r2devops hub:

* Several examples of projects using R2Devops hub:



    * Python flask based REST API permitting to play TicTacToe 👉 [tictactoe/grid-api](https://gitlab.com/tictac-toe/grid-api)

    * Vue.js project providing a client to the TicTacToe API 👉 [tictactoe/grid-frontend](https://gitlab.com/tictac-toe/grid-frontend)
@@ -92,7 +91,7 @@ Follows these steps to setup your CI/CD pipeline in less than 5 minutes ! 
              version)

        * Plug-and-play set of jobs from the hub to automatically build, push

          and test docker images:

            * [`docker_build`](https://r2devops.io/_/r2devops-bot/docker_build)

            * [`docker_build`](https://r2devops.io/_/r2devops-bot/docker_build/)

              (version `0.3.0`)

            * [`trivy_image`](https://r2devops.io/_/r2devops-bot/trivy_image/)

              (version `0.2.0`)
@@ -100,9 +99,8 @@ Follows these steps to setup your CI/CD pipeline in less than 5 minutes ! 


    ``` yaml

    stages:

    - static_tests

    - build

    - dynamic_tests

    - tests

    - provision

    - review

    - release
@@ -111,16 +109,16 @@ Follows these steps to setup your CI/CD pipeline in less than 5 minutes ! 


    # Jobs from r2devops.io (they don't need any configuration in standard cases)

    include:

    - remote: 'https://jobs.r2devops.io/latest/mkdocs.yml'

    - remote: 'https://jobs.r2devops.io/latest/lighthouse.yml'

    - remote: 'https://jobs.r2devops.io/latest/pages.yml'

    - remote: 'https://jobs.r2devops.io/0.3.0/docker_build.yml'

    - remote: 'https://jobs.r2devops.io/0.2.0/trivy_image.yml'

    - remote: 'https://api.r2devops.io/job/r/r2devops-bot/mkdocs/latest.yaml'

    - remote: 'https://api.r2devops.io/job/r/r2devops-bot/lighthouse/latest.yaml'

    - remote: 'https://api.r2devops.io/job/r/r2devops-bot/pages/latest.yaml'

    - remote: 'https://api.r2devops.io/job/r/r2devops-bot/docker_build/0.2.0.yaml'

    - remote: 'https://api.r2devops.io/job/r/r2devops-bot/trivy_image/0.2.0.yaml'



    # Locally configured job

    unit_tests:

      image: python:3.9-alpine

      stage: static_tests

      stage: tests

      before_script:

        - pip install pipenv && pipenv --bare install --dev

      script:
@@ -131,9 +129,8 @@ Follows these steps to setup your CI/CD pipeline in less than 5 minutes ! 


By default, each job from the hub is a part of on these stages:



* **🔎 Static_tests:** static testing of  repository files

* **🧱 Build:** building and packaging of software

* **🔥 Dynamic_tests:** dynamic testing of a running version of the software

* **🧱 Build:** building and packaging of the software

* **🔎 Tests:** testing your repository files with dynamic and static tests

* **🛠 Provision:** preparation of the software infrastructure

* **👌 Review:** deployment of the software in an isolated review environment

* **🏷 Release:** releasing and tagging of the software
@@ -147,8 +144,8 @@ the hub. Follow the [customization section](#jobs-customization) to do it. 
## 🔧 Jobs customization



!!! info

    All jobs from the `r2devops/hub` specify a docker image to be run in a

    docker container

    All the jobs from the `r2devops/hub` specify a docker image to be run in a

    docker container.



### 🖌 Global
@@ -156,22 +153,22 @@ Each jobs of the hub can be customized. To do it, you have to include the job 
URL as usual and, in addition, override the options you want to customize.



!!! tip

    In this way, you can override all Gitlab jobs parameters. All parameters

    This way, you can override all Gitlab jobs parameters. All parameters

    are described in [Gitlab

    documentation](https://docs.gitlab.com/ee/ci/yaml/){:target="_blank"}.



For example, if you want to use the [trivy_image](/jobs/dynamic_tests/trivy_image/) job and

For example, if you want to use the [trivy_image](https://r2devops.io/_/r2devops-bot/trivy_image) job and

customize it by:



* Redefining the `stage` to `security` to fit in your personal stages workflow

* Redefining the `stage` to `security` to fit in your personal stages workflow,

* Set the variable `TRIVY_VERSION` to `0.9.1` to use this version instead of

  the default

  the default,

* Set the variable `TRIVY_SEVERITY` to `CRITICAL` to display only CRITICAL

  issues

  issues.



```yaml

include:

  - remote: 'https://jobs.r2devops.io/trivy_image.yml'

  - remote: 'https://api.r2devops.io/job/r/r2devops-bot/trivy_image/latest.yaml'



trivy_image:

  stage: security
@@ -182,7 +179,7 @@ trivy_image: 


### ✏️  Use custom stage



If you want to use your own stage name it's possible to do so when including

If you want to use your own stage name, it's possible to do so when including

your job. Example:



```yaml
@@ -190,7 +187,7 @@ stages: 
  - security



include:

  - remote: 'https://jobs.r2devops.io/trivy_image.yml'

  - remote: 'https://api.r2devops.io/job/r/r2devops-bot/trivy_image/latest.yaml'



trivy_image:

  stage: security
@@ -205,16 +202,16 @@ to a job: [`services`](https://docs.gitlab.com/ee/ci/yaml/#services). 


To use this option, you must have access to an image of the container you want

to run as a service. For example, if you are using our

[docker_build](https://r2devops.io/_/r2devops-bot/docker_build) job to build an

[docker_build](https://r2devops.io/_/r2devops-bot/docker_build/) job to build an

image of your application, and you want to test this image using the

[nmap](/jobs/dynamic_tests/nmap/) job, just add the following configuration in

[nmap](https://r2devops.io/_/r2devops-bot/nmap) job, just add the following configuration in

your `.gitlab-ci.yml` file:



!!! info

    * The `name` option must contain your image name and tag or an image name from [Docker Hub](https://hub.docker.com){:target="_blank"}.

    * The `name` option must contain your image name and tag, or an image name from [Docker Hub](https://hub.docker.com){:target="_blank"}.

    * The `alias` option permits to the job to reach your application using a name. This name

    must be the same that the one specified inside the job target's variable.

    * You may also run some other services like a database depending on your application needs.

    * You may also run some other services, like a database depending on your application needs.



```yaml

nmap:
@@ -227,9 +224,7 @@ nmap: 
### 🎶 Multiple usage of the same job in your pipeline





If you want to reuse a job on the hub, for example launching `apiDoc` to build 2 API documentations in the same pipeline:



You can easily do so with Hub's jobs using ==extends== GitLab keyword.

If you want to reuse a job from the hub, for example launching `apiDoc` to build 2 API documentations in the same pipeline, you can easily do so with the Hub's jobs using ==extends== GitLab keyword.



``` yaml hl_lines="13"
@@ -237,7 +232,7 @@ stages: 
  - build



include:

  - remote: 'https://jobs.r2devops.io/0.2.0/apidoc.yml'

  - remote: 'https://api.r2devops.io/job/r/r2devops-bot/apidoc/0.2.0.yaml'



apidoc:

  variables:
@@ -253,6 +248,6 @@ apidoc_project2: 
```



!!! warning

    Be aware to have different artifacts path not to overwrite your first artifact by the second one.

    Be aware to have different artifacts path to not overwrite your first artifact by the second one.



--8<-- "includes/abbreviations.md"