Commit 85aa7dc5 authored by Guilhem Bonnefille's avatar Guilhem Bonnefille
Browse files

feat: initiate publish job to Gitlab Packages

parent 0c7bffa2

.gitlab-ci.yml

+1 −1
Original line number Diff line number Diff line
@@ -20,7 +20,7 @@ stages: 
  - publish



variables:

  GITLAB_CI_FILES: "templates/gitlab-ci-xxx.yml"

  GITLAB_CI_FILES: "templates/gitlab-ci-packages.yml"

  BASH_SHELLCHECK_FILES: "*.sh"



semantic-release:

README.md

+9 −86
Original line number Diff line number Diff line 
# GitLab CI template for XXX

# GitLab CI template for Gitlab Packages



This project implements a generic GitLab CI template for [XXX](https://link.to.tool.com/).

This project implements a generic GitLab CI template for [Gitlab Package Registry](https://docs.gitlab.com/ee/user/packages/package_registry/index.html).



It provides several features, usable in different modes (by configuration).
@@ -10,104 +10,27 @@ In order to include this template in your project, add the following to your `gi 


```yaml

include:

  - project: 'to-be-continuous/xxx'

  - project: 'to-be-continuous/glpck'

    ref: '1.0.0'

    file: '/templates/gitlab-ci-xxx.yml'

    file: '/templates/gitlab-ci-packages.yml'

```



## Global configuration



The XXX template uses some global configuration used throughout all jobs.

The Gitlab Packages template uses some global configuration used throughout all jobs.



| Name                  | description                            | default value     |

| --------------------- | -------------------------------------- | ----------------- |

| `XXX_IMAGE`           | The Docker image used to run XXX       | `xxx:latest` |

| `PACKAGES_IMAGE`      | The Docker image used to access Gitlab packages  | `curl:latest` |



## Jobs



### `xxx-build` job

### `packages-publish` job



This job performs **build and tests** at once.

This job **publish** files to [Gitlab Generic Packages](https://docs.gitlab.com/ee/user/packages/generic_packages/).



It uses the following variable:



| Name                  | description                              | default value     |

| --------------------- | ---------------------------------------- | ----------------- |

| `XXX_BUILD_ARGS`      | Arguments used by the build job          | `build --with-default-args` |



### SonarQube analysis



If you're using the SonarQube template to analyse your XXX code, here are 2 sample `sonar-project.properties` files.



```properties

# see: https://docs.sonarqube.org/latest/analysis/languages/xxx/

# set your source directory(ies) here (relative to the sonar-project.properties file)

sonar.sources=.

# exclude unwanted directories and files from being analysed

sonar.exclusions=output/**,**/*_test.xxx



# set your tests directory(ies) here (relative to the sonar-project.properties file)

sonar.tests=.

sonar.test.inclusions=**/*_test.xxx



# tests report (TODO)

sonar.xxx.testExecutionReportPaths=reports/sonar_test_report.xml

# coverage report (TODO)

sonar.xxx.coverage.reportPaths=reports/coverage.cov

```



More info:



* [XXX language support](https://docs.sonarqube.org/latest/analysis/languages/xxx/)

* [test coverage & execution parameters](https://docs.sonarqube.org/latest/analysis/coverage/)

* [third-party issues](https://docs.sonarqube.org/latest/analysis/external-issues/)



### `xxx-lint` job



This job performs a [lint](link-to-the-tool) analysis of your code, mapped to the `build` stage.



It uses the following variables:



| Name                  | description                                | default value     |

| --------------------- | ------------------------------------------ | ----------------- |

| `XXX_LINT_IMAGE`      | The Docker image used to run the lint tool | `xxx-lint:latest` |

| `XXX_LINT_DISABLED`   | Set to `true` to disable the `lint` analysis| _none_ (enabled) |

| `XXX_LINT_ARGS`       | Lint [options and arguments](link-to-the-cli-options) | `--serevity=medium` |



### `xxx-depcheck` job



This job enables a manual [dependency check](link-to-the-tool) analysis of your code, mapped to the `test` stage.



It uses the following variables:



| Name                  | description                                | default value     |

| --------------------- | ------------------------------------------ | ----------------- |

| `XXX_DEPCHECK_IMAGE`  | The Docker image used to run the dependency check tool | `xxx-depcheck:latest` |

| `XXX_DEPCHECK_ARGS`   | Dependency check [options and arguments](link-to-the-cli-options) | _none_ |



### `xxx-publish` job



This job is **disabled by default** and performs a publish of your built binaries.



It uses the following variables:



| Name                  | description                            | default value     |

| --------------------- | -------------------------------------- | ----------------- |

| `XXX_PUBLISH_ENABLED` | Variable to enable the publish job     | _none_ (disabled) |

| `XXX_PUBLISH_ARGS`    | Arguments used by the publish job      | `publish --with-default-args` |

| :lock: `XXX_PUBLISH_LOGIN` | Login to use to publish           | **has to be defined** |

| :lock: `XXX_PUBLISH_PASSWORD` | Password to use to publish     | **has to be defined** |



### Secrets management



Here are some advices about your **secrets** (variables marked with a :lock:):



1. Manage them as [project or group CI/CD variables](https://docs.gitlab.com/ee/ci/variables/#create-a-custom-variable-in-the-ui):

    * [**masked**](https://docs.gitlab.com/ee/ci/variables/#mask-a-custom-variable) to prevent them from being inadvertently

      displayed in your job logs,

    * [**protected**](https://docs.gitlab.com/ee/ci/variables/#protect-a-custom-variable) if you want to secure some secrets

      you don't want everyone in the project to have access to (for instance production secrets).

2. In case a secret contains [characters that prevent it from being masked](https://docs.gitlab.com/ee/ci/variables/#masked-variable-requirements), 

  simply define its value as the [Base64](https://en.wikipedia.org/wiki/Base64) encoded value prefixed with `@b64@`:

  it will then be possible to mask it and the template will automatically decode it prior to using it.

3. Don't forget to escape special characters (ex: `$` -> `$$`).
 
| `PACKAGES_FILES_PATTERN` | Shell pattern to identify files to publish  | _none_ (mandatory) |

kicker.json

+9 −72
Original line number Diff line number Diff line 
{

  "name": "XXX",

  "description": "GitLab CI template for XXX",

  "template_path": "templates/gitlab-ci-xxx.yml",

  "name": "Gitlab Packages",

  "description": "GitLab CI template for Gitlab Packages",

  "template_path": "templates/gitlab-ci-packages.yml",

  "kind": "build",

  "variables": [

    {

      "name": "XXX_IMAGE",

      "description": "The Docker image used to run XXX",

      "default": "xxx:1.2.3"

      "name": "PACKAGES_IMAGE",

      "description": "The Docker image used to publish packages",

      "default": "curl:latest"

    },

    {

      "name": "XXX_BUILD_ARGS",

      "description": "Arguments used by the build job",

      "default": "build --with-default-args",

      "advanced": true

    }

  ],

  "features": [

    {

      "id": "lint",

      "name": "XXX lint",

      "description": "[XXX lint](link-to-the-tool) analysis",

      "disable_with": "XXX_LINT_DISABLED",

      "variables": [

        {

          "name": "XXX_LINT_IMAGE",

          "description": "The Docker image used to run the lint tool",

          "default": "xxx-lint:latest"

        },

        {

          "name": "XXX_LINT_ARGS",

          "description": "Lint [options and arguments](link-to-the-cli-options)",

          "default": "--serevity=medium",

          "advanced": true

        }

      ]

    },

    {

      "id": "depcheck",

      "name": "XXX dependency check",

      "description": "[XXX dependency check](link-to-the-tool) analysis",

      "variables": [

        {

          "name": "XXX_DEPCHECK_IMAGE",

          "description": "The Docker image used to run the dependency check tool",

          "default": "xxx-depcheck:latest"

        },

        {

          "name": "XXX_DEPCHECK_ARGS",

          "description": "Dependency check [options and arguments](link-to-the-cli-options)",

          "advanced": true

        }

      ]

    },

    {

      "id": "publish",

      "name": "Publish",

      "description": "Publish your package to a repository",

      "enable_with": "XXX_PUBLISH_ENABLED",

      "variables": [

        {

          "name": "XXX_PUBLISH_ARGS",

          "description": "Arguments used by the publish job",

          "default": "publish --with-default-args",

          "advanced": true

        },

        {

          "name": "XXX_PUBLISH_LOGIN",

          "description": "Login to use to publish",

          "secret": true

        },

        {

          "name": "XXX_PUBLISH_PASSWORD",

          "description": "Password to use to publish",

          "secret": true

        }

      ]

      "name": "PACKAGES_FILES_PATTERN",

      "description": "Pattern to identify files to publish",

      "default": "bin/*/*/*"

    }

  ]

}

templates/gitlab-ci-xxx.yml→templates/gitlab-ci-packages.yml

+24 −110
Original line number Diff line number Diff line
@@ -45,18 +45,10 @@ variables: 
  TBC_TRACKING_IMAGE: "$CI_REGISTRY/to-be-continuous/tools/tracking:master"



  # Default Docker image (use a public image - can be overridden)

  XXX_IMAGE: "xxx:latest"

  PACKAGES_IMAGE: "curl:latest"



  # Default arguments for 'build' command

  XXX_BUILD_ARGS: "build --with-default-args"



  # Default arguments for 'publish' command

  XXX_PUBLISH_ARGS: "publish --with-default-args"



  XXX_LINT_IMAGE: "xxx-lint:latest"

  XXX_LINT_ARGS: "--serevity=medium"



  XXX_DEPCHECK_IMAGE: "xxx-depcheck:latest"

  # Pattern of files to publish

  # PACKAGES_FILES_PATTERN: bin/*/*/*



  # default production ref name (pattern)

  PROD_REF: '/^(master|main)$/'
@@ -65,11 +57,9 @@ variables: 


# allowed stages depend on your template type (see: orange-opensource.gitlab.io/tbc/doc/dev-guidelines/#stages)

stages:

  - build

  - test

  - publish



.xxx-scripts: &xxx-scripts |

.packages-scripts: &packages-scripts |

  # BEGSCRIPT

  set -e
@@ -298,9 +288,16 @@ stages: 
    fi

  }



  function output_coverage() {

    echo "[TODO]: compute and output global coverage result"

    echo "11% covered"

  function publish() {

    if [ -z "$PACKAGES_FILES_PATTERN" ]

    then

      fail "The variable \\e[33;1m${PACKAGES_FILES_PATTERN}\\e[0m is mandatory"

    fi

    for file in $PACKAGES_FILES_PATTERN

    do

      log_info "Publishing \\e[33;1m${file}\\e[0m"

      curl -sSf --header "JOB-TOKEN: $CI_JOB_TOKEN" --upload-file "$file" "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/generic/${CI_PROJECT_NAME}/${CI_COMMIT_TAG}/$(basename '$file')"

    done

  }



  unscope_variables
@@ -310,110 +307,27 @@ stages: 


# job prototype

# defines default Docker image, tracking probe, cache policy and tags

.xxx-base:

  image: $XXX_IMAGE

.packages-base:

  image: $PACKAGES_IMAGE

  services:

    - name: "$TBC_TRACKING_IMAGE"

      command: ["--service", "xxx", "1.0.0"]

      command: ["--service", "packages", "1.0.0"]

  before_script:

    - *xxx-scripts

    - *packages-scripts

    - install_ca_certs "${CUSTOM_CA_CERTS:-$DEFAULT_CA_CERTS}"

  # Cache downloaded dependencies and plugins between builds.

  # To keep cache across branches add 'key: "$CI_JOB_NAME"'

  # TODO (if necessary): define cache policy here

  # No need for cache, only artifacts

  cache:

    # cache shall be per branch per template

    key: "$CI_COMMIT_REF_SLUG-xxx"

    paths:

      - .cache/



# (example) build & test job

xxx-build:

  extends: .xxx-base

  stage: build

  script:

    # TODO (if possible): $TRACE set enables debug logs on the tool

    # TODO (if possible): force test tool to produce JUnit report(s)

    # TODO (if possible): force test tool to compute code coverage with report

    - xxx ${TRACE+--verbose} --coverage --junit --output=reports/ $XXX_BUILD_ARGS

    - output_coverage

  # TODO: code coverage support and GitLab integration (see: https://docs.gitlab.com/ee/ci/yaml/#coverage)

  coverage: '/^(\d+.\d+\%) covered$/'

  # keep build artifacts and test reports (see: https://docs.gitlab.com/ee/ci/yaml/#artifactsreportsjunit)

  artifacts:

    name: "$CI_JOB_NAME artifacts from $CI_PROJECT_NAME on $CI_COMMIT_REF_SLUG"

    expire_in: 1 day

    reports:

      # TODO: Unit tests use JUnit format and GitLab integration (see: https://docs.gitlab.com/ee/ci/yaml/#artifactsreports)

      junit:

        - reports/TEST-*.xml

    paths:

        - build/

        - reports/



# (example) linter job

xxx-lint:

  extends: .xxx-base

  stage: build

  image: $XXX_LINT_IMAGE

  # force no dependency

  dependencies: []

  script:

    - xxx_lint ${TRACE+--verbose} --output=reports/ $XXX_LINT_ARGS

  artifacts:

    name: "$CI_JOB_NAME artifacts from $CI_PROJECT_NAME on $CI_COMMIT_REF_SLUG"

    expire_in: 1 day

    when: always

    paths:

      - reports/

  rules:

    # exclude if $XXX_LINT_DISABLED

    - if: '$XXX_LINT_DISABLED == "true"'

      when: never

    # .test-policy rules

    - !reference [.test-policy, rules]



# (example) dependency check job

xxx-depcheck:

  extends: .xxx-base

  stage: test

  image: $XXX_DEPCHECK_IMAGE

  # force no dependency

  dependencies: []

  script:

    - xxx_depcheck ${TRACE+--verbose} --output=reports/ $XXX_DEPCHECK_ARGS

  artifacts:

    name: "$CI_JOB_NAME artifacts from $CI_PROJECT_NAME on $CI_COMMIT_REF_SLUG"

    expire_in: 1 day

    when: always

    paths:

      - reports/

  rules:

    # on schedule: auto

    - if: '$CI_PIPELINE_SOURCE == "schedule"'

      allow_failure: true

      when: always

    # all other cases: manual & non-blocking

    - when: manual

      allow_failure: true



# (example) publish job activated on env ($XXX_PUBLISH_ENABLED), with required $XXX_PUBLISH_LOGIN and $XXX_PUBLISH_PASSWORD env verification

xxx-publish:

  extends: .xxx-base

packages-publish:

  extends: .packages-base

  stage: publish

  before_script:

    - *xxx-scripts

    # verify $XXX_PUBLISH_LOGIN and $XXX_PUBLISH_PASSWORD are set

    - assert_defined "$XXX_PUBLISH_LOGIN" 'Missing required env $XXX_PUBLISH_LOGIN'

    - assert_defined "$XXX_PUBLISH_PASSWORD" 'Missing required env $XXX_PUBLISH_PASSWORD'

    - xxx login --login=$XXX_PUBLISH_LOGIN --password=$XXX_PUBLISH_PASSWORD

    - *packages-scripts

  script:

    - xxx $XXX_PUBLISH_ARGS

    - publish

  rules:

    # exclude if $XXX_PUBLISH_ENABLED unset

    - if: '$XXX_PUBLISH_ENABLED != "true"'

      when: never

    # on integration or production branch(es): manual & non-blocking

    - if: '$CI_COMMIT_REF_NAME =~ $INTEG_REF || $CI_COMMIT_REF_NAME =~ $PROD_REF'

      when: manual

      allow_failure: true

    - if: '$CI_COMMIT_TAG'
 
 No newline at end of file