Merge branch 'devel' into 'main'

# GitLab CI template for pre-commit

This project implements a GitLab CI/CD template to build, test and analyse your [pre-commit](https://gitlab.com/to-be-continuous) projects.

This project implements a GitLab CI/CD template to integrate [pre-commit](https://pre-commit.com/) in your pipelines.

## Template explained

> :warning: this template is _not_ necessarily what you may think it is, please bear with us and read on!

### What this template *is* designed for

A CI job in the `build` stage that runs the `pre-commit` framework and:

- ignores the default `.pre-commit-config.yaml` file,

- runs the `pre-commit` command with a sanitized list of pre-commit checks

  we think many teams can agree to implement as a sensible baseline.

### What this template *is not* designed for

- a way to enforce all developper/team pre-commits hooks in CI,

- a way to implement checks that would've better been implemented through dedicated

  _to-be-continuous_ templates.

### How it is designed

The job follows [adaptive pipeline](https://to-be-continuous.gitlab.io/doc/understand/#adaptive-pipeline) workflow rules that balances

speed vs. quality:

- manually triggered and allowed to fail in non-MR feature branch pipelines,

- triggered but don't fail the pipeline in Draft MR pipelines,

- triggered and fails the pipeline in Ready MR and eternal branches pipelines.

Unless a specific `.pre-commit-ci.yaml` file is present in the repository,

the template uses its own predefined following configuration:

```yaml

repos:

  - repo: https://github.com/pre-commit/pre-commit-hooks

    rev: v4.5.0

    hooks:

      - id: check-merge-conflict

      - id: check-executables-have-shebangs

      - id: check-shebang-scripts-are-executable

      - id: destroyed-symlinks

      - id: end-of-file-fixer

      - id: fix-byte-order-marker

      - id: mixed-line-ending

      - id: trailing-whitespace

```

> :information_source: as stated above, this template will ignore the default `.pre-commit-config.yaml` 

> configuration file, and will only take into account the custom `.pre-commit-ci.yaml` instead.

> This is done on purpose as we believe the `pre-commit` configuration in the developers environment will/should

> be different than the one run in CI/CD.

## Building your own pre-commit image

Building a pre-configured image is a must for pipeline speed, reproducibility

and supply-chain security.

To build a preconfigured image, use the following guidelines:

- use the smallest base image possible

- embed all required pre-commit hooks (reminder: use only basic checks)

See the sample Dockerfile in the `sample` folder.

## Usage

variables:

  # 2: set/override template variables

  # ⚠ this is only an example

  PRE_COMMIT_ARGS: "build --with-my-args"

  PRE_COMMIT_SKIP: "check-byte-order-marker,no-commit-to-branch"

  PRE_COMMIT_ARGS: "-v --show-diff-on-failure"

```

## Global configuration

| Input / Variable      | Description                            | Default value     |

| --------------------- | -------------------------------------- | ----------------- |

| `pre-commit-image` / `PRE_COMMIT_IMAGE` | The Docker image used to run `pre-commit` | `registry.hub.docker.com/pre-commit:1.0.0` |

| `pre-commit-image` / `PRE_COMMIT_IMAGE` | The Docker image used to run `pre-commit` | `registry.hub.docker.com/library/python:3-alpine` |

## Jobs

| Input / Variable      | Description                              | Default value     |

| --------------------- | ---------------------------------------- | ----------------- |

| `pre-commit-disabled` / `PRE_COMMIT_DISABLED` | Disable pre-commit job | `false` |

| `pre-commit-args` / `PRE_COMMIT_ARGS` | Additionnal arguments for the pre-commit command | `` |

| `pre-commit-disabled` / `PRE_COMMIT_DISABLED` | Disable pre-commit run | `false` |

| `pre-commit-skip` / `PRE_COMMIT_SKIP` | pre-commit `SKIP` environment variable (see https://pre-commit.com/#temporarily-disabling-hooks) | `no-commit-to-branch` |

| `pre-commit-args` / `PRE_COMMIT_ARGS` | Additionnal arguments for the `pre-commit run` command | `` |

### Secrets management

  simply define its value as the [Base64](https://en.wikipedia.org/wiki/Base64) encoded value prefixed with `@b64@`:

  it will then be possible to mask it and the template will automatically decode it prior to using it.

3. Don't forget to escape special characters (ex: `$` -> `$$`).

## Building your own pre-commit image

Building a pre-configured image is a must for pipeline speed, reproducibility

and supply-chain security.

To build a preconfigured image, use the following guidelines:

- use the smallest base image possible

- embed all required pre-commit hooks (reminder: use only basic checks)

Bellow is an example `Dockerfile` to build such an image:

```Dockerfile

FROM registry.hub.docker.com/library/python:3-alpine

RUN mkdir /build

RUN apk add --no-cache git && \

    rm -rf /var/cache/apk/*

WORKDIR /build

RUN pip install --no-cache-dir pre-commit==3.5.0

RUN cat <<EOF > hook-install-config.yaml

repos:

    - repo: https://github.com/pre-commit/pre-commit-hooks

      rev: v4.5.0

      hooks:

        - id: no-commit-to-branch

        - id: trailing-whitespace

        - id: check-merge-conflict

        - id: check-yaml

        - id: end-of-file-fixer

        - id: fix-byte-order-marker

        - id: mixed-line-ending

EOF

COPY hook-install-config.yaml ./.pre-commit-config.yaml

RUN git init && \

    pre-commit install-hooks && \

    rm ./.pre-commit-config.yaml && \

    rm -rf .git

CMD ["--help"]

ENTRYPOINT ["pre-commit"]

```

  "variables": [

    {

      "name": "PRE_COMMIT_IMAGE",

      "description": "The Docker image used to run `pre-commit`",

      "default": "registry.hub.docker.com/pre-commit:1.0.0"

      "description": "The Docker image used to run `pre-commit`\n\n:information_source: You may build your own pre-configured image to speed-up things and prevent the tool and plugins from being pip-installed (see documentation).",

      "default": "registry.hub.docker.com/library/python:3-alpine"

    }

  ],

  "features": [

    {

      "id": "run-all",

      "name": "Run all pre-commit hooks",

      "id": "pre-commit-run",

      "name": "pre-commit run",

      "description": "[pre-commit](https://pre-commit.com/) analysis",

      "disable_with": "PRE_COMMIT_DISABLED",

      "variables": [

        {

          "name": "PRE_COMMIT_ARGS",

          "description": "Additionnal arguments for the pre-commit command",

          "default": "",

          "description": "Additionnal arguments for the `pre-commit run` command",

          "advanced": true

        },

        {

          "name": "PRE_COMMIT_SKIP",

          "description": "pre-commit `SKIP` environment variable that allows to disable some hooks (see https://pre-commit.com/#temporarily-disabling-hooks)",

          "default": "no-commit-to-branch",

          "advanced": true

        }

      ]

spec:

  inputs:

    pre-commit-image:

      description: The Docker image used to run `pre-commit`

      default: registry.hub.docker.com/pre-commit:1.0.0

      description: "The Docker image used to run `pre-commit`\n\n:information_source: You may build your own pre-configured image to speed-up things and prevent the tool and plugins from being pip-installed (see documentation)."

      default: registry.hub.docker.com/library/python:3-alpine

    pre-commit-args:

      description: pre-commit custom args

      description: "Additionnal arguments for the `pre-commit run` command"

      default: ''

    pre-commit-skip:

      description: "pre-commit `SKIP` environment variable that allows to disable some hooks (see https://pre-commit.com/#temporarily-disabling-hooks)"

      default: 'no-commit-to-branch'

    pre-commit-disabled:

      description: disable the pre-commit job

      description: Disable pre-commit run

      type: boolean

      default: false

  # Default Docker image (use a public image - can be overridden)

  PRE_COMMIT_IMAGE: $[[ inputs.pre-commit-image ]]

  PRE_COMMIT_ARGS: $[[ inputs.pre-commit-args ]]

  PRE_COMMIT_SKIP: $[[ inputs.pre-commit-skip ]]

  PRE_COMMIT_DISABLED: $[[ inputs.pre-commit-disabled ]]

  # default production ref name (pattern)

    fi

  }

  function output_coverage() {

    echo "[TODO]: compute and output global coverage result"

    echo "11% covered"

  function pre_commit_setup() {

    log_info "Performing pre-commit setup..."

    # maybe install default hooks

    if [[ -e .pre-commit-ci.yaml ]]

    then

      log_info "file \\e[32m.pre-commit-ci.yaml\\e[0m found: use"

    else

      log_info "file \\e[32m.pre-commit-ci.yaml\\e[0m not found: initializing default config"

      # Would love to not have an hardcoded config here but it is not possible

      # to use external ressources from included components (yet?).

      # see https://stackoverflow.com/questions/77801510/how-to-access-other-files-in-a-gitlab-ci-cd-component-when-including-it

      cat <<EOF > .pre-commit-ci.yaml

  repos:

  - repo: https://github.com/pre-commit/pre-commit-hooks

    rev: main

    hooks:

      - id: check-merge-conflict

      - id: check-executables-have-shebangs

      - id: check-shebang-scripts-are-executable

      - id: destroyed-symlinks

      - id: end-of-file-fixer

      - id: fix-byte-order-marker

      - id: mixed-line-ending

      - id: trailing-whitespace

        args:

          - --markdown-linebreak-ext=md

  EOF

    fi

    # maybe install pre-commit

    if command -v pre-commit > /dev/null

    then

      log_info "\\e[32mpre-commit\\e[0m found, assuming \\e[33;1m${PRE_COMMIT_IMAGE}\\e[0m is a preconfigured image"

    else

      log_info "\\e[32mpre-commit\\e[0m not found in image \\e[33;1m${PRE_COMMIT_IMAGE}\\e[0m, initializing..."

      apk add --no-cache git

      pip install pre-commit

      pre-commit install-hooks

    fi

  }

  function pre_commit_run() {

    log_info "Performing pre-commit run..."

    if [[ "$CI_MERGE_REQUEST_ID" ]]

    then

      # shellcheck disable=SC2153

      log_info "MR pipeline for \\e[33;1m!${CI_MERGE_REQUEST_IID}\\e[0m: checking only touched files since \\e[33;1m${CI_MERGE_REQUEST_DIFF_BASE_SHA}\\e[0m"

      _scope_opts="--from-ref $CI_MERGE_REQUEST_DIFF_BASE_SHA --to-ref HEAD"

    else

      log_info "non-MR pipeline: checking all files"

      _scope_opts="--all-files"

    fi

    # shellcheck disable=SC2086

    SKIP=${PRE_COMMIT_SKIP} pre-commit run ${_scope_opts} --config .pre-commit-ci.yaml $PRE_COMMIT_ARGS

    log_info "... done"

  }

  unscope_variables

  # ENDSCRIPT

# job prototype

# defines default Docker image, tracking probe, cache policy and tags

.pre-commit-base:

# pre-commit job

pre-commit:

  image:

    name: "$PRE_COMMIT_IMAGE"

    entrypoint: [""]

  services:

    - name: "$TBC_TRACKING_IMAGE"

      command: ["--service", "pre-commit", "1.0.0"]

  stage: build

  needs: []

  before_script:

    - !reference [.pre-commit-scripts]

    - install_ca_certs "${CUSTOM_CA_CERTS:-$DEFAULT_CA_CERTS}"

  # Cache downloaded dependencies and plugins between builds.

  # To keep cache across branches add 'key: "$CI_JOB_NAME"'

  # TODO (if necessary): define cache policy here

  variables:

    # override pip and pre-commit cache dirs

    PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"

    PRE_COMMIT_HOME: "$CI_PROJECT_DIR/.cache/pre-commit"

  cache:

    # cache shall be per branch per template

    key: "$CI_COMMIT_REF_SLUG-pre-commit"

    # cache across all pre-commit jobs in repository

    key: pre-commit

    paths:

      - .cache/

# (example) linter job

pre-commit:

  extends: .pre-commit-base

  stage: build

  # force no dependency

  dependencies: []

  script:

    - mkdir -p -m 777 reports

    - SKIP=no-commit-to-branch pre-commit run -a $PRE_COMMIT_ARGS

    - pre_commit_setup

    - pre_commit_run

  rules:

    # exclude if $PRE_COMMIT_DISABLED

    - if: '$PRE_COMMIT_DISABLED == "true"'