This project implements a generic GitLab CI template for running [XXX](https://link.to.tool.com/) automated tests.
It provides several features, usable in different modes (by configuration).
This project implements a GitLab CI/CD template to continuously analyse your web apps and web pages performances and developer best practices with [Lighthouse CI](https://github.com/GoogleChrome/lighthouse-ci).
## Usage
@@ -10,43 +8,110 @@ In order to include this template in your project, add the following to your `gi
```yaml
include:
-project:'to-be-continuous/xxx'
-project:'to-be-continuous/lighthouse'
ref:'1.0.0'
file:'/templates/gitlab-ci-xxx.yml'
file:'/templates/gitlab-ci-lighthouse.yml'
```
:warning: depending on your needs and environment, you might have to use [one of the template variants](#variants).
## `xxx` job
## `lighthouse` job
This job starts [XXX](https://link.to.tool.com/) tests.
This job runs the [Lighthouse CI](https://github.com/GoogleChrome/lighthouse-ci) analysis.
| `LHCI_IMAGE` | The Docker image used to run Lighthouse CI (use [browser images](https://github.com/cypress-io/cypress-docker-images/tree/master/browsers) only). | `registry.hub.docker.com/cypress/browsers:latest` |
| `LHCI_VERSION` | Lighthouse CI version to run | `latest` |
| `REVIEW_ENABLED` | Set to `true` to enable Lighthouse tests on review environments (dynamic environments instantiated on development branches) | _none_ (disabled) |
All the rest of your Lighthouse CI configuration shall be defined either as [`LHCI_` environment variables](https://github.com/GoogleChrome/lighthouse-ci/blob/main/docs/configuration.md#environment-variables)
or in one of the supported [configuration files](https://github.com/GoogleChrome/lighthouse-ci/blob/main/docs/configuration.md#configuration-file),
located at the root of your Git repository.
In addition to a textual report in the console, this job produces the following reports, kept for one day:
| `reports/lhci/*.{html|json}` | HTML report(s) | n/a |
### `$LHCI_RUN_OPTS` late expansion
The `$LHCI_RUN_OPTS` variable supports a **late variable expansion mechanism** with the `%{somevar}` syntax.
Therefore if an upstream job in the pipeline deployed your code to a server and propagated the deployed server url
as `$environment_url` variable (through a [dotenv artifact](https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html#artifactsreportsdotenv)),
then you can easily run Lighthouse CI against this server, simply by using the `%{environment_url}` syntax in the `$LHCI_RUN_OPTS` variable.
:warning: all _to be continuous_ depoyment templates implement this design. Therefore even purely dynamic environments (such as review
environments) will automatically be propagated to your Lighthouse CI analysis.
If you're not using your own deployment job, you may:
1. implement the common _to be continuous_ design (i.e. propagate the environment base url as `$environment_url` variable through a [dotenv artifact](https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html#artifactsreportsdotenv))
2. use any programmatic solution of your choice with a JavaScript-based [configuration file](https://github.com/GoogleChrome/lighthouse-ci/blob/main/docs/configuration.md#configuration-file).
3. explicitly set the `collect.url` option in the `LHCI_RUN_OPTS` variable (hardcoded to a single server),
## Examples
### Unit tests report integration
### Analysing multiple urls
XXX test reports are [integrated to GitLab by generating JUnit reports](https://docs.gitlab.com/ee/ci/junit_test_reports.html).
This example demonstrates 2 techniques for a project willing to analyse several urls, also reusing the _to be continuous_ `$environment_url` dynamic variable.
This is done using the following CLI options: `--junit --output=reports/`
#### Solution 1: `LHCI_RUN_OPTS`-based
### Base URL auto evaluation
The easiest solution is to declare all the analysed urls in the `$LHCI_RUN_OPTS` variable using the late variable expansion mechanism.
By default, the XXX template tries to auto-evaluate the base URL (i.e. the variable pointing at server under test) by
looking either for a `$environment_url` variable or for an `environment_url.txt` file.
```yaml
include:
-project:'to-be-continuous/lighthouse'
ref:'1.0.0'
file:'/templates/gitlab-ci-lighthouse.yml'
variables:
# pass 3 collect urls, using late %{environment_url} expansion
LHCI_RUN_OPTS:>-
--upload.target=filesystem
--collect.settings.chromeFlags="--no-sandbox"
--collect.url="%{environment_url}"
--collect.url="%{environment_url}/admin"
--collect.url="%{environment_url}/profile"
```
#### Solution 2: Javascript-based
Another option could be to handle it in your JavaScript-based [configuration file](https://github.com/GoogleChrome/lighthouse-ci/blob/main/docs/configuration.md#configuration-file):
Therefore if an upstream job in the pipeline deployed your code to a server and propagated the deployed server url,
either through a [dotenv](https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html#artifactsreportsdotenv) variable `$environment_url`
or through a basic `environment_url.txt` file, then the XXX test will automatically be run on this server.
`.gitlab-ci.yaml`:
:warning: all our deployment templates implement this design. Therefore even purely dynamic environments (such as review
environments) will automatically be propagated to your XXX tests.
```yaml
include:
-project:'to-be-continuous/lighthouse'
ref:'1.0.0'
file:'/templates/gitlab-ci-lighthouse.yml'
If you're not using a smart deployment job, you may still explicitly declare the `XXX_BASE_URL` variable (but that
will be unfortunately hardcoded to a single server).
Security fixes and updates are only applied to the latest released version. So always try to be up to date.
## Reporting a Vulnerability
In order to minimize risks of attack while investigating and fixing the issue, any vulnerability shall be reported by
opening a [**confidential** issue on gitlab.com](https://gitlab.com/to-be-continuous/lighthouse/-/issues/new?issue[confidential]=true&issue[description]=%28type+in+the+vulnerability+details+here%29%0A%0A%2Flabel%20~%22kind%3A%3Avulnerability%22).
Follow-up and fixing will be made on a _best effort_ basis.
If you have doubts about a potential vulnerability, please reach out one of the maintainers on Discord.