@@ -10,6 +10,28 @@ Using the official _to be continuous_ templates, you may have some specific need
* or maybe develop and share some template adjustments to address your specific technical context (ex: no default untagged runner, need to access the internet through a proxy, ...),
* or even develop your own internal templates.
## Installing tbc in a custom group
By default and preferably, _to be continuous_ shall be installed:
* in the `to-be-continuous` root group on your GitLab server,
* with **public** visibility.
If one or both of these requirements can't be met (because you're not allowed to create a root group in your organization and/or
not allowed to create projects with public visibility), then you'll have a couple of extra things to do to have _to be continuous_
working in your self-managed server:
1. Use the right GitLab Synchronization option(s) when [running the GitLab sync script](./basic.md#run-the-gitlab-synchronization-script) for first time:
*`--dest-sync-path` to override the GitLab destination root group path,
*`--max-visibility` to override the maximum visibility of projects in the destination group.
For more info about GitLab Synchronization Script options, please [refer to the doc](https://gitlab.com/to-be-continuous/tools/gitlab-sync#usage-script).
2. Set the right variable(s) in your local copy of the [tools/gitlab-sync](https://gitlab.com/to-be-continuous/tools/gitlab-sync) project
when [configuring the TBC synchronization](./basic.md#sync-your-local-copy-of-tbc) for first time:
*`$DEST_SYNC_PATH` to override the GitLab destination root group path,
*`$MAX_VISIBILITY` to override the maximum visibility of projects in the destination group.
3. TBC configuration shall be overridden accordingly in the `KICKER_RESOURCE_GROUPS` variable in your local copy of the
[doc](https://gitlab.com/to-be-continuous/doc) project (see the (_Have your own doc + kicker_ chapter)[#have-your-own-doc-kicker]).
## Variable presets
Variable presets are groups of _to be continuous_ variable values that can be used within your company.
@@ -210,6 +232,29 @@ define the `TRACKING_CONFIGURATION` CI/CD project variable as follows:
```
4. Manually start a pipeline on the `main` (or `master`) branch: this will (re)generate a new Docker image with your
configuration that will now be used by every template job.
5. Set the following as an [instance-level CI/CD variable](https://docs.gitlab.com/ee/ci/variables/#for-an-instance):
* name: `TBC_TRACKING_IMAGE`
* value: `$CI_REGISTRY/to-be-continuous/tools/tracking:master` (adapt the path if you've [installed TBC in a custom root group](#installing-tbc-in-a-custom-group))<br/>
_This will override the default tracking image used by all TBC templates_
## Use custom service images
Apart from the tracking image (see [previous chapter](#setup-tracking)), TBC also uses a couple of extra service images.
By default, all latest versions of the TBC templates are pulling those images from the gitlab.com public registry, but you might override this behavior and use your own built/hosted images.
Proceed the same way as explained in the [previous chapter](#setup-tracking):
1. (re)build the image(s) locally in your GitLab server,
2. override the default image by setting the right [instance-level CI/CD variable](https://docs.gitlab.com/ee/ci/variables/#for-an-instance)(see variable names below).
Here are the service images used by TBC templates:
| Project | Description | Image variable |
| ------- | ----------- | -------------- |
| [tools/tracking](https://gitlab.com/to-be-continuous/tools/tracking) | This image can be used to collect statistics about template jobs execution.<br/>**Used by all TBC templates.** | `TBC_TRACKING_IMAGE` |
| [tools/vault-secrets-provider](https://gitlab.com/to-be-continuous/tools/vault-secrets-provider) | This image can be used to retrieve secrets from a [Vault](https://www.vaultproject.io/) server.<br/>Used by TBC Vault variants. | `TBC_VAULT_IMAGE` |
| [tools/aws-auth-provider](https://gitlab.com/to-be-continuous/tools/aws-auth-provider) | This image can be used to retrieve an authorization token for AWS.<br/>Used by TBC AWS variants. | `TBC_AWS_PROVIDER_IMAGE` |
| [tools/gcp-auth-provider](https://gitlab.com/to-be-continuous/tools/gcp-auth-provider) | This image can be used to retrieve an access token for GCP.<br/>Used by TBC GCP variants. | `TBC_GCP_PROVIDER_IMAGE` |
## Use Docker registry mirrors
@@ -241,14 +286,12 @@ Example:
To refer to `aquasec/trivy:latest`, _to be continuous_ templates will always specify `registry.hub.docker.com/aquasec/trivy:latest`
### Drawbacks
When using [`containerd`](https://containerd.io/) as a container runtime, this should have no impact, containerd will still try to use the configured Docker registry mirrors if any.
On the other hand, when using Docker as a container runtime, specifying the registry name when pulling a Docker image prevents Docker from using a Docker registry mirror. Instead, the Docker daemon will directly pull the image from the specified registry. As a consequence, Docker Hub quotas may be reached sooner.
### Workarounds
You can simply override the image names specifying your own Docker registry mirror.
@@ -263,4 +306,3 @@ variables:
```
In this case, both `containerd` and the Docker daemon will try to pull the `aquasec/trivy:latest` image through your Docker registry mirror.
@@ -20,19 +20,6 @@ The GitLab Synchronization Script has the following requirements:
*[curl tool](https://curl.se/) installed and accessible as `curl` command from the Bash interpreter
*[jq tool](https://stedolan.github.io/jq/download/) installed and accessible as `jq` command from the Bash interpreter
:warning: by default and preferably, the GitLab Synchronization Script will clone the tbc group:
* to the `to-be-continuous` root group on your GitLab server,
* with public visibility.
If one or both of these requirements can't be met (because you're not allowed to create a root group in your organization and/or
not allowed to create projects with public visibility), then you'll have to override the default options or variables:
*`--dest-sync-path` / `$DEST_SYNC_PATH` to override the GitLab destination root group path,
*`--max-visibility` / `$MAX_VISIBILITY` to override the maximum visibility of projects in the destination group.
For more info, [see script doc](https://gitlab.com/to-be-continuous/tools/gitlab-sync#usage-script).
### Run the GitLab Synchronization Script
1. Create an empty `to-be-continuous` root group with `public` visibility.
@@ -46,17 +33,25 @@ For more info, [see script doc](https://gitlab.com/to-be-continuous/tools/gitlab
That should take a while, but hopefully at the end you'll have cloned the complete _to be continuous_
group and projects :tada:.
## Build the tracking image
!!! WARNING "Installing _to be continuous_ in a custom root group"
By default and preferably, _to be continuous_ shall be installed:
* in the `to-be-continuous` root group on your GitLab server,
* with **public** visibility.
Every _to be continuous_ template uses the [tracking](https://gitlab.com/to-be-continuous/tools/tracking) Docker image as a [service container](https://docs.gitlab.com/ee/ci/services/).
If one or both of these requirements can't be met (because you're not allowed to create a root group in your organization and/or
not allowed to create projects with public visibility), please
[read the advanced usage chapter](./advanced.md#installing-tbc-in-a-custom-group).
## Build the tracking image
As a result you won't be able to run any pipeline as long as don't have any tracking image available in the registry.
!!! WARNING "Deprecated"
_to be continuous_ used to need the [tracking](https://gitlab.com/to-be-continuous/tools/tracking) Docker image to be successfully built and available locally in your
Docker registry because it is used as a [service container](https://docs.gitlab.com/ee/ci/services/) by all templates.
Go in your local copy of the [tools/tracking](https://gitlab.com/to-be-continuous/tools/tracking) project, and manually click
the last `docker-publish` job that should be pending (if it's not the case, manually start a fresh new pipeline, then click the `docker-publish` job when it's pending). It may happen that the publish job fails the first time: simply retry it.
This is no longer required as in its latest versions, TBC is - by default - pulling the image [from the gitlab.com public registry](https://gitlab.com/to-be-continuous/tools/tracking/container_registry).
We'll see later [how to configure the tracking project](advanced.md#setup-tracking) to collect statistics about template jobs execution.
For now it is unconfigured, and will not track anything.
This can be overridden. For more info, please [read the advanced usage chapter](./advanced.md#setup-tracking).
## Sync. your local copy of tbc
@@ -91,7 +86,7 @@ As explained in the [usage guide](../usage.md#certificate-authority-configuratio
define a `CUSTOM_CA_CERTS` variable either as a group/project variable, or in its `.gitlab-ci.yml` file to declare the custom company Certificate Authorities.
But there is also a global way to fix this. Ask your GitLab administrator to declare `DEFAULT_CA_CERTS`
as an [instance CI/CD variable](https://docs.gitlab.com/ee/ci/variables/#instance-cicd-variables).
as an [instance-level CI/CD variable](https://docs.gitlab.com/ee/ci/variables/#for-an-instance).
Similar to `CUSTOM_CA_CERTS`, `DEFAULT_CA_CERTS` shall contain one or several certificates in [PEM format](https://en.wikipedia.org/wiki/Privacy-Enhanced_Mail).
Every _to be continuous_ template job - prior to executing - determines whether a `$CUSTOM_CA_CERTS` or else `$DEFAULT_CA_CERTS` is defined and adds its content to the trusted CA certificates.