Merge branch 'feat/add-aws-variant' into 'master' (209b329e) · Commits · to be continuous... / Docker

README.md

+77 −6

Original line number	Diff line number	Diff line
		@@ -77,7 +77,6 @@ The snapshot and release images are defined by the following variables:
		\| `DOCKER_SNAPSHOT_IMAGE` \| Docker snapshot image \| `$CI_REGISTRY_IMAGE/snapshot:$CI_COMMIT_REF_SLUG` \|
		\| `DOCKER_RELEASE_IMAGE` \| Docker release image \| `$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME` \|


		As you can see, the Docker template is configured by default to use the GitLab container registry.
		You may perfectly override this and use another Docker registry, but be aware of a few things:

		@@ -104,7 +103,6 @@ variables:
		\| :lock: `DOCKER_REGISTRY_USER` \| Docker registry username for image registry \|
		\| :lock: `DOCKER_REGISTRY_PASSWORD`\| Docker registry password for image registry \|


		#### Using different registries for snapshot and release

		If you use different registries for snapshot and release images, you shall use separate configuration variables:
		@@ -116,8 +114,6 @@ If you use different registries for snapshot and release images, you shall u
		\| :lock: `DOCKER_REGISTRY_RELEASE_USER` \| Docker registry username for release image registry \|
		\| :lock: `DOCKER_REGISTRY_RELEASE_PASSWORD`\| Docker registry password for release image registry \|



		#### Setting your own Docker configuration file (advanced)

		There might be cases where you need to provide the complete [Docker configuration file](https://docs.docker.com/engine/reference/commandline/cli/#configuration-files):
		@@ -641,6 +637,9 @@ List of requirements before using this variant for publishing your container ima
		\| `GCP_RELEASE_OIDC_PROVIDER` \| Workload Identity Provider to push the release image _(only define if different from default)_ \| _none_ \|
		\| `GCP_RELEASE_OIDC_ACCOUNT` \| Service Account to use to push the release image _(only define if different from default)_ \| _none_ \|

		:warning: if using Kaniko, don't forget to either create the cache repository (snapshot image repository + `/cache`) or override `$KANIKO_SNAPSHOT_IMAGE_CACHE`
		to use the snapshot image repository (will host your snapshot image as well as cached layers).

		#### Example

		```yaml
		@@ -654,9 +653,10 @@ include:

		variables:
		# untested & unverified container image
		DOCKER_SNAPSHOT_IMAGE: "{GCP_REGION}-docker.pkg.dev/{GCP_PROJECT_ID}/{YOUR_REPOSITORY}/{YOUR_IMAGE_NAME}/snapshot"
		DOCKER_SNAPSHOT_IMAGE: "{GCP_REGION}-docker.pkg.dev/{GCP_PROJECT_ID}/{YOUR_REPOSITORY}/{YOUR_IMAGE_NAME}/snapshot:$CI_COMMIT_REF_SLUG"
		# ⚠ don't forget to create the '{GCP_REGION}-docker.pkg.dev/{GCP_PROJECT_ID}/{YOUR_REPOSITORY}/{YOUR_IMAGE_NAME}/snapshot/cache' repo for Kaniko
		# validated container image (published)
		DOCKER_RELEASE_IMAGE: "{GCP_REGION}-docker.pkg.dev/{GCP_PROJECT_ID}/{YOUR_REPOSITORY}/{YOUR_IMAGE_NAME}"
		DOCKER_RELEASE_IMAGE: "{GCP_REGION}-docker.pkg.dev/{GCP_PROJECT_ID}/{YOUR_REPOSITORY}/{YOUR_IMAGE_NAME}:$CI_COMMIT_REF_NAME"
		# default WIF provider
		GCP_OIDC_PROVIDER: "projects/{GCP_PROJECT_NUMBER}/locations/global/workloadIdentityPools/{YOUR_WIF_POOL_NAME}/providers/gitlab-diod"
		# default GCP Service Account
		@@ -667,3 +667,74 @@ variables:
		GCP_SNAPSHOT_OIDC_ACCOUNT: "{YOUR_REGISTRY_SA}@{GCP_PROJECT_ID}.iam.gserviceaccount.com"
		DOCKER_BUILD_TOOL: "kaniko" # Only Kaniko has been proved to work for this use case YET
		```

		### Amazon Elastic Container Registry

		This variant allows publishing your container images to Amazon's [Elastic Container Registry](https://docs.aws.amazon.com/ecr/).

		It takes care of retrieving an [ECR authorization token](https://docs.aws.amazon.com/AmazonECR/latest/APIReference/API_GetAuthorizationToken.html)
		that will be used as a temporary credential to login to the ECR registry.

		In order to use the AWS APIs, the variant supports two authentication methods:

		1. [federated authentication using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/aws/) (recommended method),
		2. or basic authentication with AWS access key ID & secret access key.

		:warning: when using this variant, you must have created the ECR repositories to push the snapshot and/or the release images.

		#### Configuration

		\| Name \| description \| default value \|
		\| ------------------------ \| -------------------------------------- \| ----------------- \|
		\| `TBC_AWS_PROVIDER_IMAGE` \| The [AWS Auth Provider](https://gitlab.com/to-be-continuous/tools/aws-auth-provider) image to use (can be overridden) \| `$CI_REGISTRY/to-be-continuous/tools/aws-auth-provider:master` \|
		\| `AWS_REGION` \| Default region (where the ECR registry is located) \| _none_ \|
		\| `AWS_SNAPSHOT_REGION` \| Region of the ECR registry for the snapshot image _(only define if different from default)_ \| _none_ \|
		\| `AWS_RELEASE_REGION` \| Region of the ECR registry for the release image _(only define if different from default)_ \| _none_ \|

		:warning: if using Kaniko, don't forget to either create the cache repository (snapshot image repository + `/cache`) or override `$KANIKO_SNAPSHOT_IMAGE_CACHE`
		to use the snapshot image repository (will host your snapshot image as well as cached layers).

		##### OIDC authentication config

		This is the recommended authentication method. In order to use it, first carefuly follow [GitLab's documentation](https://docs.gitlab.com/ee/ci/cloud_services/aws/),
		then set the required configuration.

		\| Name \| description \| default value \|
		\| ------------------------ \| -------------------------------------- \| ----------------- \|
		\| `AWS_OIDC_AUD` \| The `aud` claim for the JWT token \| `$CI_SERVER_URL` \|
		\| `AWS_OIDC_ROLE_ARN` \| Default IAM Role ARN associated with GitLab \| _none_ \|
		\| `AWS_SNAPSHOT_OIDC_ROLE_ARN`\| IAM Role ARN associated with GitLab for the snapshot image _(only define if different from default)_\| _none_ \|
		\| `AWS_RELEASE_OIDC_ROLE_ARN`\| IAM Role ARN associated with GitLab for the release image _(only define if different from default)_\| _none_ \|

		##### Basic authentication config

		\| Name \| description \| default value \|
		\| ------------------------ \| -------------------------------------- \| ----------------- \|
		\| `AWS_ACCESS_KEY_ID` \| Default access key ID \| _none_ (disabled) \|
		\| `AWS_SECRET_ACCESS_KEY` \| Default secret access key \| _none_ (disabled) \|
		\| `AWS_SNAPSHOT_ACCESS_KEY_ID`\| Access key ID for the snapshot image _(only define if different from default)_ \| _none_ \|
		\| `AWS_SNAPSHOT_SECRET_ACCESS_KEY`\| Secret access key for the snapshot image _(only define if different from default)_ \| _none_ \|
		\| `AWS_RELEASE_ACCESS_KEY_ID`\| Access key ID for the release image _(only define if different from default)_ \| _none_ \|
		\| `AWS_RELEASE_SECRET_ACCESS_KEY`\| Secret access key for the release image _(only define if different from default)_ \| _none_ \|

		#### Example

		```yaml
		include:
		- project: 'to-be-continuous/docker'
		ref: "5.2.0"
		file: '/templates/gitlab-ci-docker.yml'
		- project: 'to-be-continuous/docker'
		ref: "5.2.0"
		file: '/templates/gitlab-ci-docker-ecr.yml'

		variables:
		AWS_REGION: "us-east-1"
		# untested & unverified container image
		DOCKER_SNAPSHOT_IMAGE: "123456789012.dkr.ecr.us-east-1.amazonaws.com/$CI_PROJECT_PATH/snapshot:$CI_COMMIT_REF_SLUG"
		# ⚠ don't forget to create the '123456789012.dkr.ecr.us-east-1.amazonaws.com/$CI_PROJECT_PATH/snapshot/cache' repo for Kaniko
		# validated container image (published)
		DOCKER_RELEASE_IMAGE: "123456789012.dkr.ecr.us-east-1.amazonaws.com/$CI_PROJECT_PATH:$CI_COMMIT_REF_NAME"
		# default Role ARN (using OIDC authentication method)
		AWS_OIDC_ROLE_ARN: "arn:aws:iam::123456789012:role/gitlab-ci"
		```

kicker.json

+85 −1

Original line number	Diff line number	Diff line
		@@ -269,7 +269,7 @@
		{
		"name": "TBC_GCP_PROVIDER_IMAGE",
		"description": "The [GCP Auth Provider](https://gitlab.com/to-be-continuous/tools/gcp-auth-provider) image to use",
		"default": "$CI_REGISTRY/to-be-continuous/tools/vault-secrets-provider:master",
		"default": "$CI_REGISTRY/to-be-continuous/tools/gcp-auth-provider:main",
		"advanced": true
		},
		{
		@@ -301,6 +301,90 @@
		"advanced": true
		}
		]
		},
		{
		"id": "ecr",
		"name": "Amazon ECR",
		"description": "Retrieves a registry authentication for the Amazon's [Elastic Container Registry](https://docs.aws.amazon.com/ecr/)",
		"template_path": "templates/gitlab-ci-docker-ecr.yml",
		"variables": [
		{
		"name": "TBC_AWS_PROVIDER_IMAGE",
		"description": "The [AWS Auth Provider](https://gitlab.com/to-be-continuous/tools/aws-auth-provider) image to use",
		"default": "$CI_REGISTRY/to-be-continuous/tools/aws-auth-provider:master",
		"advanced": true
		},
		{
		"name": "AWS_REGION",
		"description": "Default region (where the ECR registry is located)"
		},
		{
		"name": "AWS_SNAPSHOT_REGION",
		"description": "Region of the ECR registry for the snapshot image _(only define if different from default)_",
		"advanced": true
		},
		{
		"name": "AWS_RELEASE_REGION",
		"description": "Region of the ECR registry for the release image _(only define if different from default)_",
		"advanced": true
		},
		{
		"name": "AWS_OIDC_AUD",
		"description": "The `aud` claim for the JWT token _(only required for [OIDC authentication](https://docs.gitlab.com/ee/ci/cloud_services/aws/))_",
		"default": "$CI_SERVER_URL",
		"advanced": true
		},
		{
		"name": "AWS_OIDC_ROLE_ARN",
		"description": "Default IAM Role ARN associated with GitLab _(only required for [OIDC authentication](https://docs.gitlab.com/ee/ci/cloud_services/aws/))_"
		},
		{
		"name": "AWS_SNAPSHOT_OIDC_ROLE_ARN",
		"description": "IAM Role ARN associated with GitLab for the snapshot image _(only required for [OIDC authentication](https://docs.gitlab.com/ee/ci/cloud_services/aws/) and if different from default)_",
		"advanced": true
		},
		{
		"name": "AWS_RELEASE_OIDC_ROLE_ARN",
		"description": "IAM Role ARN associated with GitLab for the release image _(only required for [OIDC authentication](https://docs.gitlab.com/ee/ci/cloud_services/aws/) and if different from default)_",
		"advanced": true
		},
		{
		"name": "AWS_ACCESS_KEY_ID",
		"description": "Default access key ID (only required for basic authentication)",
		"secret": true,
		"advanced": true
		},
		{
		"name": "AWS_SECRET_ACCESS_KEY",
		"description": "Default secret access key (only required for basic authentication)",
		"secret": true,
		"advanced": true
		},
		{
		"name": "AWS_SNAPSHOT_ACCESS_KEY_ID",
		"description": "Access key ID for the snapshot image (only required for basic authentication and if different from default)",
		"secret": true,
		"advanced": true
		},
		{
		"name": "AWS_SNAPSHOT_SECRET_ACCESS_KEY",
		"description": "Secret access key for the snapshot image (only required for basic authentication and if different from default)",
		"secret": true,
		"advanced": true
		},
		{
		"name": "AWS_RELEASE_ACCESS_KEY_ID",
		"description": "Access key ID for the release image (only required for basic authentication and if different from default)",
		"secret": true,
		"advanced": true
		},
		{
		"name": "AWS_RELEASE_SECRET_ACCESS_KEY",
		"description": "Secret access key for the release image (only required for basic authentication and if different from default)",
		"secret": true,
		"advanced": true
		}
		]
		}
		]
		}

templates/gitlab-ci-docker-ecr.yml

0 → 100644

+34 −0

Original line number	Diff line number	Diff line
		# =====================================================================================================================
		# === AWS Auth template variant
		# =====================================================================================================================
		variables:
		TBC_AWS_AUTH_PROVIDER: "$CI_REGISTRY/to-be-continuous/tools/aws-auth-provider:master"
		AWS_OIDC_AUD: "$CI_SERVER_URL"

		.docker-base:
		services:
		- name: "$TBC_TRACKING_IMAGE"
		command: ["--service", "docker", "5.4.1"]
		- name: "$TBC_AWS_AUTH_PROVIDER"
		alias: "aws-auth-provider"
		id_tokens:
		# required for OIDC auth
		AWS_JWT:
		aud: "$AWS_OIDC_AUD"
		variables:
		# DOCKER_REGISTRY_SNAPSHOT_USER: "@url@http://aws-auth-provider/ecr/auth/username?env_ctx=SNAPSHOT"
		# DOCKER_REGISTRY_RELEASE_USER: "@url@http://aws-auth-provider/ecr/auth/username?env_ctx=RELEASE"
		DOCKER_REGISTRY_SNAPSHOT_USER: "AWS" # GetAuthorizationToken API always generate token for user 'AWS'
		DOCKER_REGISTRY_RELEASE_USER: "AWS" # GetAuthorizationToken API always generate token for user 'AWS'
		DOCKER_REGISTRY_SNAPSHOT_PASSWORD: "@url@http://aws-auth-provider/ecr/auth/password?env_ctx=SNAPSHOT"
		DOCKER_REGISTRY_RELEASE_PASSWORD: "@url@http://aws-auth-provider/ecr/auth/password?env_ctx=RELEASE"
		# secrets have to be explicitly declared in the YAML to be exported to the service
		AWS_JWT: "$AWS_JWT"
		# can't use AWS_ACCESS_KEY_ID as it is read by boto3
		AWS_DEFAULT_ACCESS_KEY_ID: "$AWS_ACCESS_KEY_ID"
		# can't use AWS_SECRET_ACCESS_KEY as it is read by boto3
		AWS_DEFAULT_SECRET_ACCESS_KEY: "$AWS_SECRET_ACCESS_KEY"
		AWS_SNAPSHOT_ACCESS_KEY_ID: "$AWS_SNAPSHOT_ACCESS_KEY_ID"
		AWS_SNAPSHOT_SECRET_ACCESS_KEY: "$AWS_SNAPSHOT_SECRET_ACCESS_KEY"
		AWS_RELEASE_ACCESS_KEY_ID: "$AWS_RELEASE_ACCESS_KEY_ID"
		AWS_RELEASE_SECRET_ACCESS_KEY: "$AWS_RELEASE_SECRET_ACCESS_KEY"