feat: add parallel:matrix jobs support (7df144fe) · Commits · to be continuous... / Azure

README.md

+48 −0

Original line number	Diff line number	Diff line
		@@ -203,6 +203,43 @@ The static way can be implemented simply by setting the appropriate configur
		To implement the dynamic way, your deployment script shall simply generate a `environment_url.txt` file in the working directory, containing only
		the dynamically generated url. When detected by the template, it will use it as the newly deployed environment url.

		### Multiple environments support

		The Azure template allows deploying multiple environments in parallel. Use cases of this include:

		- monorepo, where a single Git repository might host several separate deployable components or apps,
		- multi-instances deployment of the same application.

		This feature can be enabled using the [parallel matrix jobs](https://docs.gitlab.com/ee/ci/yaml/#parallelmatrix)
		pattern at the `.azure-env-base` job level (this is the top parent job of all deployment jobs).
		Environments namespacing is ensured by the `AZURE_ENVIRONMENT_NAMESPACE` variable (must start with a slash `/`).

		Here is the example of the `.gitlab-ci.yml` file for a project deploying both a frontend and a backend applications:

		```yaml
		.azure-env-base:
		parallel:
		matrix:
		- AZURE_ENVIRONMENT_NAMESPACE: "/front"
		# Azure deployment scripts are located in the ./front/ directory
		AZURE_SCRIPTS_DIR: "front"
		- AZURE_ENVIRONMENT_NAMESPACE: "/back"
		# Azure deployment scripts are located in the ./back/ directory
		AZURE_SCRIPTS_DIR: "back"

		# ⚠ on_stop must be unset when defining parallel:matrix environments
		# see: https://gitlab.com/gitlab-org/gitlab/-/issues/332247
		azure-review:
		environment:
		on_stop: null
		```

		The above configuration will deploy 2 environments on each pipeline:

		- on feature branches: `review/front/$CI_COMMIT_REF_NAME` and `review/back/$CI_COMMIT_REF_NAME`
		- on the integration branch: `integration/front` and `integration/back`
		- on the production branch: `staging/front` and `staging/back` (and finally `production/front` and `production/back`)

		### Deployment output variables

		Each deployment job produces _output variables_ that are propagated to downstream jobs (using [dotenv artifacts](https://docs.gitlab.com/ci/yaml/artifacts_reports/#artifactsreportsdotenv)):
		@@ -215,6 +252,16 @@ Those variables may be freely used in downstream jobs (for instance to run accep

		You may also add and propagate your own custom variables, by pushing them to the `azure.env` file in your [deployment script](#deployment-and-cleanup-scripts).

		> [!important]
		> If [multiple environments](#multiple-environments-support) are configured, the output variables are prefixed with a
		> sluggified value of the `AZURE_ENVIRONMENT_NAMESPACE` variable (stripped of punctuation characters and converted to lowercase):
		>
		> * `<namespace_slug>_environment_type`: set to the type of environment (`review`, `integration`, `staging` or `production`),
		> * `<namespace_slug>_environment_name`: the application name (see below),
		> * `<namespace_slug>_environment_url`: set to the environment URL (whether determined statically or dynamically).
		>
		> The output dotenv file will be `azure.env.<namespace_slug>` instead, and the dynamic variable `${environment_namespace}` can be used in your scripts and manifests to access the contextual value of `<namespace_slug>`.

		## Configuration reference

		### Secrets management
		@@ -240,6 +287,7 @@ The Azure template uses some global configuration used throughout all jobs and e
		\| `cli-image` / `AZURE_CLI_IMAGE` \| the Docker image used to run Azure CLI commands\| `mcr.microsoft.com/azure-cli:latest` <br/>[![Trivy Badge](https://to-be-continuous.gitlab.io/doc/secu/trivy-badge-AZURE_CLI_IMAGE.svg)](https://to-be-continuous.gitlab.io/doc/secu/trivy-AZURE_CLI_IMAGE) \|
		\| `base-app-name` / `AZURE_BASE_APP_NAME` \| Base application name \| `$CI_PROJECT_NAME` ([see GitLab doc](https://docs.gitlab.com/ci/variables/predefined_variables/)) \|
		\| `environment-url` / `AZURE_ENVIRONMENT_URL` \| Default environments url _(only define for static environment URLs declaration)_<br/>_supports late variable expansion (ex: `https://%{environment_name}.azure.acme.com`)_ \| _none_ \|
		\| `environment-namespace` / `AZURE_ENVIRONMENT_NAMESPACE` \| Extra [GitLab environments](https://docs.gitlab.com/ci/environments/) namespace _(only required when deploying [multiple environments](#multiple-environments-support))_<br/>:warning: must start with a slash `/` \| _none_ \|
		\| `scripts-dir` / `AZURE_SCRIPTS_DIR` \| Directory where Azure scripts (deploy & cleanup) are located \| `.` _(root project dir)_ \|
		\| `sp-client-id` / `AZURE_SP_CLIENT_ID` \| Default Service Principal client ID _(only define if using Service Principal authentication with credentials)_ \| _none_ \|
		\| :lock: `AZURE_SP_PASSWORD`\| Default Service Principal password (client secret or certificate (File type)) _(only define if using Service Principal authentication with credentials)_ \| _none_ \|

kicker.json

+5 −0

Original line number	Diff line number	Diff line
		@@ -22,6 +22,11 @@
		"type": "url",
		"description": "The default environments url _(only define for static environment URLs declaration)_\n\n_supports late variable expansion (ex: `https://%{environment_name}.azure.acme.com`)_"
		},
		{
		"name": "AZURE_ENVIRONMENT_NAMESPACE",
		"description": "Extra [GitLab environments](https://docs.gitlab.com/ci/environments/) namespace _(only required when deploying multiple environments)_\n\n:warning: must start with a slash `/`",
		"advanced": true
		},
		{
		"name": "AZURE_SCRIPTS_DIR",
		"description": "Directory where Azure scripts (deploy & cleanup) are located",

templates/gitlab-ci-azure-oidc.yml

+1 −1

Original line number	Diff line number	Diff line
		@@ -51,7 +51,7 @@ variables:
		AZURE_PROD_OIDC_CLIENT_ID: $[[ inputs.prod-oidc-client-id ]]
		AZURE_PROD_OIDC_TENANT_ID: $[[ inputs.prod-oidc-tenant-id ]]

		.azure-base:
		.azure-env-base:
		id_tokens:
		AZURE_JWT:
		aud: "$AZURE_OIDC_AUD"

templates/gitlab-ci-azure.yml

+47 −32

Original line number	Diff line number	Diff line
		@@ -28,6 +28,12 @@ spec:

		_supports late variable expansion (ex: `https://%{environment_name}.azure.acme.com`)_
		default: ''
		environment-namespace:
		description: \|-
		Extra [GitLab environments](https://docs.gitlab.com/ci/environments/) namespace _(only required when deploying multiple environments)_

		:warning: must start with a slash `/`
		default: ''
		scripts-dir:
		description: Directory where Azure scripts (deploy & cleanup) are located
		default: .
		@@ -172,6 +178,7 @@ variables:
		INTEG_REF: /^develop$/

		AZURE_ENVIRONMENT_URL: $[[ inputs.environment-url ]]
		AZURE_ENVIRONMENT_NAMESPACE: $[[ inputs.environment-namespace ]]
		AZURE_SP_CLIENT_ID: $[[ inputs.sp-client-id ]]
		AZURE_SP_TENANT_ID: $[[ inputs.sp-tenant-id ]]
		AZURE_REVIEW_ENABLED: $[[ inputs.review-enabled ]]
		@@ -538,13 +545,17 @@ stages:
		hostname=$(echo "$environment_url" \| awk -F[/:] '{print $4}')
		export hostname

		# compute environment namespace (strip punctuation, lowercase)
		environment_namespace=$(echo "$AZURE_ENVIRONMENT_NAMESPACE" \| tr -d '[:punct:]' \| tr '[:upper:]' '[:lower:]')
		export environment_namespace

		log_info "--- \\e[32mdeploy\\e[0m"
		log_info "--- \$environment_type: \\e[33;1m${environment_type}\\e[0m"
		log_info "--- \$environment_name: \\e[33;1m${environment_name}\\e[0m"
		log_info "--- \$hostname: \\e[33;1m${hostname}\\e[0m"

		# unset any upstream deployment env & artifacts
		rm -f azure.env
		rm -f azure.env*
		rm -f environment_url.txt

		deployscript=$(ls -1 "$AZURE_SCRIPTS_DIR/azure-deploy-${environment_type}.sh" 2>/dev/null \|\| ls -1 "$AZURE_SCRIPTS_DIR/azure-deploy.sh" 2>/dev/null \|\| echo "")
		@@ -566,8 +577,15 @@ stages:
		else
		echo "$environment_url" > environment_url.txt
		fi
		echo -e "environment_type=$environment_type\\nenvironment_name=$environment_name\\nenvironment_url=$environment_url" >> azure.env
		chmod 644 environment_url.txt azure.env
		# var prefix ('_' if namespace)
		prefix="${environment_namespace:+${environment_namespace}_}"
		dotenvfile="azure.env${environment_namespace:+.${environment_namespace}}"
		{
		echo "${prefix}environment_type=${environment_type}"
		echo "${prefix}environment_name=${environment_name}"
		echo "${prefix}environment_url=${environment_url}"
		} >> "$dotenvfile"
		chmod 644 environment_url.txt "$dotenvfile"
		}

		# environment cleanup function
		@@ -609,20 +627,31 @@ stages:
		before_script:
		- !reference [.azure-scripts]
		- install_ca_certs "${CUSTOM_CA_CERTS:-$DEFAULT_CA_CERTS}"
		- auth

		# Deploy job prototype
		# Env management job prototype
		# Can be extended to define a concrete environment
		#
		# @arg ENV_TYPE : environment type
		# @arg ENV_APP_NAME : env-specific application name
		# @arg ENV_APP_SUFFIX : env-specific application suffix
		# @arg ENV_URL : env-specific application url
		.azure-deploy:
		.azure-env-base:
		extends: .azure-base
		stage: deploy
		variables:
		ENV_APP_SUFFIX: "-$CI_ENVIRONMENT_SLUG"
		before_script:
		- !reference [.azure-base, before_script]
		- auth
		environment:
		name: ${ENV_TYPE}${AZURE_ENVIRONMENT_NAMESPACE}
		resource_group: ${ENV_TYPE}${AZURE_ENVIRONMENT_NAMESPACE}

		# Deploy job prototype
		# Can be extended for each deployable environment
		#
		# @arg ENV_URL : env-specific application url
		.azure-deploy:
		extends: .azure-env-base
		script:
		- deploy
		artifacts:
		@@ -630,23 +659,16 @@ stages:
		paths:
		- environment_url.txt
		reports:
		dotenv: azure.env
		dotenv: azure.env*
		environment:
		url: "$environment_url" # can be either static or dynamic

		# Cleanup job prototype
		# Can be extended for each deletable environment
		#
		# @arg ENV_TYPE : environment type
		# @arg ENV_APP_NAME : env-specific application name
		# @arg ENV_APP_SUFFIX: env-specific application suffix
		.azure-cleanup:
		extends: .azure-base
		stage: deploy
		extends: .azure-env-base
		# force no dependencies
		dependencies: []
		variables:
		ENV_APP_SUFFIX: "-$CI_ENVIRONMENT_SLUG"
		script:
		- delete
		environment:
		@@ -668,10 +690,12 @@ azure-review:
		ENV_OIDC_CLIENT_ID: "$AZURE_REVIEW_OIDC_CLIENT_ID"
		ENV_OIDC_TENANT_ID: "$AZURE_REVIEW_OIDC_TENANT_ID"
		environment:
		name: review/$CI_COMMIT_REF_NAME
		name: ${ENV_TYPE}${AZURE_ENVIRONMENT_NAMESPACE}/${CI_COMMIT_REF_NAME}
		# ⚠ on_stop must be unset when defining parallel:matrix environments
		# see: https://gitlab.com/gitlab-org/gitlab/-/issues/332247
		on_stop: azure-cleanup-review
		auto_stop_in: "$AZURE_REVIEW_AUTOSTOP_DURATION"
		resource_group: review/$CI_COMMIT_REF_NAME
		resource_group: ${ENV_TYPE}${AZURE_ENVIRONMENT_NAMESPACE}/${CI_COMMIT_REF_NAME}
		rules:
		# exclude tags
		- if: $CI_COMMIT_TAG
		@@ -696,9 +720,9 @@ azure-cleanup-review:
		ENV_OIDC_CLIENT_ID: "$AZURE_REVIEW_OIDC_CLIENT_ID"
		ENV_OIDC_TENANT_ID: "$AZURE_REVIEW_OIDC_TENANT_ID"
		environment:
		name: review/$CI_COMMIT_REF_NAME
		name: ${ENV_TYPE}${AZURE_ENVIRONMENT_NAMESPACE}/${CI_COMMIT_REF_NAME}
		action: stop
		resource_group: review/$CI_COMMIT_REF_NAME
		resource_group: ${ENV_TYPE}${AZURE_ENVIRONMENT_NAMESPACE}/${CI_COMMIT_REF_NAME}
		rules:
		# exclude tags
		- if: $CI_COMMIT_TAG
		@@ -722,9 +746,6 @@ azure-integration:
		# service principal login (OIDC token)
		ENV_OIDC_CLIENT_ID: "$AZURE_INTEG_OIDC_CLIENT_ID"
		ENV_OIDC_TENANT_ID: "$AZURE_INTEG_OIDC_TENANT_ID"
		environment:
		name: integration
		resource_group: integration
		rules:
		# only on integration branch(es), with $AZURE_INTEG_ENABLED set
		- if: '$AZURE_INTEG_ENABLED == "true" && $CI_COMMIT_REF_NAME =~ $INTEG_REF'
		@@ -743,9 +764,6 @@ azure-staging:
		# service principal login (OIDC token)
		ENV_OIDC_CLIENT_ID: "$AZURE_STAGING_OIDC_CLIENT_ID"
		ENV_OIDC_TENANT_ID: "$AZURE_STAGING_OIDC_TENANT_ID"
		environment:
		name: staging
		resource_group: staging
		rules:
		# only on production branch(es), with $AZURE_STAGING_ENABLED set
		- if: '$AZURE_STAGING_ENABLED == "true" && $CI_COMMIT_REF_NAME =~ $PROD_REF'
		@@ -765,9 +783,6 @@ azure-production:
		# service principal login (OIDC token)
		ENV_OIDC_CLIENT_ID: "$AZURE_PROD_OIDC_CLIENT_ID"
		ENV_OIDC_TENANT_ID: "$AZURE_PROD_OIDC_TENANT_ID"
		environment:
		name: production
		resource_group: production
		rules:
		# exclude non-production branches
		- if: '$CI_COMMIT_REF_NAME !~ $PROD_REF'