Commit 9df38ac3 authored by Pierre Smeyers's avatar Pierre Smeyers
Browse files

Merge branch 'feat/terraform-matrix' into 'main' 

feat: add support for parallel:matrix - #114

Closes #114

See merge request to-be-continuous/terraform!167
parents aed56118 74337516

README.md

+83 −13
Original line number Diff line number Diff line
@@ -216,6 +216,65 @@ variables: 
  # OS_USERNAME & OS_PASSWORD are overridden as secret GitLab CI variables

```



### Multiple environments support



The terraform template allows deploying multiple environments in parallel.

Use cases of this include:



- monorepo, where a single Git repository might host several separate deployable components or apps,

- multi-instances deployment of the same application.



This feature can be enabled using the [parallel matrix jobs](https://docs.gitlab.com/ee/ci/yaml/#parallelmatrix) pattern

at the `.tf-base` or `.tf-workspace` job level.

The job on which put the matrix depends on your needs (whether to use multiple terraform modules in the same repository or only one).



Environments namespacing is ensured by the `TF_ENVIRONMENT_NAMESPACE` variable (must start with a `/`).



Here is an example of the `.gitlab-ci.yml` file for a project deploying both a frontend and a backend application:



```yaml

variables:

  # Terraform deployment scripts are located in the ./scripts/ directory

  TF_SCRIPTS_DIR: $CI_PROJECT_DIR/scripts



# Overriding .tf-base to get all security and deployments jobs for every terraform module

.tf-base:

  parallel:

    matrix:

      - TF_ENVIRONMENT_NAMESPACE: /front

        TF_PROJECT_DIR: front

      - TF_ENVIRONMENT_NAMESPACE: /back

        TF_PROJECT_DIR: back

```



The above configuration will deploy 2 environments on each pipeline:



- on feature branches: `review/front/$CI_COMMIT_REF_NAME` and `review/back/$CI_COMMIT_REF_NAME`

- on the integration branch: `integration/front` and `integration/back`

- on the production branch: `staging/front` and `staging/back` (and finally `production/front` and `production/back`)



Here's another example of the `.gitlab-ci.yml` file for a project deploying the same application on multiple environments:



```yaml

variables:

  # Terraform deployment scripts are located in the ./scripts/ directory

  TF_SCRIPTS_DIR: $CI_PROJECT_DIR/scripts



# Overriding .tf-workspace to get only deployments jobs for each terraform module

.tf-workspace:

  parallel:

    matrix:

      - TF_ENVIRONMENT_NAMESPACE: /some-project-id

        GCP_PROJECT_ID: some-project-id

      - TF_ENVIRONMENT_NAMESPACE: /some-other-project-id

        GCP_PROJECT_ID: some-other-project-id

```



The above configuration will deploy 2 enviroments on deployments jobs (`tf-plan-*`, `tf-review`, `tf-destroy-*`, etc.).



:info: When using both this feature and [GitLab backend](#gitlab-managed-terraform-state-default), the state name will be `<namespace>_<environment_slug>`

where `namespace` is built from `TF_ENVIRONMENT_NAMESPACE` (stripped of punctuation characters and converted to lowercase).



### Supported output artifacts



The Terraform template supports [job artifacts](https://docs.gitlab.com/ci/jobs/job_artifacts/) that your Terraform
@@ -231,6 +290,16 @@ Examples: 
* When used in conjuction with Ansible template, your Terraform script may [generate the Ansible inventory file](https://www.percona.com/blog/how-to-generate-an-ansible-inventory-with-terraform/) into the `$TF_OUTPUT_DIR` directory.

* When dynamically obtaining a floating IP address, your Terraform script may generate the `terraform.env` file to propated it as an environment variables.



> [!important]

> If [multiple environments](#multiple-environments-support) are configured, the output variables are prefixed with a

> sluggified value of the `TF_ENVIRONMENT_NAMESPACE` variable (stripped of punctuation characters and converted to lowercase):

>

> * `<namespace_slug>_environment_type`: set to the type of environment (`review`, `integration`, `staging` or `production`),

> * `<namespace_slug>_environment_name`: the application name (see below),

> * `<namespace_slug>_environment_url`: set to the environment URL (whether determined statically or dynamically).

>

> The output dotenv file will be `terraform.env.<namespace_slug>` instead, and the dynamic variable `${environment_namespace}` can be used in your scripts and manifests to access the contextual value of `<namespace_slug>`.



### Terraform integration in Merge Requests



This template enables [Terraform integration in Merge Requests](https://docs.gitlab.com/user/infrastructure/iac/mr_integration/).
@@ -376,7 +445,7 @@ Here are some advices about your **secrets** (variables marked with a :lock:): 
The Terraform template uses some global configuration used throughout all jobs.



| Input / Variable                                         | Description                                                                                                                                                                                                        | Default value                                                                                                                                                                                  |

| -------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |

| -------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |

| `image` / `TF_IMAGE`                                     | the Docker image used to run Terraform CLI commands <br/>:warning: **set the version required by your project**                                                                                                    | `docker.io/hashicorp/terraform:latest` <br/>[![Trivy Badge](https://to-be-continuous.gitlab.io/doc/secu/trivy-badge-TF_IMAGE.svg)](https://to-be-continuous.gitlab.io/doc/secu/trivy-TF_IMAGE) |

| `gitlab-backend-disabled` / `TF_GITLAB_BACKEND_DISABLED` | Set to `true` to disable [GitLab managed Terraform State](https://docs.gitlab.com/user/infrastructure/iac/terraform_state/)                                                                                        | _none_ (enabled)                                                                                                                                                                               |

| `project-dir` / `TF_PROJECT_DIR`                         | Terraform project root directory                                                                                                                                                                                   | `.`                                                                                                                                                                                            |
@@ -388,6 +457,7 @@ The Terraform template uses some global configuration used throughout all jobs. 
| `apply-opts` / `TF_APPLY_OPTS`                           | Default Terraform extra [apply options](https://developer.hashicorp.com/terraform/cli/commands/apply)                                                                                                              | _none_                                                                                                                                                                                         |

| `destroy-opts` / `TF_DESTROY_OPTS`                       | Default Terraform extra [destroy options](https://developer.hashicorp.com/terraform/cli/commands/destroy)                                                                                                          | _none_                                                                                                                                                                                         |

| `apk-extra-opts` / `TF_APK_EXTRA_OPTS`                   | Extra [`apk add` options](https://www.mankier.com/8/apk) (`apk` is used to install `jq` and/or `curl` if necessary)                                                                                                | _none_                                                                                                                                                                                         |

| `environment-namespace` / `TF_ENVIRONMENT_NAMESPACE`     | Extra [GitLab environments](https://docs.gitlab.com/ci/environments/) namespace _(only required when deploying [multiple environments](#multiple-environments-support))_<br/>:warning: must start with a slash `/` | _none_                                                                                                                                                                                         |



### Review environments configuration

kicker.json

+5 −0
Original line number Diff line number Diff line
@@ -70,6 +70,11 @@ 
      "name": "TF_APK_EXTRA_OPTS",

      "description": "Extra [`apk add` options](https://www.mankier.com/8/apk) (`apk` is used to install `jq` and/or `curl` if necessary)",

      "advanced": true

    },

    {

      "name": "TF_ENVIRONMENT_NAMESPACE",

      "description": "Extra [GitLab environments](https://docs.gitlab.com/ci/environments/) namespace _(only required when deploying multiple environments)_\n\n:warning: must start with a slash `/`",

      "advanced": true

    }

  ],

  "features": [

templates/gitlab-ci-terraform-aws.yml

+2 −2
Original line number Diff line number Diff line
@@ -23,7 +23,7 @@ spec: 
      default: ''

---

.tf-commands:

  init:

  gitlab_auth:

    # init Assume Role with Web Identity Configuration

    # see: https://registry.terraform.io/providers/hashicorp/aws/latest/docs#assume-role-with-web-identity-configuration-reference

    - |
@@ -46,7 +46,7 @@ spec: 
        export AWS_ROLE_ARN="${env_role_arn:-$AWS_OIDC_ROLE_ARN}"

        export AWS_ROLE_SESSION_NAME="GitLabRunner-${CI_PROJECT_ID}-${CI_PIPELINE_ID}"

      fi

    - !reference [ .tf-commands, default, init ]

    - !reference [ .tf-commands, default, gitlab_auth ]



variables:

  AWS_OIDC_AUD: $[[ inputs.aws-oidc-aud ]]

templates/gitlab-ci-terraform-gcp.yml

+42 −47
Original line number Diff line number Diff line
@@ -51,7 +51,8 @@ variables: 
  GCP_PROD_OIDC_ACCOUNT: $[[ inputs.gcp-prod-oidc-account ]]

  GCP_PROD_OIDC_PROVIDER: $[[ inputs.gcp-prod-oidc-provider ]]



.tf-gcp-adc:

.tf-commands:

  gitlab_auth:

    - |- 

      if [[ "$GCP_JWT" ]]

      then
@@ -92,15 +93,9 @@ variables: 
      else

        echo '[WARN] $GCP_JWT is not set: cannot setup Application Default Credentials (ADC) authentication'

      fi

    - !reference [ .tf-commands, default, gitlab_auth ]



.tf-workspace:

  extends: .tf-base

.tf-base:

  id_tokens:

    GCP_JWT:

      aud: "$GCP_OIDC_AUD"
 
  before_script:

    - !reference [.tf-base, before_script]

    - !reference [.tf-gcp-adc]

    - guess_tf_system

    - !reference [.tf-commands, init]

    - !reference [.tf-commands, select_workspace]

templates/gitlab-ci-terraform.yml

+58 −68
Original line number Diff line number Diff line
@@ -170,6 +170,12 @@ spec: 
    module-files:

      description: 'Glob patterns matching files to include into the Terraform module (:warning: does not support double star)'

      default: '*.tf *.tpl *.md'

    environment-namespace:

      description: |-

        Extra [GitLab environments](https://docs.gitlab.com/ci/environments/) namespace _(only required when deploying multiple environments)_

        

        :warning: must start with a slash `/`

      default: ''

    review-enabled:

      description: Enable Review

      type: boolean
@@ -414,6 +420,7 @@ variables: 
  TF_DOCS_ENABLED: $[[ inputs.docs-enabled ]]

  TF_DOCS_EXTRA_OPTS: $[[ inputs.docs-extra-opts ]]

  TF_PUBLISH_ENABLED: $[[ inputs.publish-enabled ]]

  TF_ENVIRONMENT_NAMESPACE: $[[ inputs.environment-namespace ]]

  TF_REVIEW_ENABLED: $[[ inputs.review-enabled ]]

  TF_REVIEW_EXTRA_OPTS: $[[ inputs.review-extra-opts ]]

  TF_REVIEW_INIT_OPTS: $[[ inputs.review-init-opts ]]
@@ -805,7 +812,8 @@ stages: 
      fi



      # If TF_ADDRESS is unset then default to GitLab backend in current project

      TF_ADDRESS="${TF_ADDRESS:-${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/terraform/state/${environment_slug}}"

      prefix="${environment_namespace:+${environment_namespace}_}"

      TF_ADDRESS="${TF_ADDRESS:-${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/terraform/state/${prefix}${environment_slug}}"



      # Set variables for the HTTP backend to default to TF_* values

      # see: https://developer.hashicorp.com/terraform/language/settings/backends/http
@@ -844,6 +852,8 @@ stages: 
    log_info "--- \$environment_name: \\e[33;1m${environment_name}\\e[0m"

    # shellcheck disable=SC2154

    log_info "--- \$environment_slug: \\e[33;1m${environment_slug}\\e[0m"

    # shellcheck disable=SC2154

    log_info "--- \$environment_namespace: \\e[33;1m${environment_namespace}\\e[0m"



    # maybe enable debug log

    if [[ "$TRACE" ]]; then
@@ -854,12 +864,11 @@ stages: 
    export TF_IN_AUTOMATION=true

    export TF_INPUT=0 # same as '-input=false' option



    # define environment_type, environment_name and environment_slug as TF variables through env rather than on CLI (fails if not declared)

    # define environment_type, environment_name, environment_slug and environment_namespace as TF variables through env rather than on CLI (fails if not declared)

    export TF_VAR_environment_type=$environment_type

    export TF_VAR_environment_name=$environment_name

    export TF_VAR_environment_slug=$environment_slug





    export TF_VAR_environment_namespace=$environment_namespace



    # make output dir

    mkdir -p "$TF_OUTPUT_DIR"
@@ -994,7 +1003,7 @@ stages: 
    log_info "--- \\e[32mapply\\e[0m"



    # unset any upstream deployment env & artifacts

    rm -f terraform.env

    rm -f terraform.env*



    # maybe execute pre apply script

    prescript="$TF_SCRIPTS_DIR/tf-pre-apply.sh"
@@ -1045,8 +1054,15 @@ stages: 


    # finally propagate environment info

    # /!\ append is important here as it may have been created by a hook script

    echo -e "environment_type=$environment_type\\nenvironment_name=$CI_ENVIRONMENT_NAME\\nenvironment_slug=$CI_ENVIRONMENT_SLUG" >> terraform.env

    chmod 644 terraform.env

    # var prefix ('_' if namespace)

    prefix="${environment_namespace:+${environment_namespace}_}"

    dotenvfile="terraform.env${environment_namespace:+.${environment_namespace}}"

    {

      echo "${prefix}environment_type=$environment_type"

      echo "${prefix}environment_name=$environment_name"

      echo "${prefix}environment_slug=$environment_slug"

    } >> "$dotenvfile"

    chmod 644 "$dotenvfile"

  }



  function tf_destroy() {
@@ -1158,6 +1174,8 @@ stages: 
    - export environment_type=$ENV_TYPE

    - export environment_name=$CI_ENVIRONMENT_NAME

    - export environment_slug=$CI_ENVIRONMENT_SLUG

    - environment_namespace=$(echo "$TF_ENVIRONMENT_NAMESPACE" | tr -d '[:punct:]' | tr '[:upper:]' '[:lower:]')

    - export environment_namespace

    - !reference [.tf-scripts]

    - !reference [ .tf-commands, gitlab_auth ]

    - install_ca_certs "${CUSTOM_CA_CERTS:-$DEFAULT_CA_CERTS}"
@@ -1173,6 +1191,7 @@ stages: 
# @arg ENV_EXTRA_OPTS: environment specific tf extra options (all commands)

.tf-workspace:

  extends: .tf-base

  resource_group: tf-${ENV_TYPE}${TF_ENVIRONMENT_NAMESPACE}

  before_script:

    - !reference [ .tf-base, before_script ]

    - guess_tf_system
@@ -1183,9 +1202,13 @@ stages: 
# @arg ENV_PLAN_ENABLED        : environment specific plan to apply (if transfered from upstream jobs)

# @arg ENV_PLAN_OPTS           : environment specific tf plan options (if plan must be refreshed)

# @arg ENV_APPLY_OPTS          : environment specific tf apply options

# @arg TF_ENVIRONMENT_NAMESPACE: env-specific namespace (to support parallel:matrix)

.tf-create:

  extends: .tf-workspace

  stage: infra

  environment:

    name: ${ENV_TYPE}${TF_ENVIRONMENT_NAMESPACE}

    action: start

  script:

    - if [[ "$ENV_PLAN_ENABLED" == "true" ]]; then tf_plan="${ENV_TYPE}.tfplan"; fi

    - !reference [ .tf-commands, apply ]
@@ -1194,13 +1217,17 @@ stages: 
    paths:

      - $TF_PROJECT_DIR/$TF_OUTPUT_DIR/

    reports:

      dotenv: $TF_PROJECT_DIR/terraform.env

      dotenv: $TF_PROJECT_DIR/terraform.env*



# Abstract TF plan job

# Plan job prototype

# @arg ENV_PLAN_OPTS           : environment specific tf plan options

# @arg TF_ENVIRONMENT_NAMESPACE: env-specific namespace (to support parallel:matrix)

.tf-plan:

  extends: .tf-workspace

  stage: build

  environment:

    name: ${ENV_TYPE}${TF_ENVIRONMENT_NAMESPACE}

    action: prepare

  script:

    - !reference [ .tf-commands, plan ]

  artifacts:
@@ -1212,6 +1239,8 @@ stages: 
    reports:

      terraform: $TF_PROJECT_DIR/${ENV_TYPE}-plan.json



# Test job prototype

# @arg TF_ENVIRONMENT_NAMESPACE: env-specific namespace (to support parallel:matrix)

.tf-test:

  extends: .tf-workspace

  stage: test
@@ -1224,7 +1253,6 @@ stages: 
    reports:

      junit: $TF_PROJECT_DIR/reports/${ENV_TYPE}-tftest.xunit.xml





# Destroy job prototype

# @arg ENV_DESTROY_OPTS: environment tf destroy arguments

.tf-destroy:
@@ -1232,6 +1260,9 @@ stages: 
  stage: infra

  # force no dependencies

  dependencies: []

  environment:

    name: ${ENV_TYPE}${TF_ENVIRONMENT_NAMESPACE}

    action: stop

  script:

    - !reference [ .tf-commands, destroy ]
@@ -1502,9 +1533,8 @@ tf-plan-review: 
    ENV_WORKSPACE: $TF_REVIEW_WORKSPACE

    ENV_PLAN_OPTS: $TF_REVIEW_PLAN_OPTS

  environment:

    name: review/$CI_COMMIT_REF_NAME

    action: prepare

  resource_group: tf-review/$CI_COMMIT_REF_NAME

    name: ${ENV_TYPE}${TF_ENVIRONMENT_NAMESPACE}/${CI_COMMIT_REF_NAME}

  resource_group: ${ENV_TYPE}${TF_ENVIRONMENT_NAMESPACE}/${CI_COMMIT_REF_NAME}

  rules:

    # exclude tags

    - if: $CI_COMMIT_TAG
@@ -1525,7 +1555,7 @@ tf-test-review: 
    ENV_TFTEST_OPTS: $TF_REVIEW_TFTEST_OPTS

    ENV_INIT_OPTS: $TF_REVIEW_INIT_OPTS

  extends: .tf-test

  resource_group: tf-review/$CI_COMMIT_REF_NAME

  resource_group: ${ENV_TYPE}${TF_ENVIRONMENT_NAMESPACE}/${CI_COMMIT_REF_NAME}

  rules:

    # skip if tftest not enabled

    - if: '$TF_TFTEST_STRATEGY != "single" && $TF_TFTEST_STRATEGY != "cascading"'
@@ -1542,7 +1572,6 @@ tf-test-review: 
    # apply global test policy (adaptive pipeline)

    - !reference [.test-policy, rules]





# create review env (only on feature branches)

tf-review:

  extends: .tf-create
@@ -1555,10 +1584,9 @@ tf-review: 
    ENV_PLAN_ENABLED: $TF_REVIEW_PLAN_ENABLED

    ENV_PLAN_OPTS: $TF_REVIEW_PLAN_OPTS

  environment:

    name: review/$CI_COMMIT_REF_NAME

    action: start

    name: ${ENV_TYPE}${TF_ENVIRONMENT_NAMESPACE}/${CI_COMMIT_REF_NAME}

    auto_stop_in: "$TF_REVIEW_AUTOSTOP_DURATION"

  resource_group: tf-review/$CI_COMMIT_REF_NAME

  resource_group: ${ENV_TYPE}${TF_ENVIRONMENT_NAMESPACE}/${CI_COMMIT_REF_NAME}

  rules:

    # exclude tags

    - if: $CI_COMMIT_TAG
@@ -1585,9 +1613,8 @@ tf-destroy-review: 
    ENV_WORKSPACE: $TF_REVIEW_WORKSPACE

    ENV_DESTROY_OPTS: $TF_REVIEW_DESTROY_OPTS

  environment:

    name: review/$CI_COMMIT_REF_NAME

    action: stop

  resource_group: tf-review/$CI_COMMIT_REF_NAME

    name: ${ENV_TYPE}${TF_ENVIRONMENT_NAMESPACE}/${CI_COMMIT_REF_NAME}

  resource_group: ${ENV_TYPE}${TF_ENVIRONMENT_NAMESPACE}/${CI_COMMIT_REF_NAME}

  rules:

    # exclude tags

    - if: $CI_COMMIT_TAG
@@ -1609,10 +1636,6 @@ tf-plan-integration: 
    ENV_INIT_OPTS: $TF_INTEG_INIT_OPTS

    ENV_WORKSPACE: $TF_INTEG_WORKSPACE

    ENV_PLAN_OPTS: $TF_INTEG_PLAN_OPTS

  environment:

    name: integration

    action: prepare

  resource_group: tf-integration

  rules:

    # exclude non-integration branches

    - if: '$CI_COMMIT_REF_NAME !~ $INTEG_REF'
@@ -1625,12 +1648,11 @@ tf-plan-integration: 


# terraform test job for integration

tf-test-integration:

  extends: .tf-test

  variables:

    ENV_TYPE: integration

    ENV_TFTEST_OPTS: $TF_INTEG_TFTEST_OPTS

    ENV_INIT_OPTS: $TF_INTEG_INIT_OPTS

  extends: .tf-test

  resource_group:  tf-integration

  rules:

    # skip if tftest not enabled

    - if: '$TF_TFTEST_STRATEGY != "single" && $TF_TFTEST_STRATEGY != "cascading"'
@@ -1662,10 +1684,7 @@ tf-integration: 
    ENV_PLAN_ENABLED: $TF_INTEG_PLAN_ENABLED

    ENV_PLAN_OPTS: $TF_INTEG_PLAN_OPTS

  environment:

    name: integration

    action: start

    auto_stop_in: "$TF_INTEG_AUTOSTOP_DURATION"

  resource_group: tf-integration

  rules:

    # exclude non-integration branches

    - if: '$CI_COMMIT_REF_NAME !~ $INTEG_REF'
@@ -1688,10 +1707,6 @@ tf-destroy-integration: 
    ENV_INIT_OPTS: $TF_INTEG_INIT_OPTS

    ENV_WORKSPACE: $TF_INTEG_WORKSPACE

    ENV_DESTROY_OPTS: $TF_INTEG_DESTROY_OPTS

  environment:

    name: integration

    action: stop

  resource_group: tf-integration

  rules:

    # only on integration branch(es), with $TF_INTEG_ENABLED set

    - if: '$TF_INTEG_ENABLED == "true" && $CI_COMMIT_REF_NAME =~ $INTEG_REF'
@@ -1710,10 +1725,6 @@ tf-plan-staging: 
    ENV_INIT_OPTS: $TF_STAGING_INIT_OPTS

    ENV_WORKSPACE: $TF_STAGING_WORKSPACE

    ENV_PLAN_OPTS: $TF_STAGING_PLAN_OPTS

  environment:

    name: staging

    action: prepare

  resource_group: tf-staging

  rules:

    # exclude non-production branches

    - if: '$CI_COMMIT_REF_NAME !~ $PROD_REF'
@@ -1726,12 +1737,11 @@ tf-plan-staging: 


# terraform test job for staging

tf-test-staging:

  extends: .tf-test

  variables:

    ENV_TYPE: staging

    ENV_TFTEST_OPTS: $TF_STAGING_TFTEST_OPTS

    ENV_INIT_OPTS: $TF_STAGING_INIT_OPTS

  extends: .tf-test

  resource_group: tf-staging

  rules:

    # skip if tftest not enabled

    - if: '$TF_TFTEST_STRATEGY != "single" && $TF_TFTEST_STRATEGY != "cascading"'
@@ -1748,7 +1758,6 @@ tf-test-staging: 
    # apply global test policy (adaptive pipeline)

    - !reference [.test-policy, rules]





# create staging env (only on master branch)

tf-staging:

  extends: .tf-create
@@ -1761,10 +1770,7 @@ tf-staging: 
    ENV_PLAN_ENABLED: $TF_STAGING_PLAN_ENABLED

    ENV_PLAN_OPTS: $TF_STAGING_PLAN_OPTS

  environment:

    name: staging

    action: start

    auto_stop_in: "$TF_STAGING_AUTOSTOP_DURATION"

  resource_group: tf-staging

  rules:

    # exclude non-production branches

    - if: '$CI_COMMIT_REF_NAME !~ $PROD_REF'
@@ -1787,10 +1793,6 @@ tf-destroy-staging: 
    ENV_INIT_OPTS: $TF_STAGING_INIT_OPTS

    ENV_WORKSPACE: $TF_STAGING_WORKSPACE

    ENV_DESTROY_OPTS: $TF_STAGING_DESTROY_OPTS

  environment:

    name: staging

    action: stop

  resource_group: tf-staging

  rules:

    # only on production branch(es), with $TF_STAGING_ENABLED set

    - if: '$TF_STAGING_ENABLED == "true" && $CI_COMMIT_REF_NAME =~ $PROD_REF'
@@ -1809,10 +1811,6 @@ tf-plan-production: 
    ENV_INIT_OPTS: $TF_PROD_INIT_OPTS

    ENV_WORKSPACE: $TF_PROD_WORKSPACE

    ENV_PLAN_OPTS: $TF_PROD_PLAN_OPTS

  environment:

    name: production

    action: prepare

  resource_group: tf-production

  rules:

    # exclude tags

    - if: $CI_COMMIT_TAG
@@ -1833,15 +1831,13 @@ tf-plan-production: 
    # enabled on production branches

    - if: '$CI_COMMIT_REF_NAME =~ $PROD_REF'





# terraform test job for production

tf-test-production:

  extends: .tf-test

  variables:

    ENV_TYPE: production

    ENV_TFTEST_OPTS: $TF_PROD_TFTEST_OPTS

    ENV_INIT_OPTS: $TF_PROD_INIT_OPTS

  extends: .tf-test

  resource_group: tf-production

  rules:

    # skip if tftest not enabled

    - if: '$TF_TFTEST_STRATEGY != "single" && $TF_TFTEST_STRATEGY != "cascading"'
@@ -1858,8 +1854,6 @@ tf-test-production: 
    # apply global test policy (adaptive pipeline)

    - !reference [.test-policy, rules]







# create production env if on branch master and variable TF_PROD_ENABLED defined

tf-production:

  extends: .tf-create
@@ -1872,10 +1866,6 @@ tf-production: 
    ENV_APPLY_OPTS: $TF_PROD_APPLY_OPTS

    ENV_PLAN_ENABLED: $TF_PROD_PLAN_ENABLED

    ENV_PLAN_OPTS: $TF_PROD_PLAN_OPTS

  environment:

    name: production

    action: start

  resource_group: tf-production

  rules:

    # exclude non-production branches

    - if: '$CI_COMMIT_REF_NAME !~ $PROD_REF'