Commit 3d215014 authored by Pierre Smeyers's avatar Pierre Smeyers
Browse files

Merge branch '74-gcp-variant-using-adc' into 'master' 

refactor: setup GCP credentials through ADC

Closes #74

See merge request to-be-continuous/terraform!110
parents eeec0eaf c7b6d211

README.md

+20 −3
Original line number Diff line number Diff line
@@ -687,9 +687,27 @@ variables: 


### Google Cloud variant



This variant allows retrieving an [OAuth access token](https://developers.google.com/identity/protocols/oauth2) for the [Google Cloud Platform Provider for Terraform](https://registry.terraform.io/providers/hashicorp/google/latest/docs) (using the [GCP Auth Provider](https://gitlab.com/to-be-continuous/tools/gcp-auth-provider) as a _service container_).

This variant uses [Application Default Credentials][gcp-adc] through the `GOOGLE_APPLICATION_CREDENTIALS` variable

as explained in the Google Terraform provider configuration [Running Terraform Outside of Google Cloud][gcp-provider]

using Workload Identity federation.



Provided you successfully configured the [federated authentication using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/), this variant automatically obtains a temporary OAuth token and stores it in the `$GOOGLE_OAUTH_ACCESS_TOKEN` variable (supported by the [Google Cloud Platform Provider for Terraform](https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/provider_reference#full-reference) as an authentication credential).

List of requirements before using this variant:



1. You must have a Workload Identity Federation Pool and Provider configured,

2. You must have a Service Account with the `roles/iam.workloadIdentityUser` IAM role

   granted to the Workload Identity [principal][gcp-iam-principals] matching your Gitlab project or group,

3. Optionally, you can set the `GOOGLE_CLOUD_PROJECT` template variable

   to define the default Google Cloud project.



The Gitlab documentation has some [details about Workload Identity Federation integration][gcp-gitlab-wif].



This [blog post about OIDC impersonation through Workload Identify Federation][gcp-wif-example] might also be of help.



[gcp-adc]: https://cloud.google.com/docs/authentication/client-libraries

[gcp-provider]: https://registry.terraform.io/providers/hashicorp/google/latest/docs/guides/provider_reference#running-terraform-outside-of-google-cloud

[gcp-iam-principals]: https://cloud.google.com/iam/docs/principal-identifiers

[gcp-gitlab-wif]: https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/

[gcp-wif-example]: https://blog.salrashid.dev/articles/2021/understanding_workload_identity_federation/#oidc-impersonated



#### Configuration
@@ -697,7 +715,6 @@ The variant requires the additional configuration parameters: 


| Input / Variable  | Description                            | Default value     |

| ----------------- | -------------------------------------- | ----------------- |

| `TBC_GCP_PROVIDER_IMAGE` | The [GCP Auth Provider](https://gitlab.com/to-be-continuous/tools/gcp-auth-provider) image to use (can be overridden) | `registry.gitlab.com/to-be-continuous/tools/gcp-auth-provider:latest` |

| `gcp-oidc-aud` / `GCP_OIDC_AUD` | The `aud` claim for the JWT token      | `$CI_SERVER_URL` |

| `gcp-oidc-provider` / `GCP_OIDC_PROVIDER` | Default Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) | _none_ |

| `gcp-oidc-account` / `GCP_OIDC_ACCOUNT` | Default Service Account to which impersonate with OpenID Connect authentication | _none_ |

kicker.json

+0 −6
Original line number Diff line number Diff line
@@ -477,12 +477,6 @@ 
      "description": "Retrieves an [OAuth access token](https://developers.google.com/identity/protocols/oauth2) for the [Google Cloud Platform Provider for Terraform](https://registry.terraform.io/providers/hashicorp/google/latest/docs)",

      "template_path": "templates/gitlab-ci-terraform-gcp.yml",

      "variables": [

        {

          "name": "TBC_GCP_PROVIDER_IMAGE",

          "description": "The [GCP Auth Provider](https://gitlab.com/to-be-continuous/tools/gcp-auth-provider) image to use",

          "default": "registry.gitlab.com/to-be-continuous/tools/gcp-auth-provider:latest",

          "advanced": true

        },

        {

          "name": "GCP_OIDC_AUD",

          "description": "The `aud` claim for the JWT token _(only required for [OIDC authentication](https://docs.gitlab.com/ee/ci/cloud_services/aws/))_",

templates/gitlab-ci-terraform-gcp.yml

+46 −11
Original line number Diff line number Diff line
@@ -39,7 +39,6 @@ spec: 
---

variables:

  # variabilized gcp-auth-provider image

  TBC_GCP_PROVIDER_IMAGE: registry.gitlab.com/to-be-continuous/tools/gcp-auth-provider:latest

  GCP_OIDC_AUD: $[[ inputs.gcp-oidc-aud ]]

  GCP_OIDC_ACCOUNT: $[[ inputs.gcp-oidc-account ]]

  GCP_OIDC_PROVIDER: $[[ inputs.gcp-oidc-provider ]]
@@ -52,16 +51,52 @@ variables: 
  GCP_PROD_OIDC_ACCOUNT: $[[ inputs.gcp-prod-oidc-account ]]

  GCP_PROD_OIDC_PROVIDER: $[[ inputs.gcp-prod-oidc-provider ]]



.tf-base:

  services:

    - name: "$TBC_TRACKING_IMAGE"

      command: ["--service", "terraform", "5.3.0" ]

    - name: "$TBC_GCP_PROVIDER_IMAGE"

      alias: "gcp-auth-provider"

  variables:

    #  have to be explicitly declared in the YAML to be exported to the service

    GCP_JWT: $GCP_JWT

    GOOGLE_OAUTH_ACCESS_TOKEN: "@url@http://gcp-auth-provider/token"

.gcp-provider-auth:

  before_script:

    - echo "Installing GCP authentication with env GOOGLE_APPLICATION_CREDENTIALS file"

    - echo $GCP_JWT > "$CI_BUILDS_DIR/.auth_token.jwt"

    - |-

      if [[ "$ENV_TYPE" ]]

      then

        case "$ENV_TYPE" in

        review*)

          env_prefix=REVIEW;;

        integ*)

          env_prefix=INTEG;;

        staging*)

          env_prefix=STAGING;;

        prod*)

          env_prefix=PROD;;

        *)

          ;;

        esac

        env_oidc_provider=$(eval echo "\$GCP_${env_prefix}_OIDC_PROVIDER")

        env_oidc_account=$(eval echo "\$GCP_${env_prefix}_OIDC_ACCOUNT")

      fi

      oidc_provider="${env_oidc_provider:-$GCP_OIDC_PROVIDER}"

      oidc_account="${env_oidc_account:-$GCP_OIDC_ACCOUNT}"

    - |-

      cat << EOF > "$CI_BUILDS_DIR/google_application_credentials.json"

      {

        "type": "external_account",

        "audience": "//iam.googleapis.com/${oidc_provider}",

        "subject_token_type": "urn:ietf:params:oauth:token-type:jwt",

        "token_url": "https://sts.googleapis.com/v1/token",

        "credential_source": {

          "file": "$CI_BUILDS_DIR/.auth_token.jwt"

        },

        "service_account_impersonation_url": "https://iamcredentials.googleapis.com/v1/projects/-/serviceAccounts/${oidc_account}:generateAccessToken"

      }

      EOF

    - export GOOGLE_APPLICATION_CREDENTIALS="$CI_BUILDS_DIR/google_application_credentials.json"



.tf-workspace:

  extends: .tf-base

  id_tokens:

    GCP_JWT:

      aud: "$GCP_OIDC_AUD"

  before_script:

    - !reference [.tf-base, before_script]

    - !reference [.gcp-provider-auth, before_script]

    - !reference [.tf-commands, init]

    - !reference [.tf-commands, select_workspace]