Commit b5342c44 authored by Pierre's avatar Pierre Committed by Pierre Smeyers
Browse files

feat: authenticate with OpenID Connect

parent c9bb5fb9

README.md

+37 −1
Original line number Diff line number Diff line
@@ -66,6 +66,8 @@ The Google Cloud template uses some global configuration used throughout all job 
| ------------------------ | -------------------------------------- | ----------------- |

| `GCP_CLI_IMAGE`          | the Docker image used to run Google Cloud CLI commands| `google/cloud-sdk:latest` |

| :lock: `GCP_KEY_FILE`    | Default [Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file) | _none_ |

| `GCP_OIDC_PROVIDER`      | Default Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) | none|

| `GCP_OIDC_ACCOUNT`       | Default Service Account to which impersonate with OpenID Connect authentication | none  |

| `GCP_BASE_APP_NAME`      | Base application name                  | `$CI_PROJECT_NAME` ([see GitLab doc](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html)) |

| `GCP_SCRIPTS_DIR`        | Directory where Google Cloud scripts (deploy & cleanup) are located | `.` _(root project dir)_ |
@@ -83,6 +85,27 @@ Here are some advices about your **secrets** (variables marked with a :lock:): 
  it will then be possible to mask it and the template will automatically decode it prior to using it.

3. Don't forget to escape special characters (ex: `$` -> `$$`).



### Federated authentication using OpenID Connect



The GCP template supports a [federated authentication using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/).



If you wish to use this authentication mode, please follow carefully [the GitLab guide](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/), then configure appropriately the related variables:



* `GPC_OIDC_PROVIDER` / `GPC_OIDC_ACCOUNT` for any global/common access,

* `GPC_<env>_OIDC_PROVIDER` / `GPC_<env>_OIDC_ACCOUNT` if you wish to use separate settings with any of your environments.



The `GPC_OIDC_PROVIDER` & `GPC_<env>_OIDC_PROVIDER` variable shall be of the form:



```

projects/<PROJECT_NUMBER>/locations/global/workloadIdentityPools/<POOL_ID>/providers/<PROVIDER_ID>

```



The following commands may help you retrieve the different values: 



- `gcloud projects describe $GCP_PROJECT --format="value(projectNumber)"` will return the `PROJECT_NUMBER` value

- `gcloud iam workload-identity-pools list  --location=global --format="value(name)"` will list you POOL_IDs available on your `GCP_PROJECT`

- `gcloud iam workload-identity-pools providers list --workload-identity-pool=<my-pool>  --location=global --format="value(name)"` will return the list of available `PROVIDER_ID` for one `POOL_ID`



### Deployment and cleanup jobs



The GitLab CI template for Google Cloud requires you to provide a shell script that fully implements your application
@@ -171,8 +194,14 @@ Here are variables supported to configure review environments: 
| `GCP_REVIEW_APP_NAME`    | Application name for `review` env      | `"${GCP_BASE_APP_NAME}-${CI_ENVIRONMENT_SLUG}"` (ex: `myproject-review-fix-bug-12`) |

| `GCP_REVIEW_ENVIRONMENT_SCHEME`| The review environment protocol scheme.<br/>_For static environment URLs declaration_ | `https` |

| `GCP_REVIEW_ENVIRONMENT_DOMAIN`| The review environment domain.<br/>_For static environment URLs declaration_ | _none_ |

| `GCP_REVIEW_OIDC_PROVIDER` | Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `review` environment | none|

| `GCP_REVIEW_OIDC_ACCOUNT`  | Service Account to which impersonate with OpenID Connect authentication on `review` environment | none  |









Note: If you're managing your environment URLs statically, review environment URLs will be built as `${AWS_REVIEW_ENVIRONMENT_SCHEME}://${$CI_PROJECT_NAME}-${CI_ENVIRONMENT_SLUG}.${AWS_REVIEW_ENVIRONMENT_DOMAIN}`



#### Integration environment



The integration environment is the environment associated to your integration branch (`develop` by default).
@@ -187,6 +216,9 @@ Here are variables supported to configure the integration environment: 
| :lock: `GCP_INTEG_KEY_FILE`|[Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file) to authenticate on `integration` env  _(only define if different from default)_    | `$GCP_KEY_FILE` |

| `GCP_INTEG_APP_NAME`     | Application name for `integration` env | `${GCP_BASE_APP_NAME}-integration` |

| `GCP_INTEG_ENVIRONMENT_URL`| The integration environment url (ex: `https://my-application-integration.nonpublic.domain.com`).<br/>_For static environment URLs declaration_ | _none_ |

| `GCP_INTEG_OIDC_PROVIDER` | Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `integration` environment | none|

| `GCP_INTEG_OIDC_ACCOUNT`  | Service Account to which impersonate with OpenID Connect authentication on `integration` environment | none  |





#### Staging environment
@@ -203,6 +235,9 @@ Here are variables supported to configure the staging environment: 
| :lock: `GCP_STAGING_KEY_FILE`|[Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file) to authenticate on `staging` env  _(only define if different from default)_    | `$GCP_KEY_FILE` |

| `GCP_STAGING_APP_NAME`   | Application name for `staging` env     | `${GCP_BASE_APP_NAME}-staging` |

| `GCP_STAGING_ENVIRONMENT_URL` | The staging environment url (ex: `https://my-application-staging.nonpublic.domain.com`).<br/>_For static environment URLs declaration_ | _none_ |

| `GCP_STAGING_OIDC_PROVIDER` | Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `staging` environment | none|

| `GCP_STAGING_OIDC_ACCOUNT`  | Service Account to which impersonate with OpenID Connect authentication on `staging` environment | none  |





#### Production environment
@@ -219,7 +254,8 @@ Here are variables supported to configure the production environment: 
| `GCP_PROD_APP_NAME`       | Application name for `production` env  | `$GCP_BASE_APP_NAME` |

| `GCP_PROD_ENVIRONMENT_URL`| The production environment url (ex: `https://my-application.public.domain.com`).<br/>_For static environment URLs declaration_ | _none_ |

| `AUTODEPLOY_TO_PROD`      | Set this variable to auto-deploy to production. If not set deployment to production will be `manual` (default behaviour). | _none_ (disabled) |



| `GCP_PROD_OIDC_PROVIDER` | Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `production ` environment | none|

| `GCP_PROD_OIDC_ACCOUNT`  | Service Account to which impersonate with OpenID Connect authentication on `production ` environment | none  |

## Examples



### Google AppEngine application

kicker.json

+61 −0
Original line number Diff line number Diff line
@@ -15,6 +15,26 @@ 
      "secret": true,

      "mandatory": true

    },

    {

      "name": "GCP_WORKLOAD_IDENTITY_PROVIDER",

      "description": "Default [Workload Identity Provider](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) associated with GitLab to authenticate\n\n(has format `projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/POOL_ID/providers/PROVIDER_ID`)",

      "advanced": true

    },

    {

      "name": "GCP_SERVICE_ACCOUNT",

      "description": "Default Service Account to which impersonate with WIF authentication",

      "advanced": true

    },

    {

      "name": "GCP_OIDC_PROVIDER",

      "description": "Global Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/)",

      "advanced": true

    },

    {

      "name": "GCP_OIDC_ACCOUNT",

      "description": "Global Service Account to which impersonate with OpenID Connect authentication",

      "advanced": true

    },

    {

      "name": "GCP_BASE_APP_NAME",

      "description": "Base application name",
@@ -27,6 +47,7 @@ 
      "default": ".",

      "advanced": true

    }



  ],

  "features": [

    {
@@ -57,6 +78,16 @@ 
          "name": "GCP_REVIEW_KEY_FILE",

          "description": "Service Account key file to authenticate on review env (only define if different from global)",

          "secret": true

        },

        {

          "name": "GCP_REVIEW_OIDC_PROVIDER",

          "description": "Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `review` environment\n\n_(only define if different from global)_",

          "advanced": true

        },

        {

          "name": "GCP_REVIEW_OIDC_ACCOUNT",

          "description": "Service Account to which impersonate with OpenID Connect authentication on `review` environment",

          "advanced": true

        }

      ]

    },
@@ -84,6 +115,16 @@ 
          "name": "GCP_INTEG_KEY_FILE",

          "description": "Service Account key file to authenticate on integration env (only define if different from global)",

          "secret": true

        },   

        {

          "name": "GCP_INTEG_OIDC_PROVIDER",

          "description": "Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `integration` environment\n\n_(only define if different from global)_",

          "advanced": true

        },

        {

          "name": "GCP_INTEG_OIDC_ACCOUNT",

          "description": "Service Account to which impersonate with OpenID Connect authentication on `integration` environment",

          "advanced": true

        }

      ]

    },
@@ -111,6 +152,16 @@ 
          "name": "GCP_STAGING_KEY_FILE",

          "description": "Service Account key file to authenticate on staging env (only define if different from global)",

          "secret": true

        },        

        {

          "name": "GCP_STAGING_OIDC_PROVIDER",

          "description": "Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `staging` environment\n\n_(only define if different from global)_",

          "advanced": true

        },

        {

          "name": "GCP_STAGING_OIDC_ACCOUNT",

          "description": "Service Account to which impersonate with OpenID Connect authentication on `staging` environment",

          "advanced": true

        }

      ]

    },
@@ -143,6 +194,16 @@ 
          "name": "GCP_PROD_KEY_FILE",

          "description": "Service Account key file to authenticate on production env (only define if different from global)",

          "secret": true

        },

        {

          "name": "GCP_PROD_OIDC_PROVIDER",

          "description": "Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `production` environment\n\n_(only define if different from global)_",

          "advanced": true

        },

        {

          "name": "GCP_PROD_OIDC_ACCOUNT",

          "description": "Service Account to which impersonate with OpenID Connect authentication on `production` environment",

          "advanced": true

        }

      ]

    }

templates/gitlab-ci-gcloud.yml

+51 −9
Original line number Diff line number Diff line
@@ -33,6 +33,7 @@ variables: 
  GCP_BASE_APP_NAME: "$CI_PROJECT_NAME"

  GCP_REVIEW_ENVIRONMENT_SCHEME: "https"





  # default production ref name (pattern)

  PROD_REF: '/^(master|main)$/'

  # default integration ref name (pattern)
@@ -256,6 +257,38 @@ stages: 
    awk '{while(match($0,"[$]{[^}]*}")) {var=substr($0,RSTART+2,RLENGTH -3);val=ENVIRON[var];gsub(/["\\]/,"\\\\&", val);gsub("\n", "\\n", val);gsub("\r", "\\r", val);gsub("[$]{"var"}",val)}}1'

  }



  

  # Google Cloud Authentication

  function gcp_auth() {

    gcp_key_file="$1"

    oidc_provider="$2"

    oidc_account="$3"



    log_info "--- \\e[32moidc_provider\\e[0m (env: \\e[33;1m${oidc_provider}\\e[0m)"

    log_info "--- \$oidc_account: \\e[33;1m${oidc_account}\\e[0m"

  

    if [[ "$oidc_provider" ]]

    then

      # Use Workload Identity Federation to authenticate

      # see: https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/

      log_info "Authenticating with OpenID Connect..."

      assert_defined "$oidc_account" 'Missing required OpenID Connect service account'

      echo "${CI_JOB_JWT_V2}" > /tmp/.ci_job_jwt_file 

      gcloud iam workload-identity-pools create-cred-config "$oidc_provider" \

        --service-account="$oidc_account" \

        --output-file=/tmp/.gcp_temp_cred.json \

        --credential-source-file=/tmp/.ci_job_jwt_file

      gcloud auth login --cred-file=/tmp/.gcp_temp_cred.json

    else

      # Use gcp_key_file to authenticate

      log_info "Authenticating with Service Account key file..."

      assert_defined "$gcp_key_file" 'Missing required GCP key file (JSON)'

      as_content "$gcp_key_file" > /tmp/gcp_key.json

      gcloud auth activate-service-account --key-file /tmp/gcp_key.json    

    fi

  }





  # application deployment function

  function deploy() {

    export env=$1
@@ -324,6 +357,7 @@ stages: 
    fi

  }





  # export tool functions (might be used in after_script)

  export -f log_info log_warn log_error assert_defined awkenvsubst
@@ -360,9 +394,8 @@ stages: 
  before_script:

    - *gcp-scripts

    - install_ca_certs "${CUSTOM_CA_CERTS:-$DEFAULT_CA_CERTS}"

    - assert_defined "${ENV_KEY_FILE:-$GCP_KEY_FILE}" 'Missing required GCP key file (JSON)'

    - as_content "${ENV_KEY_FILE:-$GCP_KEY_FILE}" > /tmp/gcp.key

    - gcloud auth activate-service-account --key-file /tmp/gcp.key

    - gcp_auth "${ENV_KEY_FILE:-$GCP_KEY_FILE}" "${ENV_OIDC_PROVIDER:-$GCP_OIDC_PROVIDER}}" "${ENV_OIDC_ACCOUNT:-$GCP_OIDC_ACCOUNT}"



  script:

    - deploy "$ENV_TYPE" "${ENV_APP_NAME:-${GCP_BASE_APP_NAME}${ENV_APP_SUFFIX}}" "$ENV_PROJECT" "$ENV_URL"

  artifacts:
@@ -392,9 +425,7 @@ stages: 
  before_script:

    - *gcp-scripts

    - install_ca_certs "${CUSTOM_CA_CERTS:-$DEFAULT_CA_CERTS}"

    - assert_defined "${ENV_KEY_FILE:-$GCP_KEY_FILE}" 'Missing required GCP key file (JSON)'

    - as_content "${ENV_KEY_FILE:-$GCP_KEY_FILE}" > /tmp/gcp.key

    - gcloud auth activate-service-account --key-file /tmp/gcp.key

    - gcp_auth "${ENV_KEY_FILE:-$GCP_KEY_FILE}" "${ENV_OIDC_PROVIDER:-$GCP_OIDC_PROVIDER}}" "${ENV_OIDC_ACCOUNT:-$GCP_OIDC_ACCOUNT}"

  script:

    - delete "$ENV_TYPE" "${ENV_APP_NAME:-${GCP_BASE_APP_NAME}${ENV_APP_SUFFIX}}" "$ENV_PROJECT"

  environment:
@@ -409,6 +440,8 @@ gcp-review: 
    ENV_TYPE: review

    ENV_APP_NAME: "$GCP_REVIEW_APP_NAME"

    ENV_PROJECT: "$GCP_REVIEW_PROJECT"

    ENV_OIDC_PROVIDER: "$GCP_REVIEW_OIDC_PROVIDER"

    ENV_OIDC_ACCOUNT: "$GCP_REVIEW_OIDC_ACCOUNT"

    ENV_KEY_FILE: "$GCP_REVIEW_KEY_FILE"

    ENV_URL: "${GCP_REVIEW_ENVIRONMENT_SCHEME}://${CI_PROJECT_NAME}-${CI_ENVIRONMENT_SLUG}.${GCP_REVIEW_ENVIRONMENT_DOMAIN}"

  environment:
@@ -432,7 +465,10 @@ gcp-cleanup-review: 
    ENV_TYPE: review

    ENV_APP_NAME: "$GCP_REVIEW_APP_NAME"

    ENV_PROJECT: "$GCP_REVIEW_PROJECT"

    ENV_OIDC_PROVIDER: "$GCP_REVIEW_OIDC_PROVIDER"

    ENV_OIDC_ACCOUNT: "$GCP_REVIEW_OIDC_ACCOUNT"

    ENV_KEY_FILE: "$GCP_REVIEW_KEY_FILE"

    ENV_URL: "${GCP_REVIEW_ENVIRONMENT_URL}"

  environment:

    name: review/$CI_COMMIT_REF_NAME

    action: stop
@@ -453,6 +489,8 @@ gcp-integration: 
    ENV_TYPE: integration

    ENV_APP_NAME: "$GCP_INTEG_APP_NAME"

    ENV_PROJECT: "$GCP_INTEG_PROJECT"

    ENV_OIDC_PROVIDER: "$GCP_INTEG_OIDC_PROVIDER"

    ENV_OIDC_ACCOUNT: "$GCP_INTEG_OIDC_ACCOUNT"

    ENV_KEY_FILE: "$GCP_INTEG_KEY_FILE"

    ENV_URL: "${GCP_INTEG_ENVIRONMENT_URL}"

  environment:
@@ -469,6 +507,8 @@ gcp-staging: 
    ENV_TYPE: staging

    ENV_APP_NAME: "$GCP_STAGING_APP_NAME"

    ENV_PROJECT: "$GCP_STAGING_PROJECT"

    ENV_OIDC_PROVIDER: "$GCP_STAGING_OIDC_PROVIDER"

    ENV_OIDC_ACCOUNT: "$GCP_STAGING_OIDC_ACCOUNT"

    ENV_KEY_FILE: "$GCP_STAGING_KEY_FILE"

    ENV_URL: "${GCP_STAGING_ENVIRONMENT_URL}"

  environment:
@@ -487,6 +527,8 @@ gcp-production: 
    ENV_APP_SUFFIX: "" # no suffix for prod

    ENV_APP_NAME: "$GCP_PROD_APP_NAME"

    ENV_PROJECT: "$GCP_PROD_PROJECT"

    ENV_OIDC_PROVIDER: "$GCP_PROD_OIDC_PROVIDER"

    ENV_OIDC_ACCOUNT: "$GCP_PROD_OIDC_ACCOUNT"

    ENV_KEY_FILE: "$GCP_PROD_KEY_FILE"

    ENV_URL: "${GCP_PROD_ENVIRONMENT_URL}"

  environment: