Commit 23681d53 authored by Pierre Smeyers's avatar Pierre Smeyers
Browse files

Merge branch 'feat/deprecated-jwt-variables' into 'master' 

OIDC authentication support now requires  explicit configuration(see doc)

Closes #15

See merge request to-be-continuous/gcloud!47
parents c881d73f 2e8032a3

README.md

+67 −26
Original line number Diff line number Diff line
@@ -77,22 +77,7 @@ Can be provided globally and/or per environment. 


The GCP template supports a [federated authentication using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/).



If you wish to use this authentication mode, please follow carefully [the GitLab guide](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/), then configure appropriately the related variables:



* `GPC_OIDC_PROVIDER` / `GPC_OIDC_ACCOUNT` for any global/common access,

* `GPC_<env>_OIDC_PROVIDER` / `GPC_<env>_OIDC_ACCOUNT` if you wish to use separate settings with any of your environments.



The `GPC_OIDC_PROVIDER` & `GPC_<env>_OIDC_PROVIDER` variable shall be of the form:



```

projects/<PROJECT_NUMBER>/locations/global/workloadIdentityPools/<POOL_ID>/providers/<PROVIDER_ID>

```



The following commands may help you retrieve the different values: 



- `gcloud projects describe $GCP_PROJECT --format="value(projectNumber)"` will return the `PROJECT_NUMBER` value

- `gcloud iam workload-identity-pools list  --location=global --format="value(name)"` will list you POOL_IDs available on your `GCP_PROJECT`

- `gcloud iam workload-identity-pools providers list --workload-identity-pool=<my-pool>  --location=global --format="value(name)"` will return the list of available `PROVIDER_ID` for one `POOL_ID`

If you wish to use this authentication mode, please activate and configure the [OIDC variant](#oidc-variant). 



The template supports two ways to retrieve the JSON web token (JWT):
@@ -237,8 +222,6 @@ The Google Cloud template uses some global configuration used throughout all job 
| ------------------------ | -------------------------------------- | ----------------- |

| `GCP_CLI_IMAGE`          | the Docker image used to run Google Cloud CLI commands| `gcr.io/google.com/cloudsdktool/cloud-sdk:latest` |

| :lock: `GCP_KEY_FILE`    | Default [Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file) | _none_ |

| `GCP_OIDC_PROVIDER`      | Default Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) | none|

| `GCP_OIDC_ACCOUNT`       | Default Service Account to which impersonate with OpenID Connect authentication | none  |

| `GCP_BASE_APP_NAME`      | Base application name                  | `$CI_PROJECT_NAME` ([see GitLab doc](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html)) |

| `GCP_ENVIRONMENT_URL`    | Default environments url _(only define for static environment URLs declaration)_<br/>_supports late variable expansion (ex: `https://%{environment_name}.gcloud.acme.com`)_ | _none_ |

| `GCP_SCRIPTS_DIR`        | Directory where Google Cloud scripts (deploy & cleanup) are located | `.` _(root project dir)_ |
@@ -258,8 +241,6 @@ Here are variables supported to configure review environments: 
| :lock: `GCP_REVIEW_KEY_FILE`| [Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file) to authenticate on `review` env  _(only define if different from default)_    | `$GCP_KEY_FILE` |

| `GCP_REVIEW_APP_NAME`    | Application name for `review` env      | `"${GCP_BASE_APP_NAME}-${CI_ENVIRONMENT_SLUG}"` (ex: `myproject-review-fix-bug-12`) |

| `GCP_REVIEW_ENVIRONMENT_URL`| The review environments url _(only define for static environment URLs declaration and if different from default)_ | `$GCP_ENVIRONMENT_URL` |

| `GCP_REVIEW_OIDC_PROVIDER` | Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `review` environment | none|

| `GCP_REVIEW_OIDC_ACCOUNT`  | Service Account to which impersonate with OpenID Connect authentication on `review` environment | none  |



### Integration environment configuration
@@ -275,8 +256,6 @@ Here are variables supported to configure the integration environment: 
| :lock: `GCP_INTEG_KEY_FILE`|[Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file) to authenticate on `integration` env  _(only define if different from default)_    | `$GCP_KEY_FILE` |

| `GCP_INTEG_APP_NAME`     | Application name for `integration` env | `${GCP_BASE_APP_NAME}-integration` |

| `GCP_INTEG_ENVIRONMENT_URL`| The integration environment url _(only define for static environment URLs declaration and if different from default)_ | `$GCP_ENVIRONMENT_URL` |

| `GCP_INTEG_OIDC_PROVIDER` | Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `integration` environment | none|

| `GCP_INTEG_OIDC_ACCOUNT`  | Service Account to which impersonate with OpenID Connect authentication on `integration` environment | none  |



### Staging environment configuration
@@ -293,8 +272,6 @@ Here are variables supported to configure the staging environment: 
| :lock: `GCP_STAGING_KEY_FILE`|[Service Account key file](https://cloud.google.com/bigquery/docs/authentication/service-account-file) to authenticate on `staging` env  _(only define if different from default)_    | `$GCP_KEY_FILE` |

| `GCP_STAGING_APP_NAME`   | Application name for `staging` env     | `${GCP_BASE_APP_NAME}-staging` |

| `GCP_STAGING_ENVIRONMENT_URL`| The staging environment url _(only define for static environment URLs declaration and if different from default)_ | `$GCP_ENVIRONMENT_URL` |

| `GCP_STAGING_OIDC_PROVIDER` | Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `staging` environment | none|

| `GCP_STAGING_OIDC_ACCOUNT`  | Service Account to which impersonate with OpenID Connect authentication on `staging` environment | none  |



### Production environment configuration
@@ -311,8 +288,6 @@ Here are variables supported to configure the production environment: 
| `GCP_PROD_APP_NAME`       | Application name for `production` env  | `$GCP_BASE_APP_NAME` |

| `GCP_PROD_ENVIRONMENT_URL`| The production environment url _(only define for static environment URLs declaration and if different from default)_ | `$GCP_ENVIRONMENT_URL` |

| `GCP_PROD_DEPLOY_STRATEGY`| Defines the deployment to production strategy. One of `manual` (i.e. _one-click_) or `auto`. | `manual` |

| `GCP_PROD_OIDC_PROVIDER` | Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `production ` environment | none|

| `GCP_PROD_OIDC_ACCOUNT`  | Service Account to which impersonate with OpenID Connect authentication on `production ` environment | none  |



## Examples
@@ -434,6 +409,69 @@ gcloud --quiet app versions delete --project=${gcp_project_id} --service=coocked 


The Google Cloud template can be used in conjunction with template variants to cover specific cases.



### OIDC variant



This variant enables [federated authentication using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/).



If you wish to use this authentication mode, please follow carefully [the GitLab guide](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/), then configure appropriately the related variables:



* `GPC_OIDC_PROVIDER` / `GPC_OIDC_ACCOUNT` for any global/common access,

* `GPC_<env>_OIDC_PROVIDER` / `GPC_<env>_OIDC_ACCOUNT` if you wish to use separate settings with any of your environments.



The `GPC_OIDC_PROVIDER` & `GPC_<env>_OIDC_PROVIDER` variable shall be of the form:



```

projects/<PROJECT_NUMBER>/locations/global/workloadIdentityPools/<POOL_ID>/providers/<PROVIDER_ID>

```



The following commands may help you retrieve the different values: 



- `gcloud projects describe $GCP_PROJECT --format="value(projectNumber)"` will return the `PROJECT_NUMBER` value

- `gcloud iam workload-identity-pools list  --location=global --format="value(name)"` will list you POOL_IDs available on your `GCP_PROJECT`

- `gcloud iam workload-identity-pools providers list --workload-identity-pool=<my-pool>  --location=global --format="value(name)"` will return the list of available `PROVIDER_ID` for one `POOL_ID`



#### Configuration



The variant supports the following configuration:



| Name              | description                            | default value     |

| ----------------- | -------------------------------------- | ----------------- |

| `GCP_OIDC_AUD`    | The `aud` claim for the JWT | `$CI_SERVER_URL` |

| `GCP_OIDC_PROVIDER`      | Default Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) | _none_ |

| `GCP_OIDC_ACCOUNT`       | Default Service Account to which impersonate with OpenID Connect authentication | _none_ |

| `GCP_REVIEW_OIDC_PROVIDER` | Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `review` environment | _none_ |

| `GCP_REVIEW_OIDC_ACCOUNT`  | Service Account to which impersonate with OpenID Connect authentication on `review` environment | _none_ |

| `GCP_INTEG_OIDC_PROVIDER` | Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `integration` environment | _none_ |

| `GCP_INTEG_OIDC_ACCOUNT`  | Service Account to which impersonate with OpenID Connect authentication on `integration` environment | _none_ |

| `GCP_STAGING_OIDC_PROVIDER` | Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `staging` environment | _none_ |

| `GCP_STAGING_OIDC_ACCOUNT`  | Service Account to which impersonate with OpenID Connect authentication on `staging` environment | _none_ |

| `GCP_PROD_OIDC_PROVIDER` | Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `production ` environment | _none_ |

| `GCP_PROD_OIDC_ACCOUNT`  | Service Account to which impersonate with OpenID Connect authentication on `production ` environment | _none_ |



#### Example



```yaml

include:

  # main template

  - project: 'to-be-continuous/gcloud'

    ref: '3.1.0'

    file: '/templates/gitlab-ci-gcloud.yml'

  # Vault variant

  - project: 'to-be-continuous/gcloud'

    ref: '3.1.0'

    file: '/templates/gitlab-ci-gcloud-oidc.yml'



variables:

  # audience claim for JWT

  GCP_OIDC_AUD: "https://iam.googleapis.com"

  # common OIDC config for non-prod envs

  GCP_OIDC_PROVIDER: "projects/<gcp_nonprod_proj_id>/locations/global/workloadIdentityPools/<pool_id>/providers/<provider_id>"

  GCP_OIDC_ACCOUNT: "<name>@$<gcp_nonprod_proj_id>.iam.gserviceaccount.com"

  # specific OIDC config for prod

  GCP_PROD_OIDC_PROVIDER: "projects/<gcp_prod_proj_id>/locations/global/workloadIdentityPools/<pool_id>/providers/<provider_id>"

  GCP_PROD_OIDC_ACCOUNT: "<name>@$<gcp_prod_proj_id>.iam.gserviceaccount.com"

```



### Vault variant



This variant allows delegating your secrets management to a [Vault](https://www.vaultproject.io/) server.
@@ -446,6 +484,7 @@ In order to be able to communicate with the Vault server, the variant requires t 
| ----------------- | -------------------------------------- | ----------------- |

| `TBC_VAULT_IMAGE` | The [Vault Secrets Provider](https://gitlab.com/to-be-continuous/tools/vault-secrets-provider) image to use (can be overridden) | `$CI_REGISTRY/to-be-continuous/tools/vault-secrets-provider:master` |

| `VAULT_BASE_URL`  | The Vault server base API url          | _none_ |

| `VAULT_OIDC_AUD`  | The `aud` claim for the JWT | `$CI_SERVER_URL` |

| :lock: `VAULT_ROLE_ID`   | The [AppRole](https://www.vaultproject.io/docs/auth/approle) RoleID | **must be defined** |

| :lock: `VAULT_SECRET_ID` | The [AppRole](https://www.vaultproject.io/docs/auth/approle) SecretID | **must be defined** |
@@ -495,6 +534,8 @@ include: 
    file: '/templates/gitlab-ci-gcloud-vault.yml'



variables:

    # audience claim for JWT

    VAULT_OIDC_AUD: "https://vault.acme.host"

    # Secrets managed by Vault

    SOME_SECRET_USED_IN_MY_APP: "@url@http://vault-secrets-provider/api/secrets/b7ecb6ebabc231/prod/gcloud/secret?field=my.app.secret"

    VAULT_BASE_URL: "https://vault.acme.host/v1"

kicker.json

+61 −44
Original line number Diff line number Diff line
@@ -25,16 +25,6 @@ 
      "description": "Default Service Account to which impersonate with WIF authentication",

      "advanced": true

    },

    {

      "name": "GCP_OIDC_PROVIDER",

      "description": "Global Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/)",

      "advanced": true

    },

    {

      "name": "GCP_OIDC_ACCOUNT",

      "description": "Global Service Account to which impersonate with OpenID Connect authentication",

      "advanced": true

    },

    {

      "name": "GCP_BASE_APP_NAME",

      "description": "Base application name",
@@ -52,7 +42,6 @@ 
      "default": ".",

      "advanced": true

    }



  ],

  "features": [

    {
@@ -80,16 +69,6 @@ 
          "name": "GCP_REVIEW_KEY_FILE",

          "description": "Service Account key file to authenticate on review env (only define if different from global)",

          "secret": true

        },

        {

          "name": "GCP_REVIEW_OIDC_PROVIDER",

          "description": "Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `review` environment\n\n_(only define if different from global)_",

          "advanced": true

        },

        {

          "name": "GCP_REVIEW_OIDC_ACCOUNT",

          "description": "Service Account to which impersonate with OpenID Connect authentication on `review` environment",

          "advanced": true

        }

      ]

    },
@@ -118,16 +97,6 @@ 
          "name": "GCP_INTEG_KEY_FILE",

          "description": "Service Account key file to authenticate on integration env (only define if different from global)",

          "secret": true

        },   

        {

          "name": "GCP_INTEG_OIDC_PROVIDER",

          "description": "Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `integration` environment\n\n_(only define if different from global)_",

          "advanced": true

        },

        {

          "name": "GCP_INTEG_OIDC_ACCOUNT",

          "description": "Service Account to which impersonate with OpenID Connect authentication on `integration` environment",

          "advanced": true

        }

      ]

    },
@@ -156,16 +125,6 @@ 
          "name": "GCP_STAGING_KEY_FILE",

          "description": "Service Account key file to authenticate on staging env (only define if different from global)",

          "secret": true

        },        

        {

          "name": "GCP_STAGING_OIDC_PROVIDER",

          "description": "Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `staging` environment\n\n_(only define if different from global)_",

          "advanced": true

        },

        {

          "name": "GCP_STAGING_OIDC_ACCOUNT",

          "description": "Service Account to which impersonate with OpenID Connect authentication on `staging` environment",

          "advanced": true

        }

      ]

    },
@@ -201,6 +160,61 @@ 
          "name": "GCP_PROD_KEY_FILE",

          "description": "Service Account key file to authenticate on production env (only define if different from global)",

          "secret": true

        }

      ]

    }

  ],

  "variants": [

    {

      "id": "oidc",

      "name": "OpenID Connect",

      "description": "Enables [federated authentication using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/)",

      "template_path": "templates/gitlab-ci-gcloud-oidc.yml",

      "variables": [

        {

          "name": "GCP_OIDC_AUD",

          "description": "The `aud` claim for the JWT",

          "default": "$CI_SERVER_URL"

        },

        {

          "name": "GCP_OIDC_PROVIDER",

          "description": "Global Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/)",

          "advanced": true

        },

        {

          "name": "GCP_OIDC_ACCOUNT",

          "description": "Global Service Account to which impersonate with OpenID Connect authentication",

          "advanced": true

        },

        {

          "name": "GCP_REVIEW_OIDC_PROVIDER",

          "description": "Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `review` environment\n\n_(only define if different from global)_",

          "advanced": true

        },

        {

          "name": "GCP_REVIEW_OIDC_ACCOUNT",

          "description": "Service Account to which impersonate with OpenID Connect authentication on `review` environment",

          "advanced": true

        },

        {

          "name": "GCP_INTEG_OIDC_PROVIDER",

          "description": "Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `integration` environment\n\n_(only define if different from global)_",

          "advanced": true

        },

        {

          "name": "GCP_INTEG_OIDC_ACCOUNT",

          "description": "Service Account to which impersonate with OpenID Connect authentication on `integration` environment",

          "advanced": true

        },

        {

          "name": "GCP_STAGING_OIDC_PROVIDER",

          "description": "Workload Identity Provider associated with GitLab to [authenticate with OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/google_cloud/) on `staging` environment\n\n_(only define if different from global)_",

          "advanced": true

        },

        {

          "name": "GCP_STAGING_OIDC_ACCOUNT",

          "description": "Service Account to which impersonate with OpenID Connect authentication on `staging` environment",

          "advanced": true

        },

        {

          "name": "GCP_PROD_OIDC_PROVIDER",
@@ -213,9 +227,7 @@ 
          "advanced": true

        }

      ]

    }

  ],

  "variants": [

    },

    {

      "id": "vault",

      "name": "Vault",
@@ -228,6 +240,11 @@ 
          "default": "$CI_REGISTRY/to-be-continuous/tools/vault-secrets-provider:master",

          "advanced": true

        },

        {

          "name": "VAULT_OIDC_AUD",

          "description": "The `aud` claim for the JWT",

          "default": "$CI_SERVER_URL"

        },

        {

          "name": "VAULT_BASE_URL",

          "description": "The Vault server base API url",

templates/gitlab-ci-gcloud-oidc.yml

0 → 100644
+10 −0
Original line number Diff line number Diff line 
# =====================================================================================================================

# === OIDC authentication template variant

# =====================================================================================================================

variables:

  GCP_OIDC_AUD: "$CI_SERVER_URL"



.gcp-base:

  id_tokens:

    GCP_JWT:

      aud: "$GCP_OIDC_AUD"

templates/gitlab-ci-gcloud-vault.yml

+6 −0
Original line number Diff line number Diff line
@@ -8,6 +8,7 @@ variables: 
  VAULT_JWT_TOKEN: "$VAULT_JWT_TOKEN"

  VAULT_ROLE_ID: "$VAULT_ROLE_ID"

  VAULT_SECRET_ID: "$VAULT_SECRET_ID"

  VAULT_OIDC_AUD: "$CI_SERVER_URL"



.gcp-base:

  services:
@@ -15,3 +16,8 @@ variables: 
      command: ["--service", "gcloud", "3.2.0" ]

    - name: "$TBC_VAULT_IMAGE"

      alias: "vault-secrets-provider"

  variables:

    VAULT_JWT_TOKEN: "$VAULT_JWT_TOKEN"

  id_tokens:

    VAULT_JWT_TOKEN:

      aud: "$VAULT_OIDC_AUD"