Commit 4a9b94c3 authored by Cédric OLIVIER's avatar Cédric OLIVIER
Browse files

Merge branch 'feat/component' into 'master' 

feat: migrate to CI/CD component

See merge request to-be-continuous/aws!49
parents 14957f5d 5a520950

.gitlab-ci.yml

+1 −1
Original line number Diff line number Diff line
@@ -10,7 +10,7 @@ include: 
    file: '/templates/validation.yml'

  - project: 'to-be-continuous/bash'

    ref: '3.3'

    file: 'templates/gitlab-ci-bash.yml'

    file: '/templates/gitlab-ci-bash.yml'

  - project: 'to-be-continuous/semantic-release'

    ref: '3.7'

    file: '/templates/gitlab-ci-semrel.yml'

README.md

+100 −93
Original line number Diff line number Diff line
@@ -4,13 +4,44 @@ This project implements a GitLab CI/CD template to deploy your application to [A 


## Usage



In order to include this template in your project, add the following to your `gitlab-ci.yml`:

This template can be used both as a [CI/CD component](https://docs.gitlab.com/ee/ci/components/#use-a-component-in-a-cicd-configuration) 

or using the legacy [`include:project`](https://docs.gitlab.com/ee/ci/yaml/index.html#includeproject) syntax.



### Use as a CI/CD component



Add the following to your `gitlab-ci.yml`:



```yaml

include:

  # 1: include the component

  - component: gitlab.com/to-be-continuous/aws/gitlab-ci-aws@5.1.0

    # 2: set/override component inputs

    inputs:

      # ⚠ this is only an example

      base-app-name: wonderapp

      review-enabled: true

      staging-enabled: true

      prod-enabled: true

```



### Use as a CI/CD template (legacy)



Add the following to your `gitlab-ci.yml`:



```yaml

include:

  # 1: include the template

  - project: 'to-be-continuous/aws'

    ref: '5.1.0'

    file: '/templates/gitlab-ci-aws.yml'



variables:

  # 2: set/override template variables

  # ⚠ this is only an example

  AWS_BASE_APP_NAME: wonderapp

  AWS_REVIEW_ENABLED: "true"

  AWS_STAGING_ENABLED: "true"

  AWS_PROD_ENABLED: "true"

```



## Understand
@@ -219,12 +250,12 @@ Here are some advices about your **secrets** (variables marked with a :lock:): 


The AWS template uses some global configuration used throughout all jobs and environments.



| Name                     | Description                            | Default value     |

| Input / Variable | Description                            | Default value     |

| ------------------------ | -------------------------------------- | ----------------- |

| `AWS_CLI_IMAGE`          | the Docker image used to run AWS CLI commands| `registry.hub.docker.com/amazon/aws-cli:latest` |

| `AWS_BASE_APP_NAME`      | Base application name                  | `$CI_PROJECT_NAME` ([see GitLab doc](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html)) |

| `AWS_ENVIRONMENT_URL`    | Default environments url _(only define for static environment URLs declaration)_<br/>_supports late variable expansion (ex: `https://%{environment_name}.aws.acme.com`)_ | _none_ |

| `AWS_SCRIPTS_DIR`        | Directory where AWS scripts (deploy & cleanup) are located | `.` _(root project dir)_ |

| `cli-image` / `AWS_CLI_IMAGE` | the Docker image used to run AWS CLI commands| `registry.hub.docker.com/amazon/aws-cli:latest` |

| `base-app-name` / `AWS_BASE_APP_NAME` | Base application name                  | `$CI_PROJECT_NAME` ([see GitLab doc](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html)) |

| `environment-url` / `AWS_ENVIRONMENT_URL`    | Default environments url _(only define for static environment URLs declaration)_<br/>_supports late variable expansion (ex: `https://%{environment_name}.aws.acme.com`)_ | _none_ |

| `scripts-dir` / `AWS_SCRIPTS_DIR` | Directory where AWS scripts (deploy & cleanup) are located | `.` _(root project dir)_ |



### Review environments configuration
@@ -235,12 +266,12 @@ They are **disabled by default** and can be enabled by setting the `AWS_REVIEW_E 


Here are variables supported to configure review environments:



| Name                     | Description                            | Default value     |

| Input / Variable | Description                            | Default value     |

| ------------------------ | -------------------------------------- | ----------------- |

| `AWS_REVIEW_ENABLED`     | AWS project ID for `review` env | _none_ (disabled) |

| `AWS_REVIEW_APP_NAME`    | Application name for `review` env      | `"${AWS_BASE_APP_NAME}-${CI_ENVIRONMENT_SLUG}"` (ex: `myproject-review-fix-bug-12`) |

| `AWS_REVIEW_ENVIRONMENT_URL`| The review environments url _(only define for static environment URLs declaration and if different from default)_ | `$AWS_ENVIRONMENT_URL` |

| `AWS_REVIEW_AUTOSTOP_DURATION`| The amount of time before GitLab will automatically stop `review` environments | `4 hours` |

| `review-enabled` / `AWS_REVIEW_ENABLED` | AWS project ID for `review` env | _none_ (disabled) |

| `review-app-name` / `AWS_REVIEW_APP_NAME` | Application name for `review` env      | `"${AWS_BASE_APP_NAME}-${CI_ENVIRONMENT_SLUG}"` (ex: `myproject-review-fix-bug-12`) |

| `review-environment-url` / `AWS_REVIEW_ENVIRONMENT_URL`| The review environments url _(only define for static environment URLs declaration and if different from default)_ | `$AWS_ENVIRONMENT_URL` |

| `review-autostop-duration` / `AWS_REVIEW_AUTOSTOP_DURATION` | The amount of time before GitLab will automatically stop `review` environments | `4 hours` |



### Integration environment configuration
@@ -250,11 +281,11 @@ It is **disabled by default** and can be enabled by setting the `AWS_INTEG_ENABL 


Here are variables supported to configure the integration environment:



| Name                     | Description                            | Default value     |

| Input / Variable | Description                            | Default value     |

| ------------------------ | -------------------------------------- | ----------------- |

| `AWS_INTEG_ENABLED`      | AWS project ID for `integration` env | _none_ (disabled) |

| `AWS_INTEG_APP_NAME`     | Application name for `integration` env | `${AWS_BASE_APP_NAME}-integration` |

| `AWS_INTEG_ENVIRONMENT_URL`| The integration environment url _(only define for static environment URLs declaration and if different from default)_ | `$AWS_ENVIRONMENT_URL` |

| `integ-enabled` / `AWS_INTEG_ENABLED` | AWS project ID for `integration` env | _none_ (disabled) |

| `integ-app-name` / `AWS_INTEG_APP_NAME` | Application name for `integration` env | `${AWS_BASE_APP_NAME}-integration` |

| `integ-environment-url` / `AWS_INTEG_ENVIRONMENT_URL`| The integration environment url _(only define for static environment URLs declaration and if different from default)_ | `$AWS_ENVIRONMENT_URL` |



### Staging environment configuration
@@ -265,11 +296,11 @@ It is **disabled by default** and can be enabled by setting the `AWS_STAGING_ENA 


Here are variables supported to configure the staging environment:



| Name                     | Description                            | Default value     |

| Input / Variable | Description                            | Default value     |

| ------------------------ | -------------------------------------- | ----------------- |

| `AWS_STAGING_ENABLED`    | AWS project ID for `staging` env | _none_ (disabled) |

| `AWS_STAGING_APP_NAME`   | Application name for `staging` env     | `${AWS_BASE_APP_NAME}-staging` |

| `AWS_STAGING_ENVIRONMENT_URL`| The staging environment url _(only define for static environment URLs declaration and if different from default)_ | `$AWS_ENVIRONMENT_URL` |

| `staging-enabled` / `AWS_STAGING_ENABLED` | AWS project ID for `staging` env | _none_ (disabled) |

| `staging-app-name` / `AWS_STAGING_APP_NAME` | Application name for `staging` env     | `${AWS_BASE_APP_NAME}-staging` |

| `staging-environment-url` / `AWS_STAGING_ENVIRONMENT_URL`| The staging environment url _(only define for static environment URLs declaration and if different from default)_ | `$AWS_ENVIRONMENT_URL` |



### Production environment configuration
@@ -279,12 +310,12 @@ It is **disabled by default** and can be enabled by setting the `AWS_PROD_ENABLE 


Here are variables supported to configure the production environment:



| Name                      | Description                            | Default value     |

| Input / Variable | Description                            | Default value     |

| ------------------------- | -------------------------------------- | ----------------- |

| `AWS_PROD_ENABLED`        | AWS project ID for `production` env | _none_ (disabled) |

| `AWS_PROD_APP_NAME`       | Application name for `production` env  | `$AWS_BASE_APP_NAME` |

| `AWS_PROD_ENVIRONMENT_URL`| The production environment url _(only define for static environment URLs declaration and if different from default)_ | `$AWS_ENVIRONMENT_URL` |

| `AWS_PROD_DEPLOY_STRATEGY`| Defines the deployment to production strategy. One of `manual` (i.e. _one-click_) or `auto`. | `manual` |

| `prod-enabled` / `AWS_PROD_ENABLED` | AWS project ID for `production` env | _none_ (disabled) |

| `prod-app-name` / `AWS_PROD_APP_NAME` | Application name for `production` env  | `$AWS_BASE_APP_NAME` |

| `prod-environment-url` / `AWS_PROD_ENVIRONMENT_URL`| The production environment url _(only define for static environment URLs declaration and if different from default)_ | `$AWS_ENVIRONMENT_URL` |

| `prod-deploy-strategy` / `AWS_PROD_DEPLOY_STRATEGY` | Defines the deployment to production strategy. One of `manual` (i.e. _one-click_) or `auto`. | `manual` |



## Examples
@@ -298,20 +329,12 @@ It enables review, staging and production environments. 


```yaml

include:

  # Include AWS template

  - project: 'to-be-continuous/aws'

    ref: '5.1.0'

    file: '/templates/gitlab-ci-aws.yml'

  ...



# Global variables

variables:

  # AWS

  # AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY defined as secret CI/CD variable

  AWS_REVIEW_ENABLED: "true" # enable review env

  AWS_STAGING_ENABLED: "true" # enable staging env

  AWS_PROD_ENABLED: "true" # enable production env

  ...

  - component: gitlab.com/to-be-continuous/aws/gitlab-ci-aws@5.1.0

    inputs:

      review-enabled: true

      staging-enabled: true

      prod-enabled: true

      # authentication means defined as secret CI/CD variable

```



#### AWS scripts
@@ -389,22 +412,14 @@ It enables review, staging and production environments. 


```yaml

include:

  # Include AWS template

  - project: 'to-be-continuous/aws'

    ref: '5.1.0'

    file: '/templates/gitlab-ci-aws.yml'

  ...



# Global variables

variables:

  # AWS

  - component: gitlab.com/to-be-continuous/aws/gitlab-ci-aws@5.1.0

    inputs:

      # use an image with both aws and sam CLI

  AWS_CLI_IMAGE: "pahud/aws-sam-cli:latest"

  # AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY defined as secret CI/CD variable

  AWS_REVIEW_ENABLED: "true" # enable review env

  AWS_STAGING_ENABLED: "true" # enable staging env

  AWS_PROD_ENABLED: "true" # enable production env

  ...

      cli-image: "pahud/aws-sam-cli:latest"

      review-enabled: true

      staging-enabled: true

      prod-enabled: true

      # authentication means defined as secret CI/CD variable

```



#### AWS scripts
@@ -483,35 +498,30 @@ then configure appropriately the related variables: 


The variant supports the following configuration:



| Name              | Description                            | Default value     |

| Input / Variable | Description                            | Default value     |

| ----------------- | -------------------------------------- | ----------------- |

| `AWS_OIDC_AUD`    | The `aud` claim for the JWT | `$CI_SERVER_URL` |

| `AWS_OIDC_ROLE_ARN`    | Default IAM Role ARN associated with GitLab to [authenticate using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/aws/) | _none_ (disabled) |

| `AWS_REVIEW_OIDC_ROLE_ARN`| IAM Role ARN associated with GitLab to [authenticate using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/aws/) on `review` env _(only define if different from global)_ | _none_ (disabled) |

| `AWS_INTEG_OIDC_ROLE_ARN`| IAM Role ARN associated with GitLab to [authenticate using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/aws/) on `integration` env _(only define if different from global)_ | _none_ (disabled) |

| `AWS_STAGING_OIDC_ROLE_ARN`| IAM Role ARN associated with GitLab to [authenticate using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/aws/) on `staging` env _(only define if different from global)_ | _none_ (disabled) |

| `AWS_PROD_OIDC_ROLE_ARN`| IAM Role ARN associated with GitLab to [authenticate using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/aws/) on `production` env _(only define if different from global)_ | _none_ (disabled) |

| `oidc-aud` / `AWS_OIDC_AUD` | The `aud` claim for the JWT | `$CI_SERVER_URL` |

| `oidc-role-arn` / `AWS_OIDC_ROLE_ARN` | Default IAM Role ARN associated with GitLab to [authenticate using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/aws/) | _none_ (disabled) |

| `review-oidc-role-arn` / `AWS_REVIEW_OIDC_ROLE_ARN` | IAM Role ARN associated with GitLab to [authenticate using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/aws/) on `review` env _(only define to override default)_ | _none_ (disabled) |

| `integ-oidc-role-arn` / `AWS_INTEG_OIDC_ROLE_ARN` | IAM Role ARN associated with GitLab to [authenticate using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/aws/) on `integration` env _(only define to override default)_ | _none_ (disabled) |

| `staging-oidc-role-arn` / `AWS_STAGING_OIDC_ROLE_ARN` | IAM Role ARN associated with GitLab to [authenticate using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/aws/) on `staging` env _(only define to override default)_ | _none_ (disabled) |

| `prod-oidc-role-arn` / `AWS_PROD_OIDC_ROLE_ARN` | IAM Role ARN associated with GitLab to [authenticate using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/aws/) on `production` env _(only define to override default)_ | _none_ (disabled) |



#### Example



```yaml

include:

  # main template

  - project: 'to-be-continuous/aws'

    ref: '5.1.0'

    file: '/templates/gitlab-ci-aws.yml'

  # Vault variant

  - project: 'to-be-continuous/aws'

    ref: '5.1.0'

    file: '/templates/gitlab-ci-aws-oidc.yml'



variables:

  - component: gitlab.com/to-be-continuous/aws/gitlab-ci-aws@5.1.0

  # OIDC variant

  - component: gitlab.com/to-be-continuous/aws/gitlab-ci-aws-oidc@5.1.0

    inputs:

      # audience claim for JWT

  AWS_OIDC_AUD: "https://gitlab.acme.com"

      oidc-aud: "https://gitlab.acme.com"

      # common OIDC role ARN for non-prod envs

  AWS_OIDC_ROLE_ARN: "arn:aws:iam::111111111111:role/cicd-role"

      oidc-role-arn: "arn:aws:iam::111111111111:role/cicd-role"

      # specific OIDC role ARN for prod

  AWS_PROD_OIDC_ROLE_ARN: "arn:aws:iam::222222222222:role/cicd-role"

      prod-oidc-role-arn: "arn:aws:iam::222222222222:role/cicd-role"

```



### Vault variant
@@ -522,11 +532,11 @@ This variant allows delegating your secrets management to a [Vault](https://www. 


In order to be able to communicate with the Vault server, the variant requires the additional configuration parameters:



| Name              | Description                            | Default value     |

| Input / Variable | Description                            | Default value     |

| ----------------- | -------------------------------------- | ----------------- |

| `TBC_VAULT_IMAGE` | The [Vault Secrets Provider](https://gitlab.com/to-be-continuous/tools/vault-secrets-provider) image to use (can be overridden) | `registry.gitlab.com/to-be-continuous/tools/vault-secrets-provider:master` |

| `VAULT_BASE_URL`  | The Vault server base API url          | _none_ |

| `VAULT_OIDC_AUD`  | The `aud` claim for the JWT | `$CI_SERVER_URL` |

| `vault-base-url` / `VAULT_BASE_URL` | The Vault server base API url          | _none_ |

| `vault-oidc-aud` / `VAULT_OIDC_AUD` | The `aud` claim for the JWT | `$CI_SERVER_URL` |

| :lock: `VAULT_ROLE_ID`   | The [AppRole](https://www.vaultproject.io/docs/auth/approle) RoleID | **must be defined** |

| :lock: `VAULT_SECRET_ID` | The [AppRole](https://www.vaultproject.io/docs/auth/approle) SecretID | **must be defined** |
@@ -540,7 +550,7 @@ Then you may retrieve any of your secret(s) from Vault using the following synta 


With:



| Name                             | Description                            |

| Parameter                        | Description                            |

| -------------------------------- | -------------------------------------- |

| `secret_path` (_path parameter_) | this is your secret location in the Vault server |

| `field` (_query parameter_)      | parameter to access a single basic field from the secret JSON payload |
@@ -550,20 +560,17 @@ With: 
```yaml

include:

  # main template

  - project: 'to-be-continuous/aws'

    ref: '5.1.0'

    file: '/templates/gitlab-ci-aws.yml'

  - component: gitlab.com/to-be-continuous/aws/gitlab-ci-aws@5.1.0

  # Vault variant

  - project: 'to-be-continuous/aws'

    ref: '5.1.0'

    file: '/templates/gitlab-ci-aws-vault.yml'

  - component: gitlab.com/to-be-continuous/aws/gitlab-ci-aws-vault@5.1.0

    inputs:

      # audience claim for JWT

      vault-oidc-aud: "https://vault.acme.host"

      vault-base-url: "https://vault.acme.host/v1"

      # $VAULT_ROLE_ID and $VAULT_SECRET_ID defined as a secret CI/CD variable



variables:

    # audience claim for JWT

    VAULT_OIDC_AUD: "https://vault.acme.host"

  # Secrets managed by Vault

  AWS_ACCESS_KEY_ID: "@url@http://vault-secrets-provider/api/secrets/b7ecb6ebabc231/aws/prod/account?field=access_key_id"

  AWS_SECRET_ACCESS_KEY: "@url@http://vault-secrets-provider/api/secrets/b7ecb6ebabc231/aws/prod/account?field=secret_access_key"

    VAULT_BASE_URL: "https://vault.acme.host/v1"

    # $VAULT_ROLE_ID and $VAULT_SECRET_ID defined as a secret CI/CD variable

```

bumpversion.sh

+2 −2
Original line number Diff line number Diff line
@@ -27,13 +27,13 @@ if [[ "$curVer" ]]; then 
  log_info "Bump version from \\e[33;1m${curVer}\\e[0m to \\e[33;1m${nextVer}\\e[0m (release type: $relType)..."



  # replace in README

  sed -e "s/ref: '$curVer'/ref: '$nextVer'/" README.md > README.md.next

  sed -e "s/ref: *'$curVer'/ref: '$nextVer'/" -e "s/ref: *\"$curVer\”/ref: \”$nextVer\”/" -e "s/component: *\(.*\)@$curVer/component: \1@$nextVer/" README.md > README.md.next

  mv -f README.md.next README.md



  # replace in template and variants

  for tmpl in templates/*.yml

  do

    sed -e "s/\"$curVer\"/\"$nextVer\"/" "$tmpl" > "$tmpl.next"

    sed -e "s/command: *\[\"--service\", \"\(.*\)\", \"$curVer\"\]/command: [\"--service\", \"\1\", \"$nextVer\"]/" "$tmpl" > "$tmpl.next"

    mv -f "$tmpl.next" "$tmpl"

  done

else

kicker.json

+10 −8
Original line number Diff line number Diff line
@@ -3,6 +3,8 @@ 
  "description": "Deploy your application to [Amazon Web Services](https://aws.amazon.com/)",

  "template_path": "templates/gitlab-ci-aws.yml",

  "kind": "hosting",

  "prefix": "aws",

  "is_component": true,

  "variables": [

    {

      "name": "AWS_CLI_IMAGE",
@@ -36,7 +38,7 @@ 
      "variables": [

        {

          "name": "AWS_REVIEW_APP_NAME",

          "description": "The application name for review env (only define if different from global)",

          "description": "The application name for review env (only define to override default)",

          "advanced": true

        },

        {
@@ -60,7 +62,7 @@ 
      "variables": [

        {

          "name": "AWS_INTEG_APP_NAME",

          "description": "The application name for integration env (only define if different from global)",

          "description": "The application name for integration env (only define to override default)",

          "advanced": true

        },

        {
@@ -79,7 +81,7 @@ 
      "variables": [

        {

          "name": "AWS_STAGING_APP_NAME",

          "description": "The application name for staging env (only define if different from global)",

          "description": "The application name for staging env (only define to override default)",

          "advanced": true

        },

        {
@@ -98,7 +100,7 @@ 
      "variables": [

        {

          "name": "AWS_PROD_APP_NAME",

          "description": "The application name for production env (only define if different from global)",

          "description": "The application name for production env (only define to override default)",

          "advanced": true

        },

        {
@@ -136,22 +138,22 @@ 
        },

        {

          "name": "AWS_REVIEW_OIDC_ROLE_ARN",

          "description": "IAM Role ARN associated with GitLab to [authenticate using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/aws/) on `review` env _(only define if different from global)_",

          "description": "IAM Role ARN associated with GitLab to [authenticate using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/aws/) on `review` env _(only define to override default)_",

          "advanced": true

        },

        {

          "name": "AWS_INTEG_OIDC_ROLE_ARN",

          "description": "IAM Role ARN associated with GitLab to [authenticate using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/aws/) on `integration` env _(only define if different from global)_",

          "description": "IAM Role ARN associated with GitLab to [authenticate using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/aws/) on `integration` env _(only define to override default)_",

          "advanced": true

        },

        {

          "name": "AWS_STAGING_OIDC_ROLE_ARN",

          "description": "IAM Role ARN associated with GitLab to [authenticate using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/aws/) on `staging` env _(only define if different from global)_",

          "description": "IAM Role ARN associated with GitLab to [authenticate using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/aws/) on `staging` env _(only define to override default)_",

          "advanced": true

        },

        {

          "name": "AWS_PROD_OIDC_ROLE_ARN",

          "description": "IAM Role ARN associated with GitLab to [authenticate using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/aws/) on `production` env _(only define if different from global)_",

          "description": "IAM Role ARN associated with GitLab to [authenticate using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/aws/) on `production` env _(only define to override default)_",

          "advanced": true

        }

      ]

logo.png

+3.18 KiB (20.5 KiB)
Loading image diff...