Commit 6c73375a authored by Pierre Smeyers's avatar Pierre Smeyers
Browse files

refactor: use $environment_type & $environment_name 

Deprecated $appname and $env dynamic variables with $environment_name & $environment_type
With backward compatibility (legacy vars still usable)

Deprecated $appname and $env dynamic variables with $environment_name & $environment_type
With backward compatibility (legacy vars still usable)
parent ea4f606a

README.md

+36 −38
Original line number Diff line number Diff line
@@ -66,7 +66,7 @@ The AWS template uses some global configuration used throughout all jobs. 
| ------------------------ | -------------------------------------- | ----------------- |

| `AWS_CLI_IMAGE`          | the Docker image used to run AWS CLI commands| `amazon/aws-cli:latest` |

| `AWS_BASE_APP_NAME`      | Base application name                  | `$CI_PROJECT_NAME` ([see GitLab doc](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html)) |

| `AWS_ENVIRONMENT_URL`    | Default environments url _(only define for static environment URLs declaration)_<br/>_supports late variable expansion (ex: `https://%{appname}.aws.acme.com`)_ | _none_ |

| `AWS_ENVIRONMENT_URL`    | Default environments url _(only define for static environment URLs declaration)_<br/>_supports late variable expansion (ex: `https://%{environment_name}.aws.acme.com`)_ | _none_ |

| `AWS_SCRIPTS_DIR`        | Directory where AWS scripts (deploy & cleanup) are located | `.` _(root project dir)_ |

| `AWS_OIDC_ROLE_ARN`    | Default IAM Role ARN associated with GitLab to [authenticate using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/aws/) | _none_ (disabled) |
@@ -93,13 +93,13 @@ deployment and cleanup using the [`aws` CLI](https://aws.amazon.com/cli/) and al 


The deployment script is searched as follows:



1. look for a specific `aws-deploy-$env.sh` in the `$AWS_SCRIPTS_DIR` directory in your project (e.g. `aws-deploy-staging.sh` for staging environment),

1. look for a specific `aws-deploy-$environment_type.sh` in the `$AWS_SCRIPTS_DIR` directory in your project (e.g. `aws-deploy-staging.sh` for staging environment),

2. if not found: look for a default `aws-deploy.sh` in the `$AWS_SCRIPTS_DIR` directory in your project,

3. if not found: the deployment job will fail.



The cleanup script is searched as follows:



1. look for a specific `aws-cleanup-$env.sh` in the `$AWS_SCRIPTS_DIR` directory in your project (e.g. `aws-cleanup-staging.sh` for staging environment),

1. look for a specific `aws-cleanup-$environment_type.sh` in the `$AWS_SCRIPTS_DIR` directory in your project (e.g. `aws-cleanup-staging.sh` for staging environment),

2. if not found: look for a default `aws-cleanup.sh` in the `$AWS_SCRIPTS_DIR` directory in your project,

3. if not found: the cleanup job will fail.
@@ -117,28 +117,26 @@ In order to be able to implement some **genericity** in your scripts and templat 
2. any [custom variable](https://docs.gitlab.com/ee/ci/variables/#custom-environment-variables)

    (ex: `${SECRET_TOKEN}` that you have set in your project CI/CD variables)

3. **dynamic variables** set by the template:

    * `${appname}`: the application target name to use for this environment (ex: `myproject-review-fix-bug-12` or `myproject-staging`) - see below

    * `${env}`: the environment type (`review`, `integration`, `staging` or `production`)

    * `${environment_type}`: the current environment type (`review`, `integration`, `staging` or `production`)

    * `${environment_name}`: the application name to use for the current environment (ex: `myproject-review-fix-bug-12` or `myproject-staging`) - see below

    * `${hostname}`: the environment hostname, extracted from the current environment url (after late variable expansion - see below)



##### `appname` generation

##### `environment_name` generation



Each environment is given an _appname_, generated programmatically by the template.



The appname is obtained using:

Each environment is given an _environment name_, generated programmatically by the template, using:



* `AWS_BASE_APP_NAME` defines the application _base_ name (defaults to `$CI_PROJECT_NAME`)

* `appname=$AWS_BASE_APP_NAME-$CI_ENVIRONMENT_SLUG` except for the production environment, where `appname` is directly `AWS_BASE_APP_NAME` (no suffix)

* each appname can be overriden per environment with the appropriate `AWS_XXX_APP_NAME` variable

* `environment_name=$AWS_BASE_APP_NAME-$CI_ENVIRONMENT_SLUG` except for the production environment, where `environment_name` is directly `AWS_BASE_APP_NAME` (no suffix)

* each environment name can be overriden per environment with the appropriate `AWS_XXX_APP_NAME` variable



Examples (with an application's base name `myapp`):



| `$environment_type` | Branch        | `$CI_ENVIRONMENT_SLUG`  | `$appname` |

|---------------------|---------------|-------------------------|------------|

| review              | `feat/blabla` | `review-feat-bla-xmuzs6`| `myapp-review-feat-bla-xmuzs6` |

| integration         | `main`        | `develop`               | `myapp-integration` |

| staging             | `main`        | `staging`               | `myapp-staging` |

| production          | `main`        | `production`            | `myapp` |

| `$environment_type` | Branch        | `$CI_ENVIRONMENT_SLUG`  | `$environment_name` |

|---------------------|---------------|-------------------------|---------------------|

| `review`            | `feat/blabla` | `review-feat-bla-xmuzs6`| `myapp-review-feat-bla-xmuzs6` |

| `integration`       | `develop`     | `integration`           | `myapp-integration` |

| `staging`           | `main`        | `staging`               | `myapp-staging` |

| `production`        | `main`        | `production`            | `myapp` |



#### AWS authentication
@@ -190,7 +188,7 @@ The **static way** can be implemented simply by setting the appropriate configur 
* `$AWS_ENVIRONMENT_URL` to define a default url pattern for all your envs,

* `$AWS_REVIEW_ENVIRONMENT_URL`, `$AWS_INTEG_ENVIRONMENT_URL`, `$AWS_STAGING_ENVIRONMENT_URL` and `$AWS_PROD_ENVIRONMENT_URL` to override the default.



> :information_source: Each of those variables support a late variable expansion mechanism with `%{somevar}` syntax, allowing you to use any dynamically evaluated variables such as `$appname`.

> :information_source: Each of those variables support a late variable expansion mechanism with `%{somevar}` syntax, allowing you to use any dynamically evaluated variables such as `$environment_name`.

>

> Example:

>
@@ -198,7 +196,7 @@ The **static way** can be implemented simply by setting the appropriate configur 
> variables:

>   AWS_BASE_APP_NAME: "wonderapp"

>   # global url for all environments

>   AWS_ENVIRONMENT_URL: "https://%{appname}.nonprod.acme.domain"

>   AWS_ENVIRONMENT_URL: "https://%{environment_name}.nonprod.acme.domain"

>   # override for prod (no need to late exand $AWS_BASE_APP_NAME here)

>   AWS_PROD_ENVIRONMENT_URL: "https://$AWS_BASE_APP_NAME.acme.domain"

> ```
@@ -326,7 +324,7 @@ stages: 


##### `aws-deploy.sh`



This script is executed by the template to perform the application(s) deployment based on `aws` CLI, and uses [dynamic variables](#dynamic-variables) provided by the template (`${appname}` is used as the CloudFormation stack name).

This script is executed by the template to perform the application(s) deployment based on `aws` CLI, and uses [dynamic variables](#dynamic-variables) provided by the template (`${environment_name}` is used as the CloudFormation stack name).



It implements [dynamic environment URLs](#static-vs-dynamic-environment-urls), by generating the `environment_url.txt` file, containing

the dynamically generated url at the end of the deployment script.
@@ -335,33 +333,33 @@ the dynamically generated url at the end of the deployment script. 
#!/usr/bin/env bash

set -e # fail on error



echo "[aws-deploy] Deploying $appname..."

echo "[aws-deploy] Deploying $environment_name..."



# disable AWS CLI pager

export AWS_PAGER=""



template_file=file://MyStack.template

# retrieve environment type ($env) from template

# retrieve environment type ($environment_type) from template

# retrieve $AWS_KEYPAIR_NAME from project secret variables

params_opts="--parameters ParameterKey=EnvType,ParameterValue=$env ParameterKey=KeyName,ParameterValue=$AWS_KEYPAIR_NAME"

params_opts="--parameters ParameterKey=EnvType,ParameterValue=$environment_type ParameterKey=KeyName,ParameterValue=$AWS_KEYPAIR_NAME"



if aws cloudformation describe-stacks --output text --stack-name "$appname" > /dev/null

if aws cloudformation describe-stacks --output text --stack-name "$environment_name" > /dev/null

then

  echo -e "Stack exists: update..."

  aws cloudformation update-stack --output text --stack-name "$appname" --template-body $template_file $parameters

  aws cloudformation update-stack --output text --stack-name "$environment_name" --template-body $template_file $parameters



  echo "Waiting for stack to be updated..."

  aws cloudformation wait stack-update-complete --stack-name "$appname"

  aws cloudformation wait stack-update-complete --stack-name "$environment_name"

else

  echo -e "Stack doesn't exist: create..."

  aws cloudformation create-stack --output text --stack-name "$appname" --template-body $template_file $parameters

  aws cloudformation create-stack --output text --stack-name "$environment_name" --template-body $template_file $parameters

  

  echo "Waiting for stack to be created..."

  aws cloudformation wait stack-create-complete --stack-name "$appname"

  aws cloudformation wait stack-create-complete --stack-name "$environment_name"

fi



# Retrieve outputs (use cloudformation query)

webserver_url=$(aws cloudformation describe-stacks --output text --stack-name "$appname" --query 'Stacks[0].Outputs[?OutputKey==`WebServerUrl`].OutputValue')

webserver_url=$(aws cloudformation describe-stacks --output text --stack-name "$environment_name" --query 'Stacks[0].Outputs[?OutputKey==`WebServerUrl`].OutputValue')



echo "Stack created/updated:"

echo " - WebServerUrl: $webserver_url"
@@ -378,12 +376,12 @@ This script is executed by the template to perform the application(s) cleanup ba 
#!/usr/bin/env bash

set -e # fail on error



echo "[aws-cleanup] Cleanup $appname..."

echo "[aws-cleanup] Cleanup $environment_name..."



# disable AWS CLI pager

export AWS_PAGER=""



aws cloudformation delete-stack --stack-name "$appname"

aws cloudformation delete-stack --stack-name "$environment_name"

```
@@ -428,7 +426,7 @@ stages: 


##### `aws-deploy.sh`



This script is executed by the template to perform the application(s) deployment based on `sam`and `aws` CLI, and uses [dynamic variables](#dynamic-variables) provided by the template (`${appname}` is used as the SAM/CloudFormation stack name).

This script is executed by the template to perform the application(s) deployment based on `sam`and `aws` CLI, and uses [dynamic variables](#dynamic-variables) provided by the template (`${environment_name}` is used as the SAM/CloudFormation stack name).



It implements [dynamic environment URLs](#static-vs-dynamic-environment-urls), by generating the `environment_url.txt` file, containing

the dynamically generated url at the end of the deployment script.
@@ -437,7 +435,7 @@ the dynamically generated url at the end of the deployment script. 
#!/usr/bin/env bash

set -e # fail on error



echo "[aws-deploy] Deploy $appname..."

echo "[aws-deploy] Deploy $environment_name..."



# disable AWS CLI pager

export AWS_PAGER=""
@@ -449,15 +447,15 @@ sam build ${TRACE+--debug} 
# AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY defined as secret CI/CD variable

# AWS_DEFAULT_REGION and AWS_SAM_BUCKET defined as project CI/CD variable

sam deploy ${TRACE+--debug} \

  --stack-name "$appname" \

  --stack-name "$environment_name" \

  --region "$AWS_DEFAULT_REGION" \

  --s3-bucket "$AWS_SAM_BUCKET" \

  --no-fail-on-empty-changeset \

  --no-confirm-changeset \

  --tags "ci-job-url=$CI_JOB_URL environment=$env"

  --tags "ci-job-url=$CI_JOB_URL environment=$environment_type"



# Retrieve outputs (use cloudformation query)

api_url=$(aws cloudformation describe-stacks --stack-name "$appname" --output text --query 'Stacks[0].Outputs[?OutputKey==`MyProjectApiUrl`].OutputValue')

api_url=$(aws cloudformation describe-stacks --stack-name "$environment_name" --output text --query 'Stacks[0].Outputs[?OutputKey==`MyProjectApiUrl`].OutputValue')



echo "Stack created/updated:"

echo " - Api URL: $api_url"
@@ -474,12 +472,12 @@ This script is executed by the template to perform the application(s) cleanup ba 
#!/usr/bin/env bash

set -e # fail on error



echo "[aws-cleanup] Cleanup $appname..."

echo "[aws-cleanup] Cleanup $environment_name..."



# disable AWS CLI pager

export AWS_PAGER=""



aws cloudformation delete-stack --stack-name "$appname"

aws cloudformation delete-stack --stack-name "$environment_name"

```



## Variants

templates/gitlab-ci-aws.yml

+24 −17
Original line number Diff line number Diff line
@@ -274,9 +274,14 @@ stages: 


  # application deployment function

  function deploy() {

    export env=$1

    export appname=$2

    export environment_type=$1

    export environment_name=$2

    environment_url=$3



    # backward compatibility

    export env=$environment_type

    export appname=$environment_name



    # variables expansion in $environment_url

    environment_url=$(echo "$environment_url" | awkenvsubst)

    export environment_url
@@ -284,18 +289,16 @@ stages: 
    hostname=$(echo "$environment_url" | awk -F[/:] '{print $4}')

    export hostname



    log_info "--- \\e[32mdeploy\\e[0m (env: \\e[33;1m${env}\\e[0m)"

    log_info "--- \$appname: \\e[33;1m${appname}\\e[0m"

    log_info "--- \$env: \\e[33;1m${env}\\e[0m"

    log_info "--- \\e[32mdeploy\\e[0m"

    log_info "--- \$environment_type: \\e[33;1m${environment_type}\\e[0m"

    log_info "--- \$environment_name: \\e[33;1m${environment_name}\\e[0m"

    log_info "--- \$hostname: \\e[33;1m${hostname}\\e[0m"



    # unset any upstream deployment env & artifacts

    unset environment_name

    unset environment_type

    rm -f aws.env

    rm -f environment_url.txt



    deployscript=$(ls -1 "$AWS_SCRIPTS_DIR/aws-deploy-${env}.sh" 2>/dev/null || ls -1 "$AWS_SCRIPTS_DIR/aws-deploy.sh" 2>/dev/null || echo "")

    deployscript=$(ls -1 "$AWS_SCRIPTS_DIR/aws-deploy-${environment_type}.sh" 2>/dev/null || ls -1 "$AWS_SCRIPTS_DIR/aws-deploy.sh" 2>/dev/null || echo "")

    if [[ -f "$deployscript" ]]

    then

      log_info "--- deploy script (\\e[33;1m${deployscript}\\e[0m) found: execute"
@@ -314,19 +317,23 @@ stages: 
    else

      echo "$environment_url" > environment_url.txt

    fi

    echo -e "environment_type=$env\\nenvironment_name=$appname\\nenvironment_url=$environment_url" > aws.env

    echo -e "environment_type=$environment_type\\nenvironment_name=$environment_name\\nenvironment_url=$environment_url" > aws.env

  }



  # environment cleanup function

  function delete() {

    export env=$1

    export appname=$2

    export environment_type=$1

    export environment_name=$2

    

    # backward compatibility

    export env=$environment_type

    export appname=$environment_name



    log_info "--- \\e[32mdelete\\e[0m (env: ${env})"

    log_info "--- \$appname: \\e[33;1m${appname}\\e[0m"

    log_info "--- \$env: \\e[33;1m${env}\\e[0m"

    log_info "--- \\e[32mdelete\\e[0m"

    log_info "--- \$environment_type: \\e[33;1m${environment_type}\\e[0m"

    log_info "--- \$environment_name: \\e[33;1m${environment_name}\\e[0m"



    cleanupscript=$(ls -1 "$AWS_SCRIPTS_DIR/aws-cleanup-${env}.sh" 2>/dev/null || ls -1 "$AWS_SCRIPTS_DIR/aws-cleanup.sh" 2>/dev/null || echo "")

    cleanupscript=$(ls -1 "$AWS_SCRIPTS_DIR/aws-cleanup-${environment_type}.sh" 2>/dev/null || ls -1 "$AWS_SCRIPTS_DIR/aws-cleanup.sh" 2>/dev/null || echo "")

    if [[ -f "$cleanupscript" ]]

    then

      log_info "--- cleanup script (\\e[33;1m${cleanupscript}\\e[0m) found: execute"