Merge branch 'feat/env-expand-enhancements' into 'master'

The Kubernetes template supports three techniques to deploy your code:

1. script-based deployment,

2. template-based deployment using raw Kubernetes manifests (with variables substitution),

2. template-based deployment using raw Kubernetes manifests (with [variables substitution](#variables-substitution-mechanism)),

3. template-based deployment using [Kustomization files](https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/).

#### 1: script-based deployment

The template processes the following steps:

1. _optionally_ executes the `k8s-pre-apply.sh` script in your project to perform specific environment pre-initialization (for e.g. create required services),

2. looks for your Kubernetes deployment file, performs [variables substitution](#using-variables) and [`kubectl apply`](https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#apply) it,

2. looks for your Kubernetes deployment file, performs [variables substitution](#variables-substitution-mechanism) and [`kubectl apply`](https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#apply) it,

    1. look for a specific `deployment-$environment_type.yml` in your project (e.g. `deployment-staging.yml` for staging environment),

    2. fallbacks to default `deployment.yml`.

3. _optionally_ executes the `k8s-post-apply.sh` script in your project to perform specific environment post-initialization stuff,

The Kubernetes template supports three techniques to destroy an environment (actually only review environments):

1. script-based deployment,

2. template-based deployment using raw Kubernetes manifests (with variables substitution),

2. template-based deployment using raw Kubernetes manifests (with [variables substitution](#variables-substitution-mechanism)),

3. template-based deployment using [Kustomization files](https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/).

#### 1: script-based cleanup

3. any [custom variable](https://docs.gitlab.com/ee/ci/variables/#for-a-project)

    (ex: `${SECRET_TOKEN}` that you have set in your project CI/CD variables)

While your scripts may simply use any of those variables, your Kubernetes and Kustomize resources can use **variable substitution**

with the syntax `${VARIABLE_NAME}`.

Each of those patterns will be dynamically replaced in your resources by the template right before using it.

#### Variables substitution mechanism

You can prevent any line from being processed by appending `# nosubst` at the end of the line.

For instance in the following example, `${REMOTE_SERVICE_NAME}` won't be replaced by its environment value during GitLab job execution:

While your scripts may freely use any of the available variables, your Kubernetes and Kustomize

resources can use a **variables substitution** mechanism implemented by the template:

- Using the syntax `${VARIABLE_NAME}` or `%{VARIABLE_NAME}`.\

  :warning: Curly braces (`{}`) are mandatory in the expression (`$VARIABLE_NAME` won't be processed).

- Each of those expressions will be **dynamically expanded** in your resource files with the variable value, right before being used.

- Variable substitution expressions **must be contained in double-quoted strings**.

  The substitution implementation takes care of escaping characters that need to be (double quote `"`, backslash `\`, tab `\t`, carriage return `\n` and line feed `\r`).

- Variable substitution can be prevented by appending `# nosubst` at the end of any line.\

  Ex:

  ```yaml

  apiVersion: v1

  kind: ConfigMap

  metadata:

    # ${environment_name} will be expanded

    labels:

    app: ${APPLICATION_NAME}

  name: ${APPLICATION_NAME}

      app: "${environment_name}"

    name: "${environment_name}"

  data:

    application.yml: |

      remote:

            name: '${REMOTE_SERVICE_NAME}' # nosubst

  ```

> :warning: In order to be properly replaced, curly braces are mandatory (ex: `${MYVAR}` and not `$MYVAR`).

> Moreover, multiline variables must be surrounded by **double quotes** (`"`).

>

> Example:

>

> ```yaml

> [...]

> containers:

> - name: restaurant-app

>   env:

>   # multiline variable

>   - name: MENU

>     value: "${APP_MENU}"

> ```

### Environments URL management

The K8S template supports two ways of providing your environments url:

    echo "$1" | tr '[:lower:]' '[:upper:]' | tr '[:punct:]' '_'

  }

  function awkenvsubst() {

    # performs variables escaping: '&' for gsub + JSON chars ('\' and '"')

    awk '!/# *nosubst/{while(match($0,"[$%]{[^}]*}")) {var=substr($0,RSTART+2,RLENGTH-3);val=ENVIRON[var];gsub(/["\\&]/,"\\\\&",val);gsub("[$%]{"var"}",val)}}1'

  function tbc_envsubst() {

    awk '

      BEGIN {

        count_replaced_lines = 0

        # ASCII codes

        for (i=0; i<=255; i++)

          char2code[sprintf("%c", i)] = i

      }

      # determine encoding (from env or from file extension)

      function encoding() {

        enc = ENVIRON["TBC_ENVSUBST_ENCODING"]

        if (enc != "")

          return enc

        if (match(FILENAME, /\.(json|yaml|yml)$/))

          return "jsonstr"

        return "raw"

      }

      # see: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/encodeURIComponent

      function uriencode(str) {

        len = length(str)

        enc = ""

        for (i=1; i<=len; i++) {

          c = substr(str, i, 1);

          if (index("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-_.!~*'\''()", c))

            enc = enc c

          else

            enc = enc "%" sprintf("%02X", char2code[c])

        }

        return enc

      }

      !/# *nosubst/ {

        orig_line = $0

        line = $0

        count_repl_in_line = 0

        # /!\ 3rd arg (match) not supported in BusyBox awk

        while (match(line, /[$%]\{([[:alnum:]_]+)\}/)) {

          expr_start = RSTART

          expr_len = RLENGTH

          # get var name

          var = substr(line, expr_start+2, expr_len-3)

          # get var value (from env)

          val = ENVIRON[var]

          # check variable is set

          if (val == "") {

            printf("[\033[1;93mWARN\033[0m] Environment variable \033[33;1m%s\033[0m is not set or empty\n", var) > "/dev/stderr"

          } else {

            enc = encoding()

            if (enc == "jsonstr") {

              gsub(/["\\]/, "\\\\&", val)

              gsub("\n", "\\n", val)

              gsub("\r", "\\r", val)

              gsub("\t", "\\t", val)

            } else if (enc == "uricomp") {

              val = uriencode(val)

            } else if (enc == "raw") {

            } else {

              printf("[\033[1;93mWARN\033[0m] Unsupported encoding \033[33;1m%s\033[0m: ignored\n", enc) > "/dev/stderr"

            }

          }

          # replace expression in line

          line = substr(line, 1, expr_start - 1) val substr(line, expr_start + expr_len)

          count_repl_in_line++

        }

        if (count_repl_in_line) {

          if (count_replaced_lines == 0)

            printf("[\033[1;94mINFO\033[0m] Variable expansion occurred in file \033[33;1m%s\033[0m:\n", FILENAME) > "/dev/stderr"

          count_replaced_lines++

          printf("> line %s: %s\n", NR, orig_line) > "/dev/stderr"

        }

        print line

      }

    ' "$@"

  }

  function exec_hook() {

        exit 1

      fi

      # replace variables (alternative for envsubst which is not present in image)

      awkenvsubst < "$deploymentfile" > generated-deployment.yml

      # variables substitution

      tbc_envsubst "$deploymentfile" > generated-deployment.yml

      log_info "--- \\e[32mkubectl $action\\e[0m"

      kubectl ${TRACE+-v=5} "$action" -f ./generated-deployment.yml

    export appname_ssc=$environment_name_ssc

    # variables expansion in $environment_url

    environment_url=$(echo "$environment_url" | awkenvsubst)

    environment_url=$(echo "$environment_url" | TBC_ENVSUBST_ENCODING=uricomp tbc_envsubst)

    export environment_url

    # extract hostname from $environment_url

    hostname=$(echo "$environment_url" | awk -F[/:] '{print $4}')

    export appname_ssc=$environment_name_ssc

    # variables expansion in $environment_url

    environment_url=$(echo "$environment_url" | awkenvsubst)

    environment_url=$(echo "$environment_url" | TBC_ENVSUBST_ENCODING=uricomp tbc_envsubst)

    export environment_url

    # extract hostname from $environment_url

    hostname=$(echo "$environment_url" | awk -F[/:] '{print $4}')

        exit 1

      fi

      # replace variables (alternative for envsubst which is not present in image)

      awkenvsubst < "$deploymentfile" > generated-deployment.yml

      # variables substitution

      tbc_envsubst "$deploymentfile" > generated-deployment.yml

      # shellcheck disable=SC2086

      /usr/bin/kube-score score $K8S_SCORE_EXTRA_OPTS generated-deployment.yml

  }

  # export tool functions (might be used in after_script)

  export -f log_info log_warn log_error assert_defined rollback awkenvsubst

  export -f log_info log_warn log_error assert_defined rollback tbc_envsubst

  unscope_variables

  eval_all_secrets