Commit da6a9d4d authored by Clement Bois's avatar Clement Bois
Browse files

Merge branch 'feat/buildah-manifest' into 'master' 

feat: manifest build with buildah allowing multi-platform images

Closes #140

See merge request to-be-continuous/docker!168
parents d54b9437 25cda525

README.md

+20 −1
Original line number Diff line number Diff line
@@ -47,7 +47,7 @@ The template supports following ways of building container images: 
2. Or using [kaniko](https://github.com/GoogleContainerTools/kaniko), an open-source, daemonless tool from Google for building Docker

   images, and that solves Docker-in-Docker security issues (and also speeds-up build times).

3. Or using [buildah](https://buildah.io/), an open-source, daemonless tool backed by RedHat for building Docker

   images, and that solves Docker-in-Docker security issues (and also speeds-up build times), and can also be configured to run rootless.

   images, and that solves Docker-in-Docker security issues (and also speeds-up build times), can also be configured to run rootless or support platform emulation.



By default, the template uses the [kaniko](https://docs.gitlab.com/ci/docker/using_kaniko/) way, but you may

select an alternate build tool by using the `DOCKER_BUILD_TOOL` variable (see below).
@@ -622,6 +622,25 @@ include: 
        DOCKER_RELEASE_IMAGE: "$CI_REGISTRY_IMAGE/back:$CI_COMMIT_REF_NAME"

```



### Building multi-platform manifest



Only `buildah` supports building multi-platform images for now.



See [Buildah documentation](https://github.com/containers/buildah/blob/main/docs/buildah-build.1.md#building-an-multi-architecture-image-using-the---manifest-option-requires-emulation-software) for more information about building arguments with different platforms.



Here is a `.gitlab-ci.yaml` that builds a multi architecture manifest to support cross-platform usage:



```yaml

include:

  - component: $CI_SERVER_FQDN/to-be-continuous/docker/gitlab-ci-docker@7.2.1

    inputs:

      build-tool: "buildah"

      build-args: "--platform linux/amd64 --platform linux/arm64"

      # select platforms or use --all-platforms to build for all platforms available in our base image.

```



*Note: The `--all-platforms` flag is incompatible with base image including both tag and digest as set by RenovateBot.*



### Using an externally built Docker image



If you want to use the template with an externally built Docker image (scan for vulnerabilities, extract SBOM, test, promote...), you can skip build jobs by setting the `DOCKER_BUILD_TOOL` variable to `external`:

templates/gitlab-ci-docker.yml

+18 −5
Original line number Diff line number Diff line
@@ -937,7 +937,7 @@ stages: 
      do

        log_info "... pushing extra tag: \\e[33;1m${extra_tag}\\e[0m..."

        # shellcheck disable=SC2086,SC2154

        skopeo copy ${TRACE+--debug} --src-authfile "$BUILDTOOL_HOME/skopeo/.docker/dest-config.json" --dest-authfile "$BUILDTOOL_HOME/skopeo/.docker/dest-config.json" ${DOCKER_PUBLISH_ARGS} "docker://$DOCKER_RELEASE_IMAGE" "docker://$docker_repository:$extra_tag"

        skopeo copy ${TRACE+--debug} --all --src-authfile "$BUILDTOOL_HOME/skopeo/.docker/dest-config.json" --dest-authfile "$BUILDTOOL_HOME/skopeo/.docker/dest-config.json" ${DOCKER_PUBLISH_ARGS} "docker://$DOCKER_RELEASE_IMAGE" "docker://$docker_repository:$extra_tag"

      done

    else

      log_info "Extra tags configured, but the released tag (\\e[33;1m${docker_tag}\\e[0m) doesn't match \$DOCKER_RELEASE_EXTRA_TAGS_PATTERN..."
@@ -1143,9 +1143,22 @@ docker-buildah-build: 
    - |

      log_info "Buildah version:"

      buildah version

    # build and push image

    - echo $DOCKER_METADATA $DOCKER_BUILD_ARGS "$(docker_context_path)" | xargs buildah build --file "$DOCKER_FILE" --tag $DOCKER_SNAPSHOT_IMAGE $buildah_cache_args --build-arg http_proxy="$http_proxy" --build-arg https_proxy="$https_proxy" --build-arg no_proxy="$no_proxy"

    - buildah push --digestfile .img-digest.txt $DOCKER_PUSH_ARGS "$DOCKER_SNAPSHOT_IMAGE"

    # build and push image or manifest

    - |

      buildah_build_target_kind="--tag"

      exp_build_args="$(echo "${DOCKER_BUILD_ARGS}" | tr ' ' '\n')"

      if [[ $(echo "$exp_build_args" | grep -c -- "--all-platforms") -ge 1 || \

            $(echo "$exp_build_args" | grep -c -- "--platform") -ge 2 || \

            $(echo "$exp_build_args" | grep -c -- "--os") -ge 2 || \

            $(echo "$exp_build_args" | grep -c -- "--arch") -ge 2 || \

            $(echo "$exp_build_args" | grep -c -- "--variant") -ge 2 ]]

      then

        log_info "Multiple platforms detected: enabling buildah manifest mode"

        buildah_build_target_kind="--manifest"

        buildah_push_mode="manifest"

      fi

    - echo $DOCKER_METADATA $DOCKER_BUILD_ARGS "$(docker_context_path)" | xargs buildah build --file "$DOCKER_FILE" $buildah_build_target_kind $DOCKER_SNAPSHOT_IMAGE $buildah_cache_args --build-arg http_proxy="$http_proxy" --build-arg https_proxy="$https_proxy" --build-arg no_proxy="$no_proxy"

    - buildah $buildah_push_mode push --digestfile .img-digest.txt $DOCKER_PUSH_ARGS "$DOCKER_SNAPSHOT_IMAGE"

    # display digest of the resulting image

    - cat .img-digest.txt

    # create dotenv file
@@ -1383,7 +1396,7 @@ docker-publish: 
    - BUILDTOOL_HOME=${BUILDTOOL_HOME:-$HOME}

    # 1: push main image

    - log_info "Copying ${DOCKER_SNAPSHOT_IMAGE} to ${DOCKER_RELEASE_IMAGE}..."

    - skopeo copy ${TRACE+--debug} --src-authfile "$BUILDTOOL_HOME/skopeo/.docker/src-config.json" --dest-authfile "$BUILDTOOL_HOME/skopeo/.docker/dest-config.json" ${DOCKER_PUBLISH_ARGS} "docker://$DOCKER_SNAPSHOT_IMAGE" "docker://$DOCKER_RELEASE_IMAGE"

    - skopeo copy ${TRACE+--debug} --all --src-authfile "$BUILDTOOL_HOME/skopeo/.docker/src-config.json" --dest-authfile "$BUILDTOOL_HOME/skopeo/.docker/dest-config.json" ${DOCKER_PUBLISH_ARGS} "docker://$DOCKER_SNAPSHOT_IMAGE" "docker://$DOCKER_RELEASE_IMAGE"

    - |

      if [[ ${DOCKER_COSIGN_STRATEGY} == "onrelease" ]] || [[ ${DOCKER_COSIGN_STRATEGY} == "always" ]]

      then