feat: add buildah support for building images (f8de5636) · Commits · to be continuous... / Docker

README.md

+17 −12

Original line number	Diff line number	Diff line
		@@ -15,16 +15,19 @@ include:

		## Understanding the Docker template

		The template supports two ways of building your Docker images:
		The template supports following ways of building container images:

		1. The former Docker-in-Docker technique, that was widely used for years because of no other alternative, but that
		1. The former Docker-in-Docker (DinD) technique, that was widely used for years because of no other alternative, but that
		is now commonly recognized to have significant security issues ([read this post](https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/) for more info),
		2. Or using [kaniko](https://github.com/GoogleContainerTools/kaniko), an open-source tool from Google for building Docker
		2. Or using [kaniko](https://github.com/GoogleContainerTools/kaniko), an open-source, daemonless tool from Google for building Docker
		images, and that solves Docker-in-Docker security issues (and also speeds-up build times).
		3. Or using [buildah](https://buildah.io/), an open-source, daemonless tool backed by RedHat for building Docker
		images, and that solves Docker-in-Docker security issues (and also speeds-up build times), and can also be configured to run rootless.

		By default, the template uses the [kaniko](https://docs.gitlab.com/ee/ci/docker/using_kaniko.html) way, but you may
		activate the Docker-in-Docker build at your own risks by setting `DOCKER_DIND_BUILD` to `true` (see below).
		:warning: In that case, make sure your runner has required privileges to run Docker-in-Docker ([see GitLab doc](https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#use-docker-in-docker-workflow-with-docker-executor)).
		select an alternate build tool by using the `DOCKER_BUILD_TOOL` variable (see below).

		:warning: If you choose to use 'Docker-in-Docker' option considering the associated security risks, make sure your runner has required privileges to run Docker-in-Docker ([see GitLab doc](https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#use-docker-in-docker-workflow-with-docker-executor)).

		### Global variables

		@@ -32,8 +35,9 @@ The Docker template uses some global configuration used throughout all jobs.

		\| Name \| Description \| Default value \|
		\| --------------------- \| -------------------------------------- \| ----------------- \|
		\| `DOCKER_DIND_BUILD` \| Set to enable Docker-in-Docker build (:warning: unsecured, requires privileged runners). \| _(none)_ (kaniko build by default) \|
		\| `DOCKER_KANIKO_IMAGE` \| The Docker image used to run kaniko - _for kaniko build only_ \| `gcr.io/kaniko-project/executor:debug` (use `debug` images for GitLab) \|
		\| `DOCKER_BUILD_TOOL` \| The build tool to use for building container image, possible values are `kaniko`, `buildah` or `dind` \| `kaniko` \|
		\| `DOCKER_KANIKO_IMAGE` \| The image used to run `kaniko` - _for kaniko build only_ \| `gcr.io/kaniko-project/executor:debug` (use `debug` images for GitLab) \|
		\| `DOCKER_BUILDAH_IMAGE` \| The image used to run `buildah` - _for buildah build only_ \| `quay.io/buildah/stable` \|
		\| `DOCKER_IMAGE` \| The Docker image used to run the docker client (see [full list](https://hub.docker.com/r/library/docker/)) - _for Docker-in-Docker build only_ \| `registry.hub.docker.com/library/docker:latest` \|
		\| `DOCKER_DIND_IMAGE` \| The Docker image used to run the Docker daemon (see [full list](https://hub.docker.com/r/library/docker/)) - _for Docker-in-Docker build only_ \| `registry.hub.docker.com/library/docker:dind` \|
		\| `DOCKER_FILE` \| The path to your `Dockerfile` \| `./Dockerfile` \|
		@@ -216,11 +220,11 @@ You can do so by adding a patch to the `.docker-base` job in your `.gitlab-ci.ym
		DOCKER_RELEASE_IMAGE: "$CI_REGISTRY/$CI_PROJECT_PATH/back:$CI_COMMIT_REF_NAME"
		```

		If you need to redefine a variable with the same value for all your Dockerfiles, you can just declare this variable as a global variable. For example, if you want to build all your images using Docker-in-Docker, you can simply define the `DOCKER_DIND_BUILD` variable as a global variable:
		If you need to redefine a variable with the same value for all your Dockerfiles, you can just declare this variable as a global variable. For example, if you want to build all your images using `buildah`, you can simply define the `DOCKER_BUILD_TOOL` variable as a global variable with value `buildah`:

		```yaml
		variables:
		DOCKER_DIND_BUILD: "true"
		DOCKER_BUILD_TOOL: "buildah"
		```

		### Secrets management
		@@ -282,7 +286,7 @@ In addition to a textual report in the console, this job produces the following
		\| `reports/docker-hadolint-*.native.json` \| native hadolint test report (json) \| [DefectDojo integration](https://defectdojo.github.io/django-DefectDojo/integrations/parsers/#hadolint)<br/>_This report is generated only if DefectDojo template is detected_ \|
		\| `reports/docker-hadolint-*.codeclimate.json` \| hadolint (GitLab) codeclimate format \| [GitLab integration](https://docs.gitlab.com/ee/ci/yaml/artifacts_reports.html#artifactsreportscodequality) \|

		### `docker-build` job
		### `docker-*-build` jobs

		This job builds the image and publishes it to the _snapshot_ repository.

		@@ -290,8 +294,9 @@ It is bound to the `package-build` stage, and uses the following variables:

		\| Name \| Description \| Default value \|
		\| ------------------------------- \| ------------------------------------------------------------------------------------------------------------- \| ------------------------------ \|
		\| `DOCKER_BUILD_ARGS` \| Additional `docker build`/`kaniko` arguments \| _(none)_ \|
		\| `DOCKER_REGISTRY_MIRROR` \| URL of a Docker registry mirror to use during the image build (instead of default `https://index.docker.io`) \| _(none)_ \|
		\| `DOCKER_BUILD_ARGS` \| Additional `docker/kaniko/buildah` `build` arguments \| _(none)_ \|
		\| `DOCKER_REGISTRY_MIRROR` \| URL of a Docker registry mirror to use during the image build (instead of default `https://index.docker.io`) <br>:warning: Used by the `kaniko` and `dind` options only \| _(none)_ \|
		\| `CONTAINER_REGISTRIES_CONFIG_FILE` \| The [`registries.conf`](https://www.redhat.com/sysadmin/manage-container-registries) configuration to be used<br>:warning: Used by the `buildah` build only \| _(none)_ \|
		\| `DOCKER_METADATA` \| Additional `docker build`/`kaniko` arguments to set label \| OCI Image Format Specification \|

		This job produces _output variables_ that are propagated to downstream jobs (using [dotenv artifacts](https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html#artifactsreportsdotenv)):

kicker.json

+24 −13

Original line number	Diff line number	Diff line
		@@ -4,24 +4,36 @@
		"template_path": "templates/gitlab-ci-docker.yml",
		"kind": "package",
		"variables": [
		{
		"name": "DOCKER_BUILD_TOOL",
		"type": "enum",
		"values": ["kaniko", "buildah", "dind"],
		"description": "The build tool to use for building container image",
		"default": "kaniko"
		},
		{
		"name": "DOCKER_KANIKO_IMAGE",
		"description": "The Docker image used to run kaniko\n\n_for kaniko build only_",
		"description": "The image used to run kaniko\n\n_for kaniko build only_",
		"default": "gcr.io/kaniko-project/executor:debug"
		},
		{
		"name": "DOCKER_BUILDAH_IMAGE",
		"description": "The image used to run buildah\n\n_for buildah build only_",
		"default": "quay.io/buildah/stable:latest"
		},
		{
		"name": "DOCKER_IMAGE",
		"description": "The Docker image used to run the docker client\n\n_for Docker-in-Docker build only_",
		"description": "The image used to run the docker client\n\n_for Docker-in-Docker(dind) build only_",
		"default": "registry.hub.docker.com/library/docker:latest"
		},
		{
		"name": "DOCKER_DIND_IMAGE",
		"description": "The Docker image used to run the Docker daemon\n\n_for Docker-in-Docker build only_",
		"description": "The image used to run the Docker daemon\n\n_for Docker-in-Docker(dind) build only_",
		"default": "registry.hub.docker.com/library/docker:dind"
		},
		{
		"name": "DOCKER_SKOPEO_IMAGE",
		"description": "The docker image used to publish docker image with Skopeo",
		"description": "The image used to publish docker image with Skopeo",
		"default": "quay.io/skopeo/stable:latest"
		},
		{
		@@ -52,11 +64,11 @@
		},
		{
		"name": "DOCKER_BUILD_ARGS",
		"description": "Additional docker build/kaniko arguments"
		"description": "Additional docker/kaniko/buildah build arguments"
		},
		{
		"name": "DOCKER_METADATA",
		"description": "Additional docker build/kaniko arguments to set labels",
		"description": "Additional metadata to set as labels",
		"default": "--label org.opencontainers.image.url=${CI_PROJECT_URL} --label org.opencontainers.image.source=${CI_PROJECT_URL} --label org.opencontainers.image.title=${CI_PROJECT_PATH} --label org.opencontainers.image.ref.name=${CI_COMMIT_REF_NAME} --label org.opencontainers.image.revision=${CI_COMMIT_SHA} --label org.opencontainers.image.created=${CI_JOB_STARTED_AT}",
		"advanced": true
		},
		@@ -78,7 +90,12 @@
		},
		{
		"name": "DOCKER_REGISTRY_MIRROR",
		"description": "URL of a Docker registry mirror to use instead of default `https://index.docker.io`"
		"description": "URL of a Docker registry mirror to use instead of default `https://index.docker.io`\n\n_Used by `kaniko` and `dind` builds only_"
		},
		{
		"name": "CONTAINER_REGISTRIES_CONFIG_FILE",
		"description": "The [registries.conf](https://www.redhat.com/sysadmin/manage-container-registries) configuration to be used\n\n_Used by the `buildah` build only_",
		"advanced": true
		}
		],
		"features": [
		@@ -118,12 +135,6 @@
		}
		]
		},
		{
		"id": "dind",
		"name": "Docker-in-Docker",
		"description": "Use Docker-in-Docker to build the image (instead of Kaniko)\n\n_Warning: unsecured, requires privileged runners_",
		"enable_with": "DOCKER_DIND_BUILD"
		},
		{
		"id": "healthcheck",
		"name": "Health Check",

templates/gitlab-ci-docker.yml

+57 −11

Original line number	Diff line number	Diff line
		@@ -21,6 +21,7 @@ workflow:
		when: never
		- when: always


		# test job prototype: implement adaptive pipeline rules
		.test-policy:
		rules:
		@@ -50,6 +51,7 @@ variables:
		DOCKER_DIND_IMAGE: "registry.hub.docker.com/library/docker:dind"
		DOCKER_KANIKO_IMAGE: "gcr.io/kaniko-project/executor:debug"
		DOCKER_SKOPEO_IMAGE: "quay.io/skopeo/stable:latest"
		DOCKER_BUILDAH_IMAGE: "quay.io/buildah/stable:latest"

		# for retro-compatibility (deprecated & undocumented)
		DOCKER_DOCKERFILE_PATH: "."
		@@ -90,6 +92,15 @@ variables:
		--label org.opencontainers.image.revision=${CI_COMMIT_SHA}
		--label org.opencontainers.image.created=${CI_JOB_STARTED_AT}

		# default to kaniko, possible options : kaniko\|buildah\|dind
		DOCKER_BUILD_TOOL:
		value: "kaniko"
		options:
		- "kaniko"
		- "buildah"
		- "dind"
		description: "The build tool to use for building container image"

		# ==================================================
		# Stages definition
		# ==================================================
		@@ -327,15 +338,12 @@ stages:
		docker_release_config_json=$(echo -n "{\"auths\":{\"$docker_release_registry_host\":{\"auth\":\"$docker_release_authent_token\"},\"HttpHeaders\":{\"User-Agent\":\"$USER_AGENT\"}}}")

		# Create the configuration file for Docker and Kaniko
		mkdir -p /root/.docker
		mkdir -p /kaniko/.docker
		mkdir -p "$HOME/.docker"
		if [ -f "${DOCKER_CONFIG_FILE}" ]
		then
		awkenvsubst < "${DOCKER_CONFIG_FILE}" > /root/.docker/config.json
		awkenvsubst < "${DOCKER_CONFIG_FILE}" > /kaniko/.docker/config.json
		awkenvsubst < "${DOCKER_CONFIG_FILE}" > "$HOME/.docker/config.json"
		else
		echo "${docker_snapshot_config_json}" > /root/.docker/config.json
		echo "${docker_snapshot_config_json}" > /kaniko/.docker/config.json
		echo "${docker_snapshot_config_json}" > "$HOME/.docker/config.json"
		fi

		# Create the configuration file for Skopeo
		@@ -405,6 +413,16 @@ stages:
		/kaniko/executor --context "$(docker_context_path)" --dockerfile "$DOCKER_FILE" --destination "$docker_image" --cache --cache-dir="$KANIKO_CACHE_DIR" --verbosity $DOCKER_KANIKO_VERBOSITY $kaniko_registry_mirror_option $DOCKER_METADATA $DOCKER_BUILD_ARGS "$@"
		}

		# Used by containers tools like buildah, skopeo.
		function configure_containers_registries() {
		if [[ -n "$CONTAINER_REGISTRIES_CONFIG_FILE" ]]
		then
		mkdir -p "$HOME/.config/containers"
		echo "${CONTAINER_REGISTRIES_CONFIG_FILE}" > "$HOME/.config/containers/registries.conf"
		log_info "Configured $HOME/.config/containers/registries.conf"
		fi
		}

		init_workspace

		# ENDSCRIPT
		@@ -453,7 +471,7 @@ stages:
		if [[ -n "${DOCKER_REGISTRY_MIRROR}" ]]; then dockerd-entrypoint.sh --registry-mirror ${DOCKER_REGISTRY_MIRROR}; else dockerd-entrypoint.sh; fi \|\| exit
		before_script:
		- *docker-scripts
		- if ! is_runner_dind_capable; then fail "Docker-in-Docker is not enabled on this runner. Either use a Docker-in-Docker capable runner, or disable this job by unsetting \$DOCKER_DIND_BUILD"; fi
		- if ! is_runner_dind_capable; then fail "Docker-in-Docker is not enabled on this runner. Either use a Docker-in-Docker capable runner, or disable this job by setting \$DOCKER_BUILD_TOOL to a different value"; fi

		# ==================================================
		# Stage: build
		@@ -530,8 +548,7 @@ docker-kaniko-build:
		dotenv:
		- docker.env
		rules:
		# execute if $DOCKER_DIND_BUILD not set
		- if: '$DOCKER_DIND_BUILD == null \|\| $DOCKER_DIND_BUILD == ""'
		- if: '$DOCKER_BUILD_TOOL == "kaniko"'

		docker-dind-build:
		extends: .docker-dind-base
		@@ -560,7 +577,36 @@ docker-dind-build:
		dotenv:
		- docker.env
		rules:
		- if: $DOCKER_DIND_BUILD
		- if: '$DOCKER_BUILD_TOOL == "dind"'

		docker-buildah-build:
		extends: .docker-base
		stage: package-build
		image: "$DOCKER_BUILDAH_IMAGE"
		variables:
		BUILDAH_BUILD_CACHE: "$CI_REGISTRY_IMAGE/snapshot/cache"
		script:
		- configure_containers_registries
		# build and push image
		- buildah build --file "$DOCKER_FILE" --tag $DOCKER_SNAPSHOT_IMAGE --layers --cache-from $BUILDAH_BUILD_CACHE --cache-to $BUILDAH_BUILD_CACHE --build-arg http_proxy="$http_proxy" --build-arg https_proxy="$https_proxy" --build-arg no_proxy="$no_proxy" $DOCKER_METADATA $DOCKER_BUILD_ARGS "$(docker_context_path)"
		- buildah push --digestfile .img-digest.txt "$DOCKER_SNAPSHOT_IMAGE"
		# display digest of the resulting image
		- cat .img-digest.txt
		# create dotenv file
		- docker_digest=$(cat .img-digest.txt)
		- docker_repository=${DOCKER_SNAPSHOT_IMAGE%:*}
		- docker_tag=${DOCKER_SNAPSHOT_IMAGE##*:}
		- echo "docker_image=$DOCKER_SNAPSHOT_IMAGE" > docker.env
		- echo "docker_image_digest=$docker_repository@$docker_digest" >> docker.env
		- echo "docker_repository=$docker_repository" >> docker.env
		- echo "docker_tag=$docker_tag" >> docker.env
		- echo "docker_digest=$docker_digest" >> docker.env
		artifacts:
		reports:
		dotenv:
		- docker.env
		rules:
		- if: '$DOCKER_BUILD_TOOL == "buildah"'

		# ==================================================
		# Stage: package-test
		@@ -626,7 +672,7 @@ docker-healthcheck:
		rules:
		- if: '$DOCKER_HEALTHCHECK_DISABLED == "true"'
		when: never
		- if: '$DOCKER_DIND_BUILD == null \|\| $DOCKER_DIND_BUILD == ""'
		- if: '$DOCKER_BUILD_TOOL != "dind"'
		when: never
		- !reference [.test-policy, rules]