Multi-stage Docker Builds

An introduction to leveraging multi-stage container image builds.

Problem

Your application is deployed via a minimal OCI image, such as one produced by a Docker build. However, its build and test pipeline consists of many stages, each of which utilizes disparate technologies, system dependencies, and testing techniques. Consistently managing the build pipeline – and its dependencies – across local development environments and CI/CD systems is complicated. Perhaps its configuration and dependency management is spread across package.json scripts, Make targets, Github Actions YML, a Jenkinsfile, a homebrew Brewfile, a Dockerfile, and/or Linux package managers with no consistent entrypoint.

A potential solution

Consider leveraging multi-stage docker image builds and leaning into Docker (or another OCI image technology) as the standard, streamlined entrypoint and lowest common dependency used to consistently build and test your software across all environments.

A basic example

Consider a simple, contrived example in which a hello application is deployed via a container image.

When built and run, the hello application prints hello:

docker build --tag hello .
[+] Building 0.9s (7/7) FINISHED
...

docker run hello
hello

Before

Prior to leveraging multi-stage Docker builds, hello is built via a minimal Dockerfile:

FROM alpine

COPY hello.sh /hello.sh

ENTRYPOINT ["/hello.sh"]

…where hello.sh looks like this:

#!/bin/sh

echo "hello"

However, before building the hello image, the CI/CD and local build processes might subject hello.sh to shellcheck analysis, ensuring that the script conforms to common shell script conventions and contains no syntax errors.

In local development, the installation and invocation of shellcheck might be managed via a Makefile. In CI/CD, shellcheck might be invoked via the action-shellcheck GitHub Action prior to a GitHub Actions-based Docker build.

After

Rather than manage multiple shellcheck installation and invocation techniques, perhaps the build process could be streamlined via a multi-stage image build in which shellcheck is invoked directly from within the docker build. The following Dockerfile does so via a shellchecker stage that must succeed before building the final image:

FROM koalaman/shellcheck-alpine AS shellchecker

COPY hello.sh /hello.sh

RUN shellcheck /*.sh

FROM alpine

COPY --from=shellchecker /hello.sh /hello.sh

ENTRYPOINT ["/hello.sh"]

In leveraging the multi-stage Dockerfile:

docker becomes the sole build-and-run-time dependency across all environments
shellcheck is installed and invoked in a standard way across both local development and CI/CD environments; the disparate Makefile-based and GitHub Actions-based shellcheck installation/invocation code can be deleted

Also note that the final hello image remains sufficiently minimal; shellcheck is only installed during the shellchecker stage and is not present in the final hello image.

Summary

While the hello example is fairly simplistic and contrived, multi-stage Docker builds could be far more sophisticated, executing application compilation, unit tests, functional tests, and even integration tests from within the Docker build.

Utilizing multi-stage Docker builds to streamline build and test processes may not be appropriate in all contexts. However, the technique’s worth considering and may simplify many workflows.