An introduction to leveraging multi-stage container image builds.
Your application is deployed via a minimal OCI image, such as one produced by a Docker build. However, its build and test pipeline consists of many stages, each of which utilizes disparate technologies, system dependencies, and testing techniques. Consistently managing the build pipeline – and its dependencies – across local development environments and CI/CD systems is complicated. Perhaps its configuration and dependency management is spread across package.json scripts, Make targets, Github Actions YML, a Jenkinsfile, a homebrew
Dockerfile, and/or Linux package managers with no consistent entrypoint.
A potential solution
Consider leveraging multi-stage
docker image builds and leaning into Docker (or another OCI image technology) as the standard, streamlined entrypoint and lowest common dependency used to consistently build and test your software across all environments.
A basic example
Consider a simple, contrived example in which a
hello application is deployed via a container image.
When built and run, the
hello application prints
docker build --tag hello . [+] Building 0.9s (7/7) FINISHED ...
docker run hello hello
Prior to leveraging multi-stage Docker builds,
hello is built via a minimal
FROM alpine COPY hello.sh /hello.sh ENTRYPOINT ["/hello.sh"]
hello.sh looks like this:
#!/bin/sh echo "hello"
However, before building the
hello image, the CI/CD and local build processes might subject
hello.sh to shellcheck analysis, ensuring that the script conforms to common shell script conventions and contains no syntax errors.
In local development, the installation and invocation of
shellcheck might be managed via a
Makefile. In CI/CD,
shellcheck might be invoked via the action-shellcheck GitHub Action prior to a GitHub Actions-based Docker build.
Rather than manage multiple
shellcheck installation and invocation techniques, perhaps the build process could be streamlined via a multi-stage image build in which
shellcheck is invoked directly from within the
docker build. The following
Dockerfile does so via a
shellchecker stage that must succeed before building the final image:
FROM koalaman/shellcheck-alpine AS shellchecker COPY hello.sh /hello.sh RUN shellcheck /*.sh FROM alpine COPY --from=shellchecker /hello.sh /hello.sh ENTRYPOINT ["/hello.sh"]
In leveraging the multi-stage
dockerbecomes the sole build-and-run-time dependency across all environments
shellcheckis installed and invoked in a standard way across both local development and CI/CD environments; the disparate
Makefile-based and GitHub Actions-based
shellcheckinstallation/invocation code can be deleted
Also note that the final
hello image remains sufficiently minimal;
shellcheck is only installed during the
shellchecker stage and is not present in the final
hello example is fairly simplistic and contrived, multi-stage Docker builds could be far more sophisticated, executing application compilation, unit tests, functional tests, and even integration tests from within the Docker build.
Utilizing multi-stage Docker builds to streamline build and test processes may not be appropriate in all contexts. However, the technique’s worth considering and may simplify many workflows.