Multi-Stage Docker Builds on CircleCI 2.0

Docker’s multi-stage builds, introduced in 2017, have become an essential feature for creating efficient and streamlined Docker images. By allowing multiple FROM statements within a single Dockerfile, multi-stage builds enable the separation of build and runtime environments, resulting in smaller and more secure images.

With a focus on Golang applications, this tutorial explores how to leverage multi-stage Docker builds on CircleCI 2.1 to optimize your CI/CD pipelines. We’ll cover the benefits of multi-stage builds, demonstrate their implementation, and provide practical examples for deploying a sample Golang project on CircleCI.

Also check out guide to using Docker for your CI/CD pipelines.

Understanding multi-stage Docker builds

In traditional Docker workflows, it’s common to use separate Dockerfiles for building the application and for running it. Multi-stage builds consolidate these steps into one Dockerfile, reducing complexity and improving maintainability.

Here’s an example of a multi-stage Dockerfile for a Go application:

# First Stage: Build the Go application
FROM golang:1.23 AS builder
WORKDIR /app

# Copy application source code
COPY . .

# Build the Go application
RUN go build -o main .

# Second Stage: Create a minimal runtime image
FROM alpine:latest
WORKDIR /root/

# Install CA certificates
RUN apk --no-cache add ca-certificates

# Copy the compiled binary from the first stage
COPY --from=builder /app/main .

# Expose port 8080 and run the application
EXPOSE 8080
CMD ["./main"]

In this Dockerfile, the first stage (builder) compiles the Go application using the official Go image. The second stage starts with a minimal Alpine image, adds necessary certificates, and copies the compiled binary from the builder stage. This approach ensures that the final image contains only the necessary runtime dependencies, leading to a smaller and more secure image.

Implementing multi-stage builds on CircleCI 2.1

To build multi-stage Dockerfiles on CircleCI 2.1, ensure that your build environment has a Docker version that supports multi-stage builds. Most modern CircleCI environments already include a compatible version by default.

Using the docker executor

The docker executor in CircleCI allows you to specify the Docker image version directly in your configuration. Using cimg/base:stable ensures that CircleCI selects a compatible version automatically. When using the stable base image, you don’t need to specify a version for setup_remote_docker in your configuration.

version: 2.1

jobs:
  build:
    docker:
      - image: cimg/base:stable
    steps:
      - setup_remote_docker
      - checkout
      - run:
          name: Build Docker Image
          command: |
            docker build -t myapp .

Using the machine executor

Alternatively, the machine executor provides a dedicated virtual machine for your builds, offering more control over the environment. Here’s how to configure it:

version: 2.1

jobs:
  build:
    machine:
      image: ubuntu-2004:202201-02
    steps:
      - checkout
      - run:
          name: Install Docker
          command: |
            curl -fsSL https://get.docker.com -o get-docker.sh
            sh get-docker.sh
      - run:
          name: Build Docker Image
          command: |
            docker build -t myapp .

In this setup:

• The machine executor uses an Ubuntu 20.04 image.
• Docker is installed manually within the build job, ensuring you have the desired Docker version that supports multi-stage builds.

Enhancing build performance with BuildKit

Docker’s BuildKit is a modern build subsystem that offers improved performance and advanced features for Docker builds. Enabling BuildKit can further optimize your multi-stage builds on CircleCI.

To enable BuildKit in your CircleCI configuration:

version: 2.1

jobs:
  build:
    docker:
      - image: cimg/base:stable
    environment:
      DOCKER_BUILDKIT: 1
    steps:
      - setup_remote_docker
      - checkout
      - run:
          name: Build Docker Image with BuildKit
          command: |
            docker build -t myapp .

Setting the environment variable DOCKER_BUILDKIT to 1 enables BuildKit for your builds, potentially reducing build times and providing access to advanced features.

Creating a Sample Golang Project

Create a directory for the project:

mkdir golang-multistage-docker
cd golang-multistage-docker

Initialize a Go module:

go mod init github.com/CIRCLECI-GWP/golang-multistage-docker

Create a main.go file:

package main

import (
	"fmt"
	"log"
	"net/http"
)

func handler(w http.ResponseWriter, r *http.Request) {
	fmt.Fprintf(w, "Hello, World! Multi-stage builds on CircleCI!")
}

func main() {
	http.HandleFunc("/", handler)
	log.Println("Server running on port 8080")
	log.Fatal(http.ListenAndServe(":8080", nil))
}

Ensure you use the Dockerfile created earlier for this project.

Next, create a main_test.go file to add a basic unit test:

package main

import (
	"net/http"
	"net/http/httptest"
	"testing"
)

func TestHandler(t *testing.T) {
	req := httptest.NewRequest("GET", "/", nil)
	w := httptest.NewRecorder()
	handler(w, req)

	if w.Body.String() != "Hello, World! Multi-stage builds on CircleCI!" {
		t.Errorf("Expected response body to be 'Hello, World! Multi-stage builds on CircleCI!' but got %s", w.Body.String())
	}
}

Setting up CircleCI

CircleCI is a CI/CD tool that automates workflows, runs tests, and much more.

Create a .circleci folder at the root of your project and add a config.yml file to it. The config.yml file determines the execution of your pipeline. Add this:

version: 2.1

jobs:
  build:
    docker:
      - image: cimg/go:1.23.4
    steps:
      - checkout
      - run:
          name: Run Unit Tests
          command: go test ./... -v

  docker-build:
    docker:
      - image: cimg/base:stable
    steps:
      - setup_remote_docker
      - checkout
      - run:
          name: Build and Tag Docker Image
          command: docker build -t myapp .
      - run:
          name: List Docker Images
          command: docker images

workflows:
  version: 2
  build-and-docker:
    jobs:
      - build
      - docker-build

This configuration defines two jobs: build for running unit tests and docker-build for building the Docker image. The workflows section specifies the order in which these jobs are executed.

For this Golang project, we use the Docker executor in CircleCI. This executor provides a lightweight and flexible environment for running builds inside Docker containers. Alternatively, the machine executor can be used if you require full control over the environment.

Pushing to GitHub and setting up CircleCI

Commit and push your local changes to GitHub.

Next, sign in to your CircleCI account to set up the project. Click Set Up Project next to the name of your remote repository golang-multistage-docker.

Select fastest, then click Set Up Project.

The build job will then be run by CircleCI, and if everything is okay you will have a green build.

Click the docker-build job to view the Docker build step in CircleCI. You should see the multi-stage build process successfully executed

Expand it to see the details of the build process.

Conclusion

By leveraging multi-stage builds and BuildKit on CircleCI 2.1, you can create efficient, maintainable, and performant CI/CD pipelines for your Dockerized applications!

The complete source code for this tutorial is available on GitHub.