Weeknote 17: Caching Docker builds on GitHub Actions

In the past few weeks, I’ve spent a lot of time building Docker images on GitHub Actions. Anyone who has done it knows what it is like: I’ve been waiting a lot.

To avoid waiting, you’ll want to set up build caching. Here’s a few things I learned about it.

Building Docker Compose targets

For one of our services, we have a Docker Compose based setup for running it locally. To ensure that it continues to work, I created a CI workflow with GitHub Actions that builds the images, starts the system, and executes a few tests to check that everything is okay.

The first version of the build step looked like this:

- run: docker compose build

It worked great except for one thing: this does not cache anything between CI jobs. The way to go for caching Docker builds is to use BuildKit’s nice cache backends. But how to use them with Docker Compose?

Docker now has the docker buildx bake command for building the targets defined in a docker-compose.yml file. There’s an official action docker/bake-action for it.

GitHub Actions cache backend (type=gha) is the easiest backend to use as it needs no setup. Use the set input to enable it like this:

- uses: docker/bake-action@v5
  with:
    set: |
      *.cache-from: type=gha
      *.cache-to: type=gha,mode=max      

• The *. wildcard means it will be enabled for all targets.
• If you’re building a multi-stage image, you’ll want to set the cache mode to max with mode=max to cache the earlier stages in addition to the last one.

See the docs for all the settings, but you’ll probably want to use one or both of these:

• To push the images, set push: true.
• To make the images available for local docker commands, set load: true.

The complete test workflow could look something like this:

name: Test the local setup

on: [push]

jobs:
  bake:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: docker/setup-buildx-action@v3
      - uses: docker/bake-action@v5
        with:
          load: true
          set: |
            *.cache-from: type=gha
            *.cache-to: type=gha,mode=max            
      - name: Start the services
        run: docker compose up -d --wait
      - name: Test the services
        run: echo TODO

The registry backend

When building Docker images, you can easily run into the 10GB size limit of GitHub Actions cache. As a workaround, you can use the registry backend (type=registry) which uses a container registry as the cache backend.

Since you’re already on GitHub, the easiest solution can be to use GitHub Container Registry (GitHub Packages), but for example AWS ECR works too.

The setup is a bit more involved - see the docs for your registry on how to allow GitHub Actions to push there.

Gotcha: Multi-arch builds

If you’re caching a multi-arch build, note that it does not work out of the box, at least not with type=registry. You have to jump through hoops to cache it.

Photo: An Independence Day view on the snow-covered lake Vähä-Parikas in Vihti, Finland. It is a color photo.