Tuning a Build Cache Until It Stopped Lying

For about four months our CI cache had a personality. Mostly it was a good friend: builds dropped from eleven minutes to three. Then, roughly once a week, it handed someone a build that contained last Thursday's code, and everyone spent an afternoon looking at the wrong thing.

We tuned it twice. The first round made it faster. The second round made it honest, and honesty turned out to be where most of the speed was hiding.

What a cache key is for

A cache is a claim: "the inputs haven't changed, so the output hasn't either." The key is the claim's evidence. Ours was built from the branch name plus the date, a choice somebody made on a Friday afternoon and left behind. It said "same branch, same day, same output", which is true until two commits land on the same branch in the same day, and that is every day.

A good key contains everything that can change the output, and nothing that can't. Both halves matter. Too little in the key and you get stale answers. Too much and you get misses, because a key that includes the commit hash is a key that never matches.

Hash the contents, not the clock

Timestamps lie in both directions. A fresh checkout gives every file a new modification time, so a timestamp-keyed cache misses on files that haven't changed. A tool that preserves old times can hide a real edit. Content doesn't have that problem. This script keys on the bytes of the sources, the lockfile, the tool version and any flags:

#!/bin/bash
# Key the cache on everything that can change the output, nothing else.
KEY=$( { cat src/*.js lock.json tool.version; echo "$BUILD_FLAGS"; } | shasum -a 256 | cut -c1-12 )
if [ -d ".cache/$KEY" ]; then
  echo "hit  $KEY"
else
  echo "miss $KEY"
  mkdir -p ".cache/$KEY" && sleep 2 && cp src/app.js ".cache/$KEY/out.js"
fi

The sleep 2 stands in for a real build. On Linux, use sha256sum where I used shasum -a 256.

I ran it six times to see whether the key behaves. First run: a miss. Second: a hit. Then I ran touch on a source file, which changes the timestamp and nothing else, and it was still a hit. Appending a comment line to the file produced a new key and a miss. Setting BUILD_FLAGS=--minify produced a different key again, so a minified build can't be served to someone who asked for a plain one. Finally the plain run hit the entry from the edit, which is the cache doing its job.

The three questions we ask now

Before anything goes into the key, three questions decide whether it belongs:

1. Can this change the output? If not, leave it out.

2. Is it something the build reads, or something we assumed it reads? Check by deleting it and rebuilding.

3. Can I hash it cheaply? A key that takes a minute to compute isn't a speedup.

The lockfile passed all three. The branch name failed the first one. The compiler version was the surprise. We had never put it in the key, and question one answers itself once you ask it: of course a different compiler can change the output. That omission explained at least two of the weekly ghosts.

Results, with a caveat

Median build time fell from three minutes to just under two, because the old key missed far more often than anyone guessed. Stale builds stopped. These are our numbers on our pipeline, not a benchmark. Measure yours before and after, and keep the old key running in a shadow job for a week, just to see how often the two disagree.