Skip to content

Commit 0fc251f

Browse files
committed
fix(ci): scope the Next.js build cache sticky disk per branch
The Turbopack persistent build cache disk was keyed on github.event_name alone, so every open PR shared one mutable volume. A sticky disk mount clones the last committed snapshot and commits back last-write-wins, so each PR build restored a cache produced by a different branch. Measured on the same staging commit, two runs minutes apart: the single-writer push disk compiled in 9.3 min, the shared pull_request disk in 14.0 min. Across 22 recent runs, push builds land at 3.5-11 min and PR builds at 13.5-17.8 min. Correctness matters more than the minutes here. turbopackFileSystemCacheForBuild is beta and cross-commit restore is not a documented-supported mode - vercel/next.js#87283 reports stale HTML from a cache built at another commit, with no maintainer answer. Every other cache layer in this repo already isolates by branch: GitHub's cache cannot read sibling branches, and Blacksmith's cache product branch-scopes by default. Only this key didn't. Adds pre/post-build cache size reporting, because whether the cache was warm is otherwise invisible - the disk mounts either way and turbo buffers the build log. Adds ci-cache-cleanup.yml to delete a branch's disk when its PR closes, so per-branch disks track open PRs instead of accumulating at ~5 GB each. Also corrects the 105s-cold/22s-warm comment: those were local numbers and have never reproduced in CI.
1 parent 161b21f commit 0fc251f

2 files changed

Lines changed: 87 additions & 6 deletions

File tree

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
name: CI Cache Cleanup
2+
3+
# The Next.js build cache sticky disk in test-build.yml is keyed per branch, so
4+
# every PR leaves a ~5 GB volume behind. Branches are short-lived; the disks
5+
# aren't. Delete a branch's disk when its PR closes so storage tracks open PRs
6+
# rather than every branch the repo has ever seen.
7+
#
8+
# Only the pull_request-event disk is deleted. The push-event disks belong to
9+
# main/staging/dev and must stay warm.
10+
11+
on:
12+
pull_request:
13+
types: [closed]
14+
15+
concurrency:
16+
group: ci-cache-cleanup-${{ github.head_ref }}
17+
cancel-in-progress: false
18+
19+
permissions:
20+
contents: read
21+
22+
jobs:
23+
delete-nextjs-cache:
24+
name: Delete Next.js build cache disk
25+
# Sticky disks only exist on Blacksmith; in GitHub break-glass mode the
26+
# cache-mount fallback uses actions/cache, which expires on its own.
27+
if: vars.CI_PROVIDER == '' || vars.CI_PROVIDER == 'blacksmith'
28+
runs-on: blacksmith-2vcpu-ubuntu-2404
29+
timeout-minutes: 5
30+
31+
steps:
32+
# Key must stay byte-identical to the Mount Next.js build cache key in
33+
# test-build.yml, or this silently deletes nothing and the disks pile up.
34+
- name: Delete sticky disk
35+
uses: useblacksmith/stickydisk-delete@b41313d28b8647d72114c9ba3c96bb04061562b6 # v1
36+
continue-on-error: true
37+
with:
38+
delete-key: ${{ github.repository }}-nextjs-cache-pull_request${{ github.event.pull_request.head.repo.fork && '-fork' || '' }}-${{ github.head_ref }}

.github/workflows/test-build.yml

Lines changed: 49 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -265,14 +265,31 @@ jobs:
265265

266266
# Turbopack's persistent build cache (NEXT_TURBOPACK_BUILD_CACHE below)
267267
# writes ~5 GB into .next/cache — a sticky disk mounts it in ~1s where an
268-
# actions/cache round-trip would eat the warm-build win. Same event/fork
269-
# namespacing as the other mounts; the GitHub fallback inside cache-mount
270-
# still uses actions/cache with a run_id-suffixed key.
268+
# actions/cache round-trip would eat the warm-build win.
269+
#
270+
# Scoped per branch, not just per event. A sticky disk is a single mutable
271+
# volume per key: mounting clones the last committed snapshot and the job
272+
# commits back last-write-wins. Keyed on event alone, every open PR shared
273+
# one volume, so each run cloned a snapshot built from a *different*
274+
# branch. Measured on the same staging commit, minutes apart: the push
275+
# disk (single-writer, genuinely warm) compiled in 9.3 min while the
276+
# shared pull_request disk took 14.0 min.
277+
#
278+
# Branch scoping is also the correctness-preserving choice —
279+
# turbopackFileSystemCacheForBuild is beta, and cross-commit restore is
280+
# not a documented-supported mode (vercel/next.js#87283 reports stale HTML
281+
# from a cache built at another commit). Every other cache layer here
282+
# already isolates by branch: GitHub's cache can't read sibling branches,
283+
# and Blacksmith's own cache product branch-scopes by default. Only this
284+
# key didn't.
285+
#
286+
# Cost of per-branch disks is bounded by ci-cache-cleanup.yml, which
287+
# deletes a branch's disk when its PR closes.
271288
- name: Mount Next.js build cache
272289
uses: ./.github/actions/cache-mount
273290
with:
274291
provider: ${{ vars.CI_PROVIDER }}
275-
key: ${{ github.repository }}-nextjs-cache-${{ github.event_name }}${{ github.event.pull_request.head.repo.fork && '-fork' || '' }}
292+
key: ${{ github.repository }}-nextjs-cache-${{ github.event_name }}${{ github.event.pull_request.head.repo.fork && '-fork' || '' }}-${{ github.head_ref || github.ref_name }}
276293
path: ./apps/sim/.next/cache
277294

278295
# Running out of RAM kills the whole VM and surfaces only as "the runner
@@ -293,6 +310,20 @@ jobs:
293310
- name: Install dependencies
294311
run: bun install --frozen-lockfile --ignore-scripts
295312

313+
# Whether the Turbopack cache was warm is otherwise invisible: the disk
314+
# mounts successfully either way, and turbo buffers the build log so
315+
# compile time only surfaces at the end. Without this, "is the cache
316+
# working" can only be answered by diffing compile times across runs and
317+
# guessing. Reported, never gated — this is a measurement, not a check.
318+
- name: Report Next.js cache state (pre-build)
319+
run: |
320+
CACHE_DIR=apps/sim/.next/cache
321+
if [ -d "$CACHE_DIR" ] && [ -n "$(ls -A "$CACHE_DIR" 2>/dev/null)" ]; then
322+
echo "PRE_BUILD_CACHE=warm ($(du -sh "$CACHE_DIR" | cut -f1), $(find "$CACHE_DIR" -type f | wc -l) files)"
323+
else
324+
echo 'PRE_BUILD_CACHE=cold (empty or absent — expect a full compile)'
325+
fi
326+
296327
- name: Build application
297328
env:
298329
NODE_OPTIONS: '--no-warnings --max-old-space-size=8192'
@@ -304,7 +335,19 @@ jobs:
304335
AWS_REGION: 'us-west-2'
305336
ENCRYPTION_KEY: '7cf672e460e430c1fba707575c2b0e2ad5a99dddf9b7b7e3b5646e630861db1c' # dummy key for CI only
306337
TURBO_CACHE_DIR: .turbo
307-
# Opt into Turbopack's persistent build cache (beta) for this CI
308-
# check build only — measured 105s cold vs 22s warm locally.
338+
# Opt into Turbopack's persistent build cache (beta) for this CI check
339+
# build only. The 105s-cold/22s-warm figures behind this were measured
340+
# locally and have never reproduced in CI, where compile has ranged
341+
# 3.5-17.8 min — treat them as local numbers, not a CI target.
309342
NEXT_TURBOPACK_BUILD_CACHE: '1'
310343
run: bunx turbo run build --filter=sim
344+
345+
- name: Report Next.js cache state (post-build)
346+
if: always()
347+
run: |
348+
CACHE_DIR=apps/sim/.next/cache
349+
if [ -d "$CACHE_DIR" ]; then
350+
echo "POST_BUILD_CACHE=$(du -sh "$CACHE_DIR" | cut -f1) committed to the sticky disk for the next run on this branch"
351+
else
352+
echo 'POST_BUILD_CACHE=absent — the build wrote no cache'
353+
fi

0 commit comments

Comments
 (0)