From 38a9150c46ec0390856b5485d6bfd2ce5abd4ed5 Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Wed, 29 Jul 2026 18:26:43 -0700 Subject: [PATCH 1/2] perf(ci): disable the Turbopack persistent build cache MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit It is a net loss at this app's size. A controlled A/B on one branch (#6078), three runs with a byte-identical module graph so only cache state varied: cache OFF 113s compile, 2m53s job cache ON, cold 162s compile, 3m54s job cache ON, warm 360s compile, 8m18s job The cache made the same build 3.2x slower. It also grew 5.1 GB -> 12 GB across two runs of an unchanged tree, which explains the progressive degradation seen on longer-lived disks (up to 11.7 min): the more a disk is written, the more the next run must read and revalidate. Flag manipulation is visible in the logs — the cache-on runs print `✓ turbopackFileSystemCacheForBuild`, the cache-off run omits it — and every run used `turbo --force` so none is a replayed log. #5869 enabled this on locally-measured numbers (105s cold -> 22s warm) that never reproduced in CI and are inverted here. #6072 then branch-scoped the disk to stop PRs restoring each other's caches; that fixed a real problem, but with the cache off the disk is unnecessary, so the mount, the pre/post size reporting, and the env gate all go with it. Pins `turbopackFileSystemCacheForBuild: false` explicitly rather than relying on the Next default: upstream already flips that default to true in canary/preview builds (vercel/next.js#94616), so leaning on the default would let a version bump silently re-enable this. Keeps ci-cache-cleanup.yml, re-scoped to draining the 5-12 GB volumes that PRs opened while the per-branch key was live still hold — nothing else reclaims them. It is a no-op for new PRs and can be deleted once drained. Caveat: n=1 per cell. The 3.2x effect size and agreement with ~15 prior observations make it convincing, but this is three runs, not a distribution. --- .github/workflows/ci-cache-cleanup.yml | 12 +++++--- .github/workflows/test-build.yml | 40 ++++---------------------- apps/sim/next.config.ts | 17 ++++++++--- 3 files changed, 27 insertions(+), 42 deletions(-) diff --git a/.github/workflows/ci-cache-cleanup.yml b/.github/workflows/ci-cache-cleanup.yml index d6426b250c5..761dab94706 100644 --- a/.github/workflows/ci-cache-cleanup.yml +++ b/.github/workflows/ci-cache-cleanup.yml @@ -1,9 +1,13 @@ name: CI Cache Cleanup -# test-build.yml keys the Next.js build cache sticky disk per branch, so every PR -# leaves a ~5 GB volume behind. Branches are short-lived; the disks aren't. Only -# the pull_request disks are reclaimed — the push disks belong to main/staging/dev -# and must stay warm. +# DRAINING LEGACY DISKS ONLY. test-build.yml no longer mounts a Next.js build +# cache — the Turbopack persistent cache measured 3.2x SLOWER than no cache, so it +# is off. But every PR open while the per-branch key was live left a 5-12 GB volume +# behind, and nothing else reclaims them. This keeps deleting them as those PRs +# close. +# +# Delete this workflow once the backlog is drained (no PR predating the cache +# removal is still open). It is a no-op for new PRs, which never create a disk. on: pull_request: diff --git a/.github/workflows/test-build.yml b/.github/workflows/test-build.yml index 192337458aa..16019578e26 100644 --- a/.github/workflows/test-build.yml +++ b/.github/workflows/test-build.yml @@ -263,24 +263,12 @@ jobs: key: ${{ github.repository }}-turbo-cache-build-${{ github.event_name }}${{ github.event.pull_request.head.repo.fork && '-fork' || '' }} path: ./.turbo - # Turbopack's persistent build cache (NEXT_TURBOPACK_BUILD_CACHE below) - # writes ~5 GB into .next/cache — a sticky disk mounts it in ~1s where an - # actions/cache round-trip would eat the warm-build win. - # - # Keyed per branch, not just per event. A sticky disk is one mutable volume - # per key: mounting clones the last committed snapshot, job end commits back - # last-write-wins. An event-only key had every open PR restoring a cache - # built from a different branch — 14.0 min vs 9.3 min for the single-writer - # push disk on the same commit. Branch scoping also keeps us off - # cross-commit restore, which turbopackFileSystemCacheForBuild (beta) does - # not document as supported (vercel/next.js#87283: stale HTML from a cache - # built at another commit). ci-cache-cleanup.yml reclaims the disks. - - name: Mount Next.js build cache - uses: ./.github/actions/cache-mount - with: - provider: ${{ vars.CI_PROVIDER }} - key: ${{ github.repository }}-nextjs-cache-${{ github.event_name }}${{ github.event.pull_request.head.repo.fork && '-fork' || '' }}-${{ github.head_ref || github.ref_name }} - path: ./apps/sim/.next/cache + # No `.next/cache` mount: the Turbopack persistent build cache is off. A + # controlled A/B on one branch (PR #6078) with a byte-identical module graph + # measured compile at 113s with the cache off, 162s cold with it on, and + # 360s warm — the cache made the same build 3.2x slower, and it grew + # 5.1 GB -> 12 GB across two runs of an unchanged tree, so a disk degrades + # the more it is used. Mounting a disk nothing reads would only cost storage. # Running out of RAM kills the whole VM and surfaces only as "the runner # has received a shutdown signal" — no mention of memory, ~12 min in. Warn @@ -300,13 +288,6 @@ jobs: - name: Install dependencies run: bun install --frozen-lockfile --ignore-scripts - # The disk mounts successfully whether or not it carried anything and turbo - # buffers the build log, so cache warmth is otherwise unobservable — #5859 - # shipped a cache that carried almost nothing and it took a PR to notice. - # Reported, never gated. - - name: Report Next.js cache size (pre-build) - run: du -sh apps/sim/.next/cache 2>/dev/null || echo 'cold — no cache restored' - - name: Build application env: NODE_OPTIONS: '--no-warnings --max-old-space-size=8192' @@ -318,13 +299,4 @@ jobs: AWS_REGION: 'us-west-2' ENCRYPTION_KEY: '7cf672e460e430c1fba707575c2b0e2ad5a99dddf9b7b7e3b5646e630861db1c' # dummy key for CI only TURBO_CACHE_DIR: .turbo - # Opt into Turbopack's persistent build cache (beta) for this CI check - # build only. #5869's 105s-cold/22s-warm was measured locally and has - # never reproduced in CI (compile has ranged 3.5-17.8 min) — local - # numbers, not a CI target. - NEXT_TURBOPACK_BUILD_CACHE: '1' run: bunx turbo run build --filter=sim - - - name: Report Next.js cache size (post-build) - if: always() - run: du -sh apps/sim/.next/cache 2>/dev/null || echo 'no cache written' diff --git a/apps/sim/next.config.ts b/apps/sim/next.config.ts index d7d32d3dc5b..463c5683ce9 100644 --- a/apps/sim/next.config.ts +++ b/apps/sim/next.config.ts @@ -154,11 +154,20 @@ const nextConfig: NextConfig = { experimental: { turbopackFileSystemCacheForDev: false, /** - * Turbopack's persistent build cache (beta) — opt-in via env so only the - * CI check build uses it; production image builds stay on the default - * cold-build path until the feature stabilizes. + * Turbopack's persistent build cache (beta) stays off — it is a net loss at + * this app's size. A controlled A/B on a byte-identical module graph (PR + * #6078) measured compile at 113s with it off, 162s cold with it on, and 360s + * warm: the cache made the same build 3.2x slower. It also grew 5.1 GB -> + * 12 GB across two runs of an unchanged tree, so a cache degrades the longer + * it lives. Restoring across commits is separately undocumented-as-supported + * (vercel/next.js#87283 reports stale HTML from a cache built elsewhere). + * + * Pinned explicitly rather than left to the Next default: upstream already + * flips this default to true in canary/preview builds (vercel/next.js#94616), + * so relying on the default would let a version bump silently re-enable a + * config we measured as harmful. */ - turbopackFileSystemCacheForBuild: process.env.NEXT_TURBOPACK_BUILD_CACHE === '1', + turbopackFileSystemCacheForBuild: false, preloadEntriesOnStart: false, /** * Under Turbopack this is not a no-op: the list feeds From e539058b706f60a75889d7560d2d835110b86233 Mon Sep 17 00:00:00 2001 From: Waleed Latif Date: Wed, 29 Jul 2026 18:58:21 -0700 Subject: [PATCH 2/2] docs(ci): correct the cleanup key comment after the mount was removed Greptile P2: the delete step still claimed its key must stay byte-identical to the Mount Next.js build cache step in test-build.yml, but this PR removes that mount. It is now a hard-coded legacy drain key that mirrors nothing. --- .github/workflows/ci-cache-cleanup.yml | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/.github/workflows/ci-cache-cleanup.yml b/.github/workflows/ci-cache-cleanup.yml index 761dab94706..2888e7f0aee 100644 --- a/.github/workflows/ci-cache-cleanup.yml +++ b/.github/workflows/ci-cache-cleanup.yml @@ -26,9 +26,12 @@ jobs: timeout-minutes: 5 steps: - # Must stay byte-identical to the Mount Next.js build cache key in - # test-build.yml, or this deletes nothing and the disks accumulate. - # Non-blocking: PRs skipped by ci.yml's paths-ignore never made a disk. + # A hard-coded legacy drain key. It no longer mirrors anything — the + # Mount Next.js build cache step it used to match was removed with the + # cache. Do not retarget or delete it while PRs from before that removal + # are still open, or their 5-12 GB disks are never reclaimed. + # Non-blocking: PRs skipped by ci.yml's paths-ignore never made a disk, + # and neither does any PR opened after the removal. - name: Delete sticky disk uses: useblacksmith/stickydisk-delete@b41313d28b8647d72114c9ba3c96bb04061562b6 # v1 continue-on-error: true