Skip to content

feat(mobile): stable native build keys and CI-offloaded native builds - #5629

Merged
iscekic merged 7 commits into
mainfrom
feat/mobile-native-build-offload
Aug 28, 2026
Merged

feat(mobile): stable native build keys and CI-offloaded native builds#5629
iscekic merged 7 commits into
mainfrom
feat/mobile-native-build-offload

Conversation

@iscekic

@iscekic iscekic commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Symptom

The machine-global native build caches almost never hit. Evidence from the e2e host:

  • 23 iOS producer locks since Aug 19 with 23 distinct cache keys — every build was a full xcodebuild.
  • 15 Android entries under 15 distinct keys, including byte-identical APKs (same artifactChecksum) stored under different keys.

Cause

hashRootNativeInputs hashes all of pnpm-lock.yaml. The lockfile changed in 14 commits over the last 14 days, so any dependency change anywhere in the monorepo re-keys mobile native builds whose inputs did not change. (The generated ios//android trees were checked and are not part of the hash — the earlier prebuild-tree-cache diagnosis attributed the churn to them, but @expo/fingerprint never included them.)

Fix

  1. Key on true inputs. Hash only the apps/mobile importer block of pnpm-lock.yaml. The block pins the full resolved versions of every direct dependency, which is exactly what autolinking can see. A missing block falls back to the whole file. Pin ios/**/android/** out of the fingerprint so no future @expo/fingerprint version can key builds on generated output.
  2. Offload the build to CI. New mobile-native-build.yml builds the debug dev-client and uploads it as an artifact named mobile-native-<platform>-<nativeHash>. It runs on main pushes that can change native inputs and on workflow_dispatch for any pushed branch. A gate job skips platforms whose artifact already exists; the iOS job fails if the hash differs between Linux and macOS (a divergence would publish artifacts nobody can find).
  3. Fetch before compiling. On a local cache miss, dev:mobile:{ios,android} build now downloads that artifact — and dispatches a run for the current pushed branch when none exists — before compiling locally. The fetched binary passes the existing publish pipeline unchanged: bundle-id/package-id check, checksum, manifest, atomic rename. Any remote failure falls back to today's local build. KILO_REMOTE_NATIVE=off disables it.

Why this matters

A native build is the expensive part of an e2e slot; installing a cached binary is cheap. With stable keys plus CI-built artifacts, agent hosts stop paying local xcodebuild/gradle for unchanged native inputs, which frees CPU for more concurrent e2e slots.

Tests

  • pnpm run test:mobile-workflow — 104 pass (new: remote-fetch publish path, remote-miss fallback, lockfile-slice keying).
  • Hash stability proven empirically: web-only lockfile changes no longer re-key; apps/mobile importer changes do.
  • dev/local/tmux.test.ts has one pre-existing failure on this host on clean origin/main (timing-sensitive; unrelated).

Native build caches almost never hit: hashRootNativeInputs hashes all of
pnpm-lock.yaml, so every dependency change anywhere in the monorepo re-keys
mobile native builds whose inputs did not change. The machine cache held 15
Android entries under 15 distinct keys, with byte-identical APKs stored
under different keys, and 23 iOS producer locks with 23 distinct keys.

- Hash only the apps/mobile importer block of pnpm-lock.yaml. The block
  pins the full resolved versions of every direct dependency, which is
  what autolinking can see. Fall back to the whole file when the block
  is missing.
- Pin the generated ios/ and android/ trees out of the fingerprint with
  ignorePaths. The current @expo/fingerprint already excludes them; the
  pin keeps any future version from keying builds on generated output.
- Add .github/workflows/mobile-native-build.yml: build the debug
  dev-client on CI and upload it as an artifact named
  mobile-native-<platform>-<nativeHash>. Runs on main pushes that can
  change native inputs and on workflow_dispatch for any pushed branch.
  A gate job skips platforms whose artifact already exists, and the iOS
  job fails if the hash differs between Linux and macOS.
- On a local cache miss, fetch that artifact (and dispatch a build for
  the current pushed branch when none exists) before compiling locally.
  The fetched app passes the existing publish pipeline: bundle-id check,
  checksum, manifest, atomic rename. Any remote failure falls back to
  the local build. Set KILO_REMOTE_NATIVE=off to disable.
@iscekic iscekic self-assigned this Aug 28, 2026
Comment thread .github/workflows/mobile-native-build.yml
Comment thread .github/workflows/mobile-native-build.yml
Comment thread dev/local/mobile-remote-native.ts Outdated
@kilo-code-bot

kilo-code-bot Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Files Reviewed (1 files)
  • .github/workflows/mobile-native-build.yml
Previous Review Summaries (4 snapshots, latest commit 672e68d)

Current summary above is authoritative. Previous snapshots are kept for context only.

Previous review (commit 672e68d)

Status: No Issues Found | Recommendation: Merge

Files Reviewed (1 files)
  • .github/workflows/mobile-native-build.yml

Previous review (commit 984e620)

Status: No Issues Found | Recommendation: Merge

Files Reviewed (4 files)
  • .github/workflows/mobile-native-build.yml
  • dev/local/mobile-ios-build.ts
  • dev/local/mobile-remote-native.test.ts
  • dev/local/mobile-remote-native.ts

Previous review (commit 4d7632b)

Status: No Issues Found | Recommendation: Merge

Files Reviewed (3 files)
  • .github/workflows/mobile-native-build.yml
  • dev/local/mobile-remote-native.test.ts
  • dev/local/mobile-remote-native.ts

Previous review (commit 6d675cc)

Status: 3 Issues Found | Recommendation: Address before merge

Overview

Severity Count
CRITICAL 1
WARNING 2
SUGGESTION 0

Fix these issues in Kilo Cloud

Issue Details (click to expand)

CRITICAL

File Line Issue
.github/workflows/mobile-native-build.yml 204 Android job has no Android SDK

WARNING

File Line Issue
.github/workflows/mobile-native-build.yml 30 PR builds share the production artifact namespace and skip self-validation
dev/local/mobile-remote-native.ts 167 Dispatched run matching can attach to the wrong workflow_dispatch
Files Reviewed (10 files)
  • .github/workflows/mobile-native-build.yml - 2 issues
  • dev/local/mobile-android-build.test.ts
  • dev/local/mobile-android-build.ts
  • dev/local/mobile-android.ts
  • dev/local/mobile-ios-build.test.ts
  • dev/local/mobile-ios-build.ts
  • dev/local/mobile-native-build.test.ts
  • dev/local/mobile-native-build.ts
  • dev/local/mobile-native-hash.ts
  • dev/local/mobile-remote-native.ts - 1 issue

Reviewed by grok-4.6 · Input: 98.2K · Output: 7.6K · Cached: 174.1K

Review guidance: REVIEW.md from base branch main

… run

Review follow-ups on the native build offload:

- A pull_request run of mobile-native-build now always builds both
  platforms and uploads under a pr<N>- artifact prefix. Before, the gate
  saw the main artifact for the same nativeHash and skipped every job, so
  the self-validation validated nothing; on a miss it published into the
  namespace hosts install from.
- The dispatch watcher now takes the newest run that did not exist before
  the dispatch and that matches HEAD, instead of the newest run created
  within a two-minute window. The old window could attach to another
  host's concurrent dispatch or to a run that started before this one.
Three follow-ups on the native build offload:

- The iOS artifact now carries a toolchain.json with the Xcode build
  version and simulator SDK that produced it, and the fetcher refuses an
  artifact whose toolchain differs from the host's. The artifact name keys
  on nativeHash alone, but the local cache key also keys on the toolchain,
  so a host used to publish a runner-built .app under its own key. When
  the runner's simulator SDK is newer than the host's runtime, that app
  never launches, and the entry sticks under a key the host keeps
  recomputing. It now falls back to a local build instead.
- The macOS job checks the android nativeHash against the gate as well as
  the iOS one. Android builds on Linux but is consumed on macOS, and a
  divergence there would make every android fetch miss in silence.
- The dispatch path logs how long it waited on CI. Waiting only pays while
  it stays under a local compile; log it so that stays measurable.
A pull_request build validates the workflow and its artifact is scoped to
the PR, so a superseded one must not hold a macOS runner. A push build
produces an artifact hosts install, so it still runs to completion.
The pull_request gate forced both platform builds on every push, so each
commit to a workflow-editing PR spent a macOS runner it did not need. Key
the PR artifact on the workflow file as well as the PR number and let the
normal exists() check decide: the first push of a given workflow version
builds, a re-push of the same file skips, and an edit builds again. PR
artifacts now expire after a day, since only a push build is installed.
setup-java v4 is deprecated, and both it and upload-artifact v5 target
Node 20, which the runner now forces onto Node 24. Bump to setup-java
v5.7.0 and upload-artifact v7.0.1, both runs.using: node24. v7.0.1 is
already used elsewhere in this repo. Neither changes the inputs used
here; both need runner 2.327.1 or newer, which GitHub-hosted satisfies.
@iscekic iscekic added the human-ready The PR is ready for human review. label Aug 28, 2026
@iscekic
iscekic merged commit 0535b82 into main Aug 28, 2026
18 of 20 checks passed
@iscekic
iscekic deleted the feat/mobile-native-build-offload branch August 28, 2026 12:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

human-ready The PR is ready for human review.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants