Skip to content

Nightly macOS binary has invalid code signature (hole-punch runs after signing) → cli upgrade SIGKILL / "Setup failed with exit code 1" #1033

@sergical

Description

@sergical

Summary

On the nightly channel, sentry cli upgrade fails on macOS with Error: Setup failed with exit code 1. Root cause: the published macOS binary has an invalid code signature because script/build.ts hole-punches the binary (binpunch) after fossilize has already code-signed + notarized it. macOS AMFI then SIGKILLs the binary (exit 137) when the upgrade flow spawns it.

This is CLI-Q1 (project sentry/cli): 90 events / 55 users, first seen 2026-03-27, still unresolved, last seen 2026-05-28. It affects every nightly-channel macOS user who upgrades.

Root cause

script/build.tscompileAllTargets()postProcessTarget():

  1. fossilize compiles the Node SEA binary and signs + notarizes it in one atomic step (FOSSILIZE_SIGN=y on main/release, via rcodesign sign --for-notarization -e entitlements.plist then rcodesign notary-submit --wait). — build.ts:320
  2. Then binpunch.processBinary(outfile) hole-punches unused ICU data to improve compression — build.ts:388. This mutates bytes the signature already covers.
  3. The hole-punched binary is gzipped and published to GHCR as the nightly artifact.

Result: the shipped binary is signed, then modified:

$ codesign --verify --verbose=2 sentry
sentry: invalid signature (code or signature have been modified)
$ ./sentry --version
Killed: 9        # exit 137 (SIGKILL by AMFI)

Ad-hoc re-signing the same downloaded bytes makes it run again, confirming the bytes are intact and only the signature is stale:

$ codesign --force --sign - sentry && ./sentry --version
0.35.0-dev.1779483941   # runs fine

Why CI doesn't catch it

darwin binaries are signed with rcodesign (which works cross-platform, even on Linux runners), but the smoke test only runs on executable targets — the signed+hole-punched darwin-arm64 binary is never actually executed on macOS in CI, so the SIGKILL is invisible. (The comment at build.ts:387, "Always runs so the smoke test exercises the same binary as the release," is the opposite of what happens on the one platform that hard-fails.)

How it surfaces in upgrade

runSetupOnNewBinary() (src/commands/cli/upgrade.ts:514) spawns the freshly downloaded binary to run cli setup. macOS kills it (137); the parent only sees a non-zero child exit and throws the opaque Setup failed with exit code 1. Worse, the old binary has already been moved aside, so the install is left bricked with no rollback (only sentry.download remains, off PATH).

Two problems to fix

1. Build pipeline (root cause) — hole-punch must happen before signing. Options discussed:

  • Reorder in build.ts: run fossilize --no-sign, hole-punch, then sign + notarize in build.ts (replicating rcodesign sign --for-notarization -e entitlements.plist + notary-submit --wait, reusing fossilize's entitlements.plist). Keeps the ICU size win; moves Apple signing into the consumer.
  • Skip hole-punch on signed builds: one-line guard (if (!signing) processBinary(...)). Simplest/safest; cost is larger signed macOS/Windows binaries and bigger delta patches.
  • Fix upstream in fossilize: add a pre-sign transform hook so hole-punch runs inside fossilize's pipeline before signing. Cleanest architecturally.

Whichever path, add a CI gate that runs the darwin-arm64 binary (and/or rcodesign verify) on a real macOS arm64 runner so a broken signature can never ship again. Note: Windows binaries are Authenticode-signed and hole-punched too — they're likely also invalidly signed (Windows just doesn't hard-fail execution the way macOS AMFI does), so the fix should cover Windows as well.

2. Upgrade resilience (defensive) — independent of the signing fix:

  • Verify the new binary actually runs (or rcodesign verify) before removing/replacing the current binary; roll back on failure.
  • Detect SIGKILL/137 specifically and surface a meaningful error (e.g. "downloaded binary failed code-signature verification") instead of Setup failed with exit code 1.

Environment

  • macOS 26.5, Apple Silicon (arm64)
  • CLI 0.35.0-dev.1779385917 → nightly 0.35.0-dev.1779483941
  • Channel: nightly

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

Labels

No labels
No labels
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions