Skip to content

Commit 065eefa

Browse files
jahoomaclaude
andcommitted
Catch async startup failures in CI smoke tests
Both --version smoke tests passed on Windows even though the binary crashed for users: commander exits the process synchronously, before the Parser.init promise has a chance to reject. Three changes to close the gap: - cli/scripts/smoke-binary.ts: portable script that spawns the binary, lets it run for 5s, kills it, and asserts the captured stdout/stderr doesn't contain earlyFatalHandler markers ("Fatal error during startup", "Internal error: tree-sitter.wasm not found", unhandled rejections, missing modules). Wired into the release-build smoke step for every platform and into the freebuff-e2e build smoke step. - freebuff/e2e/tests/startup.e2e.test.ts: wait for "Pick a model to start" to render instead of just non-empty output. The model selector only appears once the binary survived module init (Parser.init included), the auth/session API call returned, and the React tree mounted, so a half-rendered crash splash no longer satisfies the assertion. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent da4f4c7 commit 065eefa

4 files changed

Lines changed: 142 additions & 14 deletions

File tree

.github/workflows/cli-release-build.yml

Lines changed: 20 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -176,11 +176,20 @@ jobs:
176176
run: |
177177
cd cli/bin
178178
if [[ "${{ runner.os }}" == "Windows" ]]; then
179-
./${{ inputs.binary-name }}.exe --version
179+
BIN="./${{ inputs.binary-name }}.exe"
180180
else
181-
./${{ inputs.binary-name }} --version
181+
BIN="./${{ inputs.binary-name }}"
182182
fi
183183
184+
# Fast path: --version exits synchronously through commander, so it
185+
# only catches early sync failures. Run it for parity with old CI.
186+
"$BIN" --version
187+
188+
# Slow path: keep the binary alive long enough for *async* startup
189+
# failures (e.g. the Parser.init rejection that crashed the
190+
# post-OpenTUI-upgrade Windows build) to surface in stdout/stderr.
191+
bun ../scripts/smoke-binary.ts "$BIN"
192+
184193
- name: Create tarball
185194
shell: bash
186195
run: |
@@ -317,7 +326,15 @@ jobs:
317326
shell: bash
318327
run: |
319328
cd cli/bin
320-
./${{ inputs.binary-name }}.exe --version
329+
BIN="./${{ inputs.binary-name }}.exe"
330+
331+
# Sync check — exits via commander before async tasks fire.
332+
"$BIN" --version
333+
334+
# Long-running check — gives async startup failures time to surface.
335+
# This is the step that would have caught the post-OpenTUI-upgrade
336+
# tree-sitter wasm crash on Windows.
337+
bun ../scripts/smoke-binary.ts "$BIN"
321338
322339
- name: Create tarball
323340
shell: bash

.github/workflows/freebuff-e2e.yml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,13 @@ jobs:
4040
- name: Smoke test binary
4141
run: |
4242
chmod +x cli/bin/freebuff
43+
# --version exits via commander synchronously and won't see async
44+
# startup failures (e.g. the Parser.init rejection from a broken
45+
# tree-sitter wasm load).
4346
cli/bin/freebuff --version
47+
# Run for a few seconds so unhandled rejections during module init
48+
# have a chance to fire and trip earlyFatalHandler.
49+
bun cli/scripts/smoke-binary.ts cli/bin/freebuff
4450
4551
- name: Upload binary
4652
uses: actions/upload-artifact@v7

cli/scripts/smoke-binary.ts

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
#!/usr/bin/env bun
2+
/**
3+
* Long-running smoke test for a compiled CLI binary.
4+
*
5+
* `--version` and `--help` exit via commander synchronously, before async
6+
* startup failures (e.g. the unhandled rejection from Parser.init when the
7+
* tree-sitter wasm load fails) get a chance to fire. This script spawns the
8+
* binary, lets it run for a few seconds, then kills it and asserts no fatal
9+
* startup markers showed up in stdout/stderr.
10+
*
11+
* Designed to run on every supported platform (Linux, macOS, Windows) without
12+
* extra deps. The binary doesn't need a TTY: `earlyFatalHandler` in
13+
* `cli/src/index.tsx` writes its diagnostic to stdout/stderr regardless.
14+
*
15+
* Usage:
16+
* bun cli/scripts/smoke-binary.ts <path-to-binary> [seconds]
17+
*
18+
* Exits 0 if no fatal markers detected, 1 otherwise.
19+
*/
20+
21+
import { spawn } from 'child_process'
22+
import { existsSync } from 'fs'
23+
24+
// Markers that indicate the CLI crashed during startup. Match what
25+
// `earlyFatalHandler` writes plus the specific tree-sitter regression.
26+
const FATAL_PATTERNS = [
27+
/Fatal error during startup/i,
28+
/Internal error: tree-sitter\.wasm not found/i,
29+
/UnhandledPromiseRejection/i,
30+
/Cannot find module/i,
31+
] as const
32+
33+
const DEFAULT_RUN_SECONDS = 5
34+
35+
async function main(): Promise<void> {
36+
const binary = process.argv[2]
37+
const runSeconds = Number(process.argv[3] ?? DEFAULT_RUN_SECONDS)
38+
39+
if (!binary) {
40+
console.error('Usage: bun smoke-binary.ts <path-to-binary> [seconds]')
41+
process.exit(2)
42+
}
43+
if (!existsSync(binary)) {
44+
console.error(`smoke-binary: binary not found: ${binary}`)
45+
process.exit(2)
46+
}
47+
if (!Number.isFinite(runSeconds) || runSeconds <= 0) {
48+
console.error(`smoke-binary: bad seconds arg: ${process.argv[3]}`)
49+
process.exit(2)
50+
}
51+
52+
console.log(`smoke-binary: spawning ${binary} for ${runSeconds}s…`)
53+
54+
const proc = spawn(binary, [], {
55+
stdio: ['ignore', 'pipe', 'pipe'],
56+
env: { ...process.env, NO_COLOR: '1', TERM: 'dumb' },
57+
})
58+
59+
let captured = ''
60+
const append = (chunk: Buffer): void => {
61+
captured += chunk.toString('utf8')
62+
}
63+
proc.stdout?.on('data', append)
64+
proc.stderr?.on('data', append)
65+
66+
let earlyExitCode: number | null = null
67+
const exited = new Promise<void>((resolve) => {
68+
proc.once('exit', (code) => {
69+
earlyExitCode = code
70+
resolve()
71+
})
72+
})
73+
74+
const killTimer = setTimeout(() => {
75+
// SIGKILL is the only signal that's portable across Linux/macOS/Windows
76+
// here; SIGTERM may be ignored by the renderer on some platforms.
77+
proc.kill('SIGKILL')
78+
}, runSeconds * 1_000)
79+
80+
await exited
81+
clearTimeout(killTimer)
82+
83+
for (const pattern of FATAL_PATTERNS) {
84+
if (pattern.test(captured)) {
85+
console.error(
86+
`smoke-binary: FAIL — output matched ${pattern} (exit code ${earlyExitCode}).`,
87+
)
88+
console.error('--- captured output (truncated to 8KB) ---')
89+
console.error(captured.slice(0, 8 * 1024))
90+
process.exit(1)
91+
}
92+
}
93+
94+
console.log(
95+
`smoke-binary: OK (exit code ${earlyExitCode}, ${captured.length} bytes captured).`,
96+
)
97+
}
98+
99+
main().catch((err: unknown) => {
100+
console.error('smoke-binary: unexpected error:', err)
101+
process.exit(2)
102+
})

freebuff/e2e/tests/startup.e2e.test.ts

Lines changed: 14 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -15,24 +15,27 @@ describe('Freebuff: Startup', () => {
1515
})
1616

1717
test(
18-
'binary starts without crashing',
18+
'binary reaches the model selection screen',
1919
async () => {
2020
const binary = requireFreebuffBinary()
2121
session = await FreebuffSession.start(binary)
22-
await session.waitForReady()
23-
24-
const output = await session.capture()
2522

26-
// Should not contain fatal errors
23+
// Wait for the model selector to render. This proves the binary survived
24+
// module init (including the eager tree-sitter Parser.init that crashed
25+
// Windows binaries after the OpenTUI 0.2.2 upgrade), passed the auth /
26+
// session API call, and successfully mounted the React tree. A pure
27+
// "non-empty output" check would pass on a half-rendered crash screen.
28+
const output = await session.waitForText('Pick a model to start')
29+
30+
// earlyFatalHandler in cli/src/index.tsx writes this to stderr on
31+
// unhandled rejections during startup. Belt-and-braces: the wait above
32+
// would already have timed out, but if some race ever surfaces a fatal
33+
// *after* the model selector renders, we still want it to fail.
34+
expect(output).not.toContain('Fatal error during startup')
35+
expect(output).not.toContain('Internal error: tree-sitter.wasm not found')
2736
expect(output).not.toContain('FATAL')
2837
expect(output).not.toContain('panic')
2938
expect(output).not.toContain('Segmentation fault')
30-
31-
// Should have some visible output (not a blank screen)
32-
const nonEmptyLines = output
33-
.split('\n')
34-
.filter((line) => line.trim().length > 0)
35-
expect(nonEmptyLines.length).toBeGreaterThan(0)
3639
},
3740
STARTUP_TIMEOUT,
3841
)

0 commit comments

Comments
 (0)