Skip to content

Commit f9f207a

Browse files
jahoomaclaude
andcommitted
Stage tree-sitter.wasm into pre-init/ for relative with { type: 'file' }
On Windows, bun --compile bundles the wasm bytes (build verification finds them at a known offset) but the JS-level binding from a node_modules subpath import returns falsy at runtime: import wasmPath from 'web-tree-sitter/tree-sitter.wasm' with { type: 'file' } // wasmPath is undefined on Windows even though the bytes are in // the binary Smoke check on the failed release confirmed it directly: tree-sitter smoke FAIL: pre-init published neither globalThis bytes nor an env path. The `with { type: 'file' }` import returned falsy. OpenTUI's own tree-sitter assets work because they're imported via *relative* paths from inside the package. Mirror that: copy the wasm into cli/src/pre-init/ before `bun build --compile`, import it relatively, remove the copy after the build. - cli/scripts/build-binary.ts: stagePreInitWasm() copies the source wasm to cli/src/pre-init/tree-sitter.wasm; cleanup runs after the compile and is also wired to process.on('exit') so a build-script crash doesn't leave a multi-MB untracked file in the working tree. The findWebTreeSitterWasm() lookup is shared with the post-build verification. - cli/src/pre-init/tree-sitter-wasm.ts: import is now `./tree-sitter.wasm` (relative). The file is .gitignored so dev-mode runs see no wasm here and fall through to init-node.ts's path-based resolution, which works locally because node_modules has the file. - cli/.gitignore: ignore the staged copy. Verified locally: build stages then cleans up the wasm, post-build verification finds the bytes, --smoke-tree-sitter exits 0 with "tree-sitter smoke ok (wasmBinary, 205488 bytes)". Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent b1bd842 commit f9f207a

3 files changed

Lines changed: 94 additions & 24 deletions

File tree

cli/.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,3 +7,7 @@ debug/
77

88
# Generated files
99
src/agents/bundled-agents.generated.ts
10+
11+
# Staged by build-binary.ts before `bun build --compile`, removed after.
12+
# See cli/src/pre-init/tree-sitter-wasm.ts for why we copy this in.
13+
src/pre-init/tree-sitter.wasm

cli/scripts/build-binary.ts

Lines changed: 74 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -145,6 +145,10 @@ async function main() {
145145
patchOpenTuiAssetPaths()
146146
await ensureOpenTuiNativeBundle(targetInfo)
147147

148+
const wasmCopy = stagePreInitWasm()
149+
// Even on a build-script crash, leave the developer's working tree clean.
150+
process.on('exit', wasmCopy.cleanup)
151+
148152
const outputFilename =
149153
targetInfo.platform === 'win32' ? `${binaryName}.exe` : binaryName
150154
const outputFile = join(binDir, outputFilename)
@@ -186,6 +190,11 @@ async function main() {
186190

187191
runCommand('bun', buildArgs, { cwd: cliRoot })
188192

193+
// Remove the staged pre-init wasm now that the build has read it. Eager
194+
// cleanup keeps a successful build clean; the exit handler above is a
195+
// backstop for crashes between stage and now.
196+
wasmCopy.cleanup()
197+
189198
// Fail the build if the wasm asset didn't actually make it into the
190199
// compiled binary. The pre-init imports tree-sitter.wasm with `with {
191200
// type: 'file' }`, which Bun should embed; this scan catches silent
@@ -211,6 +220,70 @@ main().catch((error: unknown) => {
211220
process.exit(1)
212221
})
213222

223+
/**
224+
* Find web-tree-sitter's tree-sitter.wasm in any plausible node_modules
225+
* layout — bun hoists differently across platforms and `bun install`
226+
* variants, and CI Windows lays it out differently than monorepo-root
227+
* installs.
228+
*/
229+
function findWebTreeSitterWasm(): string {
230+
const candidates = [
231+
join(cliRoot, 'node_modules', 'web-tree-sitter', 'tree-sitter.wasm'),
232+
join(cliRoot, '..', 'node_modules', 'web-tree-sitter', 'tree-sitter.wasm'),
233+
join(cliRoot, '..', 'sdk', 'node_modules', 'web-tree-sitter', 'tree-sitter.wasm'),
234+
]
235+
const found = candidates.find((p) => existsSync(p))
236+
if (found) return found
237+
try {
238+
const cliRequire = createRequire(join(cliRoot, 'package.json'))
239+
return cliRequire.resolve('web-tree-sitter/tree-sitter.wasm')
240+
} catch (err) {
241+
throw new Error(
242+
`Could not locate web-tree-sitter/tree-sitter.wasm. Searched:\n - ` +
243+
candidates.join('\n - ') +
244+
`\nAnd createRequire failed: ${err instanceof Error ? err.message : String(err)}`,
245+
)
246+
}
247+
}
248+
249+
/**
250+
* Copy `tree-sitter.wasm` into `cli/src/pre-init/` so the pre-init module
251+
* can import it via a relative `with { type: 'file' }` path. We can't
252+
* import it directly as a node_modules subpath: on Windows, bun's
253+
* `with { type: 'file' }` resolution returned falsy at runtime for
254+
* `web-tree-sitter/tree-sitter.wasm` even though the bytes ended up in
255+
* the binary, breaking the pre-init's runtime path lookup. OpenTUI's own
256+
* tree-sitter assets work because they're imported relatively from
257+
* inside the package — same trick here.
258+
*
259+
* Returns a cleanup function. The build calls it eagerly after compile
260+
* and registers it as an exit handler so a mid-build crash doesn't leave
261+
* a multi-MB untracked file in the working tree.
262+
*/
263+
function stagePreInitWasm(): { cleanup: () => void } {
264+
const sourceWasm = findWebTreeSitterWasm()
265+
const stagedPath = join(cliRoot, 'src', 'pre-init', 'tree-sitter.wasm')
266+
let cleaned = false
267+
const cleanup = (): void => {
268+
if (cleaned) return
269+
cleaned = true
270+
if (existsSync(stagedPath)) {
271+
try {
272+
rmSync(stagedPath)
273+
} catch (error) {
274+
console.error('Failed to remove staged pre-init wasm:', error)
275+
}
276+
}
277+
}
278+
279+
// Read + write rather than copyFile so we don't accidentally hardlink
280+
// (some Windows hosts fail to delete hardlinks while bun has the file
281+
// mmapped from the compile step).
282+
writeFileSync(stagedPath, readFileSync(sourceWasm))
283+
logAlways(`Staged pre-init wasm: ${sourceWasm}${stagedPath}`)
284+
return { cleanup }
285+
}
286+
214287
/**
215288
* Sanity-check the compiled binary actually contains web-tree-sitter's
216289
* tree-sitter.wasm. The pre-init imports it via `with { type: 'file' }`,
@@ -226,25 +299,7 @@ main().catch((error: unknown) => {
226299
* proves *this specific* wasm shipped.
227300
*/
228301
function verifyTreeSitterWasmEmbedded(outputFile: string): void {
229-
const candidates = [
230-
join(cliRoot, 'node_modules', 'web-tree-sitter', 'tree-sitter.wasm'),
231-
join(cliRoot, '..', 'node_modules', 'web-tree-sitter', 'tree-sitter.wasm'),
232-
join(cliRoot, '..', 'sdk', 'node_modules', 'web-tree-sitter', 'tree-sitter.wasm'),
233-
]
234-
let wasmPath = candidates.find((p) => existsSync(p))
235-
if (!wasmPath) {
236-
try {
237-
const cliRequire = createRequire(join(cliRoot, 'package.json'))
238-
wasmPath = cliRequire.resolve('web-tree-sitter/tree-sitter.wasm')
239-
} catch (err) {
240-
throw new Error(
241-
`Could not locate web-tree-sitter/tree-sitter.wasm to verify against. Searched:\n - ` +
242-
candidates.join('\n - ') +
243-
`\nAnd createRequire failed: ${err instanceof Error ? err.message : String(err)}`,
244-
)
245-
}
246-
}
247-
302+
const wasmPath = findWebTreeSitterWasm()
248303
const wasm = readFileSync(wasmPath)
249304
// Take a 64-byte slice from the middle of the file. The header has
250305
// generic wasm magic + section markers; the tail can be padding. The

cli/src/pre-init/tree-sitter-wasm.ts

Lines changed: 16 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -13,11 +13,22 @@
1313

1414
import { readFileSync } from 'fs'
1515

16-
// @ts-expect-error - Bun's `with { type: 'file' }` returns a string path; TS
17-
// has no loader for the .wasm subpath of web-tree-sitter's package exports.
18-
import treeSitterWasmPath from 'web-tree-sitter/tree-sitter.wasm' with {
19-
type: 'file',
20-
}
16+
// Important: this is a *relative* import of a wasm file the build script
17+
// copies in from `web-tree-sitter/tree-sitter.wasm` immediately before
18+
// `bun build --compile`. On Windows, bun's `with { type: 'file' }`
19+
// returned falsy at runtime when this import was a node_modules subpath
20+
// (`web-tree-sitter/tree-sitter.wasm`) even though the bytes ended up in
21+
// the binary — OpenTUI works around the same issue by using relative
22+
// paths from inside its own package, which is what we're mirroring here.
23+
//
24+
// The `.wasm` lives at `./tree-sitter.wasm` next to this file. It is
25+
// .gitignored; build-binary.ts copies it in before compile and removes
26+
// it after, so dev-mode runs see no `.wasm` here and fall back to
27+
// path-based resolution via init-node.ts (which works locally).
28+
//
29+
// @ts-expect-error - TS has no loader for .wasm; bun's `with { type: 'file' }`
30+
// returns a string path at compile time.
31+
import treeSitterWasmPath from './tree-sitter.wasm' with { type: 'file' }
2132

2233
let embeddedWasm: Uint8Array | undefined
2334

0 commit comments

Comments
 (0)