Export wasm chunks as a function so the bundler can't inline them away

Round 4 (chunked array literals) still failed on Windows: the build's
own verification step caught the first chunk missing from the compiled
binary. So either:

 - Bun's bundler reads tree-sitter-wasm-bytes.ts at static-analysis
   time, sees `export const X = []` (the committed stub), inlines `X`
   into pre-init's call sites, then DCEs the conditional branch that
   would have referenced the chunks. Whatever my embed script wrote
   later is treated as unused and dropped.
 - OR the file write doesn't propagate to disk before bun reads it on
   Windows.

Switch the export from `const` to a function. Function return values
aren't statically inlinable — the bundler can't substitute a literal
empty array at the call site. The chunks live inside the function
body, only materialized when the pre-init calls
`getTreeSitterWasmChunks()`.

Add a sanity re-read after writing the embed file: if NTFS buffers
the write and bun reads the stale stub, the embed step itself fails
*during the build*, with a clear "wrote N chunks but re-read does not
contain chunk[0]" message — instead of letting the build silently
produce a broken artifact.

Verified locally: build embeds 268 chunks, post-build verifies 3
chunks in the compiled binary, --smoke-tree-sitter exits 0,
boot smoke passes.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

Author: jahooma

cli/scripts/build-binary.ts

@@ -294,13 +294,27 @@ function embedTreeSitterWasmAsChunks(): {
 
   const generated =
     `// AUTO-GENERATED by cli/scripts/build-binary.ts during \`bun build --compile\`.\n` +
-    `// Restored to the empty stub after the build finishes — do not commit a\n` +
-    `// non-empty value here.\n` +
-    `export const TREE_SITTER_WASM_BASE64_CHUNKS: readonly string[] = [\n` +
-    chunks.map((c) => `  ${JSON.stringify(c)},`).join('\n') +
-    `\n]\n`
+    `// Restored to an empty function after the build finishes — do not commit a\n` +
+    `// non-empty body here.\n` +
+    `export function getTreeSitterWasmChunks(): string[] {\n` +
+    `  return [\n` +
+    chunks.map((c) => `    ${JSON.stringify(c)},`).join('\n') +
+    `\n  ]\n` +
+    `}\n`
 
   writeFileSync(stubPath, generated)
+  // Re-read what we just wrote so we can fail loudly if the OS buffered
+  // the write. On Windows, NTFS writes can lag, and bun --compile would
+  // then read the stale stub. Verifying here means the build fails
+  // *during embed* instead of producing a broken binary that surprises
+  // us later.
+  const onDisk = readFileSync(stubPath, 'utf8')
+  if (!onDisk.includes(chunks[0]!)) {
+    throw new Error(
+      `Embed wrote ${chunks.length} chunks but re-read of ${stubPath} ` +
+        `does not contain chunk[0]. File on disk: ${onDisk.slice(0, 200)}…`,
+    )
+  }
   logAlways(
     `Embedded tree-sitter.wasm from ${sourceWasm} (${wasmBytes.length} bytes → ${chunks.length} chunks of ~${CHUNK_SIZE} chars).`,
   )

cli/src/pre-init/tree-sitter-wasm-bytes.ts

@@ -1,14 +1,19 @@
-// Stub committed for dev mode and tests. The real wasm chunks are written
+// Stub committed for dev mode and tests. The real chunks are written
 // here by `cli/scripts/build-binary.ts` immediately before
-// `bun build --compile`, then restored to an empty array after the build
-// completes. Dev mode and unit tests see the empty stub and fall back to
-// path-based resolution in `packages/code-map/src/init-node.ts` (which
-// works locally because `node_modules/web-tree-sitter/tree-sitter.wasm`
-// exists on the filesystem).
+// `bun build --compile`, then restored to this empty stub after.
 //
-// Why an array of small chunks rather than one big string: a single
-// 274KB string literal got dropped/transformed by bun's Windows
-// minifier (the binary built clean but ran without the bytes). Many
-// small string literals slip under whatever threshold caused that. See
-// `cli/src/pre-init/tree-sitter-wasm.ts` for the full failure history.
-export const TREE_SITTER_WASM_BASE64_CHUNKS: readonly string[] = []
+// Why a *function* return rather than a top-level const: prior
+// approaches kept getting eliminated on Windows even with 268
+// individual chunks. The bundler appears to evaluate the imported
+// value at static-analysis time (we suspect either filesystem write
+// timing or an AST cache), inlines it as the empty stub, and DCEs
+// any conditional that depends on `.length > 0`. A function call's
+// return value is not statically inlinable in the same way — the
+// chunks live inside the function body, only materialized on call.
+//
+// Why a function instead of `export const X = (() => [...])()`:
+// same reason — IIFEs can be folded by aggressive minifiers, but
+// imported functions called at runtime are preserved.
+export function getTreeSitterWasmChunks(): string[] {
+  return []
+}

cli/src/pre-init/tree-sitter-wasm.ts

@@ -1,41 +1,47 @@
 // Embed tree-sitter.wasm into the bun-compile binary so the SDK's tree-sitter
 // parser singleton can find it at runtime. Must be the very first import in
 // `index.tsx`: subsequent imports (the SDK / code-map) eagerly construct the
-// parser, and its init reads what we publish here on `globalThis` and via
-// the env var.
+// parser, and its init reads what we publish here on `globalThis`.
 //
-// History of failed approaches before this one:
+// History of failed approaches before this one (all worked on macOS/Linux,
+// failed on Windows in different ways):
 //
-//  1. `with { type: 'file' }` import of `web-tree-sitter/tree-sitter.wasm`
-//     (node_modules subpath) — bun --compile on Windows embedded the
-//     bytes but bound the import variable to undefined.
-//  2. `with { type: 'file' }` import of a copied-in relative wasm file —
-//     same problem; this turns out to be a bun/Windows bug, not a
-//     subpath-vs-relative thing.
-//  3. Single 274KB base64 string literal in a generated TS module —
-//     bun's Windows minifier dropped/transformed the literal even
-//     though the embed step wrote it.
+//  1. `with { type: 'file' }` of `web-tree-sitter/tree-sitter.wasm` (node_
+//     modules subpath) — bytes ended up in the binary but the import
+//     variable was undefined at runtime. Bun/Windows bug with the import-
+//     attribute binding.
+//  2. `with { type: 'file' }` of a copied-in relative .wasm — same as #1,
+//     so it's not subpath-vs-relative.
+//  3. Single 274KB base64 string literal in a generated TS module — the
+//     literal didn't appear in the compiled binary at all. Probably the
+//     minifier transforming "huge constant" literals.
+//  4. ~268 chunked base64 string literals — same fate; the bundler
+//     appeared to evaluate the imported array as the empty stub at
+//     static-analysis time and DCE'd the conditional that consumed it.
 //
-// What works: many small base64 chunks (each well under any plausible
-// minifier threshold) joined at runtime. The build script writes the
-// chunks; this module decodes them. The committed file ships an empty
-// stub array — dev-mode runs see no chunks and fall through to
-// path-based resolution in init-node.ts (which works locally because
-// `node_modules/web-tree-sitter/tree-sitter.wasm` exists on disk).
+// What this version does: import a *function* whose body returns the
+// chunks. Function return values aren't statically inlinable the way
+// `export const` values are, so the bundler can't substitute the empty
+// stub for the call site. Reference the result unconditionally so DCE
+// can't kick in even if some inliner does fold the function.
 
-import { TREE_SITTER_WASM_BASE64_CHUNKS } from './tree-sitter-wasm-bytes'
+import { getTreeSitterWasmChunks } from './tree-sitter-wasm-bytes'
 
-let embeddedWasm: Uint8Array | undefined
-if (TREE_SITTER_WASM_BASE64_CHUNKS.length > 0) {
-  // Joined string is up to ~275KB but only lives long enough to decode.
-  const buf = Buffer.from(TREE_SITTER_WASM_BASE64_CHUNKS.join(''), 'base64')
-  embeddedWasm = new Uint8Array(buf.buffer, buf.byteOffset, buf.byteLength)
+const chunks = getTreeSitterWasmChunks()
+if (chunks.length > 0) {
+  const buf = Buffer.from(chunks.join(''), 'base64')
   // globalThis is the only cross-bundle channel: the SDK pre-built bundle
   // inlines its own copy of `init-node.ts`, so a module-level variable
-  // here isn't visible to the singleton initialized via the SDK.
+  // here isn't visible to the singleton initialized via the SDK. Slice
+  // into a fresh Uint8Array view rather than handing over Buffer's shared
+  // underlying ArrayBuffer.
   ;(
     globalThis as { __CODEBUFF_TREE_SITTER_WASM_BINARY__?: Uint8Array }
-  ).__CODEBUFF_TREE_SITTER_WASM_BINARY__ = embeddedWasm
+  ).__CODEBUFF_TREE_SITTER_WASM_BINARY__ = new Uint8Array(
+    buf.buffer,
+    buf.byteOffset,
+    buf.byteLength,
+  )
 }
 
 // `--smoke-tree-sitter` is the deterministic CI gate. The handler lives at