Dense changed-file recovery sync hits V8's spread-argument limit

[netbrah]

Summary

A recovery sync with roughly 98,000 newly parsed C/C++ files reached 100% and
then exited with:

Failed to sync: Maximum call stack size exceeded

The CLI prints only err.message, so the actual stack was unavailable. Source
inspection points to an unbounded result-array spread in
QueryBuilder.getUnresolvedReferencesByFiles():

const chunkRows = statement.all(...chunk);
rows.push(...chunkRows);

The input path chunk is kept below SQLite's parameter limit (about 900 paths),
but the number of unresolved-reference rows returned for those paths is not
bounded. A dense C/C++ chunk can therefore pass more arguments to
Array.prototype.push than V8 allows.

Suggested direction

Append chunkRows with a loop or bounded slices instead of spreading an
unbounded query result. Separately, please expose a full stack under a debug
flag; message-only CLI errors make it difficult to confirm failures in a
specific large-array boundary.

I am labeling the spread site as the source-level hypothesis because the CLI
discarded the stack. The externally observed failure itself is reproducible
after the same dense changed-file batch.

[colbymchenry]

Confirmed in the wild and fixed in #1576 — thank you for the precise diagnosis; the source inspection was exactly right.

While validating the #1541 repair (#1575) on a cpython-stdlib-sized index that the released v1.5.0 binary had damaged, the recovery sync re-indexed 919 files and died with Failed to sync: Maximum call stack size exceeded — resolution aborted after the store phase, leaving 234,440 references pending and the graph ~226k edges short of a clean index. Exactly the failure you reported, at one-hundredth the file count you saw it at: the spread passes every result row as a call argument, so it's the per-chunk row density that matters, not the input size.

Both sites now append with a loop — getUnresolvedReferencesByFiles and the failed-ref retry loader, which had the identical pattern on unbounded result rows. A regression test inserts 200k pending refs and reproduces the exact RangeError on the old code. After the fix, the same damaged-index recovery completes in one sync (939 files, 8s) with no crash.

On your suggestion about surfacing full stacks: agreed the message-only CLI error made this needlessly hard to confirm — worth doing as part of #1562's review, which also covers your other findings and remains very much wanted. Ships in the next release.