Skip to content

JSE-20: lazy shared strings for capped reads (maxRows) - #95

Merged
securedeveloper merged 1 commit into
mainfrom
jse-20-lazy-sst
Nov 16, 2026
Merged

securedeveloper merged 1 commit into
mainfrom
jse-20-lazy-sst

Conversation

@securedeveloper

@securedeveloper securedeveloper commented Nov 16, 2026 •

Copy link
Copy Markdown
Owner

JSE-20: lazy shared strings for capped reads. This is the follow-up proposed in #93 (merged), now based on main.

What changes

With ReadOptions.maxRows set (which importDataset's preview and maxRows use):

  1. While the sheets are parsed, a t="s" cell records its shared-string index instead of looking it up (DeferredStrings, parallel arrays).
  2. After the sheets, the shared strings part is decoded only up to the highest index those rows use. The parser then jumps to </sst> with the tokenizer skip from JSE-19: javascript-excel/import (importDataset, header detection, types, locales, mapping) #93, so the rest is never tokenized.
  3. The deferred cells are filled in. An index past the table gives the same INVALID_SHARED_STRING warning and the same cell outcome as the full path (dropped, or kept as { type: 'empty' } with its formula or style).

Without maxRows, nothing changes: shared strings are parsed before the sheets, as before.

Measured

100,000 × 10 sheet, styles: false, median of 7 reads, Node 26.13.1 on an Apple M2 under light load (load average about 2), same file and method before and after:

full read maxRows: 25
1M unique strings, before (#93) 924 ms 412 ms
1M unique strings, after 941 ms 166 ms (about 2.5× faster)
numbers only, after (for reference) 517 ms 77 ms

The full read is unchanged (924 vs 941 ms is run-to-run noise). The remaining time in the capped string read is mostly inflating the two parts to the end, which the zip reader still does. docs/api.md §5.2 now carries these quiet-machine numbers, replacing #93's numbers from a loaded machine.

Behaviour notes (documented in §5.2)

  • maxSharedStrings: only the decoded entries count. A preview of a file whose table exceeds the limit succeeds when the rows it reads only use early entries; the full read still fails with LIMIT_EXCEEDED. There's a test for both.
  • Invalid indexes: for a cell whose index is past the table, sheet.dimension was computed before the strings were resolved, so in a capped read it may still include that dropped cell. Only damaged files are affected.
  • Unparsed parts: the internal "retain unparsed parts" option still never lists the shared strings part, even when a capped read used no strings.

Tests (test/read/lazy-sst.test.ts)

  • Equivalence: a capped read equals the full read sliced to the same rows, on two sheets, with styles on and off.
  • Index order: high indexes used before low ones; decoding stops at the highest index used.
  • Text modes: rich text, both runs and plain.
  • Bad references: an index past the table, including a formula cell; a missing shared strings part.
  • Edge cases: maxRows: 0 (the table is never decoded and still isn't an unparsed part); the maxSharedStrings behaviour above; a later duplicate cell keeping its own value.

Locally (Node 26): 732 tests, typecheck, build, publint and docs:check all pass. A patch changeset is included.

With ReadOptions.maxRows, string cells are collected while the sheets are
parsed and the shared strings table is decoded afterwards, only up to the
highest index those rows use; the rest of the part is skipped like the rows
past the limit. Without maxRows the reader is unchanged. 100,000 x 10
unique strings, maxRows 25: 412 -> 166 ms (full read 941 ms).
@securedeveloper
securedeveloper changed the base branch from jse-19-import to main November 16, 2026 13:50
@securedeveloper
securedeveloper merged commit 8961824 into main Nov 16, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant