Repository navigation
JSE-20: lazy shared strings for capped reads (maxRows) - #95
Merged
Merged
Conversation
With ReadOptions.maxRows, string cells are collected while the sheets are parsed and the shared strings table is decoded afterwards, only up to the highest index those rows use; the rest of the part is skipped like the rows past the limit. Without maxRows the reader is unchanged. 100,000 x 10 unique strings, maxRows 25: 412 -> 166 ms (full read 941 ms).
securedeveloper
force-pushed
the
jse-20-lazy-sst
branch
from
November 16, 2026 13:50
8a702c5 to
f26fe0c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
JSE-20: lazy shared strings for capped reads. This is the follow-up proposed in #93 (merged), now based on
main.What changes
With
ReadOptions.maxRowsset (whichimportDataset'spreviewandmaxRowsuse):t="s"cell records its shared-string index instead of looking it up (DeferredStrings, parallel arrays).</sst>with the tokenizer skip from JSE-19: javascript-excel/import (importDataset, header detection, types, locales, mapping) #93, so the rest is never tokenized.INVALID_SHARED_STRINGwarning and the same cell outcome as the full path (dropped, or kept as{ type: 'empty' }with its formula or style).Without
maxRows, nothing changes: shared strings are parsed before the sheets, as before.Measured
100,000 × 10 sheet,
styles: false, median of 7 reads, Node 26.13.1 on an Apple M2 under light load (load average about 2), same file and method before and after:maxRows: 25The full read is unchanged (924 vs 941 ms is run-to-run noise). The remaining time in the capped string read is mostly inflating the two parts to the end, which the zip reader still does.
docs/api.md§5.2 now carries these quiet-machine numbers, replacing #93's numbers from a loaded machine.Behaviour notes (documented in §5.2)
maxSharedStrings: only the decoded entries count. A preview of a file whose table exceeds the limit succeeds when the rows it reads only use early entries; the full read still fails withLIMIT_EXCEEDED. There's a test for both.sheet.dimensionwas computed before the strings were resolved, so in a capped read it may still include that dropped cell. Only damaged files are affected.Tests (
test/read/lazy-sst.test.ts)maxRows: 0(the table is never decoded and still isn't an unparsed part); themaxSharedStringsbehaviour above; a later duplicate cell keeping its own value.Locally (Node 26): 732 tests, typecheck, build, publint and docs:check all pass. A patch changeset is included.