perf(pglite): reduce per-row allocation in parseResults#1050
Conversation
Hoist field names and type OIDs once per result set on rowDescription, then build rows with indexed loops instead of Object.fromEntries and per-row .map(). Adds unit tests for parseResults covering object/array row modes, multi-result sets, and custom parsers.
add5c8e to
2433e52
Compare
|
@huntiezz you mention "optimize" - do you have any benchmarks showing what the actual gains might be? |
|
Good question — I didn't include benchmarks in the initial PR, so I ran some after your comment. Scope: This only optimizes Parse-only microbenchmark (object mode, default)Synthetic workload: alternating
The main win is eliminating Array mode ( End-to-end (
|
| Rows | Old | New | E2E speedup |
|---|---|---|---|
| 10,000 | 198ms | 156ms | 1.27× (~21% faster) |
| 50,000 | 946ms | 838ms | 1.13× (~11% faster) |
As result sets grow, WASM dominates more, so the percentage gain shrinks even though the parse step itself is ~3–4× faster.
Takeaway
- Large SELECTs in object mode: modest but real end-to-end gains (~10–20%)
- Parse-heavy paths (e.g. re-parsing message batches): potentially ~3–4× on the JS parsing step
- Small queries / few rows: negligible — overhead is elsewhere
Happy to add a small benchmark script to the repo if that would be useful for review/CI.
Summary
Optimizes the hottest JS-side path in PGlite — converting PostgreSQL wire-protocol
DataRowmessages into JavaScript objects/arrays inparseResults().Problem: Every row previously allocated intermediate structures:
Object.fromEntries(msg.fields.map(...))— builds a[key, value][]array per row, then iterates it againmsg.fields.map(...)— callback-based allocation per rowField names and type OIDs were also re-read from
currentResultSet.fields[i]on every row even though they're constant for the entire result set.Fix:
fieldNames[]andfieldTypes[]once per result set onrowDescriptionforloop (noObject.fromEntries)new Array(n)and fill by indexcommandCompleteThis is a behavior-preserving micro-optimization — a natural follow-up to #495, focused on large result-set workloads (#630). Does not overlap with maintainer drafts #973 (parse redesign) or #903 (streaming parse).
Test plan
packages/pglite/tests/parse.test.ts(9 cases: object/array modes, multi-result sets, INSERT/UPDATE/DELETE counts, custom parsers, blob attachment, describe helper)pnpm vitest tests/parse.test.ts— 9 passedpnpm test:basic— 278 passed (57 files)pnpm stylecheck && pnpm typecheck— clean@electric-sql/pglitepatch