Skip to content

perf(pglite): reduce per-row allocation in parseResults#1050

Open
huntiezz wants to merge 1 commit into
electric-sql:mainfrom
huntiezz:perf/optimize-parse-results
Open

perf(pglite): reduce per-row allocation in parseResults#1050
huntiezz wants to merge 1 commit into
electric-sql:mainfrom
huntiezz:perf/optimize-parse-results

Conversation

@huntiezz

Copy link
Copy Markdown

Summary

Optimizes the hottest JS-side path in PGlite — converting PostgreSQL wire-protocol DataRow messages into JavaScript objects/arrays in parseResults().

Problem: Every row previously allocated intermediate structures:

  • Object mode (default): Object.fromEntries(msg.fields.map(...)) — builds a [key, value][] array per row, then iterates it again
  • Array mode: msg.fields.map(...) — callback-based allocation per row

Field names and type OIDs were also re-read from currentResultSet.fields[i] on every row even though they're constant for the entire result set.

Fix:

  1. Hoist fieldNames[] and fieldTypes[] once per result set on rowDescription
  2. Build object-mode rows with a tight indexed for loop (no Object.fromEntries)
  3. Preallocate array-mode rows with new Array(n) and fill by index
  4. Reset hoisted metadata on commandComplete

This is a behavior-preserving micro-optimization — a natural follow-up to #495, focused on large result-set workloads (#630). Does not overlap with maintainer drafts #973 (parse redesign) or #903 (streaming parse).

Test plan

  • New unit tests in packages/pglite/tests/parse.test.ts (9 cases: object/array modes, multi-result sets, INSERT/UPDATE/DELETE counts, custom parsers, blob attachment, describe helper)
  • pnpm vitest tests/parse.test.ts — 9 passed
  • pnpm test:basic — 278 passed (57 files)
  • pnpm stylecheck && pnpm typecheck — clean
  • Changeset added for @electric-sql/pglite patch

Hoist field names and type OIDs once per result set on rowDescription,
then build rows with indexed loops instead of Object.fromEntries and
per-row .map(). Adds unit tests for parseResults covering object/array
row modes, multi-result sets, and custom parsers.
@huntiezz
huntiezz force-pushed the perf/optimize-parse-results branch from add5c8e to 2433e52 Compare July 12, 2026 11:56
@AntonOfTheWoods

Copy link
Copy Markdown
Contributor

@huntiezz you mention "optimize" - do you have any benchmarks showing what the actual gains might be?

@huntiezz

Copy link
Copy Markdown
Author

Good question — I didn't include benchmarks in the initial PR, so I ran some after your comment.

Scope: This only optimizes parseResults() (wire-protocol DataRow → JS objects/arrays). WASM Postgres execution, planning, and I/O are unchanged, so end-to-end gains are smaller than the parse-only numbers.

Parse-only microbenchmark (object mode, default)

Synthetic workload: alternating int4 + text columns, node --expose-gc.

Scenario Old New Speedup
1,000 rows × 4 cols 250k rows/s 1.8M rows/s ~3–7×
10,000 rows × 4 cols 552k rows/s 2.0M rows/s ~3–4×
100,000 rows × 4 cols 309k rows/s 1.5M rows/s ~4×
10,000 rows × 12 cols 195k rows/s 844k rows/s ~3×

The main win is eliminating Object.fromEntries(msg.fields.map(...)), which allocated an intermediate [key, value][] array on every row in object mode.

Array mode (rowMode: 'array'): roughly neutral — sometimes slightly faster, sometimes slightly slower. The optimization is really aimed at object mode.

End-to-end (PGlite.query('SELECT * ...'))

Includes WASM + protocol + parse (median of 6 runs):

Rows Old New E2E speedup
10,000 198ms 156ms 1.27× (~21% faster)
50,000 946ms 838ms 1.13× (~11% faster)

As result sets grow, WASM dominates more, so the percentage gain shrinks even though the parse step itself is ~3–4× faster.

Takeaway

  • Large SELECTs in object mode: modest but real end-to-end gains (~10–20%)
  • Parse-heavy paths (e.g. re-parsing message batches): potentially ~3–4× on the JS parsing step
  • Small queries / few rows: negligible — overhead is elsewhere

Happy to add a small benchmark script to the repo if that would be useful for review/CI.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants