Skip to content

fix: join parameterized SQL statements in linear time - #1004

Open
MaolmeoX wants to merge 1 commit into
loopbackio:masterfrom
MaolmeoX:fix/parameterized-sql-join-linear
Open

MaolmeoX wants to merge 1 commit into
loopbackio:masterfrom
MaolmeoX:fix/parameterized-sql-join-linear

Conversation

@MaolmeoX

@MaolmeoX MaolmeoX commented Nov 9, 2026

Copy link
Copy Markdown

ParameterizedSQL.join builds the joined SQL and a single fresh params array in one pass, instead of calling ParameterizedSQL.append once per statement, which copied the accumulated params array every time. Every inq / nin clause goes through join with one statement per value, so the cost was quadratic in the number of values.

Measured on Node 24.19 with the script from the issue:

Statements joined Before After
20,000 483 ms 1 ms
35,000 2,366 ms 2 ms
50,000 5,115 ms 18 ms

buildWhere with an inq of 35,000 ids goes from 2,318 ms to 24 ms.

The output is unchanged: same separator handling (including statements with an empty sql), same params order, and plain strings and {sql, params} objects are still normalized through the constructor. 20,000 random joins compared old against new produced identical results. The params arrays of the joined statements are not modified: the constructor keeps the caller's array by reference, so the result gets its own array.

Tests added in test/sql.test.js: the joined result for mixed input, the joined statements' params left untouched, and a join of 50,000 statements under a 1 s mocha timeout. That last test takes about 5 s and fails on the current code, and about 20 ms with this change.

Fixes #1003

Checklist

  • DCO (Developer Certificate of Origin) signed in all commits
  • npm test passes on your machine
  • New tests added or existing tests modified to cover all changes
  • Code conforms with the style guide
  • Commit messages are following our guidelines

ParameterizedSQL.join builds the joined SQL and one fresh params array in
a single pass. Appending each statement through ParameterizedSQL.append
copied the accumulated params array once per statement, so building an
inq/nin clause over N values cost O(N^2): 35,000 values blocked the event
loop for about 2.3 s.

The params arrays of the joined statements are left untouched, since the
constructor keeps the caller's array by reference.

Fixes loopbackio#1003

Signed-off-by: MaolmeoX <arnaud.maurice.pro@gmail.com>
@MaolmeoX
MaolmeoX marked this pull request as ready for review November 9, 2026 10:41
@MaolmeoX
MaolmeoX requested a review from dhmlau as a code owner November 9, 2026 10:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ParameterizedSQL.join is quadratic in the number of statements, so large inq filters block the event loop

1 participant