Skip to content

Commit c526528

Browse files
authored
feat(webapp,database): bound Prisma list filter arity (#4480)
## Summary Prisma expands `in` / `notIn` into one bind parameter per element, so every distinct list length is a separate prepared statement. Where the length tracks data volume (a batch size, a run-graph fan-out, a prior query's id set) one call site can mint hundreds of them. Each is used about once, but inserting it evicts an entry that was being reused, so the cost lands on unrelated queries sharing the pooler's statement cache. An unbounded list also risks the 65535 bind-parameter ceiling. `boundedIn()` pads a filter list to the next power of two by repeating its last element. `IN` and `NOT IN` ignore duplicates, so results are unchanged, and a call site drops from one statement per length to at most `log2(cap)`. Applied to all existing sites. ## Enforcement Two oxlint rules require the helper: a list filter must be an inline array literal or a `boundedIn()` call. - The first covers filters reached through `where` / `having` / `cursor`, and deliberately never descends into `data`, `create`, `update`, `set` or `equals`. A key named `in` in those positions is user data, not a predicate, and rewriting it would corrupt what gets stored or compared. - The second covers bare filter objects passed to where-building helpers, which the first cannot see. It found five sites in the run-graph batch loaders that were otherwise invisible. Both rules follow filters through the shapes they are actually written in: conditional expressions, logical-and objects, spread-conditional properties, computed keys, and call arguments. An array literal only counts as fixed-arity when nothing spreads into it, since `[...new Set(ids)]` has a runtime length. Twelve sites were hidden behind those shapes until the rules handled them. Scoped to `in` and `notIn`. The scalar-list filters `hasSome` and `hasEvery` compile to `&& $1` and `@> $1`, passing the whole array as a single bind parameter, so their arity never reaches the statement text and there is nothing to bound. Both rules are `error`, so new call sites fail CI. That ratchet has already caught four sites added by other PRs while this one was in review. ## Notes `boundedIn` pads by repeating rather than with null: `x NOT IN (a, b, NULL)` is never true, so null-padding a `notIn` filter would silently return no rows. Lists above 32768 are returned unchanged so padding can never push a query past the parameter limit. Route modules reach the helper through `~/db.server` rather than importing the database barrel directly, since a value import of that barrel into a module that also exports a React component is only safe while dead-code elimination prunes it. Measured on a local rig: 300 distinct list lengths produce 300 prepared statements unpadded, 10 padded. Verified end-to-end against a local stack with the full task-suite sweep, which surfaced no regressions.
1 parent 63176a6 commit c526528

57 files changed

Lines changed: 576 additions & 115 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.oxlintrc.json

Lines changed: 19 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,8 @@
33
"plugins": ["typescript", "import", "react"],
44
"jsPlugins": [
55
"./oxlint-plugins/no-thrown-unawaited-redirect.mjs",
6-
"./oxlint-plugins/runops-residency.mjs"
6+
"./oxlint-plugins/runops-residency.mjs",
7+
"./oxlint-plugins/prisma-in-filter.mjs"
78
],
89
"ignorePatterns": [
910
"**/dist/**",
@@ -30,13 +31,21 @@
3031
"no-empty-pattern": "off",
3132
"no-control-regex": "off",
3233
"typescript/no-non-null-asserted-optional-chain": "off",
33-
"no-unused-expressions": ["warn", { "allowShortCircuit": true, "allowTernary": true }],
34+
"no-unused-expressions": [
35+
"warn",
36+
{
37+
"allowShortCircuit": true,
38+
"allowTernary": true
39+
}
40+
],
3441
"typescript/consistent-type-imports": "error",
3542
"import/no-duplicates": "error",
3643
"import/namespace": "off",
3744
"react-hooks/exhaustive-deps": "off",
3845
"react-hooks/rules-of-hooks": "off",
39-
"trigger/no-thrown-unawaited-redirect": "error"
46+
"trigger/no-thrown-unawaited-redirect": "error",
47+
"trigger-prisma/no-unbounded-list-filter": "error",
48+
"trigger-prisma/no-unbounded-list-filter-in-args-helper": "error"
4049
},
4150
"overrides": [
4251
{
@@ -52,6 +61,13 @@
5261
"trigger-runops/no-control-plane-run-graph-access": "off",
5362
"trigger-runops/no-control-plane-in-runops-slot": "off"
5463
}
64+
},
65+
{
66+
"files": ["**/*.test.ts", "**/*.test.tsx", "**/test/**", "**/tests/**", "**/e2e/**"],
67+
"rules": {
68+
"trigger-prisma/no-unbounded-list-filter": "off",
69+
"trigger-prisma/no-unbounded-list-filter-in-args-helper": "off"
70+
}
5571
}
5672
]
5773
}
Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
area: webapp
3+
type: improvement
4+
---
5+
6+
Database queries that filter on a list of values now reuse cached query plans more consistently, instead of forcing the database to re-plan whenever the list length changes.

apps/webapp/app/db.server.ts

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
import {
22
Prisma,
33
PrismaClient,
4+
boundedIn,
45
$transaction as transac,
56
type PrismaClientOrTransaction,
67
type PrismaReplicaClient,
@@ -122,7 +123,7 @@ async function $transactionInner<R>(
122123
}
123124
}
124125

125-
export { Prisma };
126+
export { Prisma, boundedIn };
126127

127128
type DatasourceLabel =
128129
| "control-plane-writer"

apps/webapp/app/models/api-key.server.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@ import type { PrismaClient, RuntimeEnvironment } from "@trigger.dev/database";
22
import type { HostRbacController } from "@trigger.dev/rbac";
33
import { customAlphabet } from "nanoid";
44
import { MAX_API_KEY_TASK_IDENTIFIERS } from "~/consts";
5-
import { prisma } from "~/db.server";
5+
import { boundedIn, prisma } from "~/db.server";
66
import { RuntimeEnvironmentType } from "~/database-types";
77
import { canIssueAdditionalApiKeys } from "~/services/additionalApiKeyIssuance.server";
88
import { apiKeyTelemetry, type ApiKeyTelemetry } from "~/services/apiKeyTelemetry.server";
@@ -165,7 +165,7 @@ export async function createEnvironmentApiKey(
165165
const matchingTasks = await prismaClient.taskIdentifier.count({
166166
where: {
167167
runtimeEnvironmentId: taskEnvironmentId,
168-
slug: { in: selectedTasks },
168+
slug: { in: boundedIn(selectedTasks) },
169169
runtimeEnvironment: {
170170
OR: [{ id: environment.id }, { parentEnvironmentId: environment.id }],
171171
},

apps/webapp/app/models/member.server.ts

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ import { getDefaultEnvironmentConcurrencyLimit } from "~/services/platform.v3.se
1111
import { rbac } from "~/services/rbac.server";
1212
import { ssoController } from "~/services/sso.server";
1313

14+
import { boundedIn } from "@trigger.dev/database";
1415
export const INVITE_NOT_FOUND = "Invite not found";
1516
export const INVITE_BLOCKED_DIRECTORY_MANAGED =
1617
"Membership for this organization is managed by Directory Sync, so invites can't be accepted.";
@@ -134,7 +135,7 @@ export async function inviteMembers({
134135
const existingMembers = await prisma.orgMember.findMany({
135136
where: {
136137
organizationId: org.id,
137-
user: { email: { in: [...uniqueEmails] } },
138+
user: { email: { in: boundedIn([...uniqueEmails]) } },
138139
},
139140
select: { user: { select: { email: true } } },
140141
});
@@ -233,7 +234,7 @@ export async function getProjectsMissingMemberDevelopmentEnvironments({
233234
organizationId,
234235
...memberDevelopmentEnvironmentWhere({
235236
orgMemberId: memberId,
236-
projectId: { in: projects.map((project) => project.id) },
237+
projectId: { in: boundedIn(projects.map((project) => project.id)) },
237238
}),
238239
},
239240
select: { projectId: true },

apps/webapp/app/models/vercelIntegration.server.ts

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ import {
2424
} from "~/v3/vercel/vercelProjectIntegrationSchema";
2525
import { EnvironmentVariablesRepository } from "~/v3/environmentVariables/environmentVariablesRepository.server";
2626
import { isReservedForExternalSync } from "~/v3/environmentVariableRules.server";
27+
import { boundedIn } from "@trigger.dev/database";
2728
import {
2829
callVercelWithRecovery,
2930
wrapVercelCallWithRecovery,
@@ -1415,7 +1416,7 @@ export class VercelIntegrationRepository {
14151416
variable: {
14161417
projectId: params.projectId,
14171418
key: {
1418-
in: varsToSync.map((v) => v.key),
1419+
in: boundedIn(varsToSync.map((v) => v.key)),
14191420
},
14201421
},
14211422
},

apps/webapp/app/presenters/v3/ApiBatchResultsPresenter.server.ts

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,7 @@ import type { AuthenticatedEnvironment } from "~/services/apiAuth.server";
1212
import { runStore as defaultRunStore } from "~/v3/runStore.server";
1313
import { BasePresenter } from "./basePresenter.server";
1414

15+
import { boundedIn } from "@trigger.dev/database";
1516
/**
1617
* Run-ops read-through wiring. All optional; absent (or `splitEnabled` falsy) collapses `call` to
1718
* passthrough. `legacyReplica` is a READ REPLICA handle only — there is NO legacy-primary field.
@@ -114,7 +115,7 @@ export class ApiBatchResultsPresenter extends BasePresenter {
114115

115116
const taskRuns = await this.runStore.findRuns(
116117
{
117-
where: { id: { in: taskRunIds } },
118+
where: { id: { in: boundedIn(taskRunIds) } },
118119
select: memberRunSelect,
119120
},
120121
this._prisma
@@ -181,7 +182,7 @@ export class ApiBatchResultsPresenter extends BasePresenter {
181182
const taskRunIds = batchRun.items.map((item) => item.taskRunId);
182183

183184
const newRows = (await newClient.taskRun.findMany({
184-
where: { id: { in: taskRunIds } },
185+
where: { id: { in: boundedIn(taskRunIds) } },
185186
select: memberRunSelect,
186187
})) as TaskRunWithAttempts[];
187188
const runsById = new Map(newRows.map((run) => [run.id, run]));
@@ -193,7 +194,7 @@ export class ApiBatchResultsPresenter extends BasePresenter {
193194
);
194195
if (legacyCandidateIds.length > 0) {
195196
const legacyRows = (await legacyReplica.taskRun.findMany({
196-
where: { id: { in: legacyCandidateIds } },
197+
where: { id: { in: boundedIn(legacyCandidateIds) } },
197198
select: memberRunSelect,
198199
})) as TaskRunWithAttempts[];
199200
for (const run of legacyRows) {

apps/webapp/app/presenters/v3/ApiRunListPresenter.server.ts

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
import { MachinePresetName, parsePacket, RunStatus } from "@trigger.dev/core/v3";
2-
import { type Project, type RuntimeEnvironment, type TaskRunStatus } from "@trigger.dev/database";
2+
import {
3+
type Project,
4+
type RuntimeEnvironment,
5+
type TaskRunStatus,
6+
boundedIn,
7+
} from "@trigger.dev/database";
38
import assertNever from "assert-never";
49
import { z } from "zod";
510
import type { API_VERSIONS } from "~/api/versions";
@@ -208,7 +213,7 @@ export class ApiRunListPresenter extends BasePresenter {
208213
where: {
209214
projectId: project.id,
210215
slug: {
211-
in: searchParams["filter[env]"],
216+
in: boundedIn(searchParams["filter[env]"]),
212217
},
213218
},
214219
});

apps/webapp/app/presenters/v3/BatchListPresenter.server.ts

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
import { type BatchTaskRunStatus } from "@trigger.dev/database";
1+
import { type BatchTaskRunStatus, boundedIn } from "@trigger.dev/database";
22
import { type RunOpsPrismaClient } from "@internal/run-ops-database";
33
import parse from "parse-duration";
44
import { type PrismaClientOrTransaction } from "~/db.server";
@@ -263,7 +263,7 @@ export class BatchListPresenter extends BasePresenter {
263263
: {}),
264264
...(friendlyId ? { friendlyId } : {}),
265265
...(statuses && statuses.length > 0
266-
? { status: { in: statuses }, batchVersion: { not: "v1" } }
266+
? { status: { in: boundedIn(statuses) }, batchVersion: { not: "v1" } }
267267
: {}),
268268
...(createdAtGte !== undefined || createdAtLte !== undefined
269269
? {

apps/webapp/app/presenters/v3/EnvironmentVariablesPresenter.server.ts

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ import type { SyncEnvVarsMapping, EnvSlug } from "~/v3/vercel/vercelProjectInteg
88
import { VercelIntegrationService } from "~/services/vercelIntegration.server";
99
import { loadEnvironmentVariablesEnvironments } from "./environmentVariablesEnvironments.server";
1010

11+
import { boundedIn } from "@trigger.dev/database";
1112
type Result = Awaited<ReturnType<EnvironmentVariablesPresenter["call"]>>;
1213
export type EnvironmentVariableWithSetValues = Result["environmentVariables"][number];
1314

@@ -72,7 +73,7 @@ export class EnvironmentVariablesPresenter {
7273
},
7374
where: {
7475
environmentId: {
75-
in: environmentIds,
76+
in: boundedIn(environmentIds),
7677
},
7778
},
7879
},
@@ -103,7 +104,7 @@ export class EnvironmentVariablesPresenter {
103104
? await this.#replicaClient.user.findMany({
104105
where: {
105106
id: {
106-
in: Array.from(userIds),
107+
in: boundedIn(Array.from(userIds)),
107108
},
108109
},
109110
select: {

0 commit comments

Comments
 (0)