Skip to content

Commit b98dd79

Browse files
authored
feat(webapp,run-store,database): env-configurable transaction resilience (maxWait + tx-start retry) (#4623)
## What Makes two transaction-resilience behaviors real and env-var configurable, defaults set to the good values, so we can tune during and after the Aug 15 database patch window without a redeploy: - **maxWait 2s → 10s** (TRI-12982): how long Prisma waits to borrow a connection before it can `BEGIN`. A restart freeze holds the pool full, and the only thing that errored was transaction starts giving up at 2s. - **Retry transaction-start P2028-at-acquisition** (TRI-12984): when Prisma can't borrow a connection within `maxWait` it raises P2028 (`Unable to start a transaction in the given time`) and **no SQL ran**, so retrying is safe. Scoped narrowly: only that error (never P2024 pool-exhaustion), 2 attempts, jittered backoff, and a token-bucket budget so a mass freeze can't amplify into a retry storm. ## Env vars (`DATABASE_*` convention) Generic defaults: | var | default | |---|---| | `DATABASE_TRANSACTION_MAX_WAIT_MS` | `10000` | | `DATABASE_TRANSACTION_START_RETRY_ENABLED` | `true` (kill switch) | | `DATABASE_TRANSACTION_START_RETRY_MAX_ATTEMPTS` | `2` | | `DATABASE_TRANSACTION_START_RETRY_BACKOFF_MIN_MS` | `50` | | `DATABASE_TRANSACTION_START_RETRY_BACKOFF_MAX_MS` | `250` | | `DATABASE_TRANSACTION_START_RETRY_BUDGET_PER_SEC` | `50` | | `DATABASE_TRANSACTION_START_RETRY_BUDGET_BURST` | `100` | Per-writer-pool overrides, each falling back to the generic when unset (same pattern as the per-client pool/connect-timeout work): `RUN_OPS_DATABASE_TRANSACTION_*` and `RUN_OPS_LEGACY_DATABASE_TRANSACTION_*` (all 7 knobs each). Transactions only open on writer pools, so those are the only pools with their own knobs. Each pool gets its **own** token bucket, so a storm on one pool can't drain another's retry budget. ## Design - The retry primitives live in `internal-packages/database` and never read `process.env` (IoC): a P2028-at-acquisition classifier, a `TokenBucketRetryBudget`, and `withTransactionStartRetry`, folded into the `$transaction` helper via a new `startRetry` option. Config is resolved at the app boundary and threaded in. - The `$transaction` helper is the chokepoint (wraps the whole transaction), not the per-statement `$allOperations` extension. - The run engine's writes go through `PostgresRunStore`'s own `.$transaction(...)`, not the webapp helper, so both the helper and the two `PostgresRunStore` sites apply maxWait + retry (sharing the per-pool config). Builds on the `options?: { timeout, maxWait }` seam added in #4514. - Webapp `$transaction` call sites get the default `maxWait` + retry injected at one merge point, so no call site needed editing. ## Evidence - Unit red/green in `internal-packages/database`: reverting the helper wiring turned the acquisition-retry test red (`Unable to start a transaction in the given time`), re-applying it green. Full package suite 25/25. Covers: classifier (P2028-acq yes, P2024 no, in-tx P2028 no), retry (retry-then-succeed, no-retry P2024, stop at maxAttempts, disabled, budget-exhausted, jitter bounds), token bucket, and `$transaction` wiring. - Typecheck clean: webapp, run-store, run-engine. - Full-stack run: bounded queue-ay pass (15 projects, real dev runs through the run-engine `PostgresRunStore` transaction path). 13 pass; the 2 failures are one documented known-failure and one stale-worker-state flake that passes 2/2 with this change active on a fresh app. - Boots cleanly with per-pool overrides set. ## Configuration & rollout Ship **inert** first (zero behavior change), then flip to the good values **live via env** — no redeploy needed for either. ### Inert — behaves exactly as today ``` DATABASE_TRANSACTION_MAX_WAIT_MS=2000 # Prisma's built-in default (change defaults to 10000) DATABASE_TRANSACTION_START_RETRY_ENABLED=false # disable the new retry entirely ``` `maxWait=2000` is what every path used before (Prisma's default; the run-store sites and the helper passed no maxWait). `retry=false` short-circuits `withTransactionStartRetry` to a single run and makes the serialization-retry exclusion a no-op. Verified on the pooler-freeze rig: identical fail-fast P2028 at ~2003ms with zero retries — byte-for-byte current behavior, across all pools. ### Production ("good") — the baked defaults Rely on defaults (nothing to set) or set explicitly: ``` DATABASE_TRANSACTION_MAX_WAIT_MS=10000 DATABASE_TRANSACTION_START_RETRY_ENABLED=true DATABASE_TRANSACTION_START_RETRY_MAX_ATTEMPTS=3 # 3 attempts (2 retries); ~30s acquisition tolerance covers a ~20-25s freeze DATABASE_TRANSACTION_START_RETRY_BACKOFF_MIN_MS=50 DATABASE_TRANSACTION_START_RETRY_BACKOFF_MAX_MS=250 DATABASE_TRANSACTION_START_RETRY_BUDGET_PER_SEC=50 DATABASE_TRANSACTION_START_RETRY_BUDGET_BURST=100 ``` Per-pool overrides `RUN_OPS_DATABASE_TRANSACTION_*` and `RUN_OPS_LEGACY_DATABASE_TRANSACTION_*` (all seven knobs each) are optional and fall back to the generic set — not needed for v1; the generic set covers the control-plane, run-ops, and run-ops-legacy writer pools. Readers open no transactions and take nothing. **Guardrail:** the retry only engages when a pool's `pool_timeout` > `maxWait`. Prod is fine (`DATABASE_POOL_TIMEOUT=60` >> 10). Do not set any writer pool's `pool_timeout` at or under `maxWait`, or saturation failures flip from retryable P2028 to non-retryable P2024 and the retry silently stops helping. ### Rollback Env flip (set inert) or revert. Retry only fires where no SQL ran, and the per-pool token bucket caps a storm. No migration. refs TRI-13295, TRI-12982, TRI-12984
1 parent 69f396f commit b98dd79

8 files changed

Lines changed: 770 additions & 38 deletions

File tree

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
area: webapp
3+
type: improvement
4+
---
5+
6+
Triggering tasks is now more resilient to brief, transient service interruptions, so short stalls are less likely to surface as errors.

apps/webapp/app/db.server.ts

Lines changed: 65 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,13 @@ import {
3232
import { computeRunOpsSplitReadEnabled } from "./v3/runOpsMigration/runOpsSplitReadGate";
3333
import { assertControlPlaneCoresidencyAdvisory } from "./v3/runOpsMigration/controlPlaneCoresidencySentinel.server";
3434
import { DATASOURCE_CONTEXT_KEY, startActiveSpan } from "./v3/tracer.server";
35+
import {
36+
controlPlaneTransactionResilience,
37+
registerTransactionResilience,
38+
resilienceForClient,
39+
runOpsLegacyTransactionResilience,
40+
runOpsTransactionResilience,
41+
} from "./v3/transactionResilience.server";
3542
import type { Span } from "@opentelemetry/api";
3643
import { context, trace } from "@opentelemetry/api";
3744
import { queryPerformanceMonitor } from "./utils/queryPerformanceMonitor.server";
@@ -59,6 +66,18 @@ function logTransactionPrismaError(error: Prisma.PrismaClientKnownRequestError)
5966
});
6067
}
6168

69+
function withTransactionDefaults(
70+
client: PrismaClientOrTransaction,
71+
options?: PrismaTransactionOptions
72+
): PrismaTransactionOptions {
73+
const resilience = resilienceForClient(client as object);
74+
return {
75+
maxWait: resilience.maxWait,
76+
...options,
77+
startRetry: options?.startRetry ?? resilience.startRetry,
78+
};
79+
}
80+
6281
export async function $transaction<R>(
6382
prisma: PrismaClientOrTransaction,
6483
name: string,
@@ -93,35 +112,41 @@ async function $transactionInner<R>(
93112
options?: PrismaTransactionOptions
94113
): Promise<R | undefined> {
95114
if (typeof fnOrName === "string") {
115+
const effectiveOptions = withTransactionDefaults(prisma, options);
96116
return await startActiveSpan(fnOrName, async (span) => {
97117
span.setAttribute("$transaction", true);
98118

99-
if (options?.isolationLevel) {
100-
span.setAttribute("isolation_level", options.isolationLevel);
119+
if (effectiveOptions.isolationLevel) {
120+
span.setAttribute("isolation_level", effectiveOptions.isolationLevel);
101121
}
102122

103-
if (options?.timeout) {
104-
span.setAttribute("timeout", options.timeout);
123+
if (effectiveOptions.timeout) {
124+
span.setAttribute("timeout", effectiveOptions.timeout);
105125
}
106126

107-
if (options?.maxWait) {
108-
span.setAttribute("max_wait", options.maxWait);
127+
if (effectiveOptions.maxWait) {
128+
span.setAttribute("max_wait", effectiveOptions.maxWait);
109129
}
110130

111-
if (options?.swallowPrismaErrors) {
112-
span.setAttribute("swallow_prisma_errors", options.swallowPrismaErrors);
131+
if (effectiveOptions.swallowPrismaErrors) {
132+
span.setAttribute("swallow_prisma_errors", effectiveOptions.swallowPrismaErrors);
113133
}
114134

115135
const fn = fnOrOptions as (prisma: PrismaTransactionClient, span: Span) => Promise<R>;
116136

117-
return transac(prisma, (client) => fn(client, span), logTransactionPrismaError, options);
137+
return transac(
138+
prisma,
139+
(client) => fn(client, span),
140+
logTransactionPrismaError,
141+
effectiveOptions
142+
);
118143
});
119144
} else {
120145
return transac(
121146
prisma,
122147
fnOrName,
123148
logTransactionPrismaError,
124-
typeof fnOrOptions === "function" ? undefined : fnOrOptions
149+
withTransactionDefaults(prisma, typeof fnOrOptions === "function" ? undefined : fnOrOptions)
125150
);
126151
}
127152
}
@@ -180,7 +205,10 @@ function captureInfraErrorsRunOps(client: RunOpsPrismaClient): RunOpsPrismaClien
180205
}
181206

182207
export const prisma = singleton("prisma", () =>
183-
captureInfrastructureErrors(tagDatasource("control-plane-writer", getClient()))
208+
registerTransactionResilience(
209+
captureInfrastructureErrors(tagDatasource("control-plane-writer", getClient())),
210+
controlPlaneTransactionResilience
211+
)
184212
);
185213

186214
export const $replica: PrismaReplicaClient = singleton("replica", () => {
@@ -309,15 +337,18 @@ const runOpsTopology: RunOpsTopology = singleton("runOpsTopology", () => {
309337
{
310338
controlPlane: { writer: prisma, replica: $replica },
311339
buildNewWriter: (url, clientType) =>
312-
captureInfraErrorsRunOps(
313-
tagDatasourceRunOps(
314-
"run-ops-writer",
315-
buildRunOpsWriterClient({
316-
url,
317-
clientType,
318-
useDriverAdapter: env.RUN_OPS_DATABASE_WRITER_DRIVER_ADAPTER === "1",
319-
})
320-
)
340+
registerTransactionResilience(
341+
captureInfraErrorsRunOps(
342+
tagDatasourceRunOps(
343+
"run-ops-writer",
344+
buildRunOpsWriterClient({
345+
url,
346+
clientType,
347+
useDriverAdapter: env.RUN_OPS_DATABASE_WRITER_DRIVER_ADAPTER === "1",
348+
})
349+
)
350+
),
351+
runOpsTransactionResilience
321352
),
322353
// Brand the run-ops replica (only built for a real replica URL) so routed replica reads stay
323354
// off the primary. When no replica URL is set, selectRunOpsTopology reuses the writer here —
@@ -338,17 +369,20 @@ const runOpsTopology: RunOpsTopology = singleton("runOpsTopology", () => {
338369
// Legacy client shares the exact control-plane wrapper stack (the legacy DB carries the full
339370
// control-plane schema); markReadReplicaClient only on a real replica URL, as with the NEW replica.
340371
buildLegacyWriter: (url, clientType) =>
341-
captureInfrastructureErrors(
342-
tagDatasource(
343-
"legacy-run-ops-writer",
344-
buildWriterClient({
345-
url,
346-
clientType,
347-
poolTimeout: env.RUN_OPS_LEGACY_DATABASE_WRITER_POOL_TIMEOUT,
348-
connectTimeout: env.RUN_OPS_LEGACY_DATABASE_WRITER_CONNECTION_TIMEOUT,
349-
useDriverAdapter: env.RUN_OPS_LEGACY_DATABASE_WRITER_DRIVER_ADAPTER === "1",
350-
})
351-
)
372+
registerTransactionResilience(
373+
captureInfrastructureErrors(
374+
tagDatasource(
375+
"legacy-run-ops-writer",
376+
buildWriterClient({
377+
url,
378+
clientType,
379+
poolTimeout: env.RUN_OPS_LEGACY_DATABASE_WRITER_POOL_TIMEOUT,
380+
connectTimeout: env.RUN_OPS_LEGACY_DATABASE_WRITER_CONNECTION_TIMEOUT,
381+
useDriverAdapter: env.RUN_OPS_LEGACY_DATABASE_WRITER_DRIVER_ADAPTER === "1",
382+
})
383+
)
384+
),
385+
runOpsLegacyTransactionResilience
352386
),
353387
buildLegacyReplica: (url, clientType) =>
354388
markReadReplicaClient(

apps/webapp/app/env.server.ts

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -114,6 +114,26 @@ const OptionalIntEnv = z.preprocess(
114114
z.coerce.number().int().optional()
115115
);
116116

117+
/** Optional boolean env var; blank/whitespace/unset normalises to undefined (so it falls back). */
118+
const OptionalBoolEnv = z.preprocess((v) => {
119+
if (typeof v !== "string" || v.trim() === "") return undefined;
120+
return ["true", "1"].includes(v.toLowerCase().trim());
121+
}, z.boolean().optional());
122+
123+
/** Boolean env var with a default where blank/whitespace falls back to the default instead of parsing as false. */
124+
const BoolEnvWithDefault = (defaultValue: boolean) =>
125+
z.preprocess((v) => {
126+
if (typeof v !== "string" || v.trim() === "") return undefined;
127+
return ["true", "1"].includes(v.toLowerCase().trim());
128+
}, z.boolean().default(defaultValue));
129+
130+
/** Int env var with a default where a blank/whitespace value falls back to the default instead of coercing to 0. */
131+
const IntEnvWithDefault = (defaultValue: number) =>
132+
z.preprocess(
133+
(v) => (typeof v === "string" && v.trim() === "" ? undefined : v),
134+
z.coerce.number().int().default(defaultValue)
135+
);
136+
117137
/**
118138
* Optional int env var for a limit that can be switched off. Blank, whitespace and `0` all mean
119139
* "no limit" and normalise to undefined; anything else that is set must be greater than zero.
@@ -142,6 +162,27 @@ const EnvironmentSchema = z
142162
DATABASE_WRITER_CONNECTION_TIMEOUT: OptionalIntEnv,
143163
DATABASE_READ_REPLICA_POOL_TIMEOUT: OptionalIntEnv,
144164
DATABASE_READ_REPLICA_CONNECTION_TIMEOUT: OptionalIntEnv,
165+
DATABASE_TRANSACTION_MAX_WAIT_MS: IntEnvWithDefault(10000),
166+
DATABASE_TRANSACTION_START_RETRY_ENABLED: BoolEnvWithDefault(true),
167+
DATABASE_TRANSACTION_START_RETRY_MAX_ATTEMPTS: IntEnvWithDefault(3),
168+
DATABASE_TRANSACTION_START_RETRY_BACKOFF_MIN_MS: IntEnvWithDefault(50),
169+
DATABASE_TRANSACTION_START_RETRY_BACKOFF_MAX_MS: IntEnvWithDefault(250),
170+
DATABASE_TRANSACTION_START_RETRY_BUDGET_PER_SEC: IntEnvWithDefault(50),
171+
DATABASE_TRANSACTION_START_RETRY_BUDGET_BURST: IntEnvWithDefault(100),
172+
RUN_OPS_DATABASE_TRANSACTION_MAX_WAIT_MS: OptionalIntEnv,
173+
RUN_OPS_DATABASE_TRANSACTION_START_RETRY_ENABLED: OptionalBoolEnv,
174+
RUN_OPS_DATABASE_TRANSACTION_START_RETRY_MAX_ATTEMPTS: OptionalIntEnv,
175+
RUN_OPS_DATABASE_TRANSACTION_START_RETRY_BACKOFF_MIN_MS: OptionalIntEnv,
176+
RUN_OPS_DATABASE_TRANSACTION_START_RETRY_BACKOFF_MAX_MS: OptionalIntEnv,
177+
RUN_OPS_DATABASE_TRANSACTION_START_RETRY_BUDGET_PER_SEC: OptionalIntEnv,
178+
RUN_OPS_DATABASE_TRANSACTION_START_RETRY_BUDGET_BURST: OptionalIntEnv,
179+
RUN_OPS_LEGACY_DATABASE_TRANSACTION_MAX_WAIT_MS: OptionalIntEnv,
180+
RUN_OPS_LEGACY_DATABASE_TRANSACTION_START_RETRY_ENABLED: OptionalBoolEnv,
181+
RUN_OPS_LEGACY_DATABASE_TRANSACTION_START_RETRY_MAX_ATTEMPTS: OptionalIntEnv,
182+
RUN_OPS_LEGACY_DATABASE_TRANSACTION_START_RETRY_BACKOFF_MIN_MS: OptionalIntEnv,
183+
RUN_OPS_LEGACY_DATABASE_TRANSACTION_START_RETRY_BACKOFF_MAX_MS: OptionalIntEnv,
184+
RUN_OPS_LEGACY_DATABASE_TRANSACTION_START_RETRY_BUDGET_PER_SEC: OptionalIntEnv,
185+
RUN_OPS_LEGACY_DATABASE_TRANSACTION_START_RETRY_BUDGET_BURST: OptionalIntEnv,
145186
// Dashboard-agent conversation store. Cloud points this at a dedicated
146187
// database; when unset it falls back to DATABASE_URL (OSS), where
147188
// the tables live in the isolated `trigger_dashboard_agent` schema.

apps/webapp/app/v3/runStore.server.ts

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,10 @@ import {
1212
} from "~/db.server";
1313
import { env } from "~/env.server";
1414
import { singleton } from "~/utils/singleton";
15+
import {
16+
resilienceForClient,
17+
type TransactionResilienceConfig,
18+
} from "./transactionResilience.server";
1519

1620
type BuildRunStoreDeps = {
1721
/** Boot constant: true only when both run-ops DBs are configured and the split flag is on. */
@@ -27,6 +31,10 @@ type BuildRunStoreDeps = {
2731
singleReplica: PrismaReplicaClient;
2832
/** Residency classifier; defaults to ownerEngine inside RoutingRunStore. */
2933
classify?: (id: string) => Residency;
34+
/** Per-pool transaction-resilience configs threaded into the store(s) this builds (IoC). */
35+
singleResilience?: TransactionResilienceConfig;
36+
newResilience?: TransactionResilienceConfig;
37+
legacyResilience?: TransactionResilienceConfig;
3038
};
3139

3240
/**
@@ -46,6 +54,8 @@ export function buildRunStore(deps: BuildRunStoreDeps): RunStore {
4654
return new PostgresRunStore({
4755
prisma: deps.singleWriter,
4856
readOnlyPrisma: deps.singleReplica,
57+
maxWait: deps.singleResilience?.maxWait,
58+
transactionStartRetry: deps.singleResilience?.startRetry,
4959
});
5060
}
5161

@@ -59,10 +69,14 @@ export function buildRunStore(deps: BuildRunStoreDeps): RunStore {
5969
prisma: deps.newWriter,
6070
readOnlyPrisma: deps.newReplica,
6171
schemaVariant: "dedicated",
72+
maxWait: deps.newResilience?.maxWait,
73+
transactionStartRetry: deps.newResilience?.startRetry,
6274
});
6375
const legacyStore = new PostgresRunStore({
6476
prisma: deps.legacyWriter,
6577
readOnlyPrisma: deps.legacyReplica,
78+
maxWait: deps.legacyResilience?.maxWait,
79+
transactionStartRetry: deps.legacyResilience?.startRetry,
6680
});
6781

6882
return new RoutingRunStore({
@@ -110,12 +124,16 @@ export const runStore: RunStore = singleton("RunStore", () => {
110124
splitEnabled: false,
111125
singleWriter: prisma,
112126
singleReplica: $replica,
127+
singleResilience: resilienceForClient(prisma),
113128
});
114129
}
115130
return buildRunStore({
116131
splitEnabled: true,
117132
...handles,
118133
singleWriter: prisma,
119134
singleReplica: $replica,
135+
singleResilience: resilienceForClient(prisma),
136+
newResilience: resilienceForClient(handles.newWriter),
137+
legacyResilience: resilienceForClient(handles.legacyWriter),
120138
});
121139
});
Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
import { TokenBucketRetryBudget, type TransactionStartRetryConfig } from "@trigger.dev/database";
2+
import { env } from "~/env.server";
3+
import { logger } from "~/services/logger.server";
4+
5+
/**
6+
* Resolved transaction-resilience config for one writer pool. Each pool gets its own
7+
* {@link TransactionStartRetryConfig} (with its OWN token bucket, so a storm on one pool cannot
8+
* drain another's retry budget) plus the `maxWait` applied when that pool opens a transaction.
9+
* Env is read here at the app boundary (IoC); the library never reads env.
10+
*
11+
* Kept out of `db.server` on purpose: `db.server` is mocked wholesale by ~150 tests, and a new
12+
* export there breaks every mock that does not list it. Both `db.server` and `runStore.server`
13+
* import these from here instead.
14+
*/
15+
export type TransactionResilienceConfig = {
16+
maxWait: number;
17+
startRetry: TransactionStartRetryConfig;
18+
};
19+
20+
function resolveTransactionResilience(
21+
pool: "control-plane" | "run-ops" | "run-ops-legacy",
22+
overrides: {
23+
maxWaitMs?: number;
24+
enabled?: boolean;
25+
maxAttempts?: number;
26+
backoffMinMs?: number;
27+
backoffMaxMs?: number;
28+
budgetPerSec?: number;
29+
budgetBurst?: number;
30+
}
31+
): TransactionResilienceConfig {
32+
const budgetPerSec =
33+
overrides.budgetPerSec ?? env.DATABASE_TRANSACTION_START_RETRY_BUDGET_PER_SEC;
34+
const budgetBurst = overrides.budgetBurst ?? env.DATABASE_TRANSACTION_START_RETRY_BUDGET_BURST;
35+
return {
36+
maxWait: Math.max(0, overrides.maxWaitMs ?? env.DATABASE_TRANSACTION_MAX_WAIT_MS),
37+
startRetry: {
38+
options: {
39+
enabled: overrides.enabled ?? env.DATABASE_TRANSACTION_START_RETRY_ENABLED,
40+
maxAttempts: overrides.maxAttempts ?? env.DATABASE_TRANSACTION_START_RETRY_MAX_ATTEMPTS,
41+
backoffMinMs: overrides.backoffMinMs ?? env.DATABASE_TRANSACTION_START_RETRY_BACKOFF_MIN_MS,
42+
backoffMaxMs: overrides.backoffMaxMs ?? env.DATABASE_TRANSACTION_START_RETRY_BACKOFF_MAX_MS,
43+
},
44+
budget: new TokenBucketRetryBudget({ ratePerSec: budgetPerSec, burst: budgetBurst }),
45+
onRetry: ({ attempt, delayMs }) =>
46+
logger.warn("retrying transaction start after acquisition failure", {
47+
pool,
48+
attempt,
49+
delayMs,
50+
}),
51+
},
52+
};
53+
}
54+
55+
export const controlPlaneTransactionResilience = resolveTransactionResilience("control-plane", {});
56+
57+
export const runOpsTransactionResilience = resolveTransactionResilience("run-ops", {
58+
maxWaitMs: env.RUN_OPS_DATABASE_TRANSACTION_MAX_WAIT_MS,
59+
enabled: env.RUN_OPS_DATABASE_TRANSACTION_START_RETRY_ENABLED,
60+
maxAttempts: env.RUN_OPS_DATABASE_TRANSACTION_START_RETRY_MAX_ATTEMPTS,
61+
backoffMinMs: env.RUN_OPS_DATABASE_TRANSACTION_START_RETRY_BACKOFF_MIN_MS,
62+
backoffMaxMs: env.RUN_OPS_DATABASE_TRANSACTION_START_RETRY_BACKOFF_MAX_MS,
63+
budgetPerSec: env.RUN_OPS_DATABASE_TRANSACTION_START_RETRY_BUDGET_PER_SEC,
64+
budgetBurst: env.RUN_OPS_DATABASE_TRANSACTION_START_RETRY_BUDGET_BURST,
65+
});
66+
67+
export const runOpsLegacyTransactionResilience = resolveTransactionResilience("run-ops-legacy", {
68+
maxWaitMs: env.RUN_OPS_LEGACY_DATABASE_TRANSACTION_MAX_WAIT_MS,
69+
enabled: env.RUN_OPS_LEGACY_DATABASE_TRANSACTION_START_RETRY_ENABLED,
70+
maxAttempts: env.RUN_OPS_LEGACY_DATABASE_TRANSACTION_START_RETRY_MAX_ATTEMPTS,
71+
backoffMinMs: env.RUN_OPS_LEGACY_DATABASE_TRANSACTION_START_RETRY_BACKOFF_MIN_MS,
72+
backoffMaxMs: env.RUN_OPS_LEGACY_DATABASE_TRANSACTION_START_RETRY_BACKOFF_MAX_MS,
73+
budgetPerSec: env.RUN_OPS_LEGACY_DATABASE_TRANSACTION_START_RETRY_BUDGET_PER_SEC,
74+
budgetBurst: env.RUN_OPS_LEGACY_DATABASE_TRANSACTION_START_RETRY_BUDGET_BURST,
75+
});
76+
77+
const transactionResilienceByClient = new WeakMap<object, TransactionResilienceConfig>();
78+
79+
/**
80+
* Associate a writer client with its pool's resilience config. Returns the client for inline use at
81+
* construction. Kept here (not in db.server) so nothing new lands on db.server's wholesale-mocked
82+
* export surface.
83+
*/
84+
export function registerTransactionResilience<T extends object>(
85+
client: T,
86+
resilience: TransactionResilienceConfig
87+
): T {
88+
transactionResilienceByClient.set(client, resilience);
89+
return client;
90+
}
91+
92+
/**
93+
* The resilience config registered for a writer client, or the control-plane config as a safe
94+
* fallback. Derives resilience from the ACTUAL client identity rather than an assumed routing role,
95+
* so run-ops clients aliased onto the control-plane pool (split flag off) correctly get the
96+
* control-plane config instead of a run-ops override.
97+
*/
98+
export function resilienceForClient(client: object): TransactionResilienceConfig {
99+
return transactionResilienceByClient.get(client) ?? controlPlaneTransactionResilience;
100+
}

0 commit comments

Comments
 (0)