fix(cloudflare): Enforce flush timeout across Workflow lifecycle

[matthewbjones]

Cloudflare was reporting that Sentry-owned waitUntil() tasks remained pending after instrumented Workflows had completed. The configured flush(2000) timeout did not fully bound the work associated with a flush.

Three separate lifecycle gaps contributed to the warning.

Transport requests survived a timed-out drain

IsolatedPromiseBuffer.drain(timeout) returned false when its timer expired, but the request producers it had started continued running. A slow or stalled fetch therefore remained attached to the invocation after Sentry reported that its flush had timed out.

Each drain now owns an AbortController. When that drain expires, it aborts only the requests started by that drain. makeCloudflareTransport combines the drain signal with any caller-provided fetchOptions.signal, removes both listeners when the request settles, and keeps overlapping drains isolated so one timeout cannot abort another drain's requests.

Workflow steps inherited the previous step's flush point

Cached Cloudflare clients reuse the Workflow run's isolation scope across step RPC invocations. The first boundary flush marked that scope's flushPointReached state as true, and later steps inherited it. Envelopes created by those later steps were consequently treated as post-invocation telemetry and registered additional eager waitUntil(transport.flush(2000)) drains even though every step already performs its own boundary flush.

The wrapped step callback now resets flushPointReached before it captures telemetry. This makes each Workflow step begin before its own flush point, while retaining eager delivery for telemetry that is genuinely created after that step's boundary flush.

Flush-lock finalization was outside the timeout

CloudflareClient.flush(timeout) awaited flushLock.finalize() before applying timeout to pending spans and the transport. A user waitUntil task that never settled could therefore keep the Sentry flush pending indefinitely. The pending-span and transport phases could also each receive the original timeout instead of the time remaining from one overall deadline.

flush() now computes one deadline and passes the remaining budget through flush-lock finalization, pending-span completion, and the transport drain. It returns false without beginning a later phase when the budget has already been exhausted.

Validation

Each failure has focused red/green regression coverage:

• a timed-out drain aborts its request, preserves a caller signal, and does not abort requests owned by an overlapping drain;
• every Workflow step starts with its own unreached flush point; and
• a never-settling flush lock causes flush(timeout) to return false within the deadline.

The production reproduction uses 100 trivial step.do() calls and an HTTP transport that delays every response for 120 seconds. The transport fix alone removed the warnings with cacheClient: false, but warnings remained on the current cached-client path. Disabling eager envelope delivery isolated the remaining behavior to the inherited Workflow flush point. With eager delivery enabled and both lifecycle fixes applied, 20/20 deployed Workflow instances completed with zero matching waitUntil() warnings in Cloudflare Observability.

The complete @sentry/cloudflare suite passes (955/955), along with package lint, build, and typecheck.

• If you've added code that should be tested, please add tests.
• Ensure your code lints and the test suite passes (yarn lint) & (yarn test).
• Link an issue if there is one related to your pull request. If no issue is linked, one will be auto-generated and linked.

Closes #24482

[cursor]

Stale Bugbot comment from a previous run.

[cursor]

Stale Bugbot comment from a previous run.

[JPeer264]

Thanks a lot for opening a PR for this already. I'll take a close look today - as soon we will ship v11 where this should actually not be needed anymore. It would still be good to be fixed in v10 though.

[matthewbjones]

@JPeer264 thanks. Yes, would be great to have in v10. We've been seeing this in our production app for a few months now and finally we were able to identify where these warnings were coming from. I did my best to keep the changes as minimal as possible but still enforce the 2000ms timeout as intended. Each newly added test does exercise a bug and then confirms the refactored code addresses it. We also deployed a version of this Sentry SDK into production and none of our reproduction Workflows can cause the warning to emit anymore.

[JPeer264]

Alright, I had a check over it and had couple of reproductions running for this. This seemed like a true issue also for the new cacheClient (it is actually kinda expected across the SDKs, but within Cloudflare we can't do what we do in other SDKs, because of this issue you posted).

So, I hope it is ok, to reduce review ping pong on this issue I added a change. I left the AbortController change from you as this was the actual fix, but had to adjust other parts. Since I'd like to remove cacheClient: false in the next major I wanted to only contain necessary fixes for cacheClient: false specific to them, so it's easier to remove later on. It turned out that some things were not necessary, at least with my reproductions. (in this case flushLock will be removed as it was quite hacky IMO - but is necessary to be adapted here)

I also added an integration test that shows the successful aborted requests when the endpoint is slow (or rather not receiving anything in this case).

I'll let bugbot run over it once and see what it says.

Do you think this fix is sufficient? Since you already applied this PR to your code, would you apply the current state again and verify? I'm pretty certain it would work as well.

[JPeer264]

bugbot run

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit fb63820. Configure here.}

[JPeer264]

I'd let this sit over the weekend in case another reviewer wants to go over it. I will come back on it on monday

[matthewbjones]

@JPeer264 Thanks for taking a look. I think this retains one of the three fixes: aborting in-flight transport requests. That abort applies to both cache modes, but in our deployed reproduction the transport fix alone eliminated the warnings only with cacheClient: false. With the default cacheClient: true, warnings continued until we reset flushPointReached at each Workflow step boundary.

The shared overall flush deadline has also been removed, so flush(timeout) may still take a multiple of the configured timeout. That is less relevant to our application because we do not register user waitUntil() work, but it remains a behavioral difference.

Until v11 is released, is Sentry's recommendation for anyone using Cloudflare Workflows with SDK v10 to set cacheClient: false?

[JPeer264]

With the default cacheClient: true, warnings continued until we reset flushPointReached at each Workflow step boundary.

Very interesting, I couldn't reproduce it. We could add it to make it work for you now - do you have a small reproduction where you reproduced this behavior?

I'll add this on monday and do another test on this.

The shared overall flush deadline has also been removed, so flush(timeout) may still take a multiple of the configured timeout

I think that is fine, unless I'm overlooking something critical here.

Until v11 is released, is Sentry's recommendation for anyone using Cloudflare Workflows with SDK v10 to set cacheClient: false?

There is actually no cacheClient on v10. In v11 we changed the way how clients are being generated over a workers lifetime which is cacheClient: true (the default). We kept cacheClient: false (which would be the equivalent of how v10 works) as an escape hatch in case there is something unexpected coming up. So for cacheClient in v11 it's recommended to keep the default.

[matthewbjones]

Yes. The reproduction used the v11/develop SDK with the default cacheClient: true:

class TestWorkflow extends WorkflowEntrypoint {
  async run(_event, step) {
    for (let i = 0; i < 100; i++) {
      await step.do(`step-${i}`, async () => i);
    }
  }
}

export const Workflow = Sentry.instrumentWorkflowWithSentry(
  () => ({
    dsn: "https://diagnostic@<slow-worker>/delayed/1",
    tracesSampleRate: 1,
  }),
  TestWorkflow,
);

The DSN pointed to a Worker that consumed the request body and delayed every response for 120 seconds:

export default {
  async fetch(request) {
    await request.arrayBuffer();
    await new Promise(resolve => setTimeout(resolve, 120_000));
    return new Response(null, { status: 204 });
  },
};

With only the transport-abort change, the default cached-client version continued producing Cloudflare waitUntil() tasks did not complete warnings. Adding the flushPointReached = false reset at the beginning of each wrapped step eliminated them in 20/20 runs. Setting cacheClient: false also eliminated them.

I deployed the slow endpoint and Workflow Worker, then triggered 20 Workflow instances with 100 steps each:

bunx wrangler deploy --config apps/waituntil-transport-sink/wrangler.jsonc
bunx wrangler deploy --config apps/waituntil-sentry-transport-control/wrangler.jsonc

for i in {1..20}; do
  bunx wrangler workflows trigger condux-waituntil-sentry-delayed "{\"sequence\":${i},\"stepCount\":100}" \
    --config apps/waituntil-sentry-transport-control/wrangler.jsonc
done

After allowing time for Cloudflare's events to appear, I searched that Worker's Observability events for the exact text waitUntil() tasks did not complete. Sometimes they can take ~60-80 seconds to appear in WOBS.

As for v10 vs v11, originally I was patching the latest v10 release and eliminating the bugs there, but soon as I went to try and contribute this upstream I saw this project's guidelines says PRs are to be submitted again develop, that's when I discovered code had shifted, cacheClient was introduced, etc.

You can also read the conversation between myself and Cloudflare engineering in this Discord thread: https://discord.com/channels/595317990191398933/1550147114703786087