test(e2e): add Bun runtime coverage on Vercel#2870
Draft
NathanColosimo wants to merge 5 commits into
Draft
Conversation
|
Contributor
Contributor
🧪 E2E Test Results❌ Some tests failed Summary
❌ Failed Tests▲ Vercel Production (152 failed)nextjs-turbopack-bun (151 failed):
nuxt (1 failed):
Details by Category❌ ▲ Vercel Production
✅ 💻 Local Development
✅ 📦 Local Production
✅ 🐘 Local Postgres
✅ 🪟 Windows
✅ 📋 Other
❌ Some E2E test jobs failed:
Check the workflow run for details. |
Contributor
📊 Workflow Benchmarkscommit Backend:
📜 Previous results (3)9d76b84Fri, 10 Jul 2026 05:25:05 GMT · run logs
1d96ca0Fri, 10 Jul 2026 04:56:38 GMT · run logs
605b090Thu, 09 Jul 2026 23:56:58 GMT · run logs
Avg deltas compare against the most recent benchmark run on Metrics — TTFS: time to first step body execution · STSO: step-to-step overhead (gap between consecutive step bodies) · WO: workflow overhead (time outside step bodies, client start → last step body exit) · SL: stream latency (first chunk write → visible to the reader) Scenarios — stream: one step that streams chunks back to the client; no hooks, so the run stays in turbo mode · hook + stream: registers a hook before the same streaming step, which exits turbo mode · 1020 steps: 1020 trivial sequential steps; STSO is measured between consecutive steps in the given step ranges 🟢/🔴 mark percentiles within/above target. Targets (p75/p90/p99, ms) — TTFS 200/300/600 · SL 50/60/125 · STSO (1-20) 20/30/60 · STSO (101-120) 30/45/90 · STSO (1001-1020) 40/60/120 TTFS/WO compare client vs deployment clocks and SL compares the step runner’s clock vs the client’s (NTP-synced in CI). WO ends at the last step body exit, the closest observable proxy for the final step-completion request. |
Contributor
Author
Bun regression follow-upAdded focused coverage after the baseline run:
Local production results:
The first CI run reached the Bun matrix job and waited the configured 1,000 seconds, then failed because no |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
1.xruntimenextjs-turbopacktest gateWhy
Workflow has accumulated several Bun-related reports, including #1735. This adds a durable Node-vs-Bun comparison on the same stable Next.js/Turbopack app and full E2E suite.
Baseline findings
Validated locally with Bun 1.3.14 and the current stable Next.js 16.2.10:
pnpm buildpassesWORKFLOW_PUBLIC_MANIFEST=1 pnpm --filter nextjs-turbopack-bun buildpassesfetchWorkflow:__builtin_response_jsonretries three times and fails withJSON Parse error: Unexpected EOFValidated on a manually built Vercel preview with managed Bun 1.3.12:
bun run --bun1.3.12(process.versions.bunagrees)cborReplayRegressionWorkflowcompletes with the expected result: 1,954 serialized bytes, 8 batches, and 40 documentsfetchWorkflowfails inside the Bun deployment withWORLD_CONTRACT_ERROR: Unexpected end of CBOR datarun.returnValuefrom a Bun route can also truncate the Workflow/v2/runsCBOR response; the Node CLI reads the same completed run and output successfullyThis points to a broader Bun response-body truncation problem affecting both JSON/stream bodies locally and CBOR API responses on Vercel, rather than the original #1735 workload remaining broken.
Manual preview: https://workbench-bun-workflow-aqm89s50p.labs.vercel.dev
The non-blocking behavior is confirmed in Tests run 29125724275: the Bun test command failed, runtime logs and artifacts uploaded, the job emitted
Experimental Bun E2E failure, and the Bun check concluded successfully. The manually attached preview could not accept the runner's GitHub OIDC trusted-source token, so that run validates failure reporting rather than the Bun suite itself; connecting the Vercel project to Git is still required for representative automated results.Vercel project prerequisite
The matrix points to the dormant Labs project
workbench-bun-workflow(prj_lb5X39knEIn63kRAoeNFipevjJBu), which was created for the old abandonedworkbench/bunexperiment. It currently has no Git link and still points at the removedworkbench/bunroot.A Vercel project owner needs to:
workbench/nextjs-turbopack-bunvercel/workflowwithmainas the production branchMy Labs role can read and link the project locally but receives 403 for project updates. Until the project is configured, the new non-blocking CI lane is expected to stop at
Waiting for the Vercel deployment.Validation
pnpm install --frozen-lockfile --lockfile-onlypnpm buildpnpm --filter @workflow/core typecheckWORKFLOW_PUBLIC_MANIFEST=1 pnpm --filter nextjs-turbopack-bun buildfetchWorkflowrunsgit diff --checkpnpm changeset status --since=main: no changeset needed