fix(e2e): honor E2E_TIMEOUT for cursor-cli and raise the default to 4m - #2113
Open
Soph wants to merge 1 commit into
Open
fix(e2e): honor E2E_TIMEOUT for cursor-cli and raise the default to 4m#2113Soph wants to merge 1 commit into
Soph wants to merge 1 commit into
Conversation
This runner hardcoded a 90s per-prompt ceiling and never read E2E_TIMEOUT,
which CLAUDE.md documents as the per-prompt timeout for every agent. Only
the hardcoded default and a per-test override applied, so the Cursor leg
could not be widened from the environment at all.
90s is independently too tight for current Cursor. On a healthy account with
no quota error, one TestSingleSessionManualCommit turn measured 188.7s and
passed only once the ceiling was lifted. At 90s the harness tears down tmux
mid-turn, so the `stop` hook never fires and the checkpoint is silently
absent — and the assertion names the timeout, not the missing hook, which
makes a harness limit read as agent flakiness.
Two changes:
- Read E2E_TIMEOUT. Precedence now matches opencode.go:
default < E2E_TIMEOUT < per-test.
- Raise the default 90s -> 4m (~2x the measured turn). No workflow sets
E2E_TIMEOUT, so the env knob alone would leave the default in force. A
too-high default only costs wall clock when an agent genuinely hangs.
This does NOT fix the currently red cursor-cli leg. That job's failures are
quota: the captured pane carries "You're out of usage. Switch to Auto or
Composer 2.5, or ask your admin to increase your limit to continue.", 66
times alongside 68 timeouts in the same run. The agent stops responding and
the harness then times out, so the timeout is the downstream symptom and
raising it will not help a rate-limited account.
Not addressed here, deliberately. The runner passes no --model, so the
account's Auto routing picks the model per run and turn duration is
non-deterministic by construction; an E2E_CURSOR_MODEL knob mirroring
E2E_CLAUDE_MODEL would fix that. Several per-test overrides are also below
the measured turn duration -- edge_cases_test.go:138 (120s),
split_commits_test.go:69,82 (2m), multi_session_test.go (3m). Those are
agent-agnostic, so widening them changes every agent's run time.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: 01M0T37WDH4WWCNJEYT8ADHV6A
Soph
force-pushed
the
soph/e2e-cursor-timeout
branch
from
August 24, 2026 14:37
a4615cc to
679e26d
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Updates the Cursor CLI E2E agent harness to honor the documented E2E_TIMEOUT environment variable and increases the default per-prompt timeout so Cursor’s slower auto-routed turns don’t get cut off mid-turn (which prevents the stop hook from firing and leads to missing checkpoints).
Changes:
- Raise Cursor CLI runner default prompt timeout from 90s to 4m.
- Add
E2E_TIMEOUTparsing with the same precedence as OpenCode: default <E2E_TIMEOUT< per-testPromptTimeout.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+120
to
+124
| if envTimeout := os.Getenv("E2E_TIMEOUT"); envTimeout != "" { | ||
| if parsed, err := time.ParseDuration(envTimeout); err == nil { | ||
| timeout = parsed | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This runner hardcoded a 90s per-prompt ceiling and never read
E2E_TIMEOUT, which CLAUDE.md documents as the per-prompt timeout for every agent. Only the hardcoded default and a per-test override applied, so the Cursor leg couldn't be widened from the environment at all.opencode.go:98already reads it.90s is independently too tight for current Cursor. On a healthy account with no quota error, one
TestSingleSessionManualCommitturn measured 188.7s and passed only once the ceiling was lifted. At 90s the harness tears down tmux mid-turn, so thestophook never fires and the checkpoint is silently absent — and the assertion names the timeout, not the missing hook, which makes a harness limit read as agent flakiness.Changes
E2E_TIMEOUT. Precedence now matchesopencode.go: default <E2E_TIMEOUT< per-test.E2E_TIMEOUT, so the env knob alone would leave the default in force. A too-high default only costs wall clock when an agent genuinely hangs.This does NOT fix the currently red cursor-cli leg
Worth being explicit, because I initially misdiagnosed it. The red job's failures are quota, not the timeout. The captured pane carries:
66 occurrences alongside 68 timeouts in the same run. The agent stops responding and the harness then times out, so the timeout is the downstream symptom. Raising the ceiling will not help a rate-limited account — that needs the Cursor account limit raised.
The change stands on its own merits: the documented env var should work, and 90s is below a healthy turn. But it should not be merged expecting green CI.
Deliberately not addressed
No
--model. The runner buildsagent --force --workspace <dir>with no model pinned, so Auto routing picks per run and turn duration is non-deterministic by construction. AnE2E_CURSOR_MODELknob mirroringE2E_CLAUDE_MODELwould fix that properly.Per-test overrides below the measured turn duration —
edge_cases_test.go:138(120s),split_commits_test.go:69,82(2m),multi_session_test.go(3m). Agent-agnostic, so widening them changes every agent's run time; wanted a maintainer call rather than a silent edit.Verification
E2E_TIMEOUT=7m mise run test:e2e --agent cursor-cli TestSingleSessionManualCommit→ passes in 188.7s, full lifecycle (SessionStart→TurnStart→TurnEnd→ checkpoint saved → condensed).Not re-run across the full Cursor leg — with the account currently out of usage, that run would fail on quota regardless of this change.
mise run checkgreen.🤖 Generated with Claude Code