Skip to content

fix(e2e): honor E2E_TIMEOUT for cursor-cli and raise the default to 4m - #2113

Open
Soph wants to merge 1 commit into
mainfrom
soph/e2e-cursor-timeout
Open

fix(e2e): honor E2E_TIMEOUT for cursor-cli and raise the default to 4m#2113
Soph wants to merge 1 commit into
mainfrom
soph/e2e-cursor-timeout

Conversation

@Soph

@Soph Soph commented Aug 24, 2026

Copy link
Copy Markdown
Collaborator

This runner hardcoded a 90s per-prompt ceiling and never read E2E_TIMEOUT, which CLAUDE.md documents as the per-prompt timeout for every agent. Only the hardcoded default and a per-test override applied, so the Cursor leg couldn't be widened from the environment at all. opencode.go:98 already reads it.

90s is independently too tight for current Cursor. On a healthy account with no quota error, one TestSingleSessionManualCommit turn measured 188.7s and passed only once the ceiling was lifted. At 90s the harness tears down tmux mid-turn, so the stop hook never fires and the checkpoint is silently absent — and the assertion names the timeout, not the missing hook, which makes a harness limit read as agent flakiness.

Changes

  • Read E2E_TIMEOUT. Precedence now matches opencode.go: default < E2E_TIMEOUT < per-test.
  • Raise the default 90s → 4m (~2x the measured turn). No workflow sets E2E_TIMEOUT, so the env knob alone would leave the default in force. A too-high default only costs wall clock when an agent genuinely hangs.

This does NOT fix the currently red cursor-cli leg

Worth being explicit, because I initially misdiagnosed it. The red job's failures are quota, not the timeout. The captured pane carries:

Error: Increase limits for faster responses
You're out of usage. Switch to Auto or Composer 2.5, or ask your admin to
increase your limit to continue.

66 occurrences alongside 68 timeouts in the same run. The agent stops responding and the harness then times out, so the timeout is the downstream symptom. Raising the ceiling will not help a rate-limited account — that needs the Cursor account limit raised.

The change stands on its own merits: the documented env var should work, and 90s is below a healthy turn. But it should not be merged expecting green CI.

Deliberately not addressed

No --model. The runner builds agent --force --workspace <dir> with no model pinned, so Auto routing picks per run and turn duration is non-deterministic by construction. An E2E_CURSOR_MODEL knob mirroring E2E_CLAUDE_MODEL would fix that properly.

Per-test overrides below the measured turn durationedge_cases_test.go:138 (120s), split_commits_test.go:69,82 (2m), multi_session_test.go (3m). Agent-agnostic, so widening them changes every agent's run time; wanted a maintainer call rather than a silent edit.

Verification

E2E_TIMEOUT=7m mise run test:e2e --agent cursor-cli TestSingleSessionManualCommit → passes in 188.7s, full lifecycle (SessionStartTurnStartTurnEnd → checkpoint saved → condensed).

Not re-run across the full Cursor leg — with the account currently out of usage, that run would fail on quota regardless of this change.

mise run check green.

🤖 Generated with Claude Code

Copilot AI lite review requested due to automatic review settings August 24, 2026 14:35
@Soph
Soph requested a review from a team as a code owner August 24, 2026 14:35
This runner hardcoded a 90s per-prompt ceiling and never read E2E_TIMEOUT,
which CLAUDE.md documents as the per-prompt timeout for every agent. Only
the hardcoded default and a per-test override applied, so the Cursor leg
could not be widened from the environment at all.

90s is independently too tight for current Cursor. On a healthy account with
no quota error, one TestSingleSessionManualCommit turn measured 188.7s and
passed only once the ceiling was lifted. At 90s the harness tears down tmux
mid-turn, so the `stop` hook never fires and the checkpoint is silently
absent — and the assertion names the timeout, not the missing hook, which
makes a harness limit read as agent flakiness.

Two changes:

  - Read E2E_TIMEOUT. Precedence now matches opencode.go:
    default < E2E_TIMEOUT < per-test.

  - Raise the default 90s -> 4m (~2x the measured turn). No workflow sets
    E2E_TIMEOUT, so the env knob alone would leave the default in force. A
    too-high default only costs wall clock when an agent genuinely hangs.

This does NOT fix the currently red cursor-cli leg. That job's failures are
quota: the captured pane carries "You're out of usage. Switch to Auto or
Composer 2.5, or ask your admin to increase your limit to continue.", 66
times alongside 68 timeouts in the same run. The agent stops responding and
the harness then times out, so the timeout is the downstream symptom and
raising it will not help a rate-limited account.

Not addressed here, deliberately. The runner passes no --model, so the
account's Auto routing picks the model per run and turn duration is
non-deterministic by construction; an E2E_CURSOR_MODEL knob mirroring
E2E_CLAUDE_MODEL would fix that. Several per-test overrides are also below
the measured turn duration -- edge_cases_test.go:138 (120s),
split_commits_test.go:69,82 (2m), multi_session_test.go (3m). Those are
agent-agnostic, so widening them changes every agent's run time.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Entire-Checkpoint: 01M0T37WDH4WWCNJEYT8ADHV6A
@Soph
Soph force-pushed the soph/e2e-cursor-timeout branch from a4615cc to 679e26d Compare August 24, 2026 14:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the Cursor CLI E2E agent harness to honor the documented E2E_TIMEOUT environment variable and increases the default per-prompt timeout so Cursor’s slower auto-routed turns don’t get cut off mid-turn (which prevents the stop hook from firing and leads to missing checkpoints).

Changes:

  • Raise Cursor CLI runner default prompt timeout from 90s to 4m.
  • Add E2E_TIMEOUT parsing with the same precedence as OpenCode: default < E2E_TIMEOUT < per-test PromptTimeout.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread e2e/agents/cursor_cli.go
Comment on lines +120 to +124
if envTimeout := os.Getenv("E2E_TIMEOUT"); envTimeout != "" {
if parsed, err := time.ParseDuration(envTimeout); err == nil {
timeout = parsed
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants