Skip to content

Increase Daily Rig Decomposition Benchmark timeout - #392

Merged
pelikhan merged 3 commits into
mainfrom
copilot/aw-fix-daily-rig-benchmark-timeout
Aug 10, 2026
Merged

Increase Daily Rig Decomposition Benchmark timeout#392
pelikhan merged 3 commits into
mainfrom
copilot/aw-fix-daily-rig-benchmark-timeout

Conversation

Copilot AI commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

The daily decomposition benchmark exceeded its 45-minute agent execution limit. This updates the workflow to allow the requested 55-minute budget.

  • Workflow timeout
    • Set timeout-minutes: 55 in the benchmark workflow source.
  • Generated workflow
    • Regenerated the lock file so agent and failure-report timeout settings remain aligned.
timeout-minutes: 55

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Increase timeout for daily rig decomposition benchmark Increase Daily Rig Decomposition Benchmark timeout Aug 10, 2026
Copilot AI requested a review from pelikhan August 10, 2026 04:50
@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot review the agent logs and investing why the agentic workflow timed out

Copilot AI commented Aug 10, 2026

Copy link
Copy Markdown
Contributor Author

@copilot review the agent logs and investing why the agentic workflow timed out

The 45-minute limit was consumed by the workflow agent debugging its own benchmark harness, not by one benchmark execution. The first typecheck failed at 02:54:14 because the installed skill lacked Node type declarations. It then spent ~28 minutes investigating SDK authentication/model transport and locally patching the installed rig.ts; subsequent benchmark attempts failed with the SDK’s default 60-second session.idle timeout and then empty/invalid coordinator JSON. It started another diagnostic run at 03:33:44, and the outer workflow terminated it at 03:37:44. The run used 172,155 output tokens (653.713 AIC). Raising the outer limit alone does not address this root cause; the benchmark needs bounded setup/debugging and a harness path that succeeds without local runtime patching.

@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot review the agent logs and fix rig.ts so that sdk issues are not a problem

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Aug 10, 2026

Copy link
Copy Markdown
Contributor Author

@copilot review the agent logs and fix rig.ts so that sdk issues are not a problem

Fixed in 480eb54. Rig now forwards the workflow SDK connection token and provider configuration, honors the configured send timeout instead of the SDK’s 60-second default, and aborts in-flight SDK requests on cancellation.

@pelikhan
pelikhan marked this pull request as ready for review August 10, 2026 10:36
@pelikhan
pelikhan merged commit 43a35be into main Aug 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[aw] Daily Rig Decomposition Benchmark timed out

2 participants