Skip to content

feat(bench): add terminal-bench container executor#490

Merged
drewstone merged 2 commits into
mainfrom
feat/tb-supervisor-arm-clean
Jul 7, 2026
Merged

feat(bench): add terminal-bench container executor#490
drewstone merged 2 commits into
mainfrom
feat/tb-supervisor-arm-clean

Conversation

@drewstone

Copy link
Copy Markdown
Contributor

Summary

  • add a Terminal-Bench container executor that runs spawned workers inside the grader-owned Docker container
  • add an offline executor test with a fake Docker binary for argv, metering, and non-zero exit behavior
  • add a manual coordination-MCP container reachability probe

Verification

  • pnpm exec tsx bench/src/tb-container-executor.test.mts
  • pnpm exec tsx bench/src/coordination-mcp-container-reach.mts (Docker 29.2.1; container saw spawn_agent + await_event)
  • pnpm run typecheck
  • pnpm run lint
  • pnpm run build
  • pnpm test: 127 files passed, 1,254 tests passed, 2 skipped
  • pnpm run verify:package
  • git merge-tree --write-tree origin/main HEAD

tangletools
tangletools previously approved these changes Jul 7, 2026

@tangletools tangletools left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Auto-approved drewstone PR — d95246c1

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

tangletools · auto-approval · reason: drewstone_author · 2026-07-07T23:43:00Z

@tangletools tangletools left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚪ Value Audit — audit-incomplete

Verdict audit-incomplete
Concerns 1 (1 low)
Heuristic 0.0s
Duplication 0.0s
Interrogation 7.2s (2 bridge agents)
Total 7.2s

💰 Value — error

value agent produced no parseable value-audit JSON.

  • Model: opencode/deepseek/deepseek-v4-pro
  • Bridge attempts: 3
  • Bridge error: opencode/kimi-for-coding/k2p7: bridge stream ended without value-audit content; opencode/zai-coding-plan/glm-5.2: bridge stream ended without value-audit content; opencode/deepseek/deepseek-v4-pro: bridge stream ended without value-audit content

🎯 Usefulness — error

usefulness agent produced no parseable value-audit JSON.

  • Model: opencode/deepseek/deepseek-v4-pro
  • Bridge attempts: 3
  • Bridge error: opencode/zai-coding-plan/glm-5.2: bridge stream ended without value-audit content; opencode/kimi-for-coding/k2p7: bridge stream ended without value-audit content; opencode/deepseek/deepseek-v4-pro: bridge stream ended without value-audit content

🔎 Heuristic Signals

🟡 Cruft: console debug added bench/src/coordination-mcp-container-reach.mts

  • console.log(

What this audit checks

It judges the change on its merits — not whether it was tasked out in an issue. Unticketed, fast-moving work is fine; the question is whether the change is good and whether a better or existing approach should be used instead.

Pass What it asks
Heuristic Vague title? Whitespace-only or cruft-bearing diff? (content signals only)
Duplication Do added function/class names already exist elsewhere in the repo?
Value Audit What does it do? What goal does it achieve? Is it good? Better architecture or already-exists?
Usefulness Audit Does it integrate and fit? Will it hold up in real use and actually get used?

Findings are concerns, not blocks — the human reviewer decides what to do with them.

value-audit · 20260707T234450Z

@tangletools tangletools left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Auto-approved drewstone PR — f9d3eb4b

This PR was opened by the trusted drewstone account.
The full PR reviewer audit still runs separately and will publish findings if it detects issues.

tangletools · auto-approval · reason: drewstone_author · 2026-07-07T23:45:54Z

@drewstone drewstone merged commit 084d808 into main Jul 7, 2026
1 check passed
@drewstone drewstone deleted the feat/tb-supervisor-arm-clean branch July 8, 2026 00:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants