feat(v1): persist harnesses over native runtime processes

[hallerite]

Summary

This is the native-process counter-proposal to #2116 and #2141:

• add rollout-scoped HarnessSession ownership to the harness lifecycle
• add a small RuntimeProcess contract: byte-stream stdout/stderr, stdin, wait, terminate, and kill
• implement it for subprocess, Docker, Modal, and Prime runtimes
• keep one RLM ACP runner alive across interaction turns over framed stdio
• keep sandboxed tool installation scratch space and uv caches on the runtime workdir instead of a VM's small /tmp tmpfs
• remove the ACP sidecar, Unix socket, probes, background service, and their custom networking lifecycle

Design

Rollout
  └─ HarnessSession
       └─ RuntimeProcess
            └─ ACP runner
                 └─ RLM ACP session

Persistence is a runtime capability rather than a harness-specific network service. The default HarnessSession adapts existing launch() / resume() implementations, so only stateful harnesses need a specialized session.

Prime live processes are deliberately gated to runtime.prime.vm=true. Container sandboxes fail with an actionable error because Prime containers are being deprecated.

Dependency

Depends on PrimeIntellect-ai/prime#819 for AsyncSandboxClient.open_process(). Until that SDK change is released, the Prime adapter reports that the installed SDK is too old; the other runtimes work independently.

Validation

• uv run pytest tests/v1/ -q
• full contributor suite passed after rerunning the sandbox-restricted socket/semaphore modules with normal host permissions
• uv run ruff check --fix .
• uv run pre-commit run --all-files
• real Prime VM SDK acceptance covers streaming stdout/stderr, delayed stdin, exit codes, signals, and clean shutdown
• live rlm-acp-in-prime-vm E2E passes: one ACP/RLM process survives two turns, calls its MCP tool on the resumed turn, and tears down cleanly

Note

Persist harness sessions across turns using live native runtime processes

• Introduces HarnessSession in harness.py as a rollout-scoped abstraction that replaces per-turn harness.run() calls in rollout.py; default implementation wraps existing launch/resume semantics.
• Adds RuntimeProcess abstract class and Runtime.open_process() to runtimes/base.py, with concrete implementations for Docker, Modal, Prime (VM-only), and subprocess runtimes.
• Rewrites acp/runner.py with a LiveACPSession that keeps a single ACP agent process alive across turns, communicating via a framed 8-byte size-prefixed stdio packet protocol; one-shot runs use a separate run_once path.
• Overrides RLMHarness.session() in harnesses/rlm/harness.py to return an ACPHarnessSession backed by a persistent rlm --acp process; per-rollout state is isolated under a trace-specific RLM_HOME.
• Risk: open_process is unsupported on Prime without a VM sandbox or a recent SDK version, raising SandboxError explicitly; Modal falls back to terminating the entire sandbox if PID acquisition times out.

Changes since #2249 opened

• Added Harness.run() coroutine method for one-shot segment execution and refactored result validation into Harness._check_result() helper method, with HarnessSession.turn() updated to delegate to the new validation helper [7b91006]
• Modified RLMHarness.session() to conditionally use base Harness.session() implementation when runtime does not support live processes [7b91006]
• Added supports_live_processes property to Runtime base class and PrimeRuntime to indicate live process capability support [7b91006]
• Added prime-sandboxes package as a git-based dependency source [e69250e]
• Added E2E test execution step for Prime VM persistent RLM scenario [e69250e]
• Reordered CI test execution steps to run Prime VM persistent RLM E2E before live v1 E2Es [14ef360]
• Removed CI step executing the Prime VM persistent RLM end-to-end test [0475b23]
• Added end-to-end test step for Kuhn Poker with RLM in Prime VMs to GitHub Actions workflow [c583beb]
• Added new CI workflow step for modeled user simulation with RLM and MCP [adb6bd6]
• Updated Kuhn Poker RLM evaluation step configuration and validation [adb6bd6]
• Modified Kuhn Poker test step validation logic and execution conditions [efe7ff6]
• Added unconditional execution to modeled user simulation test step [efe7ff6]
• Removed the 'Run Kuhn Poker with RLM in Prime VMs' step from the GitHub Actions workflow, including its evaluation invocation and Python validation [65d809b]
• Modified validation assertions in the 'Run modeled user simulation with RLM and MCP' step to enforce stricter success criteria and validate specific node content [65d809b]
• Removed GitHub Actions workflow step for modeled user simulation [17338f2]
• Renamed VerifiersClient class to VerifiersACPClient and updated all instantiation and usage sites [af19253]
• Converted private utility functions to public functions by removing underscore prefixes [af19253]
• Added validation logic to prompt function that raises RuntimeError when agent produces empty visible reply [02db586]
• Implemented tool call status tracking throughout VerifiersACPClient session lifecycle [02db586]
• Updated prime-sandboxes dependency version constraint and removed git source override [0ff6f9a]
• Removed feature detection logic from PrimeRuntime.open_process method [da65000]

^{Macroscope summarized 5f7dc2d.}

---

Note

High Risk
Touches core rollout lifecycle, multi-runtime process management, and Prime VM-only live processes; teardown/cancellation paths must not leak sandboxes or discard scoreable traces.

Overview
Rollouts now open a rollout-scoped HarnessSession at setup and drive each segment with turn() instead of one-shot harness.run(), so stateful harnesses can keep a live process for the whole interaction.

RuntimeProcess and Runtime.open_process() add stdin/stdout streaming with wait/terminate/kill on subprocess, Docker, Modal, and Prime (VM-only). ACP gains ACPHarnessSession: a long-lived stream runner with framed stdio packets, while once keeps single-segment behavior. RLM moves to rlm-harness with --acp, uses persistent ACP when supports_live_processes, and isolates state per trace.

Sandboxed MCP installs now use the runtime workdir for source/uv cache (avoiding Prime VM /tmp exhaustion). prime-sandboxes is bumped to 0.2.35; E2E adds RLM ACP resume on Docker and Prime VM.

^{Reviewed by Cursor Bugbot for commit da65000. Bugbot is set up for automated code reviews on this repo. Configure here.}

[macroscopeapp]

Approvability

Verdict: Needs human review

This PR introduces a significant new feature: persistent harness sessions over native runtime processes. It adds new abstractions (HarnessSession, RuntimeProcess), a streaming protocol for ACP, implementations across Docker/Modal/Prime/Subprocess runtimes, and modifies the core rollout lifecycle. The scope and complexity of these infrastructure changes warrant human review.

^{You can customize Macroscope's approvability policy. Learn more.}

[hallerite]

Integration verification: prime-sandboxes is temporarily pinned to Prime main commit 892bea7, which contains AsyncSandboxClient.open_process.

The targeted rlm-acp-in-prime-vm E2E passed against a real Prime VM and live model in Actions run 31001845264 (commit 14ef360b7). It exercised two ACP interaction segments over the persistent RLM process and retained tool access after resume.

The temporary workflow step was removed after the successful run. The immutable SDK pin remains until prime-sandboxes 0.2.35 is released; then it can be replaced with prime-sandboxes>=0.2.35 and a registry lock.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit efe7ff6. Configure here.}

[hallerite]

Real-workload validation update (the temporary workflow steps were removed afterward):

• Kuhn Poker, both seats RLM on Prime VMs: Actions run 31005016449. The diagnostic step passed 3/3 hands and 6/6 traces with clean user_closed termination, no trace errors/forfeits, and zero-sum payouts. Histories were check-bet-call (seed 0) and check-bet-fold (seeds 1–2). The same harness sessions resumed after opponent turns; one player used six sampled RLM nodes across its decisions. An earlier run's seed-0 failure was separately confirmed as the model spending all eight internal RLM turns deriving a CFR strategy (rollout stopped: max_turns), not a process/session failure.
• Modeled user + colocated MCP, assistant RLM on a Prime VM: Actions run 31005761673. The diagnostic step passed: the modeled user made the request and then returned ###DONE###; the assistant executed three internal RLM steps, invoked await echo_back.run(message="hello world") through persistent IPython, observed the hidden hello world [ok-7f3] stamp, earned reward 1.0, and both traces closed cleanly with user_closed.

Together these cover environment-level interleaving/session resume (Kuhn) and generic modeled-user + nested MCP composition (user-sim) end to end over the pinned native Prime process API. The PR no longer contains the temporary CI diagnostics.