[codex] Support raw image offload in v1 train client

[eligotts]

Design update — inline/offload image storage

This PR now follows the prime-rl multimodal image storage policy:

• offload: current behavior, rewrite base64 data images to file:// run assets and require file-backed image URLs.
• inline: keep data:image/...;base64,... URLs in the message payload and validate them without rewriting.

TrainClient now calls the policy-aware image preparation helper, so prime-rl can be the single source of truth via environment/config propagation.

Validation after latest push: uv run pytest tests/v1/test_train_client_multimodal.py -q passed (5 passed). Commit/push hooks also passed (ruff check, ruff format, generated AGENTS/CLAUDE check, ty).

Design update — dropped the None/cache-only image path

This PR and its companions (prime-rl #2836 / verifiers #1746 / renderers #89) no longer use the "send None for already-cached images" mechanism. Every image carries its raw descriptor ref at every slot (current and prior turns); /inference/v1/generate rematerializes each ref from disk every request.

Why: the None path coupled correctness to deployment (LRU cache present, single replica / DP-affinity, no eviction) and surfaced a miss as a hard vLLM EngineDeadError (qwen3-vl mrope dereferences a None image_grid_thw) that the retry net couldn't catch across the engine→API IPC. Dropping it is deployment-agnostic (a miss is impossible) and non-hacky. vLLM's mm_hash encoder cache still skips the expensive GPU re-encode for free — we only forgo the cheap IPC/CPU-reprocess dedup.

Validated: color-codeword (Qwen3-VL-4B) under DP=2, no affinity / no cache reliance: 0 crashes, 0 data=None, multi-turn accumulation correct, reward ~0.84. Also confirmed under TP.

This repo: with every image carrying its ref, no cache miss can occur — removed the retry subsystem (_generate_with_image_ref_retry, _has_descriptor_only_images, _retryable_mm_error_type, _json_error_type, _RETRYABLE_MM_ERROR_TYPES). Rollouts call renderers.client.generate directly. Obsolete retry tests removed.

---

Original description

Summary

• tighten v1 multimodal graph serialization around strict raw descriptor sidecars
• reject processed multimodal payload keys recursively, including nested pixel_values, image_embeds, and image_features
• update v1 multimodal tests to use strict prime_raw_mm_item envelopes instead of descriptor-only Qwen payloads
• keep raw image offload and retry behavior aligned with the companion Renderers and Prime-RL PRs

Companion PRs

• Prime-RL: Support v1 raw multimodal image offload prime-rl#2836
• Renderers: [codex] Support raw image refs for multimodal rendering renderers#89

Notes

• Draft/WIP: this depends on the renderer generic raw multimodal ref contract in the companion PR.
• v1 multimodal sidecars intentionally carry raw descriptors only, not processed image tensors or image-processor payloads.
• Prime/vLLM materialization happens from raw image refs rather than Verifiers-held processor outputs.

Validation

• Commit hooks: ruff check, ruff format, generated AGENTS/CLAUDE check passed.
• Push hook: ty (ci parity) passed.
• End-to-end hosted-style smoke through Prime-RL with /home/ubuntu/verifiers, /home/ubuntu/renderers, and /home/ubuntu/prime-rl-v1-raw-mm-offload completed inference, env rollouts, train batch creation, trainer step 0, and decoded strict trainer-bound raw image refs.

[!NOTE]

Support raw image offload in the v1 train client for multimodal chat requests

• Adds prepare_images_inplace in multimodal.py to canonicalize image parts (HF-style and OpenAI-style) and offload non-file:// URLs (e.g. data: URLs) to file:// run assets via renderers.mm_store.
• Adds a prepare_request_body hook to the base Client class, invoked by the interception server before request processing; TrainClient overrides it to run image offload for ChatDialect bodies.
• Extends multimodal bridging in TrainClient.get_response to pass previous_multi_modal_data to the renderer; prompts with multimodal content are no longer excluded from bridging.
• Propagates mm_placeholders through MessageNode, Branch.multi_modal_data, and _attribute_mm so placeholder ranges are carried alongside items and hashes.
• Risk: requires a version of renderers that exposes mm_store.offload_image_to_run_assets; missing support raises RuntimeError at request preparation time.

📊 Macroscope summarized 4c6f31e. 9 files reviewed, 0 issues evaluated, 0 issues filtered, 0 comments posted

🗂️ Filtered Issues

No issues evaluated.

Update: ingress hardening (2c2824ae)

• prepare_images_inplace now covers the full renderer part treaty: nested image_url dicts, direct-string image_url, direct-string image parts, and typed pydantic parts. Non-string sources raise with the part shape named; renderer-side raw mode hard-requires file:// (no second offload layer).
• Interception server labels prepare_messages failures as InterceptionError instead of misattributing them to the user simulator.
• New shape-coverage test (skips until the renderers pin ships mm_store; passes against the sibling renderers checkout). Suite: 839 passed with pre-existing test_envs/test_opencode_rlm_env failures reproduced on the base branch.

Update: merged main (2b1627d03)

Reconciled with verifiers main (111 commits ahead at the merge-base — mostly unrelated v1 harness/multi-agent, taskset/environment, and trace/data-model churn; none of it touched this branch's actual offload files, verifiers/utils/multimodal.py / clients/renderer_client.py / types.py).

• interception/server.py: main's d21100bea ("resume replaces mid-request user injection") independently moved the user-simulator loop out of the interception server entirely, into Harness.launch/resume, and added graph-atomicity retry coalescing to handle_request/_stream. Took main's structure as authoritative; dropped this branch's now-obsolete session.user/session.opening mid-request injection (and its prepare_messages call sites) since injected/resumed messages now re-enter as ordinary request bodies, already covered by prepare_request_body (kept, rewired to the top of the new single-shot handle_request).
• graph.py: kept finish_reason/usage/multi_modal_data/previous_multi_modal_data() (this branch) alongside main's SkipJsonSchema wrapping convention and commit()'s new (tools) -> assistant_node_id signature. Caught and fixed an auto-merge that silently dropped the FinishReason/Usage imports — surfaced as a pydantic "Trace not fully defined" failure at Trace construction, not a textual conflict.
• trace.py: kept this branch's more accurate multi_modal_data docstring; took main's tuple-based bridge-mutation assertion in the test suite (strictly better — reuses already-computed prior_mm/prior_counts instead of recomputing).
• verifiers/v1/ARCHITECTURE.md: main deleted this file (docs moved to a much shorter docs/v1/architecture.md that doesn't cover this depth) — accepted the deletion; the one load-bearing fact (why multi_modal_data is JSON-excluded) is now a docstring on _NODE_DUMP_EXCLUDE instead of a dedicated doc.

Validation: full suite passing with a local renderers checkout override (uv run --with-editable ../renderers pytest tests/, minus PRIME_API_KEY-gated e2e/env tests) — includes test_prepare_images_inplace_offloads_every_image_part_shape, previously skipped pending a renderers pin with mm_store.

Update: dropped the orphaned prepare_messages hook (d9e79d6c)

Main's harness-segment-resume refactor removed the interception server's mid-request user-simulator injection — the only call sites of this PR's prepare_messages hook. Resumed user turns now re-enter the server as ordinary request bodies, already covered by prepare_request_body at request ingress, so the hook (base Client + TrainClient override) is deleted rather than carried as dead code.

Update: synced with main (0e4166608)

Merged verifiers main (35 commits): the v1 structure cleanup (#2146) retiring StrictBaseModel and renaming _NODE_DUMP_EXCLUDE → EXCLUDE_FIELDS, the ruff 0.16 / ty tooling bump (#2147, #2148), runtime-on-agent with the agent config stamped on the trace (#2106), Reward score/weight records (#2119), MCP tools for Codex (#2140).

Resolutions:

• trace.py: main's restructure subsumes this branch's block wholesale — EXCLUDE_FIELDS already carries multi_modal_data, and TRACE_VERSION = 1 is main's deliberate reset (this branch never touched it). Took main.
• graph.py: kept the raw-mm sidecar validators and previous_multi_modal_data; adopted main's plain BaseModel now that StrictBaseModel is gone.
• clients/train.py: kept the is_multimodal import — still used by the bridge path that threads previous_multi_modal_data.

Ruff 0.16 flagged two spots this branch added that main's own cleanup pass never saw: a constant getattr in the ingress offload walker, and the blind except at the prepare_request_body boundary (annotated # noqa: BLE001, matching main's convention at rollout boundaries).

Full suite with the sibling renderers checkout (minus PRIME_API_KEY-gated e2e/envs): all passing.

Update: merged main — renderer pool reconciliation (ebc0e2f26)

Merged verifiers main (25 commits: process-shared renderer pool #2218, serve CLI removal / ServeConfig rename #2237, Harbor separate verifier envs #2152, unscored-reward None placeholders #2235, pydantic adapter reuse #2233, and more).

One conflict, in train.py, against #2218's restructure:

• Adopted main's RendererSlot / ElasticRendererPool and the slot.run call structure wholesale. It subsumes this branch's _maybe_offload thread-hop (main now thread-hops + locks all encode-side work, generalized).
• Dropped main's multimodal bridging gate (not _has_multimodal_content(prompt) in can_bridge). That gate exists because processed-tensor sidecars can't bridge safely; this PR's raw descriptor refs are exactly what makes multimodal bridging safe, so the gate is removed and the previous_multi_modal_data bridge kwargs are re-applied inside the pooled bridge() closure.
• The ingress prepare_request_body image-offload hook auto-merged into the new TrainClient untouched.

Full suite: 913 passed, 74 skipped.

Update: merged main (c9d1f134c)

Merged verifiers main (11 commits: per-token advantages #2245, run info moved trace-to-episode #2244, ACP harnesses #2257, persistent harness runtimes #2249, and more). One conflict, in graph.py: main's advantages field landed adjacent to this branch's finish_reason on MessageNode — union of both. Full suite: 913 passed.

Update: review fixes — ingress hook + image-part parsing (97b6af8bd)

Two confirmed finds from Bugbot/Macroscope; the rest of the review items were assessed and intentionally skipped (impossible-deployment hardening, already-loud failure paths, and the deliberate raw-only trace contract):

• Interception ingress called the hook on the wrong object (Bugbot, High): session.ctx.client is the client config; the live client is session.client. Every intercepted request died at ingress with an AttributeError-turned-rollout-failure, so image offload never ran on that path. One-word fix.
• HF-style {type: "image", image: url} parts were silently dropped (Macroscope, High): the ingress offloader and the renderer part treaty both accept the shape, but content_to_parts kept only text/image_url — the image was offloaded, then lost before rendering. The typing boundary now normalizes it to ImageUrlContentPart.

Full suite: 913 passed.

Update: canonical image-part shape at ingress (49a7dc1a0)

Every image part is rewritten at v1 training ingress to exactly {"type": "image_url", "image_url": {"url": ...}} — HF-style {type: image} normalizes into it, data images offload to file:// run assets, and anything else (http, non-string) is rejected loudly. prepare_images_inplace loses the multi-shape/pydantic tolerance (walks request-body dicts/lists only), already-offloaded file:// URLs pass without importing renderers, and the v0 RendererClient ingress call is dropped — v0 renderer-client multimodal is explicitly not a supported path on this branch. One shape after ingress means dialects, renderers, and trace validation have a single place to look for an image. Full suite: 913 passed.

---

Note

Medium Risk
Touches the v1 train interception path and multimodal trace wire format; mis-offload or descriptor validation would break multimodal rollouts, but eval/relay clients are unchanged and failures are explicit at ingress.

Overview
Adds v1 multimodal training ingress so chat requests normalize images before interception, rendering, and tracing all see the same lightweight refs.

prepare_images_inplace walks request bodies, rewrites HF-style {type: "image"} parts to canonical image_url, offloads data: URLs to file:// run assets via renderers.mm_store, and rejects unsupported sources (e.g. https://). The base Client gains prepare_request_body; TrainClient runs offload on chat dialect bodies in a thread; the interception server calls it at ingress and surfaces prep failures as InterceptionError.

Multimodal sidecars move from processed tensors to raw descriptors. Graph serialization validates raw_image_uri, rejects nested pixel_values / image_embeds / image_features, and attributes hashes, items, and placeholders per introducing node. PendingTurn.previous_multi_modal_data() feeds incremental renderer bridging; the old “no bridge when prompt has images” guard is removed when the renderer is multimodal.

Smaller follow-ons: content_to_parts accepts HF image parts; legacy v0→v1 trace mapping preserves live trajectory multi_modal_data; docstrings note raw descriptors instead of pixel tensors.

^{Reviewed by Cursor Bugbot for commit 4c6f31e. Bugbot is set up for automated code reviews on this repo. Configure here.}

[macroscopeapp]

Approvability

Verdict: Needs human review

2 blocking correctness issues found. This PR introduces new multimodal image offload functionality with significant changes to graph validation and legacy rollout handling. Unresolved review comments identify potential backwards compatibility breakage and missing field assignments that should be addressed before merging.

^{You can customize Macroscope's approvability policy. Learn more.}

[macroscopeapp]

🟡 Medium v1/graph.py:76

_validate_raw_mm_item now unconditionally rejects processed multimodal payloads containing keys like pixel_values, and deserialize_multi_modal_data runs it on every multi_modal_data field during deserialization. Loading a previously persisted multimodal v1 trace whose sidecars contain pixel_values now raises TypeError instead of round-tripping, breaking backwards compatibility for existing saved rollouts. Consider allowing processed payloads through on the deserialization path (e.g. by skipping the processed-key check in the validator's before path) so old traces can still be loaded.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @verifiers/v1/graph.py around line 76:

`_validate_raw_mm_item` now unconditionally rejects processed multimodal payloads containing keys like `pixel_values`, and `deserialize_multi_modal_data` runs it on every `multi_modal_data` field during deserialization. Loading a previously persisted multimodal v1 trace whose sidecars contain `pixel_values` now raises `TypeError` instead of round-tripping, breaking backwards compatibility for existing saved rollouts. Consider allowing processed payloads through on the deserialization path (e.g. by skipping the processed-key check in the validator's `before` path) so old traces can still be loaded.

[macroscopeapp]

🟡 Medium

verifiers/verifiers/v1/graph.py

Line 669 in 4c6f31e

MessageNode.model_construct(

_commit_turn constructs the assistant MessageNode via MessageNode.model_construct(...) but omits finish_reason=response.finish_reason and usage=response.usage, so every assistant node committed through PendingTurn.commit has finish_reason=None and usage=None — even when the Response carried them. Consumers of these newly added fields therefore cannot detect length truncation or retrieve the provider token accounting the fields are meant to preserve. Consider passing finish_reason=response.finish_reason and usage=response.usage into the MessageNode.model_construct call.

🚀 Reply "fix it for me" or copy this AI Prompt for your agent:

In file @verifiers/v1/graph.py around line 669:

`_commit_turn` constructs the assistant `MessageNode` via `MessageNode.model_construct(...)` but omits `finish_reason=response.finish_reason` and `usage=response.usage`, so every assistant node committed through `PendingTurn.commit` has `finish_reason=None` and `usage=None` — even when the `Response` carried them. Consumers of these newly added fields therefore cannot detect length truncation or retrieve the provider token accounting the fields are meant to preserve. Consider passing `finish_reason=response.finish_reason` and `usage=response.usage` into the `MessageNode.model_construct` call.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 4c6f31e. Configure here.}

[cursor]

Legacy bridge bypasses public rollout entry points

Low Severity

Switching to the private _run_rollout_state/_run_group_states skips run_rollout/run_group, which EnvGroup overrides to route each rollout into its child environment. For a nested EnvGroup, the inner group's rubric then resolves the outer route name against its own child map, finds nothing, and silently scores the rollout 0.0. Server-mode delegation through env_client is also skipped on this path.

Additional Locations (1)

• verifiers/v1/legacy.py#L485-L494

 

^{Reviewed by Cursor Bugbot for commit 4c6f31e. Configure here.}