Add fail-closed W2 restage safety probes - #6
Open
OmarB97 wants to merge 1 commit into
Open
Conversation
the-crypt-keeper
pushed a commit
to the-crypt-keeper/vLLM-sm89
that referenced
this pull request
Aug 4, 2026
… in responses
vLLM 0.25.1 renamed the response field to `reasoning` (OpenAI spelling), but
DeepSeek's API and every client honouring their thinking-mode contract read
`reasoning_content`. The block was returned all along under a name nothing
looked for, so agent harnesses silently dropped the model's monologue on every
tool-carrying turn and it re-derived state from scratch.
The request path was never broken -- _normalize_messages_before accepts either
spelling and chat_utils renders both back into the assistant turn. Only the two
response serializers omitted the alias.
Measured on deep-swe (15 tasks, seed 0, mini-swe-agent, identical sampler):
| before | after | DeepSeek cloud
PASS | 2/15 | 7/15 | 7/15
terminated | 7/15 | 12/15 | 15/15
exceptions | 10 | 3 | 0
Per-step token profile converges on cloud's: 95,521 -> 132,504 input tokens per
step (cloud 136,561) and 854 -> 655 output (cloud 636). The broken arm was
lighter on input because it discarded the monologue, and heavier on output
because the model kept re-deriving what it had already worked out.
Remaining 3 are AgentTimeoutError at 10800s -- wall clock, not crashes.
Patch count 73 -> 75. gen_patches.py --verify and --check both clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The runtime fix needs reproducible fail-closed probes, not only a successful production canary. Reviewers need focused checks for page residency, shard retry behavior, bounded pack construction, cgroup semantics, and cleanup policy.
What changed
How to review
Start with
tools/w2_restage_watchdog.py, then inspecttools/test_store_safety.pyand the two smaller residency/stream probes. Confirm the watchdog distinguishes the softmemory.highboundary from hardmemory.max/host floors and fails closed on incomplete metrics.Evidence
On the frozen exact image, store safety passed 37/37, MXFP4 stream safety passed 10/10, checkpoint mappings dropped at the intended lifecycle boundaries, and a 48-layer constrained pack build stayed below 2 GiB with no swap or cgroup failure event. The guarded real restage completed all 46 checkpoint shards and 43 expert layers without swap, hard-limit, or OOM events.
Verification
python3 -m pytest tools/ds4_eval/tests/test_watchdog.py -q— 3 passedruff checkandruff format --checkon the new safety toolspython3 -m py_compileon all files in this PRgit diff --checkRisks / gaps
The GPU/store suites require the baked CUDA environment and are therefore recorded as exact-image evidence rather than rerun by lightweight GitHub CI.
tools/test_store_backends.pyretains one upstream pre-existing unused local outside this PR's added assertions; no source byte was changed to hide that unrelated lint debt.Collaborators