fix(agents): a suppressed runtime failure must not conclude an agent-dm - #1233
fix(agents): a suppressed runtime failure must not conclude an agent-dm#1233lilyshen0722 wants to merge 2 commits into
Conversation
@sprint-review found that agentMessageService empties sanitizedContent for a runtime model-failure (:949) and a tool-failure note (:957), and the empty path returns the same `silent_or_empty` an intentional NO_REPLY returns. The label was the small half. The large half: that path also fires ADR-012 §4's recordAgentDmConclusion, which writes a system_exchanges entry for BOTH peers whose takeaway is the sender's PRECEDING message. So a model chain being exhausted wrote "this conversation concluded" into the record, attributing a takeaway the agent never reached. A failure laundered into a positive semantic event, with a console.warn as its only honest trace — a surface no agent can read. Two arrivals at `''` that mean opposite things: an intentional NO_REPLY is the agent deciding the exchange is over; a suppressed failure is the agent never having produced a reply. Now tracked and discriminated. A failure concludes nothing and names itself: runtime_model_failure_suppressed / runtime_tool_failure_suppressed. Still success+skipped, because the caller did nothing wrong and a 500 would turn a degraded model chain into a broken endpoint. Both existing silent_or_empty assertions (clawdbot-e2e :732, :748) are genuine silences and keep the old value. CONTROL in the suite: a bare NO_REPLY and whitespace-only content must STILL conclude, or the fix trades one collapse for another. Mutation- checked: reverting the discrimination reddens 4 of 7, and the two CONTROL cases stay green. 14 tests green with the chatNoise sibling suite. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@sprint-review: the exemplar is 160 lines below the defect. The phantom- upload guard appends a VISIBLE system note to the message AND warns — both audiences, deliberately. The runtime-failure suppression warns only. Same file, same function, both answers, nothing connecting them. The asymmetry is defensible and must not be "fixed" by symmetry: the phantom guard has a message to annotate, the suppression would have to manufacture a post, which is the ~30-min-per-pod flood it exists to stop. Recorded in both directions so the next author sees that the question has already been answered once here, and on what grounds. What was never defensible is two guards deciding who hears a failure, independently, in one function, with no pointer between them. 93 tests green across the agentMessageService suites. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
The exemplar is in the same function, 160 lines down — @sprint-review.
sanitizedContent += `\n\n⚠️ _(system note: ... no matching attachment was found ...)_`;
console.warn(`[agent-msg] phantom-upload-directive from agent=... `);Both audiences, deliberately. The runtime-failure suppression at The asymmetry is defensible and should not be "fixed" by symmetry. The phantom guard has a message to annotate; the suppression would have to manufacture a post, which is exactly the ~30-minute-per-pod flood it exists to stop. So the answer differs for a real reason. What was never defensible is that the reason lived nowhere: the second author could not see the first had already decided this question. Added a pointer in both directions recording the difference and its grounds. Left undecided, flagged rather than shipped. This fix stops a suppressed failure from writing a false conclusion, but the peer in an 93 tests green across the |
lilyshen0722
left a comment
There was a problem hiding this comment.
Verified the whole chain at d331b16d, including the part I only inferred when I raised it.
recordAgentDmConclusion (systemExchangeTriggers.ts:173) does what the PR body says: gated on podIsAgentDm, then "Write to BOTH peers", with the speaker's takeaway derived from findPreviousNonSilentMessage and the listener getting a @peer:-prefixed copy. So an exhausted model chain really was writing a conclusion entry into both agents' system_exchanges with a takeaway the agent never reached. The fix is correct and minimal.
Reproduced the mutation claim exactly: baseline 7 passed; deleting just the if (suppressedFailure) branch reddens 4, and both CONTROLs (a bare NO_REPLY fires the trigger, and so does genuinely empty content) stay green. The CONTROLs are the right load-bearing half.
One gap: there is a third arrival at '' and this fix does not reach it. sanitizeAgentContent returns '' for a bare runtime artifact at :1862 — BARE_RUNTIME_ARTIFACTS = new Set(['RGCTX']), whose own comment describes it as "a wrapper artifact ... runtime noise" added "only after observing it as a wrapper artifact in production". That path leaves suppressedFailure null, so it falls through to the ADR-012 §4 trigger. Measured at this head, against the PR's own harness:
RGCTX reason=silent_or_empty concludes=true
NO_REPLY reason=silent_or_empty concludes=true
Byte-identical. The same laundering the PR fixes, one branch over, inside the function the PR edits.
I am not certain it is a defect, and the uncertainty is the argument. TASK-042 describes the wrapper as having emitted RGCTX "instead of a silent reply", which reads as malformed silence — in which case concluding is right. But nothing in the artifact distinguishes "the agent chose silence and the wrapper mangled it" from "the wrapper ate a real reply", and that is precisely the ambiguity this PR argues must not be resolved in favour of a positive semantic event.
Either resolution is fine by me:
- Extend
suppressedFailureto the artifact floor (reason: 'runtime_artifact_suppressed'), or - Leave the behaviour and state in the new comment why the artifact floor is a decision while the other two are failures.
What should not ship is the third case being unmentioned next to a comment that says there are two ways to arrive here — a future reader will take that enumeration as complete. Same class as the "two readers only" fix on #1219 an hour ago.
Everything else here is right, and the reason-string split is the correct shape. Not blocking.
|
Follow-up on the item flagged-not-acted in the PR body ("emit a minimal visible note only when the suppressed message was answering a direct wake"). Two things worth pinning before that becomes a design ticket, both checked at 1. "Answering a direct wake" is not decidable at There is no event id, mention id, or wake provenance in it. And 2. The pod-type-scoped version is already free, and the premise is already in the tree.
That is the same distinction the flagged item wants ("unlike a heartbeat pod, someone is explicitly waiting"), already written down and already relied on. The empty path can call So the two framings are very different sizes of change, and the PR body currently reads as one item. Worth splitting: dm-scoped visible note is small and local; wake-scoped visible note is a kernel-surface change with new plumbing. My read is the first is what the flag actually wants. Also confirming the suite claim: 93 passed across 10 And a note on the asymmetry defence in the PR body — it is stronger than "defensible", it is structural. |
Found by @sprint-review while checking whether rule 19's audience clause was over-fitted to one site. It is not — this is the second instance and the worse one, because here the direction is wrong too, not just the audience.
The chain
Two ways to reach
''and they mean opposite things. An intentionalNO_REPLYis the agent deciding the exchange is over. A suppressed runtime failure is the agent never having produced a reply at all.Collapsing them cost more than a wrong label. The empty path fires ADR-012 §4's
recordAgentDmConclusion, which writes asystem_exchangesentry for both peers whose takeaway is the sender's preceding message. So a model-chain exhaustion wrote "this conversation concluded" into the record, attributing a takeaway the agent never reached. The caller got the samesilent_or_emptya deliberate silence returns, and the only honest trace was aconsole.warn— a surface no agent can read.That is the rule-19 shape twice over: the guard fails toward the silent outcome, and its one signal is addressed to an audience that is not the party owed the outcome.
The fix
Track why the content emptied, then discriminate at the empty path:
NO_REPLYsilent_or_emptysilent_or_emptyruntime_model_failure_suppressedruntime_tool_failure_suppressedStill
success: true, skipped: true— the caller did nothing wrong, and a 500 here would turn a degraded model chain into a broken endpoint. Both existingsilent_or_emptyassertions (clawdbot-e2e.test.js:732,:748) are genuine silences and keep the old value; no assertion inverted.Evidence
The CONTROL is the half that matters: a bare
NO_REPLYand whitespace-only content must still conclude, or the fix has traded one collapse for another.Mutation-checked per rule 1 — reverting just the discrimination branch reddens 4 of 7, and both CONTROL cases stay green:
14 green with the
chatNoisesibling suite.