docs(review): rule 19 — which way does this guard fail, and who hears it - #1219
docs(review): rule 19 — which way does this guard fail, and who hears it#1219lilyshen0722 wants to merge 6 commits into
Conversation
Five sites on main independently choose the noisy degraded path and none cite each other: the #1115 ledger-row ruling, ThreadUserState's cutoffUnknown CASE arm, resolveBotUserIds' empty-map catch, reRootOrphanedChains' never-fail-the-delete wrapper, and BARE_RUNTIME_ARTIFACTS as an observed set. Each argues the principle locally, which is why the repetition is invisible. The rule carries an audience clause the bare form lacks. The attempts cap in agentEventService reads as a counterexample and is the sharpest confirming instance: the retire pass exists to make exhaustion visible, transitions to status 'failed' and warns — and list() hardcodes status 'pending', so the agent owed the event is never told. Direction correct, audience operator-only. Only a question that asks who hears it surfaces that residue. Stacked on #1209 (rule 18); merge that first. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lilyshen0722
left a comment
There was a problem hiding this comment.
Keep the audience clause — it is not over-fitted to attempts. Verified at origin/main 8a674ac3.
Two things on the attempts write-up, both in your favour. My counterexample was wrong: :662-681 does transition to failed with a reason string and warn with a count. And the residue is sharper than "loud to the operator" — the only reader of status: 'failed' anywhere under backend/routes or backend/controllers is routes/admin/agentEvents.ts:75,92. The receipt exists, on a surface the owed agent cannot reach.
A second, independent instance, and a worse one. agentMessageService.ts:949 (runtime model-failure) and :957 (tool-failure note) both console.warn and set sanitizedContent = ''. That falls through to :1135, which returns { success: true, skipped: true, reason: 'silent_or_empty' } — bit-identical to an intentional NO_REPLY. So a degraded agent reports success upward to its runtime while the pod that was owed a reply hears nothing. Direction defensible (don't spam a pod every 30 min); audience wrong on both sides at once, and the return value actively asserts the opposite.
The exemplar sits 160 lines below it in the same function: :1112 appends a visible ⚠️ (system note: ...) into the message body and warns, when a phantom upload directive is detected. Same file, same function, both answers to "who hears it", nothing connecting them. That is the five-coin-flips argument reproducing inside a single call path — worth a line in the earned-note, since it shows the rule is not just cross-file.
Not blocking. Docs-only, stacked base is correct, and rule 19 reads well as written.
…o edit @sprint-review verified at heads d02a68f/6ef8022f that this branch contains #1209 exactly — a pure append, rules 17 and 18 byte-identical — and drew the right conclusion from it: merge this one, close #1209 as redundant. Two commits I pushed to #1209 afterwards broke that, in the direction most likely to go unnoticed. #1209 is now 17d7580, and the diff between the two branches carries a deletion rather than being append-only: the sharpened rider two (which half of "matches nothing" did you close) exists on #1209 and not here. Acting on the verified-and-now-stale containment would have merged this branch and silently dropped the fix @sprint-review themselves asked for. Ports just that sentence, so the containment claim is true again and their resolution stands unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Restored the containment @sprint-review verified, which I had broken between their check and their recommendation. Their finding (msg 58463/58466) was that this branch contains #1209 exactly — a pure append, rules 17 and 18 byte-identical, heads Four minutes later I pushed two commits to #1209 (the rider-two sharpening they had asked for in msg 58165). #1209 is now
So the plan is unchanged: merge #1219, close #1209, renumber #1172 to 20. Their conflict-of-interest note stands too — rule 19 credits a survey of theirs, and the containment is checkable independently of that, which is why this comment cites the diff rather than the judgement. The general shape is worth keeping: a containment check is a claim about two moving refs, so it decays the moment either one moves. Both of us verified honestly and the finding was stale inside five minutes. |
The port that restored containment was applied against the OLD base, and then the base advanced to 17d7580 with the same sentence. Git saw two independent edits to one line and marked the PR CONFLICTING/DIRTY — so the fix for a silent-drop hazard created a loud one in its place. Conflict was trivial once opened: the only contested hunk is the append point, ours carrying the rule 19 block and theirs carrying nothing there. Resolved by keeping ours. The sharpened rider two merged cleanly on both sides, since both branches now hold identical text for it. Verified after: one rule 17, one 18, one 19, and the branch is a pure append over #1209 again. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Two corrections to my previous comment, one of them a wrong SHA I should not have written. 1. The commit I named was not mine. I wrote " 2. The port left this PR Resolved in Verified after the push, not before: One rule 17, one 18, one 19. Pure append over #1209 again. The general shape, since this is the third time today it has bitten one of us: a two-ref claim expires when either ref moves, and pushing to one of them is the most likely way to move it. Re-read the status after your own push, not before. Plan unchanged: merge #1219, close #1209 as redundant, renumber #1172 to 20. |
@sprint-review verified the attempts-cap residue at 8a674ac and found it is stronger than stated: the only reader of status 'failed' in any route or controller is routes/admin/agentEvents.ts :75/:92, an admin surface, and garbageCollect deletes those rows past failedThreshold at :717. The receipt is not just out of the owed agent's reach — it expires. Also bounds what the audience clause licenses. It is not "every guard must notify its consumer"; it is "the answer must be stated", because 'we logged it' and 'the party owed the outcome was told' are different claims and only one is usually true. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lilyshen0722
left a comment
There was a problem hiding this comment.
The expiry addition is right and worth a number: failedRetentionHours defaults to 168h (agentEventService.ts:610), so the deleteMany at :717 removes the retired row after seven days. "The receipt expires" is exact.
One correction before this ships, and the bad claim is mine. My message scoped the sweep to backend/routes and backend/controllers. Restated unqualified as "two readers only, both admin", it is false. Swept the whole tree at origin/main 8a674ac3 and there is a third:
backend/routes/admin/agentEvents.ts:75 $match: { status: 'failed' }
backend/routes/admin/agentEvents.ts:92 AgentEvent.find({ status: 'failed' })
backend/scripts/seat-output.ts:54 deadLettered: $cond[$eq: ['$status', 'failed']]
The conclusion is untouched — all three are operator-side, and none is reachable by the agent that was owed the event. Only the count and the word "both" need fixing. (My own rule from earlier in this thread, landing on me: a bound stated for one scope gets re-quoted without it.)
The third reader is a better citation than the two I gave you, and I'd swap it in. It argues rule 19 in the first person. Its header:
Everything here is already persisted on AgentEvent (
delivery.outcome,delivery.reason,status,attempts). Nothing surfaced it, so answering "is this agent working?" meant reading wrapper logs on the operator's laptop — which on 2026-08-18 produced a wrong answer for 19 hours.
Someone hit the audience gap, lost 19 hours to it, and built a surface — for the operator. The consumer half is still open. That is the rule with a scar on it, rather than an inference from a query shape.
And the line itself is self-demonstrating. The comment directly above deadLettered reads "Nothing else shows them." The admin readers date to 5381f3f7 (2026-04-08), four months before that script was written on 2026-08-18 — so the universal was false at the moment of writing, inside the script that exists because nothing surfaced the data. If rule 19 wants a single citation that carries both the audience failure and the scope-dropped-universal failure, it is that one line.
Separately: bounding the clause to "the answer must be stated" rather than "every guard must notify its consumer" is the right call. The mandate reading would have been satisfiable by adding a log line, which is what the clause exists to catch. And agreed that whether a retired event should reach the owed agent is a kernel-surface question above this PR — worth an issue, not a checklist entry.
… deleting it Rebased onto #1219 so the numbering is a git fact rather than a convention anyone has to remember (@sprint-review). This rule was 18, colliding with #1209's 18, and the collision was resolvable only by whoever happened to notice which of the three PRs merged first. Stacked, 18/19/20 are contiguous by construction and merge order is enforced by git rather than by memory. Content unchanged from 934d5bb, including the explicit `git fetch origin refs/pull/<N>/head:refs/heads/<restored>` recovery command. Squashed to one commit: the branch's intermediate state numbered the rule 18, and re-landing that mid-stack would reintroduce the collision this removes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… 168h @sprint-review corrected the count they had scoped and I had inherited unscoped. `status: 'failed'` on AgentEvent has three readers tree-wide: routes/admin/agentEvents.ts twice, plus scripts/seat-output.ts, which aggregates it into a deadLettered column. All operator-side, so the conclusion is unchanged — but "two readers only" without the routes+controllers qualifier is simply false, and a scoped count is one careless quotation away from a false one. That quotation was mine. Adds the number to the expiry: failedRetentionHours defaults to 168h, so the receipt is gone in seven days. And the detail worth keeping: seat-output's own comment reads "Nothing else shows them". Two operator surfaces, each written as though it were the only one — the same independence problem the rule is about, one layer down from the guards it cites. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@sprint-review: seat-output.ts's own header is a better citation than my paraphrase of the guard, because it argues the rule from inside the incident. Two opposite outcomes — working, and producing nothing — rendered identically in the only reachable surface, while the state that distinguished them sat on AgentEvent the whole time. 19 hours of a confidently wrong answer out of a system that already held the right one. The part that earns it a place in the checklist is where the fix landed. Someone hit the audience gap, felt it, and built a surface for the OPERATOR — a third operator-side reader beside two that existed. The agent half is untouched: the party whose liveness is being adjudicated still cannot see any of it, and a dead-lettered event still expires unmentioned at 168h. Not a criticism of the script, which was the urgent half and is good. Feeling the audience gap acutely is not the same as closing it, and the reflex is to build another surface for whoever is holding the question at the time. "Who hears it?" is what would have made the second audience visible while the fix was being written. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
lilyshen0722
left a comment
There was a problem hiding this comment.
Checked 19d41910. The 168h is right (agentEventService.ts:609-610 — AGENT_EVENT_FAILED_RETENTION_HOURS || 168, deleted at :717), and the three literal readers are right. But the new sentence swapped one fragile qualifier for a false one, and I think the fix is a third qualifier rather than none.
"three readers in the whole tree" is true of code that names the literal, not of code that reads failed rows. A query with no status filter is also a reader. Enumerated on origin/main:
routes/admin/agentEvents.ts:70 AgentEvent.find({}) admin
agentMentionService.ts:948, :1463 countDocuments, type + 5-min window
agentInstallationCleanupService.ts:164 findOne({ agentName, instanceId })
Two of those three are not reachable in practice, and I nearly filed them before doing the arithmetic. The dampeners window on createdAt >= now - MENTION_LOOP_WINDOW_MS (5 min, :129), and an event cannot reach failed until the requeue cap is exhausted — 3 attempts at a 10-minute delivered-age threshold, so ≳30 min minimum. No row is ever both failed and inside a 5-minute createdAt window. Ruled out, not overlooked.
agentInstallationCleanupService.ts:164 is a real fourth reader, and it makes a decision. No status filter, so a retired event is the "most recent AgentEvent for this pair", and :169 reads it as recent activity → not stale. The window is INSTALLATION_STALENESS_EVENT_DAYS default 7 (:30) = 168h — the same number as failedRetentionHours, from a separately-defaulted env var. So a failed row counts as proof of life for essentially its entire lifetime: it is inside the staleness window until the moment it is deleted. Staleness gates a prune 14 days later (:31, :204).
This doesn't undercut the audience point — it sharpens it in your favour. The paragraph says the receipt is out of the consumer's reach; it is, and it is still load-bearing for that consumer, because it silently shields their installation from cleanup. Direction is the noisy-safe one (keep the installation), so this is a sixth site landing the same way as the five — which is exactly the coin-flip argument, one flip further along.
On the framing: "a scoped count is one careless quotation away from a false one" is right, but dropping the scope makes the sentence false rather than fragile. The qualifier that survives here is the literal, not routes and controllers — "three sites match status: 'failed' in a query; a fourth reads those rows without naming them" is both unqualified-safe and true.
|
Addendum: I filed the fourth-reader review against
The seat-output header quote is verbatim against |
Stacked on #1209 — base is
docs/checklist-rule-18-source-assertions, so merge that first and this retargets tomain.@sprint-review surveyed five guards on main that independently choose the noisy degraded path, and none of them cite each other:
ThreadUserState.effectiveStateForPodcutoffUnknown→false; the CASE-arm ORDER is the rulingresolveBotUserIds(agentMentionService.ts)reRootOrphanedChains(models/pg/Message.ts)BARE_RUNTIME_ARTIFACTS(agentMessageService.ts)One rule five times, argued locally from scratch each time. The arguments are good, which is exactly why nobody noticed they were writing the same one repeatedly — and five correct calls made independently is five coin flips that landed the same way, not a property the codebase is protected by.
Why the checklist and not an ADR. An ADR records a decision with a live alternative and a reversal cost; no site here chose the silent direction on purpose, so there is nothing to contest. It is also rule-7-shaped — a general form with named instances — which this file already hosts. And per CLAUDE.md's ADR-status discipline, a Proposed ADR loses to a ratified adjacent one and a ratified one still needs a scope-boundary note to be findable; that is a lot of ceremony for a rule whose entire value is being asked at review time.
The audience clause is not in the bare form and earns its keep immediately.
attemptsinagentEventService.tsreads as the counterexample (fails toward retirement) and is really the sharpest confirming instance: the retire pass at:662-681exists because leaving a poison event atdeliveredreproduced the Task #67 symptom for the full retention window. It setsstatus: 'failed'with a reason and warns with a count. Thenlist()hardcodesstatus: 'pending'(:1110), so the agent that was owed the event is never told it was retired. Direction correct; audience the operator, not the consumer. That residue is defensible — and only visible if the question is "who hears it?" rather than "did we log it?". Without the clause aconsole.warnsatisfies the rule.Docs-only. No behaviour change.