From 4cfc1ae91a9f62082fefc8f1ade3b8b5cbe20b76 Mon Sep 17 00:00:00 2001 From: Topi Jarvinen Date: Wed, 29 Jul 2026 07:32:54 +0300 Subject: [PATCH] =?UTF-8?q?chore:=20graduate=20the=20friction-log=20inbox?= =?UTF-8?q?=20to=20GitHub=20Issues=20(#149=E2=80=93#150)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fifth triage-friction-log sweep overall, dated 2026-07-29. Seven inbox entries in, seven accounted for: one graduated into two new issues (#149, #150), six routed as seven occurrence comments (#120, #138, #127, #75, #71, #45, #113). Run in LLM-only mode — the triage engine (#6) is still not vendored — so the sweep was scripted by hand against a frozen-inbox snapshot taken before any write, whose digest reproduces from 0b82ff2. Two fallback-panel rounds ran, four isolated lenses. Round 1 found a HIGH both lenses reached independently: the sweep inflated an operator remark ("CodeRabbit is currently not available here") into an assertion the operator never made — not installed, never exercised, nothing rate-limited — and used it to file a "structurally never reviews" verdict onto #45, whose whole subject is that such a verdict cannot be made from the outside. Round 2 then found the fix had over-corrected, asserting a review count that was really a count of bot comments. Independent attempts to settle that number disagreed, so no count is published; the irreducibility is the better evidence for #45. Both retractions are visible on the comment rather than the text being replaced. Also corrected after review: the sweep is the fifth, not the third; "as in the two prior sweeps" laundered a recorded #128 violation into a substitution; the post-sweep entry re-derived #121 without noticing it, and its own rewrite then made a larger false claim about config comments than the one it corrected. The marker's verification section was cut rather than corrected a third time. Five review rounds across two sweeps have gone to that section, every one finding the prose claiming more than the checks did; deletion is the remedy those sweeps recorded. The check headings that over-claim are now named as such instead of being described as narrowed — they were not narrowed, and building a better harness inside a fix round is the mechanism-creep the panel doctrine warns against. #138 and #127 stay open. The run of PRs with no CodeRabbit output is twelve, not seven. Verified with make test (503 passed), check_doc_budget (136/150, down from 203), kit_doctor (26 unchanged, 0 differ), and a six-check script published in full on the PR with its limits stated. --- docs/kit-friction-log-archive.md | 209 +++++++++++++++++++++++ docs/kit-friction-log.md | 285 ++++++++++++------------------- 2 files changed, 318 insertions(+), 176 deletions(-) diff --git a/docs/kit-friction-log-archive.md b/docs/kit-friction-log-archive.md index 0f1c445..d79ac01 100644 --- a/docs/kit-friction-log-archive.md +++ b/docs/kit-friction-log-archive.md @@ -3,6 +3,215 @@ Graduated friction entries live here after they have been routed to the tracker (GitHub Issues on this repo) or promoted into a repeated-pattern rule. +## Graduated 2026-07-29 — GitHub Issues (#149–#150) + +Swept by the `triage-friction-log` workflow, run in LLM-only mode (the engine tracked in +[#6](https://github.com/topij/agentic-dev-kit/issues/6) is not vendored yet). Seven entries in, +seven accounted for: **one graduated** into two new issues, **six** routed as seven occurrence +comments on the seven issues they are evidence for. + +Both one-into-two counts are deliberate, not miscounts. The correction-propagation entry split +into `#149` (enumerate a corrected claim's surfaces) and `#150` (a scripted replacement that +matches nothing must fail) because those are separable mechanisms; the check-heading entry +produced comments on both `#138` (the routing check's heading) and `#127` (the block-integrity +check's substring test) because it names two distinct checks. + +The two blocks below are the swept text, verbatim, with headings demoted one level. The first is +the previous sweep's graduation marker; the second is the session block that accumulated after +it. The live log's marker for this sweep — which carries the approval record and the +verification statement, per `#128` — is in `kit-friction-log.md`, not here. + +### 2026-07-28 (second sweep) — Backlog migrated to GitHub Issues (#138–#143) + +The inbox was swept by the `triage-friction-log` workflow, run in LLM-only mode (the engine +tracked in [#6](https://github.com/topij/agentic-dev-kit/issues/6) is still not vendored). +**Fourteen entries in, fourteen accounted for:** seven graduated into six new issues +([#138](https://github.com/topij/agentic-dev-kit/issues/138)–[#143](https://github.com/topij/agentic-dev-kit/issues/143)), +seven routed as five occurrence comments (`#45` ×2 entries, `#113` ×2, `#75`, `#73`, `#120`). + +Two reconciliations, stated rather than left to a reader — mis-stating exactly these is what +`#138` was filed for: seven graduated entries became **six** issues because the `pr_watch` 403 +defect was recorded in two sessions and both entries went to +[#139](https://github.com/topij/agentic-dev-kit/issues/139); seven routed entries became **five** +comments because `#45` and `#113` each carry two. + +`#23` is named as a routing target by the swept text and received nothing **from this sweep** — +it is closed, and its occurrence data is consolidated on `#45`. It is not un-commented: it +carries the *previous* sweep's comment, posted `2026-07-28T13:46:17Z`, four minutes before `#126` +merged, after a panel caught that it had been omitted. + +#### Approval record — in-session operator, no DM + +This run was **interactive**, so the operator substituted for the DM review surface rather than +the stop being bypassed. [#128](https://github.com/topij/agentic-dev-kit/issues/128) asks that +such a substitute leave the approval record somewhere **committed**, since `state/` and +`reports/` are gitignored. This block is it. Approval was given up front and unconditionally — +*"I approve all suggestions"* — so every proposal carries the same decision and the +explicit-opt-in default was never exercised. + +**Frozen-inbox snapshot:** `state/triage/frozen-inbox_2026-07-28.json` (gitignored), +`sha256 b3a168a8ba8c18dc7d254fe76d1621b6ae5afff6d757540f1316c398643a6db7` over **14,341 bytes** +(14,235 characters; 54 non-ASCII — the digest is over the bytes). The snapshot is a *copy of a +committed blob*: the inbox at `06490a1` is that text, so the digest and every check in +the PR reproduce from `git` and `gh` alone, in any session. + +| # | proposal (inbox entry, abridged) | decision | outcome | +| - | -------------------------------- | -------- | ------- | +| 1 | A routing list is a claim about tracker state, and nothing verifies it | approve | [#138](https://github.com/topij/agentic-dev-kit/issues/138) | +| 2 | `pr_watch.py`'s 403 blames the token, and neither the token nor the proxy's message is the problem | approve | [#139](https://github.com/topij/agentic-dev-kit/issues/139) | +| 3 | I filed a mechanism I had not tested, and it read as verified because it was specific | approve | [#140](https://github.com/topij/agentic-dev-kit/issues/140) | +| 4 | A rate-limited reviewer and an absent one are the same signal — fourth shape | approve | comment on #45 | +| 5 | `#113` reproduced as a setup condition, one day after being filed | approve | comment on #113 | +| 6 | The panel's worktree pointed at the wrong ref on 2 of 2 launches | approve | comment on #75 | +| 7 | `#73` gained an instance that is being kept on purpose | approve | comment on #73 | +| 8 | Four of the panel's ten findings were defects in the PR body, not the diff | approve | comment on #120 | +| 9 | A correct general argument was used to justify deleting instances it did not cover | approve | [#141](https://github.com/topij/agentic-dev-kit/issues/141) | +| 10 | `safety-critical-changes.md` rule 1 says to stop, not what to do when the change *is* the guard | approve | [#142](https://github.com/topij/agentic-dev-kit/issues/142) | +| 11 | `pr_watch.py:687` still discards the 403 body — needs a ticket | approve | [#139](https://github.com/topij/agentic-dev-kit/issues/139) (with 2) | +| 12 | `session-start`'s tracker step overflows its own tool limit at 68 open issues | approve | [#143](https://github.com/topij/agentic-dev-kit/issues/143) | +| 13 | CodeRabbit registered nothing on a fourth consecutive PR | approve | comment on #45 (with 4) | +| 14 | `#113` reproduced a third time | approve | comment on #113 (with 5) | + +#### What was verified, and what was not + +The full six-check script and its unedited output are in PR +[#144](https://github.com/topij/agentic-dev-kit/pull/144). Summarised honestly, because two +review rounds went to this block and both found the summary claiming more than the checks did: + +**Established.** The snapshot digest reconstructs from `06490a1`. All three swept blocks appear +in the archive verbatim modulo heading demotion, are **absent from the pre-change archive**, and +appear **exactly once**; zero entry bullets remain in this file; the prior archive body is +preserved. `#138`–`#143` exist, are OPEN, are authored by this account, and their titles contain +the expected fragments. Each of `#45`/`#113`/`#75`/`#73`/`#120` carries exactly one comment from +this run, matched by **author and timestamp**. `#23` is CLOSED and carries nothing from this run +(it has three comments overall, the latest being the previous sweep's). `7 + 7 = 14` against the +14 parsed bullets. + +**Not established, and worth naming.** The comment checks assert existence, not *content* — a +correct comment posted to the wrong issue would pass. The block-presence check is a substring +test over the whole archive; an adversarial lens showed it passes against an archive whose +visible text is destroyed while the real bytes hide in an HTML comment, so it rules out "archived +nothing", not "archived the wrong bytes". Nothing verifies that the approval happened as +described, or that the proposals shown were the proposals drafted — that is what the DM thread +would have carried. And no automated gate covers any of it +([#127](https://github.com/topij/agentic-dev-kit/issues/127)): a lens deleted a whole swept block +from the archive and `make mutation-test`, `check_doc_budget` and `kit_doctor` all stayed green. + +The swept entries now live in `kit-friction-log-archive.md`, under the section +`Graduated 2026-07-28 (second sweep)`. + +*(On that sentence, which took four tries. Three earlier versions carried a relative link to the +archive; the third also claimed to be "named, not linked" while doing so, and a lens caught the +contradiction. The link is now gone — but removing it is a smaller fix than it looked. +[#73](https://github.com/topij/agentic-dev-kit/issues/73)'s two recorded occurrences are both +**prose**, not markdown links: a sentence pointing "above" or naming another file, which stops +being true once the block moves. So the sentence above is still a latent instance — it names a +file it will sit inside after the next sweep. What removing the link actually bought is that it +will not also be a broken clickable target. Recorded rather than papered over, because two +versions of this parenthetical over-claimed the mitigation.)* + +### 2026-07-29 (session spanning from 2026-07-28) + +- **A correction applied to one copy of a claim, while the same claim stands on other surfaces, + was the dominant defect shape — four rounds running, on the same PR.** Round 1 found a false + `#23` sentence; it was rewritten in `docs/kit-friction-log.md`. Round 2 found the identical + sentence still published on `#45`'s occurrence comment — and that the round had amended `#73`'s + comment for a LOW in the same window, so the ability was there and the HIGH was the one missed. + Round 3 found the same claim still live in **`#140`'s issue body**. Round 4 found a round-3 fix + that had *silently matched nothing* (the target phrase wraps mid-sentence, the anchor assumed one + line) while its commit message reported it as landed. Each round fixed the surface it was pointed + at. **M** — proposed fix: when a claim is corrected, enumerate the surfaces it was published to + *at that moment*, rather than discovering them one review at a time. For this workflow the set is + fixed and short: the live log, the archive, the issue bodies the run filed, the occurrence + comments the run posted, the PR body, the commit messages. Distinct from `#138`, which asks the + *routing* to be verified — this asks a *correction* to be propagated. The silent-no-op half also + argues that a scripted text replacement should assert it changed something. +- **The verification a run writes about itself is a bigger defect source than the work it + verifies.** Across eight panel rounds and at least fifteen isolated lenses on three PRs, **no + HIGH was in executable behaviour** — every one was in prose. Some of that prose lives inside + `.py`/`.sh` files (a module docstring, a `# Requires:` header), so "prose" means wherever it + lives, not "outside the source tree". The sweep moved exactly the right bytes on its first + commit and no round ever found otherwise; three rounds went to the record describing it. The + documentation audit's edits were almost all correct; three rounds went to its evidence for them. + + **Three of the HIGHs were in prose that *ships*** — the class worth separating, because these + would reach an adopter: `pr-watch.md`'s flag table (it described `--assert-draft`/`--assert-ready` + as read-only checks when they issue `gh pr ready`, so following it flips a deliberately drafted + PR to ready), `devmodel_config.py`'s module docstring, and the `init.sh` prerequisite list, which + was wrong on **two** surfaces at once (`init.sh`'s own header *and* `README.md`). + + The mechanism is now visible: each correction round *adds prose*, and added prose is where the + next round's findings live. What broke the cycle was **deleting** the elaborate verification + transcript rather than correcting it a third time — the file went 141 → 93 lines and the defect + surface went with it. **No new fix proposed** — occurrence data for `#120`, with the + deletion-beats-correction observation attached. +- **A check whose heading is larger than its assertion reads as coverage.** The sweep's routing + check was headed *"every claimed comment exists on the issue it claims"* while asserting only + existence, author and timestamp — never content, so a comment carrying a falsehood passes (which + is exactly how the `#23` HIGH survived into round 2). Its block-integrity check was an unanchored + substring test: a lens built an archive whose visible text is `CORRUPTED` ×200 with the real bytes + hidden in an HTML comment at EOF, and **the check passed**. Both headings needed two rewrites to + match what the code does. **No new fix proposed** — occurrence data for `#138` (routing) and + `#127` (integrity). `#138` was filed by this session; `#127` was filed two sessions back + (`2026-07-28T13:46:49Z`, during `#126`'s review — the inbox-graduation session, not the + mutation-gate one between it and this). Both were reproduced inside this session's + pilot run of the checks they ask for. +- **`#75` reproduced on 14 of 14 lens launches.** Every isolated reviewer was placed in a worktree + at `main` with an empty `git diff main...HEAD`, across three PRs and eight rounds. Every one + detected it and fetched the real head, because the launch prompt required reporting path, sha and + diffstat *before* reviewing. Largest set recorded, and unanimous. **No new fix proposed** — + occurrence data for `#75`, but at 14/14 the contract item should stop saying "verify" and start + saying "assume wrong, fetch first". +- **A closing keyword in a squash message closed an issue documenting an unfixed defect — and + the check that cleared it could not see the surface that fired.** `#147`'s squash message read + *"Filed rather than fixed:"* followed directly by the two references. GitHub matched the + keyword immediately preceding the first and closed it when `030f053` landed: `gh api …/issues/145/events` returns + `event=closed commit_id=030f053`, and `commit_id` is populated only when a commit triggers the + close. The sentence was asserting the **opposite**. `#146`, filed the same way in the same + sentence, survived because no keyword happened to sit next to it. Reopened by hand. + + Two separate failures, and the second is the interesting one: + + 1. **The scan never ran on the surface that mattered.** A `close|fix|resolve`-adjacent-to-`#N` + scan was run on every PR body and on the added lines of every diff this session. A squash + message is composed at merge time, after every other gate has passed, and was never scanned. + `CLAUDE.md` names it explicitly; the habit did not. + 2. **The clearing check was structurally blind.** This entry's first version reported the + incident as a near-miss — *"no `closingIssuesReferences` were created (verified on both + PRs)"*. That field is derived from the **PR body** and cannot see a commit message, so it + returns `[]` whether or not a squash message fired. The verification was aimed at the wrong + surface and returned a confident, meaningless pass. + + Separately and more mildly: on one invocation the scan and the `gh pr edit` were chained in a + single shell command, so the edit published regardless of what the scan found. That one *was* + a near-miss — it found a `closed` adjacent to a reference in a PR body, and the body was + corrected. **M** — proposed fix: this is `#71`, and the instance sharpens where its guard must + live and what it must read. A scan the author can sequence after the thing it guards is not a + guard; and any "no harm done" check must read the issue's own **event stream** + (`gh api repos/:o/:r/issues/N/events`, looking for `closed` with a non-null `commit_id`), not a + PR-body-derived field. Second occurrence — the archive already records the same keyword firing from + an inline code span in a commit message against `#61` — and the first where the checking was also wrong. +- **CodeRabbit registered nothing on a sixth and seventh consecutive PR.** `#126`, `#129`, `#130`, + `#131`, `#137`, `#144`, `#147` — no check row, no comment, past grace on every one. The fallback + panel was the only independent pass throughout. The occurrence comment recording this pattern was + itself posted with an undercount ("four consecutive"), eight minutes after the fifth instance + merged. **No new fix proposed** — occurrence data for `#45`. +- **`#113` has a latent instance in a state path, not just a branch name.** This session ran a + *second* sweep on a date that already had one, so `chore/triage-{date}` and + `state/triage/frozen-inbox_{date}.json` were both candidates to collide. **Neither actually + did**, and for the same reason: the first sweep ran on `claude/triage-friction-log-kabrzh` + (`gh pr view 126 --json headRefName`) and wrote no snapshot at all, so the default branch name + was never taken either. The branch was renamed by hand against a collision that was not there. **No data was lost** — `stat` + reports `frozen-inbox_2026-07-28.json` with `created == modified == Jul 28 23:14:44`, this + session's write, and only the `2026-07-27` file predates it, because the first sweep never wrote + a snapshot at all. **M** — proposed fix: `#113` should cover date-patterned *state* paths as well + as branch names. The hazard is latent only because the engine that would have written the first + snapshot is not vendored (`#6`); once it is, a same-day re-run silently overwrites the artifact + the previous run's audit trail depends on. *(Recorded as latent after checking. The first draft of + this entry asserted the overwrite had happened — inferred from the shared path, with no command + run. One `stat` refuted it. That is `#140`'s shape, in the session that filed `#140`, caught this + time because the entry was checked before being committed rather than after.)* + ## Graduated 2026-07-28 (second sweep) — GitHub Issues (#138–#143) Swept by the `triage-friction-log` workflow, run in LLM-only mode (the engine tracked diff --git a/docs/kit-friction-log.md b/docs/kit-friction-log.md index 3768a9d..ec5c52e 100644 --- a/docs/kit-friction-log.md +++ b/docs/kit-friction-log.md @@ -11,193 +11,126 @@ > > Tracker board: https://github.com/topij/agentic-dev-kit/issues -## 2026-07-28 (second sweep) — Backlog migrated to GitHub Issues (#138–#143) +## 2026-07-29 — Backlog migrated to GitHub Issues (#149–#150) The inbox was swept by the `triage-friction-log` workflow, run in LLM-only mode (the engine tracked in [#6](https://github.com/topij/agentic-dev-kit/issues/6) is still not vendored). -**Fourteen entries in, fourteen accounted for:** seven graduated into six new issues -([#138](https://github.com/topij/agentic-dev-kit/issues/138)–[#143](https://github.com/topij/agentic-dev-kit/issues/143)), -seven routed as five occurrence comments (`#45` ×2 entries, `#113` ×2, `#75`, `#73`, `#120`). +**Seven entries in, seven accounted for:** one graduated into two new issues +([#149](https://github.com/topij/agentic-dev-kit/issues/149), +[#150](https://github.com/topij/agentic-dev-kit/issues/150)), six routed as seven occurrence +comments (`#120`, `#138`, `#127`, `#75`, `#71`, `#45`, `#113`). Two reconciliations, stated rather than left to a reader — mis-stating exactly these is what -`#138` was filed for: seven graduated entries became **six** issues because the `pr_watch` 403 -defect was recorded in two sessions and both entries went to -[#139](https://github.com/topij/agentic-dev-kit/issues/139); seven routed entries became **five** -comments because `#45` and `#113` each carry two. - -`#23` is named as a routing target by the swept text and received nothing **from this sweep** — -it is closed, and its occurrence data is consolidated on `#45`. It is not un-commented: it -carries the *previous* sweep's comment, posted `2026-07-28T13:46:17Z`, four minutes before `#126` -merged, after a panel caught that it had been omitted. +`#138` was filed for. Both are one-into-two. The correction-propagation entry became **two** +issues because the surface-enumeration checklist (`#149`) and the assert-your-edit-changed- +something guard (`#150`) are different mechanisms that can land independently. The +check-heading entry became **two** comments because it names two distinct checks: the routing +check's heading (`#138`) and the block-integrity check's unanchored substring test (`#127`). ### Approval record — in-session operator, no DM -This run was **interactive**, so the operator substituted for the DM review surface rather than -the stop being bypassed. [#128](https://github.com/topij/agentic-dev-kit/issues/128) asks that -such a substitute leave the approval record somewhere **committed**, since `state/` and -`reports/` are gitignored. This block is it. Approval was given up front and unconditionally — -*"I approve all suggestions"* — so every proposal carries the same decision and the -explicit-opt-in default was never exercised. - -**Frozen-inbox snapshot:** `state/triage/frozen-inbox_2026-07-28.json` (gitignored), -`sha256 b3a168a8ba8c18dc7d254fe76d1621b6ae5afff6d757540f1316c398643a6db7` over **14,341 bytes** -(14,235 characters; 54 non-ASCII — the digest is over the bytes). The snapshot is a *copy of a -committed blob*: the inbox at `06490a1` is that text, so the digest and every check in -the PR reproduce from `git` and `gh` alone, in any session. +`config/dev-model.yaml → notify.user_key` is empty, so there is no DM surface to stop on; the +operator was present and substituted for it, as in the 2026-07-28 second sweep. (This is the +**fifth** `triage-friction-log` sweep overall — the archive holds four earlier markers. Only the +second-of-07-28 also substituted; the first-of-07-28 is the run `#128` was filed *against*, and +the archive records that one as having **violated** the stop, not substituted for it.) +[#128](https://github.com/topij/agentic-dev-kit/issues/128) asks that such a substitute leave +the approval record somewhere **committed**, since `state/` and `reports/` are gitignored. This +block is it. Approval was bulk and unconditional — *"lgtm"* — so every proposal carries the +same decision and the explicit-opt-in default was never exercised. + +**Frozen-inbox snapshot:** `state/triage/frozen-inbox_2026-07-29.json` (gitignored), +`sha256 a24d1e32693a3df94f63aa5faa708c00381785c3b96587b7af9fe8fbed12a538` over **15,840 bytes** +(15,755 characters; 44 non-ASCII — the digest is over the bytes). Taken before any write, and a +*copy of a committed blob*: the inbox at `0b82ff2` is that text, so the digest reproduces from +`git` alone, in any session. | # | proposal (inbox entry, abridged) | decision | outcome | | - | -------------------------------- | -------- | ------- | -| 1 | A routing list is a claim about tracker state, and nothing verifies it | approve | [#138](https://github.com/topij/agentic-dev-kit/issues/138) | -| 2 | `pr_watch.py`'s 403 blames the token, and neither the token nor the proxy's message is the problem | approve | [#139](https://github.com/topij/agentic-dev-kit/issues/139) | -| 3 | I filed a mechanism I had not tested, and it read as verified because it was specific | approve | [#140](https://github.com/topij/agentic-dev-kit/issues/140) | -| 4 | A rate-limited reviewer and an absent one are the same signal — fourth shape | approve | comment on #45 | -| 5 | `#113` reproduced as a setup condition, one day after being filed | approve | comment on #113 | -| 6 | The panel's worktree pointed at the wrong ref on 2 of 2 launches | approve | comment on #75 | -| 7 | `#73` gained an instance that is being kept on purpose | approve | comment on #73 | -| 8 | Four of the panel's ten findings were defects in the PR body, not the diff | approve | comment on #120 | -| 9 | A correct general argument was used to justify deleting instances it did not cover | approve | [#141](https://github.com/topij/agentic-dev-kit/issues/141) | -| 10 | `safety-critical-changes.md` rule 1 says to stop, not what to do when the change *is* the guard | approve | [#142](https://github.com/topij/agentic-dev-kit/issues/142) | -| 11 | `pr_watch.py:687` still discards the 403 body — needs a ticket | approve | [#139](https://github.com/topij/agentic-dev-kit/issues/139) (with 2) | -| 12 | `session-start`'s tracker step overflows its own tool limit at 68 open issues | approve | [#143](https://github.com/topij/agentic-dev-kit/issues/143) | -| 13 | CodeRabbit registered nothing on a fourth consecutive PR | approve | comment on #45 (with 4) | -| 14 | `#113` reproduced a third time | approve | comment on #113 (with 5) | +| 1 | When a claim is corrected, enumerate every surface it was published to | approve | [#149](https://github.com/topij/agentic-dev-kit/issues/149) | +| 2 | A scripted text replacement that matches nothing must fail, not report success | approve | [#150](https://github.com/topij/agentic-dev-kit/issues/150) | +| 3 | The verification a run writes about itself outweighs the work it verifies | approve | comment on #120 | +| 4 | A check whose heading is larger than its assertion reads as coverage | approve | comment on #138 | +| 5 | The block-integrity check was an unanchored substring test | approve | comment on #127 | +| 6 | `#75` reproduced on 14 of 14 lens launches | approve | comment on #75 | +| 7 | A closing keyword in a squash message acted on an issue documenting an unfixed defect | approve | comment on #71 | +| 8 | CodeRabbit registered nothing on a sixth and seventh consecutive PR | approve | comment on #45 | +| 9 | `#113` has a latent instance in a state path, not just a branch name | approve | comment on #113 | + +**Proposal 8 was amended after approval, the amendment was wrong, and both panel lenses caught +it independently.** The operator's actual remark was narrow: CodeRabbit is *currently* not +available here, it is in use on `cs-toolkit`, and — the load-bearing part, which stands — its +absence **should not generate friction, because the fallback panel exists**. What reached the +`#45` comment was an inflation of that into *"not installed, never exercised, nothing +rate-limited, no credit run out."* `#45`'s own body records a **Pro Plus** plan for this repo, and +CodeRabbit has both reviewed PRs here and posted many `Review limit reached` notices, the last +activity of any kind being `2026-07-28T04:35:09Z` on `#101`. One `gh api` call establishes that — +exactly what [#140](https://github.com/topij/agentic-dev-kit/issues/140) asks for before writing +*"X is not available here."* + +The inflation was then used to file a **structurally-never** verdict onto `#45`, whose subject is +that structurally-absent and merely-pending are indistinguishable — committing that issue's own +confusion, on that issue. Corrected in place with both retractions visible: a **second** round +found the correction had over-claimed in the opposite direction, and independent attempts to count +how many PRs CodeRabbit actually *reviewed* (as against was refused for quota) returned different +answers. So no such count appears here. That irreducibility is the better evidence for `#45` than +any of the numbers were: from the outside, a twelve-PR silence is not distinguishable from +removal, quota exhaustion, or an infinite queue, and two successive careful readings got it wrong +in opposite directions. The swept entry's *"sixth and seventh consecutive"* is separately an +undercount — `#102`, `#103`, `#104`, `#111`, `#148` also carry nothing, making it **twelve**, the +third undercount in that series. The archive keeps the original wording; it is a verbatim record. ### What was verified, and what was not -The full six-check script and its unedited output are in PR -[#144](https://github.com/topij/agentic-dev-kit/pull/144). Summarised honestly, because two -review rounds went to this block and both found the summary claiming more than the checks did: - -**Established.** The snapshot digest reconstructs from `06490a1`. All three swept blocks appear -in the archive verbatim modulo heading demotion, are **absent from the pre-change archive**, and -appear **exactly once**; zero entry bullets remain in this file; the prior archive body is -preserved. `#138`–`#143` exist, are OPEN, are authored by this account, and their titles contain -the expected fragments. Each of `#45`/`#113`/`#75`/`#73`/`#120` carries exactly one comment from -this run, matched by **author and timestamp**. `#23` is CLOSED and carries nothing from this run -(it has three comments overall, the latest being the previous sweep's). `7 + 7 = 14` against the -14 parsed bullets. - -**Not established, and worth naming.** The comment checks assert existence, not *content* — a -correct comment posted to the wrong issue would pass. The block-presence check is a substring -test over the whole archive; an adversarial lens showed it passes against an archive whose -visible text is destroyed while the real bytes hide in an HTML comment, so it rules out "archived -nothing", not "archived the wrong bytes". Nothing verifies that the approval happened as -described, or that the proposals shown were the proposals drafted — that is what the DM thread -would have carried. And no automated gate covers any of it -([#127](https://github.com/topij/agentic-dev-kit/issues/127)): a lens deleted a whole swept block -from the archive and `make mutation-test`, `check_doc_budget` and `kit_doctor` all stayed green. - -The swept entries now live in `kit-friction-log-archive.md`, under the section -`Graduated 2026-07-28 (second sweep)`. - -*(On that sentence, which took four tries. Three earlier versions carried a relative link to the -archive; the third also claimed to be "named, not linked" while doing so, and a lens caught the -contradiction. The link is now gone — but removing it is a smaller fix than it looked. -[#73](https://github.com/topij/agentic-dev-kit/issues/73)'s two recorded occurrences are both -**prose**, not markdown links: a sentence pointing "above" or naming another file, which stops -being true once the block moves. So the sentence above is still a latent instance — it names a -file it will sit inside after the next sweep. What removing the link actually bought is that it -will not also be a broken clickable target. Recorded rather than papered over, because two -versions of this parenthetical over-claimed the mitigation.)* - -## 2026-07-29 (session spanning from 2026-07-28) - -- **A correction applied to one copy of a claim, while the same claim stands on other surfaces, - was the dominant defect shape — four rounds running, on the same PR.** Round 1 found a false - `#23` sentence; it was rewritten in `docs/kit-friction-log.md`. Round 2 found the identical - sentence still published on `#45`'s occurrence comment — and that the round had amended `#73`'s - comment for a LOW in the same window, so the ability was there and the HIGH was the one missed. - Round 3 found the same claim still live in **`#140`'s issue body**. Round 4 found a round-3 fix - that had *silently matched nothing* (the target phrase wraps mid-sentence, the anchor assumed one - line) while its commit message reported it as landed. Each round fixed the surface it was pointed - at. **M** — proposed fix: when a claim is corrected, enumerate the surfaces it was published to - *at that moment*, rather than discovering them one review at a time. For this workflow the set is - fixed and short: the live log, the archive, the issue bodies the run filed, the occurrence - comments the run posted, the PR body, the commit messages. Distinct from `#138`, which asks the - *routing* to be verified — this asks a *correction* to be propagated. The silent-no-op half also - argues that a scripted text replacement should assert it changed something. -- **The verification a run writes about itself is a bigger defect source than the work it - verifies.** Across eight panel rounds and at least fifteen isolated lenses on three PRs, **no - HIGH was in executable behaviour** — every one was in prose. Some of that prose lives inside - `.py`/`.sh` files (a module docstring, a `# Requires:` header), so "prose" means wherever it - lives, not "outside the source tree". The sweep moved exactly the right bytes on its first - commit and no round ever found otherwise; three rounds went to the record describing it. The - documentation audit's edits were almost all correct; three rounds went to its evidence for them. - - **Three of the HIGHs were in prose that *ships*** — the class worth separating, because these - would reach an adopter: `pr-watch.md`'s flag table (it described `--assert-draft`/`--assert-ready` - as read-only checks when they issue `gh pr ready`, so following it flips a deliberately drafted - PR to ready), `devmodel_config.py`'s module docstring, and the `init.sh` prerequisite list, which - was wrong on **two** surfaces at once (`init.sh`'s own header *and* `README.md`). - - The mechanism is now visible: each correction round *adds prose*, and added prose is where the - next round's findings live. What broke the cycle was **deleting** the elaborate verification - transcript rather than correcting it a third time — the file went 141 → 93 lines and the defect - surface went with it. **No new fix proposed** — occurrence data for `#120`, with the - deletion-beats-correction observation attached. -- **A check whose heading is larger than its assertion reads as coverage.** The sweep's routing - check was headed *"every claimed comment exists on the issue it claims"* while asserting only - existence, author and timestamp — never content, so a comment carrying a falsehood passes (which - is exactly how the `#23` HIGH survived into round 2). Its block-integrity check was an unanchored - substring test: a lens built an archive whose visible text is `CORRUPTED` ×200 with the real bytes - hidden in an HTML comment at EOF, and **the check passed**. Both headings needed two rewrites to - match what the code does. **No new fix proposed** — occurrence data for `#138` (routing) and - `#127` (integrity). `#138` was filed by this session; `#127` was filed two sessions back - (`2026-07-28T13:46:49Z`, during `#126`'s review — the inbox-graduation session, not the - mutation-gate one between it and this). Both were reproduced inside this session's - pilot run of the checks they ask for. -- **`#75` reproduced on 14 of 14 lens launches.** Every isolated reviewer was placed in a worktree - at `main` with an empty `git diff main...HEAD`, across three PRs and eight rounds. Every one - detected it and fetched the real head, because the launch prompt required reporting path, sha and - diffstat *before* reviewing. Largest set recorded, and unanimous. **No new fix proposed** — - occurrence data for `#75`, but at 14/14 the contract item should stop saying "verify" and start - saying "assume wrong, fetch first". -- **A closing keyword in a squash message closed an issue documenting an unfixed defect — and - the check that cleared it could not see the surface that fired.** `#147`'s squash message read - *"Filed rather than fixed:"* followed directly by the two references. GitHub matched the - keyword immediately preceding the first and closed it when `030f053` landed: `gh api …/issues/145/events` returns - `event=closed commit_id=030f053`, and `commit_id` is populated only when a commit triggers the - close. The sentence was asserting the **opposite**. `#146`, filed the same way in the same - sentence, survived because no keyword happened to sit next to it. Reopened by hand. - - Two separate failures, and the second is the interesting one: - - 1. **The scan never ran on the surface that mattered.** A `close|fix|resolve`-adjacent-to-`#N` - scan was run on every PR body and on the added lines of every diff this session. A squash - message is composed at merge time, after every other gate has passed, and was never scanned. - `CLAUDE.md` names it explicitly; the habit did not. - 2. **The clearing check was structurally blind.** This entry's first version reported the - incident as a near-miss — *"no `closingIssuesReferences` were created (verified on both - PRs)"*. That field is derived from the **PR body** and cannot see a commit message, so it - returns `[]` whether or not a squash message fired. The verification was aimed at the wrong - surface and returned a confident, meaningless pass. - - Separately and more mildly: on one invocation the scan and the `gh pr edit` were chained in a - single shell command, so the edit published regardless of what the scan found. That one *was* - a near-miss — it found a `closed` adjacent to a reference in a PR body, and the body was - corrected. **M** — proposed fix: this is `#71`, and the instance sharpens where its guard must - live and what it must read. A scan the author can sequence after the thing it guards is not a - guard; and any "no harm done" check must read the issue's own **event stream** - (`gh api repos/:o/:r/issues/N/events`, looking for `closed` with a non-null `commit_id`), not a - PR-body-derived field. Second occurrence — the archive already records the same keyword firing from - an inline code span in a commit message against `#61` — and the first where the checking was also wrong. -- **CodeRabbit registered nothing on a sixth and seventh consecutive PR.** `#126`, `#129`, `#130`, - `#131`, `#137`, `#144`, `#147` — no check row, no comment, past grace on every one. The fallback - panel was the only independent pass throughout. The occurrence comment recording this pattern was - itself posted with an undercount ("four consecutive"), eight minutes after the fifth instance - merged. **No new fix proposed** — occurrence data for `#45`. -- **`#113` has a latent instance in a state path, not just a branch name.** This session ran a - *second* sweep on a date that already had one, so `chore/triage-{date}` and - `state/triage/frozen-inbox_{date}.json` were both candidates to collide. **Neither actually - did**, and for the same reason: the first sweep ran on `claude/triage-friction-log-kabrzh` - (`gh pr view 126 --json headRefName`) and wrote no snapshot at all, so the default branch name - was never taken either. The branch was renamed by hand against a collision that was not there. **No data was lost** — `stat` - reports `frozen-inbox_2026-07-28.json` with `created == modified == Jul 28 23:14:44`, this - session's write, and only the `2026-07-27` file predates it, because the first sweep never wrote - a snapshot at all. **M** — proposed fix: `#113` should cover date-patterned *state* paths as well - as branch names. The hazard is latent only because the engine that would have written the first - snapshot is not vendored (`#6`); once it is, a same-day re-run silently overwrites the artifact - the previous run's audit trail depends on. *(Recorded as latent after checking. The first draft of - this entry asserted the overwrite had happened — inferred from the shared path, with no command - run. One `stat` refuted it. That is `#140`'s shape, in the session that filed `#140`, caught this - time because the entry was checked before being committed rather than after.)* +The six checks and their output are published on the PR. **Read them there rather than trusting a +summary here** — three review rounds went to this section in the 2026-07-28 sweep and two more +went to it here, every one finding the prose claiming more than the checks did. This version +states less on purpose; the earlier sweeps' remedy for the same loop was deletion, not a further +correction. + +**What the checks establish.** The snapshot digest reconstructs from `0b82ff2`. The archived +block, un-demoted, matches the snapshot modulo one trailing newline (hence `15,839` in the output +against `15,840` here). The prior archive body is preserved byte-for-byte. `#149` and `#150` exist +and are OPEN. Each of `#120`/`#138`/`#127`/`#75`/`#71`/`#45`/`#113` carries exactly one comment +from this run, and its body hashes to the text this run sent. The live log holds one bullet. +`1 + 6 = 7` against the 7 parsed bullets. + +**What they do not.** Several check *headings* on the PR name more than their bodies assert — +issue titles and labels are printed but not compared, and the bullet count has no notion of +"swept", so a restored swept entry would pass it. A reviewer demonstrated both, and demonstrated +that checks 1 and 2 share no trust chain: the snapshot's text field is never hashed, so a forged +snapshot produces byte-identical output. These are named rather than fixed — building a better +harness inside a fix round is the mechanism-creep the panel doctrine warns against, and `#138` and +`#127`, which ask for exactly that harness, both stay open. The content check is also the one +thing here a third party cannot re-run: its right-hand side is a local file. Nothing verifies that +the approval happened as described, and nothing here proves any posted comment is *true* — the +`#45` amendment above was caught by a reviewer, not by a check. No automated gate covers any of it +([#127](https://github.com/topij/agentic-dev-kit/issues/127)). + +The swept entries now live in the archive, under the section `Graduated 2026-07-29`. + +## 2026-07-29 (post-sweep) + +- **The sweep re-derived `#121` from scratch without noticing it existed, and got two things + wrong that `#121` would have corrected.** [#121](https://github.com/topij/agentic-dev-kit/issues/121) + is OPEN, filed by the *previous* run of this workflow, and already covers `tracker.backend: + linear` with blank `linear.*` ids, the placeholder `tracker.project_name`, and `notify.user_key` + (routed onward to `#128`) — closing by asking whether any *other* block is still unstamped + placeholder, the question this entry then asked again a day later as though it were new. The + first draft claimed each instance "has been paid for separately" and that the placeholders carry + "no comment saying they are stale". Both false; `#121` is the comment. A panel lens found it. + The one key not in `#121`, `review.bots: [coderabbit]`, turned out to need no fix at all: the + draft called the bot "not installed on this repo", which the marker above retracts, so the value + is **accurate**. **M** — proposed fix, narrowed to what survives: `#121` should absorb the + remaining question, which is not "these values are wrong" but *which* of this config is template + and which is live, and how a reader could tell. `paths.*` carries a six-line comment explaining + why **this repo** deviates; `tracker.*` and `notify.*` carry only schema hints (`# linear | + github-issues | jira | none`, `# a key into your project's own notify config`) that say nothing + either way — so a placeholder and a deliberate value are typographically identical. (The first + draft of this sentence said those keys "carry no comment", which a lens refuted; they do, just + not comments that answer the question.) + Separately, and independent of any value: `review.bots` cannot express *"expected, currently + silent, panel is the pass"* — `#45`'s subject, on which this sweep spent a HIGH. **Not from the + swept inbox:** surfaced during pre-flight, recorded on operator instruction, and + rewritten after review; it sits below the marker for the next pass, where it should be merged + into `#121` rather than filed fresh.