Wrap-up: Phase 3b — the queued-vs-unavailable ambiguity, closed#30
Conversation
Three PRs merged, three issues closed, two filed. - **#25** (closes #19 + #23) — a configured bot resolves to *unavailable* (outage on a comment body OR a status-check description, the surface that was invisible) or *pending* (a verdict still coming). Nothing reaches `converged`, which is what let both be fixed at once without touching the anti-wedge property. - **#28** (closes #10) — the state_paths suite failed from inside a lane worktree because its fixture cleared every sandbox *env* signal and not the *cwd* one. - **#29** (half of #27) — `review_bots.coverage` reports which commit each bot's last review actually saw. - **#26 / #27 filed** from the friction-log inbox, both with concrete sketches. The handoff and friction log carry the detail. The two entries worth repeating: **Eleven review rounds, ten with findings — and five of those were defects introduced by the previous round's fix**, twice at HIGH. #25 never produced a clean pass. That is not a caution about fix rounds; it is the base rate, and the doctrine currently states it as advice. **Reading is not running.** The #19 guard was dead code for the exact bot it was written for, because CodeRabbit's pending check reports the zero timestamp — no amount of re-reading found that; polling the live PR did. Same for a change that looked plainly correct and aborted `init.sh` under `set -eu`. Also recorded: `pre-push` refused this very commit's first attempt, because the narrative files had been swept into a lane branch. The guard fired on its own author with a message that made the fix obvious — logged as the positive case this log rarely captures. Claude-Session: https://claude.ai/code/session_011KVJzDj7Z2nYkUjqbGgSVp
|
Warning Review limit reached
Next review available in: 56 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughThe documentation records migrated friction items, new review-process observations, and a completed Phase 3b session. The handoff updates PR outcomes, merge-gate decisions, verification notes, open work, and the constraint that fixes do not touch ChangesDocumentation and session records
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
I had the wrap-up independently fact-checked against `git log` / `gh` rather than self-certifying it, because the handoff is read at the start of every future session. It came back with 13 issues, three HIGH. The one worth naming: **I reintroduced a number that a review round on #25 had explicitly corrected** — "four ways to corrupt an adopter's config" when `321907e` had already changed it to three, and `init.sh`'s own comment enumerates three. I did that while writing up the lesson that PR bodies keep drifting from their diffs. The friction entry recording that failure now records that it recurred inside the document describing it, and is filed at M not L. **The base rate was understated, in my own argument's favour.** I wrote "eleven rounds, ten with findings" and "#29 produced a clean pass on the fourth attempt". Neither is true: session-wide it is **13 rounds across three PRs, all 13 with findings**, and #29's fourth pass found no HIGH/MEDIUM but still found four LOW including a stale comment. Self-inflicted defects — introduced by the previous round's fix — were 7, not 5. So "re-review until a pass finds nothing new" never terminated, which sharpens rather than weakens the point: the rule needs a stopping criterion, and the one actually used was blast radius. **The proposed rule change was aimed at a rule that already exists.** `safety-critical-changes.md` rule 3 already mandates re-review after every fix round and already notes that fix rounds introduce regressions. The proposal now asks for what is genuinely missing — a termination criterion, and the blast-radius calibration that let #29 merge where #25's standard would not have. Also corrected: the mutation-proved tally (five properties session-wide, not two in one place and four in another); "the panel carried the entire review load" (CodeRabbit completed 3 reviews across 17 heads — most, not all, and one of its three caught a real bug); "five manual launches per round" (two lenses per round, ~10 rounds); a "126 report pairs" figure that exists only in a PR comment and is not reproducible from the repo, now pointing at the in-repo tests instead; a dangling "the two entries above this line" after those entries were deleted; and two inbox entries whose issues this session closed, now marked done. Claude-Session: https://claude.ai/code/session_011KVJzDj7Z2nYkUjqbGgSVp
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/kit-friction-log.md`:
- Around line 75-76: The mutation-testing totals and property lists are
inconsistent across the two documents. Update docs/kit-friction-log.md lines
75-76 and docs/kit-handoff.md lines 82-86 to use the authoritative result of
three unpinned PR `#29` properties, keeping both descriptions and lists aligned.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 91d3c1e6-d390-4842-8fd0-fc21132be843
📒 Files selected for processing (2)
docs/kit-friction-log.mddocs/kit-handoff.md
| the lens to *execute* the changed paths and to mutation-test new branches — mutation | ||
| is what proved two properties on #29 were unpinned despite tests that named them. |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== files =="
git ls-files | sed -n '/docs\/kit-friction-log.md/p;/docs\/kit-handoff.md/p'
echo "== doc snippets =="
sed -n '60,85p' docs/kit-friction-log.md | cat -n
echo "---"
sed -n '76,90p' docs/kit-handoff.md | cat -n
echo "== references to `#29` and mutation-related text across docs =="
rg -n "`#29`|mutation testing|mutation-test|unpinned|author matching|newest-review-per-bot|bots=|init\.sh|grep -qi|pinned by nothing" docs || trueRepository: topij/agentic-dev-kit
Length of output: 7826
Keep the mutation-testing totals consistent across both documents.
The handoff attributes three unpinned #29 properties, while the friction log still says two. Use the authoritative mutation-testing result for both entries and keep the property lists aligned.
docs/kit-friction-log.md#L75-L76: update the “two properties” claim or description.docs/kit-handoff.md#L82-L86: keep the PR#29count/list aligned with the authoritative result.
📍 Affects 2 files
docs/kit-friction-log.md#L75-L76(this comment)docs/kit-handoff.md#L82-L86
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@docs/kit-friction-log.md` around lines 75 - 76, The mutation-testing totals
and property lists are inconsistent across the two documents. Update
docs/kit-friction-log.md lines 75-76 and docs/kit-handoff.md lines 82-86 to use
the authoritative result of three unpinned PR `#29` properties, keeping both
descriptions and lists aligned.
CodeRabbit, on the wrap-up. My correction commit fixed the handoff's tally to five session-wide and left the friction log saying two — so the two documents the next session reads disagreed with each other about the very number the correction was about. Both now say five, with the properties enumerated (three on #29, two on #25) so the figure is checkable rather than asserted. Claude-Session: https://claude.ai/code/session_011KVJzDj7Z2nYkUjqbGgSVp
|
Merging. Receipt: This PR was fact-checked rather than self-certified, because the handoff is read at the start of every future session and a wrong number there propagates further than a wrong PR body. The check came back with 13 issues, three HIGH. The one worth naming: I had reintroduced a figure that a review round on #25 had explicitly corrected — while writing up the lesson that PR bodies keep drifting from their diffs. That entry is now filed at M instead of L, and records that the failure recurred inside the document describing it. It also corrected the base rate against my own argument's favour: not "eleven rounds, ten with findings" but 13 rounds across three PRs, all 13 with findings, and 7 self-inflicted by the previous round's fix rather than 5. So "re-review until a pass finds nothing new" never terminated this session — which sharpens the point rather than softening it, and is why the proposed rule change now asks for a stopping criterion rather than for a floor that rule 3 already states. Incidental validation: the coverage warning merged in #29 an hour ago fired on this very PR, on both surfaces — the poll render and now the receipt. That is the #22/#25 gap being caught automatically instead of by hand. |
1 participant
Session wrap-up. Narrative files only — no code.
Shipped
convergedreview_bots.coverage— which commit each bot's last review sawFiled from the inbox: #26 (fallback review as a panel spec) and #27
(receipt staleness; the cheap half shipped in #29).
The two things worth carrying forward
Eleven review rounds, ten with findings — five of them defects introduced by
the previous round's fix, twice at HIGH. #25 never produced a clean pass; #29
produced one on the fourth attempt.
safety-critical-changes.mdrule 3 alreadysays to treat "the last round found nothing" as provisional. This session says
it is not a caution, it is the base rate — which is a candidate for graduating
the rule from advice to a floor.
Reading is not running. Three defects were invisible to careful reading and
obvious on execution. The sharpest: CodeRabbit's pending check reports
startedAt: 0001-01-01T00:00:00Z, so the "unmeasurable age fails open" branchwas not an edge case but the only path that bot ever took — the #19 guard was
dead code for its own target, and only polling the live PR showed it.
Also recorded
pre-pushrefused this commit's first attempt, because the narrative files hadbeen swept into a lane branch by a
git add -A. It named both files and saidwhere lane handoff belongs. Logged as the positive case the friction log
rarely captures: a fail-closed guard firing on its own author, with a message
that made the fix obvious, at a cost of about ten seconds.
Next
#26 — make the fallback a panel spec. CodeRabbit was rate-limited on nearly
every head today, so the panel carried the entire review load and is still five
manual subagent launches per round. Two lessons belong in its prompt: require the
lens to execute the changed paths, and to mutation-test new branches — that
is what proved four separate properties were named by tests and pinned by nothing.
https://claude.ai/code/session_011KVJzDj7Z2nYkUjqbGgSVp
Summary by CodeRabbit