Make the fallback review a panel, not a command (#26)

.agents/skills/pr-watch/SKILL.md

@@ -9,5 +9,5 @@ description: Drive a pull request through the deterministic poll, fix, acknowled
 2. Read `config/dev-model.yaml` and `docs/agentic-dev-kit/workflows/pr-watch.md` completely.
 3. Follow the workflow for the PR number in the user's request, or the current branch's PR when none is supplied.
 4. Resolve the engine path from the repository root; support both `scripts/pr_watch.py` and a namespaced adopted path such as `scripts/devkit/pr_watch.py`.
-5. Use `review.fallback_commands` for the current runtime when configured. Never treat an unavailable review bot as a review waiver.
+5. When a bot is unavailable, run the `review.fallback_panel` pass — one isolated, fresh-context reviewer per lens, per `docs/agentic-dev-kit/fallback-review-panel.md`. Fall back to `review.fallback_commands` only if the runtime cannot isolate a reviewer, and record that as one lens, never as the configured `review.fallback_panel.receipt_source`. Never treat an unavailable review bot as a review waiver.
 6. For safety-critical changes, also read and apply `docs/agentic-dev-kit/safety-critical-changes.md` before recommending merge.

.gitignore

@@ -8,3 +8,6 @@ __pycache__/
 # appends these to an adopting repo's .gitignore.
 state/
 .devkit_state_root
+
+# Agent worktrees (created by isolated review lenses) are never committed.
+.claude/worktrees/

config/dev-model.yaml

@@ -93,6 +93,29 @@ review:
   # `CodeRabbit` and the author `coderabbitai` either way. Keep entries specific
   # enough not to collide with a CI job name.
   bots: [coderabbit]
+  # The independent pass to run when a configured bot can't review. TWO shapes,
+  # and the difference is whether the reviewer is independent of the author:
+  #
+  #   fallback_panel     one isolated, fresh-context reviewer PER LENS. This is
+  #                      the real substitute — `safety-critical-changes.md`
+  #                      rule 2 wants two disjoint lenses, which a single
+  #                      command cannot be.
+  #   fallback_commands  one command in the cockpit's own context. When the
+  #                      cockpit authored the diff that is the author
+  #                      re-reading their own work — kept as the DEGRADED mode
+  #                      for a runtime with no way to isolate a reviewer.
+  #
+  # Which lenses is adopter knowledge and lives here; how a lens must be run
+  # (fresh context, raw diff, no author framing, execute rather than only read,
+  # mutation-test new branches) is kit doctrine and lives in
+  # docs/agentic-dev-kit/fallback-review-panel.md.
+  fallback_panel:
+    receipt_source: "fallback:panel"   # never the same as a single-lens receipt
+    lenses:
+      - name: adversarial
+        focus: "assume the change is wrong and try to prove it — bypasses, fail-open paths, wedges, and whether the new guard actually guards"
+      - name: correctness
+        focus: "assume it works and ask what it says — stale comments, claims that overstate what is verified, tests whose names promise more than their bodies check"
   fallback_commands:
     claude: "/code-review"
     codex: "/review"

docs/CLAUDE-sections.md

@@ -65,10 +65,14 @@ infra failure) — then report the specific blocker and ask.
 
 **A review bot being unavailable is not a reason to skip review.** When a configured bot
 can't review an otherwise-ready PR — rate-limited, out of credits, or it silently skipped
-the PR — run `review.fallback_commands.claude` as the independent review pass and triage its
+the PR — run the `review.fallback_panel` pass (one isolated, fresh-context reviewer per
+lens; see `docs/agentic-dev-kit/fallback-review-panel.md`) and triage its
 findings the same way (fix if confident and small, reply-with-reason otherwise). A blocked
 review bot is **not** a waiver: the "clean" bar still requires one independent review
-pass — bot *or* the fallback command.
+pass — a configured bot, or a completed `review.fallback_panel` pass.
+`review.fallback_commands` is the DEGRADED one-lens mode for a runtime that cannot
+isolate a reviewer; it is the author re-reading their own diff, which
+`safety-critical-changes.md` rule 2 does not accept as a green light.
 
 ---
 

docs/agentic-dev-kit/fallback-review-panel.md

@@ -0,0 +1,158 @@
+# The fallback review panel
+
+What to run when your configured review bot can't. Read this whenever
+`pr_watch` reports a reviewer as unavailable, and before recording any
+`fallback:` receipt.
+
+## Why a panel and not a command
+
+`review.fallback_commands.<runtime>` runs the runtime's own review command — in
+the cockpit's context. When the cockpit *authored* the diff, that is the author
+re-reading their own work. It finds things; it does not find the things the
+author is blind to, which are exactly the ones that survive to production.
+
+`safety-critical-changes.md` rule 2 already says a single-lens verdict "is an
+incomplete review, not a green light". A single command cannot satisfy a rule
+that asks for two disjoint lenses, so the mechanism was violating the doctrine
+it was meant to serve.
+
+**The evidence is disjointness, not volume.** Run as two fresh-context lenses
+over one merge-gate change, the panel found a stale outage comment that
+reintroduced the exact bug the PR existed to fix; a future-dated clock that
+wedged the gate for 30 days *after* the fix for that class shipped; and a
+section-scoping fix applied to one of three guards in the same function. The two
+lenses overlapped on almost nothing, and the author's own passes had found none
+of it.
+
+## The lenses
+
+Configured in `review.fallback_panel.lenses` — **that is where each lens's
+brief lives**, as its `focus`, and it is what you hand the reviewer. Restating
+the briefs here would give the kit two copies to drift apart, so this section
+describes only what the two shipped lenses are *for*:
+
+- **adversarial** — starts from "this is wrong" and tries to prove it.
+- **correctness** — starts from "this works" and asks what it *says*.
+
+They are the two the doctrine names, and they are chosen to overlap as little
+as possible. Add or replace lenses for your own risk profile (a data-migration
+lens, a performance lens): two disjoint lenses is the floor, not the ceiling.
+
+## The contract every lens gets
+
+These are why the panel works. Drop any of them and it degrades toward the
+author re-reading their own diff.
+
+1. **Fresh context.** A lens that watched the change being written inherits the
+   author's model of it, including the wrong parts.
+2. **The raw diff, no framing.** Do not tell the lens what the change is *for*,
+   what you already fixed, or what you think is risky. That is the anchoring
+   the panel exists to escape.
+3. **Say it did not write the code.** Cheap, and it measurably changes what a
+   lens is willing to call wrong.
+4. **Execute, don't only read.** The highest-value findings came from *running*
+   the changed paths against hostile input, not from reading them. A guard was
+   dead code for the exact bot it was written for, because that bot reports a
+   zero timestamp — no amount of re-reading surfaced it; one live poll did.
+5. **Mutation-test new branches.** Break the new behaviour deliberately and
+   confirm a test fails. Repeatedly, properties were *named* by a test and
+   pinned by nothing — hardwiring a branch to a constant still passed the whole
+   suite.
+
+   **Beware false kills.** If your repo has a checksum/drift test over the files
+   you are mutating (this kit has one: `kit_doctor`'s self-check), *every*
+   mutation fails it regardless of behaviour, and a whole run can report 100%
+   killed while nothing behavioural caught anything. One lens's first pass here
+   reported 17/17 killed; re-run with that test excluded, 7 had survived.
+   Exclude the drift test — or regenerate the manifest after each mutation —
+   before believing a kill.
+6. **Report, do not fix.** A lens that edits loses the disjointness: it starts
+   defending its own changes on the next round.
+7. **Mutate in an isolated copy, never the shared tree.** Mutation testing needs
+   temporary writes, and lenses run concurrently. Discovered the hard way on the
+   PR that added this file: one lens's mutations appeared to the other as an
+   external process corrupting the repo, and it "restored" them mid-run — so one
+   lens's results were unreliable and the other nearly destroyed live work. Give
+   each lens a scratch copy or its own git worktree, and require it to leave the
+   shared tree byte-identical.
+8. **State what was verified clean, and how.** Absence of findings is only
+   evidence if you know what was actually checked.
+
+## Running it
+
+1. Read the change: `git diff <base>...HEAD`.
+2. Launch **one isolated reviewer per lens**, concurrently, each with the
+   contract above and its lens focus. Use whatever isolation your runtime has
+   (a subagent, a separate session, a second person). If it has none, see
+   *Degraded mode*.
+3. Triage every finding against the *current* code — some go stale across
+   rounds.
+4. Fix real findings, reply-with-reason to the rest.
+5. **Re-run the panel after the fix round.** Not optional: see below.
+6. Record the receipt with the lenses that actually ran:
+
+   ```sh
+   uv run <engine-dir>/pr_watch.py <PR#> \
+     --record-review "<review.fallback_panel.receipt_source>" \
+     --lenses <names of the lenses that actually ran> \
+     --head <polled-sha>
+   ```
+
+   **`--lenses` is self-reported, and the engine does not verify it.** You write
+   the source and the lens names in one invocation; nothing binds either to a
+   review that happened. Four rounds of this PR tried to verify it from the
+   engine — matching the source, then the lens names, then a required roster,
+   then a counted one —
+   and each was defeated, the last by a single extra character in the source,
+   after which the render cheerfully affirmed the forgery.
+   `safety-critical-changes.md` rule 1 calls that a stopgap, not a fix.
+
+   So what you get is an **audit trail, not a gate**. The poll render states the
+   claim, labelled as a claim, on every poll once a current-head receipt exists
+   (no receipt, or one bound to an older head, prints nothing):
+
+   ```text
+   review evidence: fallback:codex — ⚠ ONE lens claimed (correctness) — not a dual-lens pass
+   ```
+
+   That is genuinely useful — a one-lens pass is visible at merge time instead of
+   buried in the record command's stdout — and it is worth exactly what an honest
+   operator puts into it. Verifying coverage needs each lens to record its own
+   receipt from its own context: issue #32, not this.
+
+## Re-running, and when to stop
+
+`safety-critical-changes.md` rule 3 says to re-review after every fix round
+"until a full pass finds nothing new". Take that literally *and* know its limit:
+across one session of 13 rounds on three PRs, **every round found something**,
+and seven of those findings were defects introduced by the *previous round's
+fix*. The termination condition may never arrive.
+
+So the stopping criterion is **blast radius, not round count**:
+
+- A **gate, send path, or destructive operation** — keep going. Worst case is an
+  unreviewed change landing, and the doctrine's operator-merge rule applies.
+- Something **reported but never acted on** (a warning, a log line, a report
+  field) — a couple of rounds with the findings decaying in severity is
+  proportionate. Worst case is a wrong message.
+
+Say which one you applied **in the PR**, where a human reads it — the receipt
+carries what the review did *not* cover (`override`, `bot_signal`,
+`bots_behind_head`) and what it did (`lenses`) — not a prose rationale for
+stopping. "The last round found
+nothing" is not available as a reason if it never happened.
+
+## Degraded mode
+
+If the runtime cannot run isolated reviewers, fall back to
+`review.fallback_commands.<runtime>` — one lens, in the author's context. It is
+better than nothing and it is **not** a panel:
+
+- record it as `fallback:<runtime>`, never the panel's `receipt_source`. The
+  engine will not stop you doing otherwise — it cannot tell what ran. The
+  honesty is entirely yours, which is the reason to write it down
+- pass `--lenses` naming what actually ran, so the audit trail shows one lens
+- for anything under `safety-critical-changes.md`, say plainly in the PR that
+  rule 2 was not satisfied
+
+A receipt should never claim more coverage than the review it stands for.

docs/agentic-dev-kit/safety-critical-changes.md

@@ -21,12 +21,17 @@ skill; do not fork the doctrine into runtime-specific copies.
    is not enough: an adversarial/bypass-focused pass and a general-correctness pass
    routinely find **disjoint** holes. A send/publish gate needs BOTH lenses before
    merge. A single-lens "converged" verdict is an incomplete review, not a green
-   light.
+   light. When your review bot is unavailable, the substitute that satisfies this
+   rule is the panel in [`fallback-review-panel.md`](fallback-review-panel.md) —
+   a single fallback command run in the author's own context does not.
 
 1. **Adversarial review to convergence, not one pass.** Re-review after every fix
    round until a full pass finds nothing new. Fix rounds on gate logic routinely
    introduce their own regressions — treat "the last round found nothing" as
-   provisional, not proof of safety.
+   provisional, not proof of safety. Be aware that "finds nothing new" may never
+   arrive: see [`fallback-review-panel.md`](fallback-review-panel.md) for the
+   observed base rate and for the stopping criterion to use instead — blast
+   radius, not round count.
 
 1. **Kill/recovery paths need an integration test.** Unit tests on the handler are
    insufficient — a kill-path can pass unit tests while the wrapper-level behavior is

docs/agentic-dev-kit/workflows/pr-watch.md

@@ -7,9 +7,12 @@ run it without being asked.
 
 **Input:** an optional PR number. With none, the current branch's open PR is used.
 
-Read `config/dev-model.yaml` first. Resolve `<engine-dir>` from `paths.engines`,
-and select the current runtime's independent fallback from
-`review.fallback_commands` when needed.
+Read `config/dev-model.yaml` first. Resolve `<engine-dir>` from `paths.engines`.
+When a configured review bot is unavailable, the substitute pass is the
+**panel** in `review.fallback_panel` — read
+[`../fallback-review-panel.md`](../fallback-review-panel.md) before running it.
+`review.fallback_commands` is the degraded one-lens mode for a runtime that
+cannot isolate a reviewer.
 
 If the diff affects a customer-facing gate, destructive operation, recovery path,
 or other configured high-risk file, read and apply
@@ -65,14 +68,26 @@ Repeat until the report says **converged**:
 1. **If there are `new_comments`:** handle each with judgment —
 
    - **Reviewer unavailable** (`review_unavailable_reason` is set — rate limit,
-     skipped review, no credits): run the current runtime's configured
-     `review.fallback_commands` pass. A blocked bot is an action signal, never
-     auto-noise or a review waiver. Acknowledge the notice only after the fallback
-     review has completed and every finding from it is handled. Then record the
-     pass against the exact `head` from the poll you reviewed with `uv run
-     <engine-dir>/pr_watch.py <PR#> --record-review "fallback:<runtime>" --head
-     <polled-sha>`. For a lane, use `<engine-dir>/dev_session.sh pr-watch <scope>
-     --record-review "fallback:<runtime>" --head <polled-sha>` instead.
+     skipped review, no credits): run the **fallback review panel** —
+     `review.fallback_panel`, one isolated fresh-context reviewer per lens. Read
+     [`../fallback-review-panel.md`](../fallback-review-panel.md) for the contract
+     each lens gets; it is what makes the pass independent of you. A blocked bot
+     is an action signal, never auto-noise or a review waiver. Acknowledge the
+     notice only after every finding is handled, then bind the pass to the exact
+     `head` from the poll you reviewed:
+
+     ```sh
+     uv run <engine-dir>/pr_watch.py <PR#> \
+       --record-review "<review.fallback_panel.receipt_source>" \
+       --lenses <names of the lenses that ran> --head <polled-sha>
+     ```
+
+     `--lenses` names what actually ran, so a degraded one-lens pass is
+     distinguishable from a panel in the audit trail. It is **self-reported** —
+     the engine records it and shows it at merge time, but cannot verify it. If your
+     runtime cannot isolate a reviewer, run `review.fallback_commands` instead
+     and record it as `fallback:<runtime>` with the single lens named. For a
+     lane, use `<engine-dir>/dev_session.sh pr-watch <scope>` with the same flags.
 
    - **Real finding** (a bug, a missing guard, a correctness/clarity issue): fix it in
      the code, commit, push. Re-running the local gate first.
@@ -141,8 +156,8 @@ Self-pace on a bounded cadence — don't busy-wait:
   summaries) is filtered out by the engine. Reviewer-unavailable notices are
   deliberately *not* noise: they surface as new comments and so block `converged`;
   acknowledging one clears `converged` but still leaves the current-head
-  review-evidence blocker on `mergeable` until the configured fallback runs and
-  records its receipt.
+  review-evidence blocker on `mergeable` until the panel runs and records its
+  receipt.
 - **A bot's outage is detected on both surfaces, and a queued bot is not a finished
   one.** `review.unavailable_markers` are matched against comment bodies *and*
   against the status-check description of any check belonging to a configured
@@ -151,7 +166,7 @@ Self-pace on a bounded cadence — don't busy-wait:
   happened to use. The report's `review_bots` block resolves each bot to:
   - **unavailable** — an outage announced on either surface. Rendered as
     `⚠ review unavailable …`, and it never blocks anything: it's the action signal
-    to run `review.fallback_commands`. It stays visible after you `--mark-seen` the
+    to run the `review.fallback_panel` pass. It stays visible after you `--mark-seen` the
     notice comment, so the gap is still readable at merge time. Only a
     **check**-surface outage cancels the pending block below: a check describes
     the bot's state now, while comments are the whole PR history unscoped by
@@ -204,7 +219,26 @@ Self-pace on a bounded cadence — don't busy-wait:
   invalidating a receipt on a shape change risks wedging a repo whose bot is
   permanently unavailable.
 
+  Once a current-head receipt exists, every poll also prints what that receipt
+  **claims** to cover:
+
+  ```
+  review evidence: fallback:panel — 2 lenses claimed (adversarial, correctness)
+  review evidence: fallback:codex — ⚠ ONE lens claimed (correctness) — not a dual-lens pass
+  ```
+
+  Self-reported: `--lenses` is written by whoever ran `--record-review`, and the
+  engine records it without verifying it (issue #32). It is shown here so a
+  one-lens pass is visible when a merge is considered rather than only in the
+  record command's output. Entries that read as prose rather than a lens name
+  are recorded but not counted.
+
   **Known gaps, so you don't mistake them for coverage:**
+  - The `review evidence:` line prints on the **poll render**, which a human
+    reads. `dev_session.sh merge` consumes the JSON and gates on `mergeable`
+    alone — so on an autonomous self-merge path nobody sees it. That is the gap
+    issue #32 exists to close properly; until then, an unattended lane's review
+    coverage is only as good as what it recorded.
   - `coverage` reports only bots that have reviewed *and* whose review carried a
     commit SHA. A bot that has never reviewed at all produces no entry and no
     warning — that case is the pending/unavailable machinery's, not this one's.