Skip to content

Close the audit's findings on itself, and run application-security on Opus - #452

Merged
nedtwigg merged 19 commits into
mainfrom
audit-scope-catchall
Aug 27, 2026
Merged

Close the audit's findings on itself, and run application-security on Opus#452
nedtwigg merged 19 commits into
mainfrom
audit-scope-catchall

Conversation

@nedtwigg

Copy link
Copy Markdown
Member

Follow-up to #445, driven by #447 and by running scripts/security-audit-local.sh repeatedly until all three domains came back clean. Supersedes #449, #450, and #451 — see the last section.

The finding that started it

default_workflow_permissions was write, so tend-mention.yaml's verify job — the only job in the repo declaring no permissions: block — got a token with nine write scopes, in a job triggerable by any issue comment. Flipped to read, with can_approve_pull_request_reviews to false. It costs tend nothing: that job's steps authenticate with TEND_BOT_TOKEN via a step-level env:, which is upstream's own design, so the ambient token was never used.

Also enabled secret scanning (history backfill: 0 alerts), push protection, and Dependabot alerts — which immediately surfaced two moderate Rust advisories. Renovate gets a vulnerabilityAlerts block so a CVE doesn't wait for the Monday window.

tend 0.1.19 closes the AGENTS.md gap

Upstream released the #1005 fix on 2026-08-26, hours before this branch. pin-instruction-paths.sh now globs **/AGENTS.md at any depth where 0.1.18's flat list named it nowhere — which mattered here specifically, because this repo keeps its instructions in AGENTS.md behind a one-line CLAUDE.md pointer, so the old revert restored a pointer and left the body attacker-controlled on a fork PR. Regenerated onto 0.1.19; the FAIL IF now pins a version floor, since the control lives in upstream code and a pin moving backwards reopens it invisibly.

Product fixes the audit found

Session snapshots were world-readable. write_session_to created sessions/ and its files under the bare umask — 0644 in a 0755 directory. Those blobs carry terminal transcripts: whatever your shells printed, which is a superset of every other secret this document tracks. Now 0700/0600, applied to the temp file before any bytes land, since the atomic rename preserves the mode.

Remote resize was clamped only at the bottom. terminal.resize carried a peer-supplied number to term.resize in the owning webview, and xterm allocates rows × cols. One frame asking for a million by a million wedges every terminal in that window. Bounded in the shared helper.

Pending pairings were unbounded in two places the ceremony's cap didn't cover. Each pair frame also allocates in RemoteHost.#clients and the service's mirrored queue, keyed by a relay-chosen clientId, removed only by a client-gone a hostile relay never sends. Measured: 5000 frames retained ~16 MB, with quadratic bridge traffic. What makes it worth fixing is who reaches it — a synced or stolen passkey is documented as buying "the ability to ask", and asking was a denial of service.

requireUserVerification was unreachable on both sides while two comments asserted the halves must mirror or they "silently disagree on what a valid assertion is". Wired through rather than deleted, since the server-side enforcement exists and is tested: DORMOUSE_REQUIRE_USER_VERIFICATION=true, mirrored into HostEnrollResponse and the Host's ConnectionPolicy.

The auditor auditing itself

application-security runs on Opus, the other two on Sonnet. The mechanical domains do fine on Sonnet — they caught the window BLOCKERs and the stale hono disclosure — but the findings needing real reasoning all came from appsec: tracing a relay-minted clientId to keystroke injection, working out that an 8-character fingerprint carried ~40 bits because a P-256 point's leading byte is constant. Verified the override takes effect by pinning a session to Sonnet and watching the subagent return as Opus.

A domain's verdict is now binding. The reporting step checked fragments existed but never read them, so a subagent could return FAIL and a mis-merged PASS would sail through — worse than a missing fragment, since PASS closes the failure issue and opens the release gate. Line 1 of each fragment is the literal verdict; the step downgrades on dissent.

workflow-audit.yaml's window reaches every consumer. I widened the git log pathspec without widening own_changes or the classifier refusals, which was worse than not widening at all: commits touching only the new paths were selected, produced no attributable files, and were silently dropped while SECURITY.md claimed coverage. One WINDOW array now, with the classifiers' half derived from it.

Things this branch got wrong first

Recording these because they are the argument for the audit existing:

  • The Renovate cooldown did the opposite of what I documented. Omitting minimumReleaseAge from vulnerabilityAlerts doesn't inherit it — Renovate's default for that block is null, force-applied. Now explicit.
  • "Both signing secrets are env-only" was wrong twice: there are three, and APPLE_SIGN_PASS sits on notarytool's command line for up to 30 minutes.
  • "Server compromise — no Host access" overstated the bound. It can't create an authorized Client, but #onMsg gates on state.established for a relay-minted clientId with no per-frame auth, so it can act through one.
  • My claim that the PAT never enters the appsec context was false — it's a step-level env: every subagent inherits.
  • The local runner pinned only one side of the model split, so on a machine defaulting to opus[1m] the relation was inverted. Every local run before that fix was more capable than the nightly.

Supersedes #449, #450, #451

All three are dormouse-bot PRs on the same problems. Two found things I hadn't, and their substance is folded in here — verified present file by file:

Not done, deliberately

  • hostToken travels in the /ws/host query string. The "browsers can't set WS headers" rationale is true of /ws/client, not of /ws/host. Closing it means an Authorization header, a createWebSocket signature that carries one, and relay support across both hosts — untestable here, and rated INFO since TLS covers it in transit. Documented with the remedy named.
  • jsign --storepass and notarytool --password stay on argv; neither tool offers an alternative. Documented as accepted with the reasoning.
  • The two workflow-audit window evasions (committer-date --since, branch deleted before the fetch) remain open and documented.

Full suite green; cargo check clean against the pinned self-contained Node.

nedtwigg and others added 13 commits August 26, 2026 11:52
The permissions FAIL from #447 is cleared — `default_workflow_permissions`
is `read` and `can_approve_pull_request_reviews` is `false`. Re-running the
ci-and-secrets domain confirmed it, and turned up the next layer.

The scope-union FAIL IF was violated by its own fix. Naming the paths each
domain owns meant the list went stale the moment a top-level directory
appeared: `.vscode/` and `.impeccable/` were owned by nobody, in the prompt
files and in SECURITY.md alike. `application-security` is now the catch-all
defined by subtraction — everything `supply-chain.md` and
`ci-and-secrets.md` do not explicitly claim, worked out from `ls -A` rather
than trusted from an enumeration. The lists that remain are descriptions of
today's tree, labelled as such.

Two paths get explicit owners on the way:

- `.vscode/` to ci-and-secrets. It is configuration that executes: a
  `tasks.json` entry with `"runOn": "folderOpen"` runs on checkout when a
  maintainer opens the folder — the persistence shape `workflow-audit`
  watches workflows for, reached without touching `.github/`. There is no
  such task today; the point is that adding one should be a finding.
- `website/scripts/` to supply-chain, which already owns the output of the
  generator living there. Owning the output without the generator is half
  a check.

`workflow-audit.yaml`'s window grows to cover `.vscode/` for that reason,
and `SECURITY.md` itself — weakening a `FAIL IF` and weakening the workflow
that runs it are the same act one file apart, and only one of them was
being watched.

Also corrects a statement that had gone stale: max-sixty/tend#1005 merged
upstream on 2026-08-22, where this document still said "still open". The
gap is unchanged in practice — the latest release is 0.1.18, cut
2026-08-16, so the fix is merged but unreleased — but the text claimed a
longer path than the one that remains.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxF3HqmmoddTbmof7fTxWu
`tauri signer sign` was given `--private-key "$TAURI_SIGNING_PRIVATE_KEY"`
while the same line already exported that variable into its environment.
The flag documents itself as falling back to `[env:
TAURI_SIGNING_PRIVATE_KEY]`, so it bought nothing and cost a production
signing key on a command line — readable via `ps` by anything on the
machine for the life of the call, and `pnpm exec` means every dependency's
lifecycle scripts share that session.

`jsign --storepass` stays. Checked its `--help`: the password is only
accepted as a literal option value, with no environment or file
indirection, so there is nowhere else to put it. Documented as accepted
rather than left to be rediscovered — the residual is local `ps` for the
duration of the call, for a PIN that is inert without the physical YubiKey.

Both facts are now pinned by a FAIL IF, with jsign named as the one
exception so the rule does not read as violated.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxF3HqmmoddTbmof7fTxWu
…e act on CVEs

Repository settings, applied and verified:

- secret scanning: enabled. The history backfill finished with **0 alerts**,
  so nothing has ever been committed that GitHub recognizes as a credential.
- push protection: enabled. This is the one control that acts before a
  secret lands rather than after — it blocks a push whose diff carries a
  recognized provider token, and it applies to `dormouse-bot` as much as to
  a human, which is the point: an injected agent pasting a token into a
  file is exactly the shape it stops. Bypass exists and is logged, so it
  cannot wedge a release.
- Dependabot alerts: enabled. Alerts only — not `dependabot_security_updates`,
  which opens PRs and would duplicate Renovate.

Left off deliberately: `non_provider_patterns` (entropy-based, high false
positive rate — worth adding once the provider-pattern signal is understood,
not on the same day) and `validity_checks` (transmits candidate secrets to
third-party validation endpoints; a real choice, not a free win).

Renovate gets a `vulnerabilityAlerts` block so a known-vulnerable dependency
does not wait for the Monday batch window. The judgement worth recording is
what it does NOT override: `minimumReleaseAge` stays in force. That cooldown
guards the opposite threat — a compromised release yanked within a day — and
dropping it would trade a supply-chain control for patch latency we do not
need, since nothing auto-merges, a human reviews every Renovate PR, and the
Dependabot alert already makes the vulnerability visible the moment it is
published. The PR is the remediation, not the notification.

All three settings are now pinned by a FAIL IF, so turning any of them back
off is a finding rather than a quiet change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxF3HqmmoddTbmof7fTxWu
All three domains passed, and each returned a WARNING that this document
was asserting something untrue. Every one is verified against source
before the fix.

**The Renovate cooldown did the opposite of what I documented.** The
previous commit omitted `minimumReleaseAge` from `vulnerabilityAlerts` on
the reasoning that omission preserves the `packageRules` cooldown. It does
not: Renovate's built-in default for that block is `minimumReleaseAge:
null`, force-applied before lookup (confirmed in `lib/config/options` in
renovate's own source), so omitting the key DROPS the cooldown. It is now
set explicitly to `1 day`, and the FAIL IF requires it explicitly rather
than forbidding an override — the inverted rule that let this pass.

**"Both signing secrets are env-only" was wrong twice over.** There are
three, and two are on argv. `APPLE_SIGN_PASS` goes to `xcrun notarytool
--password` and, with `--wait --timeout 30m`, sits on the command line for
up to half an hour per architecture — a standalone credential, unlike the
PIV PIN that is inert without its YubiKey. Each of the three is now listed
with its actual state. The notarytool remedy (`store-credentials` +
`--keychain-profile`) is named but not applied: it changes the release
runbook and cannot be exercised without live Apple credentials.

**"Server compromise — no Host access" read stronger than what holds.**
It cannot create an authorized Client, which is what the sentence meant.
But `RemoteHost.#onMsg` gates `msg` frames on `state.established` for a
`clientId` the relay itself minted, with no per-frame authentication, so a
compromised Server can fabricate frames on an open session and reach
`terminal.write` — keystroke injection into a live PTY — as well as
suppress or rewrite in either direction. Architectural, following from the
relay being a dumb pipe, and the staged PRF session key is what closes it.
The bound is that compromise cannot create an authorized Client, not that
it cannot act through one.

Also: the catch-all subtraction was top-level only, so `website/`'s own
config files fell through the gap between three partial claims on that
tree. It is recursive now.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxF3HqmmoddTbmof7fTxWu
The previous commit added `.github/audit/`, `.vscode/`, and `SECURITY.md`
to `workflow-audit.yaml`'s `git log` pathspec and nowhere else. The audit
found what that actually did, and it was worse than leaving them out.

`own_changes` still listed files with the old two-path spec, and the caller
drops any commit whose list comes back empty — a `continue` written for
merge commits. So a commit touching only one of the three new paths was
matched by `git log`, returned nothing from `own_changes`, and was silently
dropped: never classified, never counted, never in the issue or the step
summary. The run stayed green, the 48-hour liveness check stayed green, and
SECURITY.md claimed the path was watched. A bot deleting a `FAIL IF`, or
adding a `runOn: folderOpen` task, would have passed unremarked.

Both classifiers had the mirror of the same hole: each refuses a commit
that touches `.config/tend.yaml`, because neither inspects it — but that
refusal was never extended to the new paths. A nightly tend regen that also
rewrote `.github/audit/` still reproduces byte-for-byte from
`uvx tend init`, so it would be classified "explained" and the audit's own
scope edited under cover of the one commit shape it is designed to wave
through.

There is now one `WINDOW` array, and every consumer reads it: the commit
list, both `own_changes` branches, and both classifier refusals (via
`WINDOW_NON_WORKFLOW`, the non-workflow half they must refuse rather than
inspect). Arrays rather than a space-joined scalar, because the unquoted
form is correct only under a word-splitting shell — it silently matches
nothing otherwise, which is the same class of failure one layer down. That
is not hypothetical: my first attempt tested clean under bash and returned
empty under zsh.

Verified by running the real functions against real commits: a
SECURITY.md-only commit is reported rather than dropped, commits touching
the new paths are refused by both classifiers, and a pure workflow commit
still reaches them as before.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxF3HqmmoddTbmof7fTxWu
…the fix

Two WARNINGs from the ci-and-secrets re-run, both cases of this document
claiming more than the code delivers. All 28 FAIL IF checks passed.

**The "one WINDOW array" claim was overstated.** There were two literal
arrays, independently maintained. Consistent today, but held by convention
rather than construction — and the drift it invites is exactly the failure
the sentence was written to rule out: add a path to `WINDOW` alone and a
commit that regenerates `tend-*.yaml` byte-for-byte while also editing that
path passes the classifiers' refusal, reproduces from `uvx tend init`, and
is classified "explained". Reachable with `TEND_BOT_TOKEN`, which is the
credential this audit exists to watch. `WINDOW_NON_WORKFLOW` is now derived
as `"${WINDOW[@]:1}"`, so the complement cannot fall behind the window.

**tend 0.1.19 shipped today**, 2026-08-26T20:51Z — minutes before the audit
ran, which is why the previous commit's "merged but unreleased" was true
when written and false by the time it was read. Verified the content, not
the tag: `shared/steps/lib/pin-instruction-paths.sh` pins
`':(glob)**/AGENTS.md'` at any depth, where 0.1.18's flat `SENSITIVE` list
named no `AGENTS.md` at all.

The gap is still live here — all eight `tend-*.yaml` still pin `@0.1.18`,
so a fork PR still hands the agent an attacker-controlled `AGENTS.md` as
authoritative instructions. What changed is that the last remaining step is
now forceable rather than something to wait on. Bumping the pin is a
separate change: it regenerates eight workflow files and deserves its own
review, not a footnote at the end of this one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxF3HqmmoddTbmof7fTxWu
Picks up max-sixty/tend#1005, which pins instruction files at any depth
rather than by a flat root-relative list. `pin-instruction-paths.sh`
globs `**/AGENTS.md` alongside `**/CLAUDE.md` and `**/.claude/**`, and
`restore-sensitive-config.sh` passes them to `pin_to_base`, so on a fork
PR the agent now reads instructions from the reviewed base branch rather
than from the PR's own tree.

That closes the gap SECURITY.md has been carrying since it was first
reported from this audit: this repo keeps its instructions in AGENTS.md
behind a one-line CLAUDE.md pointer, so 0.1.18's list reverted the
pointer and left the content it points at attacker-controlled.

Regenerated with `uvx tend@0.1.19 init`. The diff is the version header
and the action pin, two lines per file, in all eight.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxF3HqmmoddTbmof7fTxWu
…e floor

The regen onto 0.1.19 closes the fork-PR instruction-injection path this
audit reported. SECURITY.md described it as live across four commits, each
time accurately at the moment of writing — merged-but-unreleased, then
released-but-unpinned — so it now says what actually holds: the revert list
is pathspec globs covering `**/AGENTS.md` at any depth, and this repo runs
that version.

Two things survive the fix and are kept:

- The control lives in upstream code, so what protects this repo is whatever
  the pinned version implements. A pin that moves backwards reopens the path
  with no visible change here except a version number — so there is now a
  FAIL IF on the floor, not just on the pin being present.
- The local remedy still exists if it ever regresses: the regen overwrites
  the workflow, not this repo's instruction files, so moving the body into
  CLAUDE.md would close it with no upstream dependency.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxF3HqmmoddTbmof7fTxWu
…s verdict binding

Three changes from the clean run, plus the model split.

**Model split.** The nightly ran everything on Sonnet — the action's
default, since no model was set. That is right for two of three domains:
`supply-chain` and `ci-and-secrets` are mechanical, and Sonnet did them
well, including catching both window BLOCKERs and the stale `hono`
disclosure. `application-security` is the one that reads code
adversarially, and its findings are the ones that needed real reasoning —
tracing a relay-minted `clientId` to keystroke injection, or working out
that an eight-character fingerprint carried ~40 bits because a P-256
point's leading byte is constant. It now runs on Opus via a per-agent
`model` in `--agents`, verified to take effect: a session pinned to
`--model sonnet` spawned a probe agent that came back on Opus. The
orchestrator and the other two domains stay on the default, so the cost
lands where the depth is. `scripts/security-audit-local.sh` applies the
same split — local and CI disagreeing would be worst on exactly the
domain where the model matters.

**A domain's verdict is now binding.** The reporting step checked that
fragments exist and never read them, so a subagent could write
`VERDICT: FAIL`, return FAIL, and an orchestrator that mis-merged into
`PASS` would pass unremarked — worse than a missing fragment, because
`PASS` closes the open failure issue and opens the release gate, leaving
the finding only in a 14-day artifact. Line 1 of each fragment is now the
literal verdict, and the step downgrades a `PASS` that contradicts one.
FAIL rather than MISSING: a dissenting domain finished and found
something.

**Remote resize is bounded at the top.** `clampTerminalDimension` clamped
only the minimum, and `terminal.resize` carries a peer-supplied number
straight to `term.resize` in the webview that owns the pane, where xterm
allocates rows × cols. One frame asking for a million by a million wedges
every terminal in that window. Reachable by an authorized Client, or by a
compromised Server forging `msg` on an established session. Bounded at
2000 in the shared helper, so Host, client, and harness all get it.

**SECURITY.md leaves the workflow-audit window**, reversing my own
addition. #450 argues it belongs out, and it is right for a reason I
missed: that job watches code that executes from *any* branch — a
workflow on a bot-pushed branch, a folderOpen task on checkout — whereas
a `FAIL IF` is inert until merged to main, which is admin-gated. So the
branch-wide watch adds nothing over PR review, while reporting a commit
on nearly every security PR. A control that cries wolf on routine work is
one people learn to skim. `.vscode/` stays, because it does execute from
a branch.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxF3HqmmoddTbmof7fTxWu
Three bot PRs converged on the same problems this branch was fixing
locally. Two of them found things I did not, so this adopts their
substance rather than closing them as pure duplicates.

**From #449 — `dormouse-lib` is a disclosure root for a specific reason,
and my prose had it wrong.** I wrote that it is "reached as a workspace
edge". It is not: `vscode-ext` declares only `node-pty` and `ws` and
reaches the lib through relative imports into `../lib/src/` from fifteen
files, so the extension's dependency walk never arrives at it. Only
`dormouse-standalone`'s edge does — which puts the disclosure of lib's
whole subtree one refactor away from vanishing silently. Verified both
halves. Naming it a root is what makes that not matter, and the paragraph
now says so.

Also from #449: `website/` goes to supply-chain as a subtraction — all of
it except `public/` — instead of two named subdirectories. Naming `src/`
and `scripts/` left the build config owned by nobody, which is the same
orphaning shape one level down that the catch-all was rewritten to stop.
And dotfile directories are now named explicitly wherever they land,
because a catch-all has twice been read as covering them.

**From #451 — the Audit visibility paragraph named two paths where the
window has four.** It is the prose spec of that pathspec, so a reader
checking the `FAIL IF` against it got a contradiction. Both enumerations
now agree, with the reason they belong together stated: each path holds
something that executes from a branch nobody reviewed.

**From #450 — the same BLOCKER this branch already fixed**, found
independently. Its fix is equivalent; the version here derives the
classifiers' half from the window (`"${WINDOW[@]:1}"`) rather than
maintaining a second list, so the two cannot drift. #450 also argued
`SECURITY.md` does not belong in the window, which is why the previous
commit took it back out.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxF3HqmmoddTbmof7fTxWu
The `FAIL IF` added one commit ago failed on its own implementation, which
is the system working. It requires application-security to run on a
stronger model than the mechanical domains "in both" CI and the local
runner. CI satisfies it: the session default is Sonnet and only that one
agent carries an override. The local runner did not — it pinned Opus for
application-security and left the other two unpinned, inheriting whatever
`~/.claude/settings.json` names.

On this machine that default is `opus[1m]`, so locally the two mechanical
domains were running on a *larger-context Opus* while application-security
ran on plain Opus. The relation was not merely unestablished, it was
inverted.

The consequence is worth recording rather than just fixing: every local
audit run in this session executed the mechanical domains on `opus[1m]`,
not Sonnet. Those runs were therefore more capable than the nightly will
be, and a local PASS from them is not evidence that CI will pass. The
first faithful rehearsal is the run after this commit.

Both sides are pinned now — `--model sonnet` for the mechanical domains,
`--model opus` for application-security.

Also carries the bash 3.2 fix from the run before this one: macOS ships
bash 3.2, where expanding an empty array under `set -u` is an unbound
variable error, which killed the runner on the first domain.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxF3HqmmoddTbmof7fTxWu
`write_session_to` created `sessions/` and its files under the bare umask,
so terminal transcripts landed `0644` in a `0755` directory — readable by
every other account on the machine. The blob is a `PersistedWindow`
carrying scrollback: whatever the user's shells printed, which is a
superset of every other secret this document tracks. The Host's own state
file has been `0600` in a `0700` directory all along, for a strictly
smaller secret, so this closes an inconsistency rather than inventing a
rule.

The mode goes on the temp file *before* any bytes are written, because the
atomic rename preserves it — tightening after the rename would leave a
window where the transcript is world-readable. Best-effort and unix-only:
Windows ACLs are not unix modes, and a filesystem without POSIX
permissions must not fail a session save.

Type-checked with `cargo check` against the pinned self-contained Node
(`build.rs` rejects the Homebrew one, which is not relocatable).

Also fixes an overlap I introduced one commit ago: `application-security`
still named `website/`'s build config as its remainder after
`supply-chain` was widened to claim all of `website/` except `public/`.
Overlap is the safe direction — audited twice rather than never — but it
is the drift the recursive-subtraction rule exists to prevent, and the
audit caught it.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxF3HqmmoddTbmof7fTxWu
…reachable

Two WARNINGs from the application-security domain, both fixed. The third
is documented rather than fixed, for a stated reason.

**Pending pairings were unbounded in two places the ceremony's own cap did
not cover.** `MAX_PENDING_TICKETS` claims to stop a hostile relay turning
`pair` frames into unbounded memory "in the process that owns the PTYs" —
but each frame also allocates in `RemoteHost.#clients` and in the
service's mirrored queue, both keyed by a relay-chosen `clientId` and
removed only by a `client-gone` a hostile relay never sends. Measured by
the audit: 5000 frames retained 5000 requests holding ~16 MB of
relay-chosen strings while `#tickets` sat at 64, and because the service
re-serializes its whole queue to the webview per frame the bridge traffic
is quadratic.

What makes it worth fixing rather than accepting is who can reach it.
SECURITY.md documents a synced or stolen passkey as buying "sign-in, and
the ability to *ask*", with the Client reaching "nothing" — but such a
session passes `checkPair` on every attempt, gets a fresh `clientId` per
socket, and can refresh presence indefinitely. Asking was a denial of
service. Both maps are now capped at `MAX_PENDING_PAIRINGS`, oldest
evicted first and answered with a denial so the evicted client learns its
request is gone.

**`requireUserVerification` was unreachable on both sides** while two
comments asserted the halves "must mirror" or they "silently disagree on
what a valid assertion is". The server-side enforcement exists and is
tested; only the config path was missing, and the Host had no mechanism at
all. Rather than delete a tested control, it is wired through:
`DORMOUSE_REQUIRE_USER_VERIFICATION=true` on the Server, mirrored into
`HostEnrollResponse` and thence the Host's `ConnectionPolicy`. Optional
and additive — absent means off, so an older Host reading a newer Server,
or the reverse, degrades to presence rather than failing. Only the exact
string `true` enables it: a misspelling must read as off, because turning
UV on without UV-capable authenticators locks the account out of its own
server.

**`hostToken` in the `/ws/host` query string is documented, not changed.**
The rationale in the wire contract — browsers cannot set WebSocket headers
— is true of `/ws/client` and not of `/ws/host`, which only a Node process
opens. But closing it means an `Authorization` header, a
`createWebSocket` signature that can carry one, and matching relay
support, across both hosts; it cannot be integration-tested here, and the
audit rates it INFO because TLS covers it in transit and nothing in this
repo logs the URL. Named in SECURITY.md as a hardening gap with the remedy
spelled out.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxF3HqmmoddTbmof7fTxWu
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 27, 2026

Copy link
Copy Markdown

Deploying mouseterm with  Cloudflare Pages  Cloudflare Pages

Latest commit: e040281
Status: ✅  Deploy successful!
Preview URL: https://793dbc7e.mouseterm.pages.dev
Branch Preview URL: https://audit-scope-catchall.mouseterm.pages.dev

View logs

@dormouse-bot dormouse-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the diff and the merged tree, and ran a second structured pass over the code portion. Four findings, one of which narrows a claim SECURITY.md now makes.

The headline: the pairing bound closes the payload, not the map. #evictOldestPairingIfFull denies the oldest pending, but #denyPairing only does state.pending = undefined — it never removes the #clients entry. So #clients still grows by one entry per distinct pair frame, keyed by a clientId that #onFrame type-checks (typeof (frame as { clientId?: unknown }).clientId !== 'string') and never length-bounds. PairingRequest fields are capped at PAIRING_FIELD_LIMIT = 1024, so the thing that just got bounded is the bounded half; the residual key string is the unbounded one. The new test asserts denial count and savedRecords, neither of which observes map size, so it passes either way. Inline suggestion below.

That matters for the prose too: the new FAIL IF reads "the only thing that removes one is a client-gone a hostile relay simply never sends. Both are capped at MAX_PENDING_PAIRINGS" — which is true of the pending queue and not of the client map the same sentence describes.

The rest is smaller: a vacuous branch in the new verdict-dissent guard, a spec sentence that overstates what two of the three evictions do, and some mangled indentation in app.ts.

Everything else I traced held up. clampTerminalDimension is genuinely on the peer-reachable path (peer-surfaces.ts nextCols/nextRows feeding term.resize, and both remote-api.ts sites), MAX_PENDING_PAIRINGS re-exports through export * from './security/pairing.js', the requireUserVerification wiring reaches passkey.ts's if (expected.requireUserVerification && !userVerified) on both sides, restrict_to_owner on the temp file before write_all is the right ordering for a rename that preserves mode, and the new wire.test.mjs is picked up by node --test test/*.test.mjs.

Comment thread lib/src/remote/host/remote-host.ts
Comment thread .github/workflows/security-audit.yaml Outdated
Comment thread docs/specs/remote-security-model.md Outdated
Comment thread server/src/app.ts Outdated
@dormouse-bot

Copy link
Copy Markdown
Collaborator

The review I left above stopped at the code portion. Two things in workflow-audit.yaml it didn't reach — the first introduced by this PR.

The issue-body header is the last hand-maintained window enumeration, and the literal it carries is wrong. #L366 now names five paths — `.github/workflows/`, `.config/tend.yaml`, `.github/audit/`, `.vscode/`, `SECURITY.md` — where WINDOW at #L136 is four. The fifth is the one this same PR spends ten lines excluding on purpose: "SECURITY.md is deliberately NOT in the window" (#L103-L112) — the same call you described on #451 as #450's, which #452 agrees with. That string is the audit's outward face, the first line of the issue a human opens, so it would claim coverage the job does not have. That is the failure mode #L123-L128 is already watching for ("the FAIL IF then claims coverage that does not exist"), one level further out than that comment reaches — it governs the consumers that select commits, and this one only describes them.

Every other consumer is derived: WINDOW_NON_WORKFLOW is "${WINDOW[@]:1}", and git log, own_changes, and both classifier refusals all expand the array. Deriving the header too closes the drift rather than re-fixing the enumeration:

WINDOW_MD=$(printf '`%s`, ' "${WINDOW[@]}"); WINDOW_MD=${WINDOW_MD%, }
echo "$COUNT unexplained commit(s) in the audit window ($WINDOW_MD) since \`$SINCE\`."

Everything from WINDOW= to the gh issue create is one run: block (the step opens at #L76 and nothing else starts before EOF), so WINDOW is in scope there. Ran it against the current array — output is today's line minus SECURITY.md.

Second, smaller, and not yours: the regen arm still opens ".config/tend.yaml is outside this audit's window" (#L241) while #L114 says it is in. It means "outside the reproducibility check's scope" — true, and the rest of the comment makes exactly that point — but "this audit's window" is now the name of a variable twenty lines up. Unchanged by this PR; raising it here only because this is the PR that gave the window one definition.

Verification
  • The header is this PR's change, not pre-existing. git diff origin/main...pr452 -- .github/workflows/workflow-audit.yaml:

    -            echo "$COUNT unexplained commit(s) touching \`.github/workflows/\` or \`.config/tend.yaml\` since \`$SINCE\`."
    +            echo "$COUNT unexplained commit(s) in the audit window (\`.github/workflows/\`, \`.config/tend.yaml\`, \`.github/audit/\`, \`.vscode/\`, \`SECURITY.md\`) since \`$SINCE\`."

    Two paths → five; four of the five match WINDOW, and SECURITY.md is new to both.

  • SECURITY.md appears at six lines in the file at f6530bcc. Five are prose or a cross-reference; #L366 is the only one that puts it in a path list, and no assignment adds it to WINDOW. So the string is the whole of the discrepancy — selection behavior is correct as written.

  • The regen-arm sentence is verbatim from main at 59b0cedf#L198; it is outside this PR's diff for that file.

  • Snippet output, run with the array as it stands: 7 unexplained commit(s) in the audit window (`.github/workflows/`, `.config/tend.yaml`, `.github/audit/`, `.vscode/`) since `2026-08-20`.

Four findings from tend-review on #452, all correct, all fixed.

**The pairing bound closed the bounded half.** `#evictOldestPairingIfFull`
called `#denyPairing`, which clears `state.pending` and leaves the
`#clients` entry in place — so the map still grew by one slot per distinct
`pair` frame. Worse, the residual is the *unbounded* part: every
`PairingRequest` field is capped at `PAIRING_FIELD_LIMIT`, but `clientId`
was only type-checked, so a relay could make each retained key arbitrarily
large. Eviction now drops the record (guarded on the client being neither
established nor session-holding, since `#clientState` recreates it if that
client is heard from again), and `clientId` is length-bounded at the frame
boundary.

My test would not have caught it: it asserted denial counts and
`savedRecords`, neither of which observes map size. `trackedClientCount`
exists now so the bound is observable, and I verified the assertion fails
against the unfixed code (200 vs 8) before restoring it. A bound nothing
can observe is exactly how the first version passed its own test.

**The verdict-dissent guard had two cases where it needed three.** It
recognized only `^VERDICT: FAIL`, so a fragment whose first line is
anything else — a subagent that ignored the brand-new preamble, a stray
heading — fell through to an unchallenged `PASS`. That puts the verdict
back on a prompt having been followed, which is the thing the guard exists
to stop being the control, and it is the *likely* first state since the
instruction ships in this PR. A verdictless fragment now reads MISSING
rather than FAIL — a report we cannot read is an audit that did not
finish, not a finding — and deliberately not via `MISSING_FRAGMENTS`,
whose message says no fragment was left at all. Exercised all four paths.

**Two smaller ones:** the spec attributed "answered with a denial" to all
three eviction sites when only the controller does that (the ceremony and
the service's queue both bare-`delete`), and my `app.ts` insertion had
mangled indentation with the closing brace at column 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxF3HqmmoddTbmof7fTxWu

@dormouse-bot dormouse-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All four findings from the last round are genuinely closed — I re-traced each one rather than taking the replies at face value. The eviction now removes the record under the right guard, the case handles all three verdict states, the spec sentence attributes the denial where it happens, and app.ts is re-indented. MAX_CLIENT_ID_LENGTH = 256 is well-chosen: relay.ts mints toBase64Url(randomBytes(16)) (22 chars), so there is an order of magnitude of headroom, and the check sits ahead of the switch so it covers every frame type, not just pair. trackedClientCount is a real improvement over asserting on denial counts.

Three things on the new commit, one of which is mine to correct.

The verdictless branch lands on issue prose that describes a different failure. My suggestion last round said plain MISSING "lands on the generic Audit produced no verdict branch, which fits." That was wrong, and the mistake is worth naming precisely because the code took it: the issue-body branch is selected on MISSING_FRAGMENTS being non-empty, not on the cause of the MISSING. The new arm deliberately doesn't populate MISSING_FRAGMENTS — correctly, for the reason you gave — so a verdictless fragment falls through to the else, whose first sentence is "The audit step exited without writing audit-status.txt." In this path audit-status.txt said PASS; that is the precondition of the whole block. Details and a reproduction inline.

A doc comment got orphaned. trackedClientCount was inserted between #evictOldestPairingIfFull's JSDoc and the method, so the eviction block now documents the getter and the method is undocumented. Purely mechanical; inline.

The map is bounded against pair, not bounded. #evictOldestPairingIfFull runs only from #onPair and only counts entries whose pending is set. #onConnect and #onConnect2 both call #resetAuthorization, which calls #clientState — creating a #clients entry unconditionally, with no cap and nothing that evicts it. So a relay sending connect frames under distinct clientIds still grows the map one entry per frame until the socket drops (#dropTransientState clears it). Each entry is now much cheaper — MAX_CLIENT_ID_LENGTH caps the key and there is no PairingRequest attached — and this is not a regression; the path was there before. What's new is the claim: the test comment says "The map itself is bounded" and the spec lists the client map among "all three are capped". I'd narrow both rather than widen the cap here — bounding the connect path is a separate design question (evicting an entry that may be established is a different act from denying a pending pairing), and it doesn't belong in a PR scoped to the pairing queue.

Also, MAX_CLIENT_ID_LENGTH is a new acceptance rule on the hostile-relay frame path, and SECURITY.md now leans on it ("which is why clientId is length-bounded at the frame boundary too"), but docs/specs/remote-security-model.md doesn't record it — neither in the "Pending requests are bounded" paragraph nor in that section's Source of truth: list. Per AGENTS.md ("When updating code covered by a spec, update the spec to match") that's a one-sentence addition.

Happy to push a commit with all of this — the workflow fix is the only one that isn't a one-liner, and it needs the third branch in the issue body, which is outside the diff so I can't suggest it inline.

Verification
  • Verdictless prose. Ran the new case loop verbatim against three fragments where frag-a.md opens with a heading and the other two open with VERDICT: PASS, then the STATUS = MISSING branch selection:

    ::warning::frag-a.md has no VERDICT line; the dissent check cannot read it.
    STATUS=MISSING  MISSING_FRAGMENTS=''  DISSENTING=''
    --- issue body ---
    Audit produced no verdict at DATE.
    The audit step exited without writing `audit-status.txt`.
    It did leave a partial audit-report.md ... A report without a verdict is the expected
    shape of an audit that ran short ... Start at the markers: UNVERIFIABLE / _No report ..._
    

    Three claims, all false here: the status file was written and said PASS; the report did reach a verdict; and the UNVERIFIABLE / _No report …_ markers the reader is sent to won't be there, because every domain reported. frag-a.md appears only in the ::warning:: — the issue never names it.

  • The comment block above that branch anticipates exactly this. "A downgraded PASS means the named domains are the whole story — the rest reported, and the 'ran short, no verdict' prose below would be wrong about a run that did reach one." The new case is a downgraded PASS that names no domains, so it reaches the branch that sentence rules out.

  • Orphaned JSDoc. remote-host.ts reads /** Drop the oldest pending pairing … */ at the block opening Drop the oldest pending pairing when the queue is full, then immediately /** How many clients this Host is tracking … */, then get trackedClientCount(). Two stacked blocks: the second binds to the getter, the first binds to nothing, and #evictOldestPairingIfFull follows with no doc.

  • connect allocation path. #onConnect#resetAuthorization(clientId)#clientState(clientId), whose body is let state = this.#clients.get(clientId); if (!state) { state = { established: false, authGeneration: 0 }; this.#clients.set(clientId, state); }. #evictOldestPairingIfFull's counter is for (const state of this.#clients.values()) if (state.pending) pendingCount++, so entries created this way are invisible to it and the while (pendingCount >= MAX_PENDING_PAIRINGS) loop never runs on a pure-connect workload.

  • The new assertion is tight, not vacuous. Traced the steady state: at frame 8 the evict fires, deletes c0, and the add restores the count, so trackedClientCount settles at exactly MAX_PENDING_PAIRINGS (8) and denials at 200 - 8 = 192 — both assertions sit on the boundary. I could not execute the suite (pnpm install hasn't run in this checkout; vitest fails to load lib/vite.config.ts), so this is traced, not run — CI's Build & Test is green on c0af617.

  • Everything else re-checked and clean. The case arms are equivalent to the old grep -q '^VERDICT: FAIL' for the FAIL path ('VERDICT: FAIL'* matches the same prefix, and tolerates a trailing \r); [ "$STATUS" = FAIL ] || STATUS=MISSING correctly refuses to downgrade an earlier real FAIL, and a later FAIL still overrides an earlier MISSING; the eviction's !evicted.established && !evicted.session guard cannot drop a live client, and pendingCount-- is right on both branches since #denyPairing clears pending either way; app.ts re-indents cleanly with the comment and }; back in column position.

Comment thread .github/workflows/security-audit.yaml Outdated
Comment thread lib/src/remote/host/remote-host.ts
Comment thread lib/src/remote/host/remote-host.test.ts Outdated
Comment thread docs/specs/remote-security-model.md Outdated
Three findings from the second review round, all correct.

**The verdictless branch landed on prose describing a different failure.**
The reviewer's own prior suggestion said plain `MISSING` "lands on the
generic Audit produced no verdict branch, which fits" — and it does not.
That branch is selected on `MISSING_FRAGMENTS` being non-empty, which the
verdictless arm deliberately leaves alone, so it fell through to an `else`
opening "The audit step exited without writing audit-status.txt." In this
path the status file was written and said `PASS`, every domain reported,
and the `UNVERIFIABLE` / `_No report …_` markers it directs the reader to
would not exist. Three claims, all false for this cause.

There is now a third arm, keyed on its own `UNREADABLE_VERDICTS`, saying
what actually happened: the merged verdict could not be trusted because a
fragment's first line was not a verdict, the report is complete, and the
likely fix is a subagent that ignored `_preamble.md`. Verified every cause
selects its own arm, with the missing-fragment case winning when both
apply since it is the graver.

**"The map itself is bounded" overstated what the cap does.**
`#evictOldestPairingIfFull` counts only entries carrying a `pending`, and
`#onConnect` creates entries through `#resetAuthorization` that carry
none — so a pure-`connect` workload still grows the map until the socket
drops. Not a regression, and not something to fix here: evicting an entry
that may be `established` is a different act from denying a pending
request, and belongs to a change scoped to the connect path. What was
wrong was the claim, in the test comment and in two specs. All three now
say the pairing path specifically, and name what keeps the other entries
cheap — `MAX_CLIENT_ID_LENGTH`, which was also missing from the spec's
`Source of truth:` list despite SECURITY.md leaning on it.

**An orphaned doc comment**, from inserting `trackedClientCount` between
`#evictOldestPairingIfFull`'s JSDoc and the method it documented.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxF3HqmmoddTbmof7fTxWu

@dormouse-bot dormouse-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All three findings from the last round are closed at 62848c7a — re-traced each rather than taking the replies at face value.

The UNREADABLE_VERDICTS arm is correct: the variable is written in the dissent loop and read in the issue-body block, both inside the same run: step, so it is in scope; the arm sits between the missing-fragment arm and the generic else; and each of the three prose blocks is now true of the cause that selects it. The JSDoc is re-attached to #evictOldestPairingIfFull with trackedClientCount keeping its own. And the narrowing holds against the code — #evictOldestPairingIfFull counts only state.pending, #onConnect allocates through #resetAuthorization#clientState unconditionally, and #dropTransientState does this.#clients.clear(), so "cleared wholesale when the socket drops" is accurate rather than hopeful. MAX_CLIENT_ID_LENGTH is checked ahead of the switch, which is what makes the spec's "before any map is touched" literally true; adding it to the Source of truth: list was the right call given SECURITY.md leans on it.

One new finding, and it is an interaction rather than a line. The dissent loop this PR adds is gated on [ "$STATUS" = "PASS" ], and the pre-existing missing-fragment loop twenty lines up writes that same variable — so one empty fragment turns the dissent check off for every other fragment. A domain that returned VERDICT: FAIL beside a missing sibling is then never noticed, and the issue the maintainer reads points them at "the rest" precisely when a domain in the rest failed. Details, the reachable path, and the gate that matches the intent are inline.

Also still open from my comment above: workflow-audit.yaml #L366 still enumerates five paths where WINDOW is four. The head commit doesn't touch that file, so I read it as queued behind this round rather than declined — flagging only so it doesn't fall off.

Comment thread .github/workflows/security-audit.yaml Outdated
@dormouse-bot

Copy link
Copy Markdown
Collaborator

Two findings from this round that the review above didn't reach — both outside the incremental's five files, neither one a restatement of the dissent-gate finding.

#evictOldestPairingIfFull charges a stranger for a re-send. #onPair calls it unconditionally, but the line right after — this.#clientState(clientId).pending = pendingreplaces the pending for a client that already has one. So when the queue is full and an already-queued client re-sends pair (a reconnect retry, or a client that never saw its own pair-result), the count is still MAX_PENDING_PAIRINGS afterward and an unrelated client has eaten the superseded denial for it. A hostile relay gains nothing here — it can produce the same evictions with fresh clientIds — so what this costs is the legitimate re-send. One line, and it keeps the counter honest about what actually grows:

    // Skip when this client already holds a slot: the assignment below
    // replaces its own pending rather than adding one, so evicting here would
    // cost an unrelated client its place without bounding anything.
    if (!this.#clients.get(clientId)?.pending) this.#evictOldestPairingIfFull();

requireUserVerification is mirrored at enrollment and never again. performEnrollment is its only writer, RemoteHostService.#start() loads the persisted blob and hands it straight to #startHost, and RemoteHost's constructor builds #policy from it once. Nothing re-fetches HostEnrollResponse. So setting DORMOUSE_REQUIRE_USER_VERIFICATION=true on a running deployment leaves every already-enrolled Host at false indefinitely, and the operator has to re-enroll each one with nothing telling them so. SECURITY.md's new FAIL IF"reachable on one side without being mirrored to the other" — therefore holds for Hosts enrolled after the flag, not for the deployment. The Server still refuses a UV-absent connect2 itself, so what's lost is the Host-side half specifically, which is the half that survives Server compromise and the reason the mirroring exists. I'd put the caveat on the docs/specs/server.md table row this PR adds; a refresh-on-reconnect is a bigger design call than this PR is scoped for.

Happy to push either, or both alongside the workflow fix.

Verification
  • Re-send eviction. The counter is for (const state of this.#clients.values()) if (state.pending) pendingCount++, which includes the re-sending client's own entry, so while (pendingCount >= MAX_PENDING_PAIRINGS) fires; #denyPairing + this.#clients.delete(oldestId) drops the oldest other client, and #clientState(clientId).pending = pending then overwrites in place. Net size unchanged, one stranger denied.
  • The two sides stay consistent, which is why this shows up only as a fairness bug and not a divergence: #denyPairing ends in #dismissApproval(clientId) → the service's #resolvePairing, which removes the evicted entry from #pairings before #requestApproval enqueues the new one. That is also why #enqueuePairing's own while (this.#pairings.size >= MAX_PENDING_PAIRINGS) is a backstop that doesn't fire on this path — worth knowing before anyone "fixes" it there instead.
  • No enrollment refresh. performEnrollment has exactly one non-test call site (service.ts #enroll). #start() reads this.#store.loadEnrollment() and goes to #startHost(enrollment) with no server round-trip. adopt carries a blob rather than re-fetching. Server-side enforcement is unaffected — handshake.ts passes its own #requireUserVerification into the policy, covered by server/test/handshake.test.mjs.
  • Also checked and clean, for the record: MAX_CLIENT_ID_LENGTH / PAIRING_FIELD_LIMIT / MAX_PENDING_PAIRINGS all resolve where the spec's Source of truth: now points; _preamble.md's new line-1 rule has no competing instruction in the three domain files; restrict_to_owner(&tmp, 0o600) runs before write_all with the dir already 0700; model_args="--model opus" behind [ … ] && does not trip set -e (an AND-list's own status is exempt — ran it to confirm, since that one is easy to get backwards); clampTerminalDimension's new ceiling doesn't reach a local geometry-derived path that could exceed 2000; and vulnerabilityAlerts.schedule: [] is the documented "any time" value.
  • I could not execute the suites in this checkout (pnpm install hasn't run), so both findings are traced rather than run.

One finding from the third review round, and it is an interaction rather
than a line: the dissent loop this PR added was gated on
`[ "$STATUS" = "PASS" ]`, and the pre-existing missing-fragment loop
twenty lines above writes that same variable. So a single empty fragment
switched the dissent check off for every *other* fragment. A domain that
returned `VERDICT: FAIL` beside a missing sibling was never noticed, and
the issue then pointed the maintainer at "the rest" precisely when a
domain in the rest had failed.

The gate is now "not already FAIL" rather than "still PASS". The three
outcomes are ordered — FAIL (a domain found something) outranks MISSING
(the audit did not finish) outranks PASS — so a dissent can raise MISSING
to FAIL and never the reverse. Simulated all five combinations: clean,
empty-sibling-plus-FAIL (the bug, now FAIL with both sets populated),
empty-sibling-only, unreadable-plus-FAIL, and an orchestrator that already
said FAIL.

That combination is newly reachable, so the FAIL issue body now names any
missing or unreadable fragments alongside the report. Without it the
report reads as the whole story while a domain in it went unaudited —
which is the same failure the missing-fragment guard exists to prevent,
one branch over.

Also closes the stale enumeration the review flagged twice:
`workflow-audit.yaml`'s issue body listed five paths where `WINDOW` has
four, left behind when `SECURITY.md` came out of the window. It now
interpolates `${WINDOW[*]}` rather than restating it, so the two cannot
disagree again.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxF3HqmmoddTbmof7fTxWu

@dormouse-bot dormouse-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The gate change is right, and the ordering it encodes (FAIL outranks MISSING outranks PASS) is the correct reading — I re-simulated the loop and a VERDICT: FAIL beside an empty sibling now reaches the FAIL branch instead of being masked. The workflow-audit.yaml header is derived from WINDOW now, which closes the drift for good rather than re-fixing the literal (WINDOW is in scope there — the step opens at - name: Audit workflow file changes and nothing else starts before EOF).

Two things the widened gate reaches that the prose downstream still assumes it can't. Both are the same class as the finding this commit fixes: a message that was true of the only state that could select it, and now isn't. Inline, with a simulation of each in the fold.

  1. MISSING alongside an unreadable verdict says nothing about the unreadable one. MISSING_FRAGMENTS and UNREADABLE_VERDICTS were mutually exclusive before this commit — the dissent loop was gated on PASS, so one empty fragment kept it from ever running. They can now both be set, and the issue-body chain is still an elif, so the first arm wins and the unreadable fragment is dropped from the issue entirely. The reader is told "any report reproduced below covers only the rest" while a domain in "the rest" is exactly the one whose verdict couldn't be read. You handled this combination in the FAIL branch — the new Incomplete besides / Unreadable verdict notes — and the MISSING branch is the same combination one arm over.

  2. The dissent path can now be entered with no merged verdict at all, and still reports one. STATUS is MISSING before either loop when audit-status.txt is absent or unreadable; the missing-fragment loop is still gated on PASS, so it's skipped, but the dissent loop now runs. A fragment opening VERDICT: FAIL correctly escalates — that's the ordering working — but the annotation says Audit reported PASS but … and the report addendum says "The merged verdict was PASS", when the audit reported nothing. Worse on that path: MISSING_FRAGMENTS is still empty (its loop never ran), so the Incomplete besides note can't fire, and a run that never wrote a verdict and lost a fragment files as a plain FAIL with no sign either happened.

Also worth a clause: the new FAIL IF requires "A FAIL arriving alongside missing or unreadable fragments must say so in the issue" — which covers the branch you fixed but not (1), the MISSING arrival the same gate change created.

Verification

I extracted the reporting step verbatim (top-level case, both loops, the addendum, and the full issue-body chain) and ran it under set -euo pipefail with real fragment files. Six scenarios; the two that matter:

  • (1) status PASS, frag-a.md absent, frag-b.md opening with a heading, frag-c.md VERDICT: PASS:

    ::warning::Audit reported PASS but frag-a.md is missing or empty; downgrading to INCONCLUSIVE.
    ::warning::frag-b.md has no VERDICT line; the dissent check cannot read it.
    STATUS=MISSING  MISSING_FRAGMENTS='frag-a.md'  UNREADABLE_VERDICTS='frag-b.md'
    --- issue body ---
    The audit reported `PASS`, but it was downgraded: no report fragment was left for:
    frag-a.md. … The named domains are unaudited; any report reproduced below covers only the rest.
    

    frag-b.md appears only in the ::warning::. Re-ran the identical input against the pre-commit gate ([ "$STATUS" = "PASS" ], the one line changed): UNREADABLE_VERDICTS='', so the elif was sound before and is not now.

  • (2) no audit-status.txt, frag-a.md absent, frag-b.md VERDICT: FAIL, frag-c.md VERDICT: PASS:

    ::error::Audit reported PASS but frag-b.md opens with VERDICT: FAIL; downgrading to FAIL.
    STATUS=FAIL  MISSING_FRAGMENTS=''  UNREADABLE_VERDICTS=''  DISSENTING='frag-b.md'
    --- issue body ---
    Audit failed at DATE.
    # merged report
    ---
    **Downgraded to FAIL by the reporting step.** The merged verdict was
    `PASS`, but these domain reports open with `VERDICT: FAIL`: frag-b.md.
    

    Three claims, none true of this run: the audit reported nothing, no verdict was merged, and frag-a.md — unaudited — is unmentioned. Same input on the pre-commit gate never enters the loop at all.

  • Also checked and clean: the two [ -n … ] && lines in the FAIL branch don't trip set -e (a failing command in an && list is exempt unless it's the final one, and cat audit-report.md follows the group) — ran the both-empty-then-one-set combination to confirm rather than reasoning about it; [ "$STATUS" = FAIL ] || STATUS=MISSING refuses the downgrade in both loop orders; 'VERDICT: FAIL'* still tolerates a trailing \r; ${WINDOW[*]} renders the four paths in one code span (a shape change from the per-path backticks I'd sketched, not a defect — the point was deriving it); and scripts/security-audit-local.sh needs no mirror of this gate, since it exits at "merge and verdict are the orchestrator's job in CI; read the fragments directly here."

  • I could not execute the workflow itself, so this is the step's logic re-run in isolation, not a live audit run. Build & Test, both Standalone Platform Check legs, Standalone Smoketest, and the Chromatic checks are green on 47d01c98.

Comment thread .github/workflows/security-audit.yaml Outdated
Comment thread .github/workflows/security-audit.yaml Outdated
Comment thread .github/workflows/security-audit.yaml Outdated
Four review rounds in a row found the same defect wearing different
clothes: an issue-body arm whose prose was true of the only state that
could select it, and became false when the next gate widened. Round 2
found it in the verdictless arm, round 3 in the dissent gate, round 4 in
two more states that round 3's fix made reachable. Each fix was correct
and each created the next finding.

That is a structural problem, not four coincidences. The reporting step
wrote prose per *combination* of `(STATUS, MISSING_FRAGMENTS,
UNREADABLE_VERDICTS, DISSENTING)`, so widening any gate made new
combinations reachable and every one needed its text re-audited. Patching
combinations cannot converge — the combinations outnumber the patches.

So it emits one note per condition that *holds*, and no note claims
anything about the others. A fifth condition later means adding one note;
it cannot make an existing note wrong. This is the same fix-by-derivation
that ended the drift twice already in this PR: `WINDOW_NON_WORKFLOW`
derived from `WINDOW`, and the audit header interpolating `${WINDOW[*]}`
instead of restating it.

Both of round 4's findings fall out rather than being patched:

- `MISSING` alongside an unreadable verdict now reports both, because
  both conditions hold and each gets its line.
- The dissent path entered with no merged verdict now says so:
  `STATUS_FILE_VERDICT` records what the file itself said, separately
  from the `STATUS` the guards mutate, since a dissent can raise the same
  MISSING to FAIL and the original cause is otherwise unrecoverable.

The second half of that finding needed a real fix: the missing-fragment
loop was still gated on `PASS`, so a run that lost a fragment *and* never
wrote a verdict recorded nothing about the fragment. Both guards are now
gated on "not already FAIL" — they record conditions the issue reports on,
so they must run on any path that reaches it.

Verified by extracting the real step and running nine combinations,
including every one the previous four rounds patched individually.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxF3HqmmoddTbmof7fTxWu

@dormouse-bot dormouse-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The rewrite closes both of last round's findings, and closes them by removing the class rather than patching the two states. I re-ran the extracted step across the combinations rather than reading the arms: MISSING alongside an unreadable verdict now emits both notes, and STATUS_FILE_VERDICT survives the escalation, so a dissent-derived FAIL on a run that never wrote a status file says so in the body. Widening the missing-fragment gate to != "FAIL" was the necessary other half — the two guards agree about when they run now — and the belt-and-braces arm really is unreachable, since every path that sets MISSING also records a condition.

One condition the issue still cannot report, and it is the one the FAIL IF names. Both fragment guards are gated on [ "$STATUS" != "FAIL" ], so when the orchestrator itself writes FAIL, neither loop runs. A domain that left no fragment beside a real finding then produces no ::warning::, no note, and nothing in the issue — while SECURITY.md requires that "A FAIL arriving alongside missing or unreadable fragments must say so in the issue, or the report reads as the whole story." That holds only for a dissent-derived FAIL, which is the path where STATUS was still PASS or MISSING when the guards ran. Details and the run are inline; the fix has to touch the fi two lines past the hunk, so I can't suggest the whole shape — happy to push it.

The last piece of per-combination prose is the one this commit's new FAIL IF names. The addendum appended to audit-report.md still opens "Downgraded to FAIL by the reporting step. The merged verdict was PASS, but…" — the sentence this thread flagged, unchanged here. I'm not re-arguing that thread; what's new is its standing. "FAIL IF the reporting step writes issue prose per combination of conditions rather than one note per condition that holds" is repo policy as of this commit, and that addendum is prose about a combination (DISSENTING set and the merge said PASS) sitting one cat below a note that contradicts it. The notes block already carries the condition-shaped version of the same fact.

Verification

I extracted the Surface result, file or close issue step verbatim from 60ebf67f (245 lines, comments and all), stubbed gh, and ran it under set -eo pipefail against real fragment files in a scratch dir. The five that matter:

  • Orchestrator FAIL + one empty fragment (audit-status.txt = FAIL, audit-application.md present but empty):

    --- annotations --- (none)
    Audit failed at DATE. [Run](…)
    
    # merged report
    real finding here
    

    audit-application.md is unaudited and appears nowhere — not in the body, not as a ::warning:: in the job log. The artifact step's if-no-files-found: warn doesn't cover it either: the file exists, it is just empty.

  • Dissent-derived FAIL + the same empty fragment (audit-status.txt = PASS) — the contrast, and it is correct: both the A domain returned FAIL and A domain left no report notes fire, because the guards ran while STATUS was still PASS.

  • No status file + a dissenting fragment — round 4's second finding, fixed at the note level, and where the addendum shows:

    - **A domain returned `FAIL`.** audit-supply-chain.md opened with `VERDICT: FAIL` …
    - **The audit wrote no verdict.** `audit-status.txt` was absent, empty, or not `PASS`/`FAIL` …
    
    # partial report
    ---
    **Downgraded to FAIL by the reporting step.** The merged verdict was
    `PASS`, but these domain reports open with `VERDICT: FAIL`: …
    

    Two bullets and the addendum, in one issue, disagreeing about whether a verdict was merged.

  • PASS + one missing fragment + one heading-first fragment — both notes, both ::warning::s, title INCONCLUSIVE. Round 4's first finding is closed.

  • Clean PASSexit 0, Audit passed., no audit-comment.md. Dropping the [ ! -s audit-report.md ] placeholder write costs nothing: the body's No audit-report.md was produced. covers it, and the placeholder was written after the upload step anyway, so nothing observed it.

Also checked and clean: STATUS_FILE_VERDICT is assigned before either loop, so it records the file rather than the escalation; [ -n "$TRANSCRIPT_URL" ] && LINKS=… does not trip set -e (every run above had an empty TRANSCRIPT_URL and still produced a full body); NOTES=$(mktemp) is removed after the redirect closes; the elif [ "$STATUS" = "MISSING" ] arm is unreachable as its comment claims; scripts/security-audit-local.sh needs no mirror, since it stops at "merge and verdict are the orchestrator's job in CI"; and the SECURITY.md bullet's rename to "either fragment guard" matches both gates as written.

I could not execute the workflow itself, so this is the real step re-run in isolation, not a live audit run. All eight non-review checks are green on 60ebf67f.

Comment thread .github/workflows/security-audit.yaml Outdated
…dict

Round 5 found the last two instances, and both are this restructure not
having gone far enough rather than the class returning.

**Both fragment guards were still gated on the status.** Which meant that
when the orchestrator itself wrote `FAIL`, neither loop ran — so a domain
that left no report beside a real finding produced no annotation, no note,
and nothing in the issue, while `SECURITY.md` requires exactly that case
to say so. Same defect in a third costume: gated on `PASS`, one empty
fragment silenced the dissent check; widened to `!= FAIL`, a real `FAIL`
silenced both.

The root is that the loops were doing two jobs. Recording what is true of
a run and deciding its verdict are separate concerns, and gating the first
on the second is what kept producing findings. Both loops now run
unconditionally and only record; escalation happens once, afterwards, from
what they recorded. The `FAIL IF` says so, rather than naming the
particular gate that was wrong this time.

**The last per-combination prose is gone.** The addendum appended to
`audit-report.md` still opened "Downgraded to FAIL — the merged verdict was
`PASS`", which is a claim about a combination (dissent set AND the merge
said `PASS`) and became false as soon as a dissent could arrive on a run
that said nothing. It sat one `cat` below a note that contradicted it. The
notes block already carries the same fact in condition form, so the
addendum is deleted rather than reworded — which is what the `FAIL IF`
added last commit actually requires.

Verified by running the extracted step across eight combinations,
including the one round 5 reported and every one earlier rounds fixed
individually.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxF3HqmmoddTbmof7fTxWu

@dormouse-bot dormouse-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I traced the separation independently before seeing the reply already on the gate thread, so I'll skip re-publishing the same scenario table — my run agrees with it, including that the escalation elif's != "FAIL" is what holds the ceiling on an orchestrator FAIL and that dropping the addendum costs the artifact nothing (the upload step runs before it).

One thing that reply didn't reach, in the other file, and it is the same shape as the defect this commit closes. The rewritten FAIL IF bullet keeps the sentence "The missing-fragment loop writes the same variable, so a PASS gate lets one empty fragment switch the dissent check off for every other one" — present tense, and it contradicts its own third sentence two clauses earlier ("Both loops run unconditionally and only record"). After this commit that loop writes only MISSING_FRAGMENTS; STATUS is assigned in exactly one place, the escalation block. It is also a restatement of the bullet's own first historical clause ("gated on PASS, one empty fragment silenced the dissent check"), so it's redundant as well as stale.

That matters more here than it would in a comment: SECURITY.md is what the audit reads as the control, so an auditor checking this bullet goes looking for a loop that writes STATUS and finds a rule whose rationale describes code that isn't there — which is exactly "prose that was true of the shape it was written against, made false by the next change", the class this PR exists to end. The suggestion folds the sentence's one non-redundant clause (the consequence) into the historical clause that already carries that shape.

Comment thread SECURITY.md Outdated
…er exists

The bullet claiming "both loops run unconditionally and only record" still
carried, two sentences later, "the missing-fragment loop writes the same
variable, so a `PASS` gate lets one empty fragment switch the dissent check
off for every other one" — present tense, contradicting the sentence
before it, and describing code this branch deleted. `STATUS` is assigned
in exactly two places now: where the status file is parsed, and in the
single escalation block.

This is my own careless edit, and of the exact class this PR exists to
end. I replaced the bullet's first sentence and left the old tail
appended, without re-reading the whole bullet — the document equivalent of
an if-arm whose prose was true of the shape it was written against.

It matters more here than in a comment: SECURITY.md is what the audit
reads as the control, so an auditor checking this bullet goes looking for
a loop that writes `STATUS` and finds a rule whose rationale does not
match the code. A rule justified by absent code is worse than no rule,
because it reads as verified.

Rewrote the bullet whole rather than excising the sentence, and then
checked the rest of the section against the workflow mechanically — the
prompt pointer, the fragment list, the Opus override, the redaction step
and its fail-closed `rm`, and `release.yml`'s `gh run watch --exit-status`
all still match what the document claims.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FxF3HqmmoddTbmof7fTxWu
@nedtwigg
nedtwigg merged commit ef9b298 into main Aug 27, 2026
19 checks passed
@nedtwigg
nedtwigg deleted the audit-scope-catchall branch August 27, 2026 05:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants