From ad701ac43a6a46b5ecf9f00a6a909f0ced22473f Mon Sep 17 00:00:00 2001 From: Algis Dumbris Date: Sun, 23 Aug 2026 21:08:40 +0300 Subject: [PATCH 01/15] =?UTF-8?q?docs(specs):=20add=20101-tpa-db=20?= =?UTF-8?q?=E2=80=94=20versioned=20signed=20offline=20TPA=20signature=20da?= =?UTF-8?q?tabase?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Spec-only (speckit stage 1) for the roadmap P1 tpa-db epic: signed, sequence-versioned bundle format with anti-downgrade + rollback; loader verify-before-parse extending the 086 fail-closed pipeline; seed corpus strategy (>=25 signatures, provenance/license, eval sample pairs); the publication channel riding the spec-087 refresh lifecycle; post-activation informational re-scan; freshness telemetry. Eval-gated on the existing scan-eval recall/FP CI bar. --- specs/101-tpa-db/spec.md | 229 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 229 insertions(+) create mode 100644 specs/101-tpa-db/spec.md diff --git a/specs/101-tpa-db/spec.md b/specs/101-tpa-db/spec.md new file mode 100644 index 00000000..66efb97c --- /dev/null +++ b/specs/101-tpa-db/spec.md @@ -0,0 +1,229 @@ +# Feature Specification: Versioned, Signed, Offline-First TPA Signature Database (tpa-db) + +**Feature Branch**: `101-tpa-db` +**Created**: 2026-08-23 +**Status**: Draft +**Input**: Roadmap P1 epic `tpa-db` — "Build a versioned, offline-first signature/pattern database (known TPA campaigns, malicious phrase corpora, IoC hashes) that the engine consumes — bundled with the binary, refreshable out-of-band, community-contributable, and guarded by the existing scan-eval recall/FP CI gate." Tasks: `tpa-db-format` (signature DB format + loader: versioned, signed, bundled default), `tpa-db-corpus` (seed corpus of known public TPA campaigns/patterns), `tpa-db-refresh` (out-of-band refresh, offline-friendly, eval-gated). + +## Positioning *(context)* + + + +## User Scenarios & Testing *(mandatory)* + +### User Story 1 - Signed database format + trust-anchored, downgrade-proof loader (Priority: P1) + +An operator receives a TPA signature database out of band — a `scanner-bundle.json` plus its detached signature file — and drops both into the configured bundle location (or lets the opt-in Spec 087 fetch retrieve them). Before anything else touches the bytes, mcpproxy verifies the signature against its built-in publisher trust anchors (or an operator-configured key), checks that the candidate's publish sequence is not older than what is already active, and only then runs the existing validate-before-activate pipeline (version compatibility, pattern compilation, activation self-check). A tampered bundle, a bundle signed by an unknown key, or a replayed older bundle never activates: the last-known-good database keeps serving and the rejection reason is recorded and surfaced. When the operator genuinely needs to roll back (a bad signature pushed by the publisher), an explicit, loudly-logged rollback action reverts to the embedded default or a named prior bundle. + +**Why this priority**: The database is a code-adjacent security input: whoever controls its bytes controls what the scanner ignores. Without signing and anti-rollback, the "refreshable signature DB" pillar creates a new supply-chain surface (swap the file → blind the scanner; replay an old bundle → resurrect a fixed blind spot). Signing + sequence + fail-closed loading is what makes every other story safe to ship, so it goes first. + +**Independent Test**: Fully offline, no live upstream. Sign a fixture bundle with a test key configured as a trust anchor; drop bundle+signature and assert activation with `signature_verified=true` and the new sequence recorded. Then assert each of the following is rejected with the active bundle unchanged and a machine-readable reason: (a) one flipped byte in the bundle, (b) a signature from a non-trusted key, (c) a missing sidecar with `require_signed_bundle=true`, (d) a validly-signed bundle whose sequence is lower than the active one, (e) a validly-signed bundle failing the activation self-check. Finally, invoke the explicit rollback action and assert the embedded default is restored and the override is logged. + +**Acceptance Scenarios**: + +1. **Given** a candidate bundle whose detached signature verifies against a configured trust anchor and whose sequence is greater than the active bundle's, **When** refresh runs, **Then** the candidate proceeds through the existing validation pipeline and, on success, becomes active with its sequence persisted and `signature_verified=true` surfaced in bundle status. +2. **Given** a candidate whose bytes were altered after signing (signature mismatch), **When** refresh runs, **Then** the candidate is rejected BEFORE its JSON is parsed or any pattern compiled, the active bundle is unchanged, and a "signature verification failed" reason is recorded. +3. **Given** a validly-signed candidate whose sequence is lower than the highest sequence ever activated on this install, **When** refresh runs, **Then** it is refused as a downgrade, the active bundle is unchanged, and a "downgrade refused" reason names both sequences. +4. **Given** `require_signed_bundle` is enabled and a file-drop candidate has no signature sidecar, **When** refresh runs, **Then** the candidate is refused (fail-closed) with a "signature required" reason; with the setting disabled (default), an unsigned drop is accepted through the existing Spec 086 validation path but surfaced as `signature_verified=false`. +5. **Given** a network-fetched candidate (Spec 087 opt-in path), **When** it arrives, **Then** signature verification is ALWAYS mandatory regardless of `require_signed_bundle` — an unsigned or wrongly-signed fetched bundle is never activated (parity with Spec 087 FR-013/FR-014). +6. **Given** an operator invokes the explicit rollback action naming the embedded default (or a prior bundle file), **When** it runs, **Then** the target is re-validated, activated even though its sequence is lower, the anti-downgrade floor is reset to the rolled-back sequence, and the override is loudly logged and visible in bundle status. +7. **Given** an air-gapped host with no network at any point, **When** any of the above scenarios run via file drop, **Then** behavior is identical — signature verification, sequence checks, and rollback require no network. + +--- + +### User Story 2 - Seed corpus: catalog known public TPA campaigns, eval-gated (Priority: P2) + +A security-conscious developer installs mcpproxy and gets, out of the box, a signature database that actually catalogs the publicly known Tool Poisoning Attack landscape — not 6 demo signatures. Hidden-instruction blocks (the `TPA-2026-0001` `…read ~/.ssh/id_rsa…` class), hidden HTML/comment directives, tool shadowing and cross-server override instructions, exfiltration redirects ("also send the result to…"), sensitive-file coaxing, and rug-pull phrasings each have cataloged signatures with a stable `TPA-YYYY-NNNN` identity, a provenance record pointing at the public disclosure that motivated them, and a redistributable license. Every signature ships with labeled eval samples, and no corpus change — addition, tightening, or removal — can land unless the existing scan-eval CI gate stays green. + +**Why this priority**: The format (US1) is worthless empty, and the refresh channel (US3) has nothing to carry without a corpus. This is the "database" in tpa-db. It is P2 only because the P1 trust machinery must exist before a larger corpus becomes an attractive distribution target. + +**Independent Test**: Run the corpus build and assert every signature carries id, category, detector(s), provenance source, and license; assert every gated signature has ≥1 labeled malicious eval sample and ≥1 category-matched hard-negative in the eval dataset; run `go run ./cmd/scan-eval --corpus specs/065-evaluation-foundation/datasets/detect_corpus_v1.json --gate --min-recall 0.90 --max-fp 0.05` with the new embedded default and assert PASS; feed each campaign class's canonical fixture through the offline scanner and assert a hard-tier finding naming the TPA id. + +**Acceptance Scenarios**: + +1. **Given** the seed corpus is built into the embedded default bundle, **When** the offline scanner inspects a tool description carrying any cataloged campaign class's canonical payload (e.g. the TPA-2026-0001 hidden-instruction string), **Then** a hard-tier finding fires naming the matched `TPA-YYYY-NNNN` id, and a `scan`-mode gate holds the tool/server. +2. **Given** any signature in the corpus, **When** its metadata is inspected, **Then** it carries provenance (public source reference) and a license permitting redistribution; a contribution without a redistributable license is refused at corpus build time (Spec 087 FR-021 parity). +3. **Given** a proposed corpus change that would drop gated-category recall below 0.90 or push hard-negative false positives above 0.05 on the frozen eval dataset, **When** the CI gate runs, **Then** the change fails CI and cannot merge; the same regressing bundle, if force-published, is additionally rejected at activation by the Spec 087 self-check (two independent layers). +4. **Given** a new signature intended to gate approvals, **When** it is authored, **Then** it emits hard-tier signals (the eval gate scores hard tier only) and its eval samples follow the dataset validator's conventions, including `hn__*` naming for hard-negatives. +5. **Given** benign-but-spicy tool descriptions (security tooling that legitimately mentions credentials, docs that quote attack examples), **When** scanned with the full seed corpus, **Then** they do not fire hard-tier findings — each campaign class is paired with hard-negatives that pin this down. + +--- + +### User Story 3 - Publication channel + adoption loop (freshness you can see, refresh that re-protects) (Priority: P3) + +The publisher cuts a new database release (new signature for a fresh campaign), signs it, and publishes it to a stable public location. A connected install with the Spec 087 opt-in fetch enabled picks it up within one daily cycle; an air-gapped operator downloads the same two files from any machine and drops them in. In both cases the operator can see — in `mcpproxy security` CLI output, the REST status, and the Web UI — which database version/sequence is active, whether its signature verified, and how stale it is. After a new database activates, mcpproxy re-evaluates the already-approved toolsets it has cached against the new signatures off the hot path and surfaces any new hits as review findings — without auto-revoking approvals — so a signature published today protects servers approved last month. Anonymous telemetry gains the active bundle version/sequence and source so the funnel "published → fetched → active → detected" is finally measurable. + +**Why this priority**: This closes the loop that makes the DB worth updating. Verified adoption analysis (2026-08-22) showed TPA adoption is structurally gated: refresh as spec'd in 087 only swaps the bundle and never re-scans, so a refreshed DB would protect only future admissions. P3 because it strictly builds on US1's artifacts and 087's lifecycle, and the product is already safer with US1+US2 alone. + +**Independent Test**: Serve a signed release from a local test endpoint; enable the 087 fetch against it; assert activation within one refresh cycle and that bundle status (CLI, REST, Web UI) reports the new version, sequence, source=fetch, signature_verified=true, and generated_at. Separately: with a server already approved whose cached tool description matches ONLY a signature added in the new bundle, activate the new bundle and assert a review finding naming the TPA id appears for that tool without the approval being revoked or the server quarantined. Assert heartbeat telemetry (when enabled) carries the active bundle version/sequence/source, and carries nothing when telemetry is opted out. + +**Acceptance Scenarios**: + +1. **Given** a signed database release published at the channel's stable location, **When** an install with the opt-in fetch enabled completes its next daily cycle, **Then** the new database is verified, gated, and active, and the same artifact pair downloaded manually and file-dropped on an air-gapped install activates identically. +2. **Given** an active database, **When** the operator inspects bundle status on any surface (CLI, REST, Web UI), **Then** they see bundle version, sequence, fingerprint, source (embedded/file/fetch), signature-verified state, generated_at, and last refresh outcome incl. the last rejection reason. +3. **Given** a newly-activated database containing a signature absent from the previous one, **When** the post-activation re-scan runs over cached, already-approved tool metadata, **Then** tools matching the new signature surface as review findings naming the TPA id — approvals are NOT auto-revoked and servers are NOT auto-quarantined by refresh alone. +4. **Given** telemetry is enabled, **When** the heartbeat fires, **Then** it includes active bundle version, sequence, and source alongside the existing anonymous TPA-scanner stats; **Given** telemetry is disabled, **Then** nothing bundle-related is sent (existing opt-out respected). + +--- + +### Edge Cases + +- **Air-gapped host, forever**: every P1 behavior (verification, sequence check, rollback, activation self-check) is network-free; the trust anchors ship inside the binary; staleness is surfaced but never blocks scanning — an old database is degraded coverage, not an outage. +- **Signature key rotation**: the trust anchor is a SET of publisher keys embedded per binary release. Rotation is a binary-release event: a release ships old+new keys, bundles are signed with the new key, the old key is removed in a later release. Keys are NEVER delivered over the network, and rotation is not done casually — every shipped binary pins its key set, so a hasty rotation strands older installs on unsigned-refresh only (same philosophy as the app-update signing keys). Key compromise → remove the key in the next release AND publish a higher-sequence bundle signed by the surviving key. +- **Bundle signed by a formerly-trusted, now-removed key**: verification fails on binaries that dropped the key (fail-closed to last-known-good); older binaries still trusting it keep working — the downgrade floor still blocks sequence replay. +- **Tampered bundle / tampered sidecar / swapped pair** (valid signature belonging to a different bundle's bytes): all reduce to signature-verification failure before parse; never a crash, never partial load. +- **Downgrade attack via the embedded default**: a fresh binary whose embedded bundle sequence is LOWER than a previously-activated external bundle's persisted floor does not silently downgrade — the external candidate re-validates and wins by precedence; if only the embedded bundle is available, it serves (scanning must never stop) with the floor retained and the situation surfaced. +- **Anti-downgrade state lost** (data directory wiped/reset): the floor resets; this is accepted — the floor is best-effort hardening on top of signing, not the sole defense, and a wiped data dir already resets approvals. +- **Equal sequence, different bytes** (two distinct bundles claiming the same sequence): refused as suspicious unless it is the currently-active fingerprint (idempotent re-drop stays a no-op per Spec 087 FR-011). +- **Clock skew**: no validity windows or expiry in v1 — sequence ordering, not wall-clock time, is the trust signal; `generated_at` is advisory freshness only. +- **Huge or adversarial database** (thousands of rules, pathological regex): size and rule-count ceilings are enforced at load; every pattern remains RE2 (linear-time, no catastrophic backtracking by construction); a candidate exceeding ceilings is rejected whole, keeping last-known-good. +- **Post-activation re-scan storms**: the re-scan is off the hot path, rate-limited, runs once per activation over cached metadata only (no upstream reconnects), and produces at most one review finding per tool per bundle activation — no notification storm. +- **Signature removed from a newer bundle** (false-positive retired): tools previously held by it are NOT auto-approved by refresh; the held state persists for human review (state changes only flow toward review, never silently toward approval). + +## Requirements *(mandatory)* + +### Functional Requirements + +**Database format & signing** + +- **FR-001**: The publishable database artifact MUST be the existing compiled bundle (`scanner-bundle.json`, Scanner Bundle Contract) plus a detached signature sidecar over the exact bundle bytes. Bundle bytes MUST remain deterministic for a given corpus (byte-identical rebuilds), so the signature and the fingerprint are stable. +- **FR-002**: The bundle manifest MUST gain additive metadata within the supported schema line: `generated_at` (RFC3339 build stamp — the loader already surfaces it), `sequence` (a strictly-increasing integer publish counter), and a publisher key identifier. These keys MUST be additive so existing v0.1 loaders ignore them (contract §4 forward-compat preserved). +- **FR-003**: The signature scheme MUST be Ed25519 detached signatures (or an equivalent modern EdDSA scheme selected in design); the sidecar MUST carry the algorithm identifier and signing key id so the scheme can evolve without breaking verification of existing artifacts. +- **FR-004**: The set of trusted verification keys MUST be the union of (a) publisher public keys embedded in the binary at build time and (b) operator-configured keys (the Spec 087 FR-013 key). Keys MUST NOT be retrieved over the network, and verification MUST work fully offline. +- **FR-005**: Key rotation MUST be supported by trusting multiple keys simultaneously: a binary release MAY add a new key while retaining the old for at least one release cycle, and key removal is also a binary-release event. The design MUST document the compromise procedure (drop key next release + publish higher-sequence bundle under surviving key) and the "do not rotate casually" operational stance. + +**Loader & activation** + +- **FR-006**: For any candidate carrying a signature, verification MUST run BEFORE JSON parsing or pattern compilation; a candidate failing verification MUST be rejected without further processing, keeping last-known-good (extends the Spec 086 fail-closed pipeline with a new first stage). +- **FR-007**: Signing policy MUST be: network-fetched candidates ALWAYS require a valid signature (Spec 087 FR-013/FR-014 unchanged); file-drop candidates are verified whenever a sidecar is present; a new `require_signed_bundle` setting (default off for back-compat with existing unsigned drops) makes the sidecar mandatory for file drops too; the embedded default needs no sidecar (its integrity rides on binary distribution integrity). Bundle status MUST always report whether the active bundle was signature-verified. +- **FR-008**: The full validate-before-activate order MUST be: signature (per policy) → sequence/anti-downgrade → version compatibility → parse → compile-all-or-reject → runnable-rules > 0 → activation self-check (Spec 087 FR-008) → atomic activation. Any failure at any stage MUST keep last-known-good and record a machine-readable reason and timestamp (surfaced via the existing bundle status), never an empty or partial rule set. +- **FR-009**: The system MUST persist the highest sequence ever activated (per data directory) and refuse any candidate with a lower sequence — and any candidate with an equal sequence but different content fingerprint — as a downgrade/replay, unless an explicit operator rollback (FR-010) is in effect. The embedded default competes under the same rule; when no higher-sequence valid candidate exists, the embedded default still serves (scanning never stops) with the retained floor surfaced. +- **FR-010**: An explicit rollback action MUST let an operator revert to the embedded default or a named bundle file: the target is re-validated (signature policy and self-check still apply), activated despite a lower sequence, the persisted floor is reset to the rolled-back sequence, and the override is loudly logged and visible in bundle status. Rollback MUST be the ONLY path that lowers the floor. +- **FR-011**: The loader MUST enforce load ceilings (maximum bundle byte size and maximum rule count, fixed in design) and reject a candidate exceeding them as a whole; all patterns remain RE2-only. A database at the ceilings MUST NOT measurably regress the scan hot path (see SC-008). + +**Seed corpus** + +- **FR-012**: The seed corpus MUST catalog known public TPA campaign/technique classes as signatures with stable `TPA-YYYY-NNNN` identities, covering at minimum: hidden-instruction blocks (the TPA-2026-0001 class), hidden HTML/comment directives, tool shadowing and cross-server override, exfiltration redirects, sensitive-file coaxing (SSH keys, cloud credentials), and rug-pull phrasing patterns. Each signature MUST carry category, detector(s), level, confidence, provenance (public source reference), and a redistributable license; corpus build MUST refuse contributions lacking either (Spec 087 FR-021 parity). +- **FR-013**: Every signature intended to gate approvals MUST ship with eval evidence in the frozen eval dataset: at least one labeled gated-malicious sample and at least one category-matched hard-negative following the dataset validator's conventions (including `hn__*` naming), so the gate can measure both its recall contribution and its false-positive risk. +- **FR-014**: Gating signatures MUST emit hard-tier signals (the eval gate and the `scan`-mode approval gates score hard tier only); any soft-tier-only signature MUST be explicitly marked non-gating in its metadata and MUST NOT be counted toward gate coverage. +- **FR-015**: Signatures MUST be authored in the corpus source-of-truth pipeline (signature sources compiled to the bundle); mcpproxy MUST continue to consume only the compiled bundle and never parse signature sources (Spec 087 FR-002 parity). The contribution pipeline MUST run the same validation the loader runs (compile, schema, license, eval gate) before a bundle can be signed. +- **FR-016**: The CI eval gate (`cmd/scan-eval --gate`, recall ≥ 0.90 over gated categories, hard-negative FP rate ≤ 0.05, existing vacuity guard) MUST pass for: every change to the embedded default corpus (merge-blocking, existing `eval.yml` D2 job) and every published bundle BEFORE it is signed — a bundle that fails the gate is never signed or published. Thresholds are reused verbatim, not redefined. + +**Publication & refresh channel** + +- **FR-017**: Published databases MUST be versioned, signed artifact pairs (bundle + sidecar) at a stable public location with a discoverable "latest" reference; the Spec 087 opt-in fetch consumes this channel unchanged, and manual download + file drop MUST remain a first-class, documented, equally-capable path. [NEEDS CLARIFICATION: hosting location for the published artifacts — GitHub Releases on the tpa-db corpus repo vs. a mcpproxy.app static path (affects URL stability, availability SLO, and bandwidth, not behavior)] +- **FR-018**: This feature MUST NOT introduce a second refresh lifecycle: cadence, single-flight, on-demand trigger, activation self-check, and fail-safe behavior are Spec 087's; this spec only inserts the signature and sequence stages (FR-008) and defines the artifacts flowing through it. +- **FR-019**: After a new database activates, the system MUST run a post-activation informational re-scan over cached tool metadata of already-approved toolsets, off the hot path, without connecting to upstreams: new hard-tier hits surface as review findings naming the TPA id. Refresh alone MUST NOT auto-revoke approvals, auto-quarantine servers, or auto-approve previously-held tools; all state changes from re-scan flow toward human review only. + +**Observability, telemetry & docs** + +- **FR-020**: Bundle status on all existing surfaces (CLI security commands, REST security overview, Web UI) MUST additionally report sequence, signature-verified state, and staleness (age since `generated_at`), alongside the existing version/fingerprint/source/load-error fields, and MUST show the last rejection reason (tamper, downgrade, gate failure) so an operator can tell "current and verified" from "stuck on last-known-good and why". +- **FR-021**: The existing anonymous TPA-scanner telemetry MUST gain the active bundle version, sequence, and source (and signature-verified state) so database adoption and freshness are measurable fleet-wide; the existing telemetry opt-out MUST fully cover these fields, and no new identifying data is introduced. +- **FR-022**: Documentation MUST cover the database lifecycle end to end for operators (verify, drop, fetch, roll back, read status) and for contributors (author a signature, provenance/license rules, eval-sample requirements, how the gate blocks a bad corpus), updating the existing security-quarantine/tool-scanner docs. + +### Key Entities *(include if feature involves data)* + +- **Signature Database (bundle + sidecar)**: the publishable unit — the deterministic compiled `scanner-bundle.json` plus a detached signature over its exact bytes. Identified by (bundle_version, sequence, fingerprint). +- **Bundle Manifest metadata**: `bundle_version`/`schema_version` (existing), plus additive `generated_at`, `sequence`, publisher key id. +- **Publisher Trust Anchor**: the set of publisher public keys embedded in a binary release, unioned with operator-configured keys; the only roots of trust for verification, never network-delivered. +- **Signature (TPA record)**: one cataloged campaign/technique — `TPA-YYYY-NNNN` id, detectors, category, level, confidence, tier intent (gating vs non-gating), provenance reference, license. +- **Eval Sample Pair**: the labeled gated-malicious sample(s) plus category-matched hard-negative(s) a gating signature must contribute to the frozen eval dataset. +- **Sequence Floor**: the persisted highest-activated sequence per install; the anti-downgrade/anti-replay state, lowered only by explicit rollback. +- **Publication Channel**: the stable public location of versioned signed artifact pairs with a "latest" reference; consumed by the Spec 087 fetch or by manual download. +- **Post-Activation Re-scan**: the one-shot, off-hot-path evaluation of cached approved tool metadata against a newly-activated database, emitting review findings only. +- **Active Database / Last-Known-Good**: unchanged from Spec 086/087 — the serving bundle and the fail-closed fallback target. + +## Success Criteria *(mandatory)* + +### Measurable Outcomes + +- **SC-001**: 100% of tampered candidates (any byte of bundle or sidecar altered, or mismatched pair) are rejected before parsing, with the last-known-good database still serving and the reason surfaced — demonstrated by fixture in CI. +- **SC-002**: A validly-signed but lower-sequence bundle is never activated by any non-rollback path (drop, fetch, restart, embedded fallback race), and an explicit rollback both succeeds and leaves an audit trail — demonstrated by fixture in CI. +- **SC-003**: An air-gapped install (zero network, ever) reaches full P1 capability: verified activation of a file-dropped signed database, downgrade refusal, and rollback, with behavior byte-identical to a connected install's file-drop path. +- **SC-004**: The shipped seed corpus contains at least 25 signatures spanning at least 8 distinct public campaign/technique classes, every one carrying provenance + redistributable license, and every gating signature backed by its eval sample pair; the canonical fixture of each class produces a hard-tier finding naming its TPA id. +- **SC-005**: The scan-eval CI gate (recall ≥ 0.90 gated categories, hard-negative FP ≤ 0.05) passes with the full seed corpus embedded, and a deliberately-regressing corpus change is blocked twice independently: at CI (cannot merge) and at activation (self-check rejects it) — both demonstrated by fixture. +- **SC-006**: Fleet freshness is answerable from telemetry: for opted-in installs, the distribution of active database version/sequence/source is reportable, enabling the metric "% of active installs on a database ≤ 30 days old" — a metric that is impossible to compute today. +- **SC-007**: A newly published signature reaches detection within one daily refresh cycle on a fetch-enabled install (and immediately on manual drop), measured end to end: publish → active → the new signature's fixture fires. +- **SC-008**: With a database at the design load ceilings (≥ 500 regex rules), p95 scan latency over a 100-tool server changes by less than 10% versus the 6-signature baseline, and bundle load-plus-verify completes under one second on commodity hardware. +- **SC-009**: After activating a database containing a new signature, an already-approved tool whose cached description matches it is surfaced for review within the same refresh cycle, with zero approvals auto-revoked and zero servers auto-quarantined by the refresh itself. + +## Assumptions + +- Spec 087's refresh lifecycle (daily tick, single-flight, activation self-check, opt-in fetch, fail-safe) is implemented before or together with this feature's refresh story; this spec inserts stages into that pipeline rather than duplicating it. US1's signature/sequence verification is also exercised by the existing hot-reload/file-drop path (Spec 086 `ConfigureBundle`), so US1 does not hard-depend on 087 landing first. +- The Scanner Bundle Contract v0.1 line remains the wire format; signing metadata is additive and the sidecar is a separate file, so existing loaders and the byte-determinism guarantee are unaffected. +- The corpus source-of-truth pipeline (signature sources → compiled bundle) lives outside this repo's runtime (the tpa-db authoring pipeline); this repo consumes compiled bundles only and embeds one at build time (a release build with no embeddable bundle fails, per Spec 087 FR-001). +- The eval gate's thresholds, dataset conventions (including hard-negative naming), vacuity guard, and hard-tier-only scoring are reused verbatim; growing the corpus means growing the dataset alongside it. +- The offline tier's runnable surface remains `engine: regex` × `target: tool_description` for v1; `structural_diff`, `resource_content`, and `server_manifest` rules stay declared-not-runnable (skipped, never clean coverage) exactly as today. +- Cached tool metadata already held by mcpproxy (approval baselines, index) is sufficient for the post-activation re-scan; no upstream connection is initiated by refresh. +- [NEEDS CLARIFICATION: back-compat window for unsigned file drops — should `require_signed_bundle` flip to default-on after one or two release cycles once signed publishing is live, or remain opt-in indefinitely for air-gapped/self-built-corpus operators?] + +## Out of Scope + +- Any change to the detect engine's check semantics, tiers, thresholds, or the shared position classifier (a corpus change must never require touching `ClassifyPosition`; recall fixes belong in signatures + eval samples, not classifier cues). +- Running `structural_diff`/stateful rules, or adding `resource_content`/`server_manifest` scan surfaces. +- LLM-assisted or networked detection tiers; this database feeds the deterministic offline tier only. +- Remote/networked trust-anchor distribution, certificate hierarchies, or transparency logs (key set ships with the binary; revisit only if the publisher set ever grows beyond the project). +- Auto-revoking approvals, auto-quarantining, or auto-approving anything as a side effect of a database refresh (re-scan is informational; state moves toward review only). +- IoC hash feeds and package/registry reputation data (the roadmap note's "IoC hashes" are deferred: v1 targets are description-borne patterns; the format's additive versioning leaves room for later rule engines). +- mcpproxy binary self-update or release-awareness changes (Spec 087 US3 owns that surface). +- Changing the scan-eval gate's thresholds or scoring. + +## Constitution Check *(note)* + +Principle IV (Security by Default) governs this spec: the database is treated as a hostile input until proven otherwise (verify-before-parse, fail-closed to last-known-good, anti-downgrade floor, mandatory signatures on the network path), and refresh can never silently widen approvals. Principle III (Configuration-Driven Architecture): `require_signed_bundle`, operator keys, and the existing bundle path/fetch settings live in `mcp_config.json` with env override and hot-reload; no hardcoded URLs or paths. Principle V (TDD): every rejection class (tamper, downgrade, unsigned, gate regression, ceiling breach) is built against failing fixtures first, and the corpus itself is test-gated by scan-eval. Principle I (Performance at Scale): verification, activation, and the post-activation re-scan stay off the scan hot path with atomic swaps (SC-008). + +## Commit Message Conventions *(mandatory)* + +When committing changes for this feature, follow these guidelines: + +### Issue References +- ✅ **Use**: `Related #[issue-number]` - Links the commit to the issue without auto-closing +- ❌ **Do NOT use**: `Fixes #[issue-number]`, `Closes #[issue-number]`, `Resolves #[issue-number]` - These auto-close issues on merge + +**Rationale**: Issues should only be closed manually after verification and testing in production, not automatically on merge. + +### Co-Authorship +- ❌ **Do NOT include**: `Co-Authored-By: Claude ` +- ❌ **Do NOT include**: "🤖 Generated with [Claude Code](https://claude.com/claude-code)" + +**Rationale**: Commit authorship should reflect the human contributors, not the AI tools used. + +### Example Commit Message +``` +feat(security): [brief description of change] + +Related #[issue-number] + +[Detailed description of what was changed and why] + +## Changes +- [Bulleted list of key changes] +- [Each change on a new line] + +## Testing +- [Test results summary] +- [Key test scenarios covered] +``` From 91d3becfe37a2269fe3ce7dc337e688c7ba53ca1 Mon Sep 17 00:00:00 2001 From: Algis Dumbris Date: Sun, 23 Aug 2026 21:18:16 +0300 Subject: [PATCH 02/15] docs(specs): close trust-model and eval-gate gaps in 101-tpa-db MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cross-model review (opencode/gpt-5.6-sol) plus local verification against the shipped loader and eval harness found ten genuine gaps in the stage-1 spec. All were confirmed against the code before being addressed. ## Changes Trust model / anti-downgrade - FR-009: the sequence floor is now a property of SIGNED artifacts only. `sequence` lives in the manifest, so an unsigned drop's sequence is attacker-chosen; an unsigned candidate could previously claim a huge sequence and permanently lock out every legitimate signed release. Unsigned candidates no longer advance the floor, and a signature ratchet stops signing from being stripped off an install that already ran signed. - FR-010: rollback no longer lowers the floor. Resetting the floor to the rolled-back sequence meant the next refresh could immediately re-activate the exact release the operator just rejected. Rollback is now a pinned exception over a monotonic high-watermark, with the rejected (sequence, fingerprint) deny-listed. - FR-009a: the embedded-default availability fallback is stated as the single explicit exemption to SC-002 (it previously contradicted it outright) and must serve as DEGRADED coverage, so deleting the external bundle cannot be used to obtain scan-mode auto-approvals under a weaker corpus. - FR-007: an active unsigned/unverified bundle likewise marks coverage degraded. `inprocess.go` derives `coverageOK` from bundle *presence*, so without this an attacker-supplied unsigned corpus keeps auto-approving. - FR-003: key ids must be canonical public-key fingerprints, duplicates rejected, and manifest id / sidecar id / verifying-key id must all match. - FR-006/FR-008: corrected an impossible pipeline order — the spec required checking the manifest-resident `sequence` before parsing the manifest. Verification is what precedes parsing; it runs over the raw bytes that are then parsed from memory, closing a verify/parse swap race. - FR-011a: activation must be crash-consistent. Spec 087 FR-010's atomicity is in-memory only; a crash between activation and floor persistence reopens the replay window FR-009 exists to close. Eval gate (the gate was measuring nothing) - FR-016a/SC-005/US2: `cmd/scan-eval/gate.go`'s `gateChecks()` registers only the four built-in detect checks — `scanner.BundleCheck` is absent and scan-eval never loads a bundle, so FR-016's "merge-blocking on corpus change" and SC-005 would both have passed vacuously. The harness must load a candidate through the production loader, register the bundle check, and map each new campaign category (`gatedCategory()` enforces a category only when its mapped check id is registered). SC-005 gains a negative control. Telemetry - FR-021: the schema-v8 anonymity backstop hard-whitelists `tpa_scanner` to four integer counters, so string/bool bundle fields would be rejected before transmit. Requires typed whitelist entries, an enum-bounded `source`, and publisher-issued `bundle_version` values only (a self-built corpus version string would itself be a fleet identifier). Consistency - FR-007: states explicitly that this spec supersedes Spec 087 FR-014's precondition, since embedding publisher keys means a fetch is no longer refused merely because the operator configured no key of their own. Related #1028 --- specs/101-tpa-db/spec.md | 34 +++++++++++++++++++--------------- 1 file changed, 19 insertions(+), 15 deletions(-) diff --git a/specs/101-tpa-db/spec.md b/specs/101-tpa-db/spec.md index 66efb97c..9ae31e27 100644 --- a/specs/101-tpa-db/spec.md +++ b/specs/101-tpa-db/spec.md @@ -40,16 +40,17 @@ An operator receives a TPA signature database out of band — a `scanner-bundle. **Why this priority**: The database is a code-adjacent security input: whoever controls its bytes controls what the scanner ignores. Without signing and anti-rollback, the "refreshable signature DB" pillar creates a new supply-chain surface (swap the file → blind the scanner; replay an old bundle → resurrect a fixed blind spot). Signing + sequence + fail-closed loading is what makes every other story safe to ship, so it goes first. -**Independent Test**: Fully offline, no live upstream. Sign a fixture bundle with a test key configured as a trust anchor; drop bundle+signature and assert activation with `signature_verified=true` and the new sequence recorded. Then assert each of the following is rejected with the active bundle unchanged and a machine-readable reason: (a) one flipped byte in the bundle, (b) a signature from a non-trusted key, (c) a missing sidecar with `require_signed_bundle=true`, (d) a validly-signed bundle whose sequence is lower than the active one, (e) a validly-signed bundle failing the activation self-check. Finally, invoke the explicit rollback action and assert the embedded default is restored and the override is logged. +**Independent Test**: Fully offline, no live upstream. Sign a fixture bundle with a test key configured as a trust anchor; drop bundle+signature and assert activation with `signature_verified=true` and the new sequence recorded. Then assert each of the following is rejected with the active bundle unchanged and a machine-readable reason: (a) one flipped byte in the bundle, (b) a signature from a non-trusted key, (c) a missing sidecar with `require_signed_bundle=true`, (d) a validly-signed bundle whose sequence is lower than the active one, (e) a validly-signed bundle failing the activation self-check, (f) an unsigned bundle dropped after a signed one was active (ratchet), and (g) an unsigned bundle claiming a huge sequence — assert the floor is unchanged so a later legitimate signed release still activates. Finally, invoke the explicit rollback action and assert the embedded default is restored, the override is logged, the high-watermark is unchanged, and a subsequent refresh does not re-activate the deny-listed release. **Acceptance Scenarios**: 1. **Given** a candidate bundle whose detached signature verifies against a configured trust anchor and whose sequence is greater than the active bundle's, **When** refresh runs, **Then** the candidate proceeds through the existing validation pipeline and, on success, becomes active with its sequence persisted and `signature_verified=true` surfaced in bundle status. 2. **Given** a candidate whose bytes were altered after signing (signature mismatch), **When** refresh runs, **Then** the candidate is rejected BEFORE its JSON is parsed or any pattern compiled, the active bundle is unchanged, and a "signature verification failed" reason is recorded. 3. **Given** a validly-signed candidate whose sequence is lower than the highest sequence ever activated on this install, **When** refresh runs, **Then** it is refused as a downgrade, the active bundle is unchanged, and a "downgrade refused" reason names both sequences. -4. **Given** `require_signed_bundle` is enabled and a file-drop candidate has no signature sidecar, **When** refresh runs, **Then** the candidate is refused (fail-closed) with a "signature required" reason; with the setting disabled (default), an unsigned drop is accepted through the existing Spec 086 validation path but surfaced as `signature_verified=false`. +4. **Given** `require_signed_bundle` is enabled and a file-drop candidate has no signature sidecar, **When** refresh runs, **Then** the candidate is refused (fail-closed) with a "signature required" reason; with the setting disabled (default) and no signature-verified bundle ever having been active on this install, an unsigned drop is accepted through the existing Spec 086 validation path but surfaced as `signature_verified=false`, does NOT advance the sequence floor, and marks coverage degraded so `scan`-mode auto-approval is suspended while it is active. +4a. **Given** an install that has already activated a signature-verified bundle, **When** an unsigned candidate is dropped (with `require_signed_bundle` still off), **Then** it is refused by the signature ratchet — signing cannot be stripped back off an install by whoever can write the bundle path — and an unsigned candidate claiming an arbitrarily high sequence never becomes the floor, so it cannot lock out subsequent legitimate signed releases. 5. **Given** a network-fetched candidate (Spec 087 opt-in path), **When** it arrives, **Then** signature verification is ALWAYS mandatory regardless of `require_signed_bundle` — an unsigned or wrongly-signed fetched bundle is never activated (parity with Spec 087 FR-013/FR-014). -6. **Given** an operator invokes the explicit rollback action naming the embedded default (or a prior bundle file), **When** it runs, **Then** the target is re-validated, activated even though its sequence is lower, the anti-downgrade floor is reset to the rolled-back sequence, and the override is loudly logged and visible in bundle status. +6. **Given** an operator invokes the explicit rollback action naming the embedded default (or a prior bundle file), **When** it runs, **Then** the target is re-validated and activated even though its sequence is lower, the historical high-watermark is NOT lowered, the artifact being rolled away from is deny-listed so the next refresh cycle cannot silently re-activate the very release the operator just rejected, and the override is loudly logged and visible in bundle status. 7. **Given** an air-gapped host with no network at any point, **When** any of the above scenarios run via file drop, **Then** behavior is identical — signature verification, sequence checks, and rollback require no network. --- @@ -60,7 +61,7 @@ A security-conscious developer installs mcpproxy and gets, out of the box, a sig **Why this priority**: The format (US1) is worthless empty, and the refresh channel (US3) has nothing to carry without a corpus. This is the "database" in tpa-db. It is P2 only because the P1 trust machinery must exist before a larger corpus becomes an attractive distribution target. -**Independent Test**: Run the corpus build and assert every signature carries id, category, detector(s), provenance source, and license; assert every gated signature has ≥1 labeled malicious eval sample and ≥1 category-matched hard-negative in the eval dataset; run `go run ./cmd/scan-eval --corpus specs/065-evaluation-foundation/datasets/detect_corpus_v1.json --gate --min-recall 0.90 --max-fp 0.05` with the new embedded default and assert PASS; feed each campaign class's canonical fixture through the offline scanner and assert a hard-tier finding naming the TPA id. +**Independent Test**: Run the corpus build and assert every signature carries id, category, detector(s), provenance source, and license; assert every gated signature has ≥1 labeled malicious eval sample and ≥1 category-matched hard-negative in the eval dataset; run `go run ./cmd/scan-eval --corpus specs/065-evaluation-foundation/datasets/detect_corpus_v1.json --gate --min-recall 0.90 --max-fp 0.05` with the new embedded default and assert PASS — and, because that command today scores built-in checks only, assert the accompanying NEGATIVE CONTROL that proves the bundle is being scored at all: with a gating signature removed from the bundle the same command must FAIL. Feed each campaign class's canonical fixture through the offline scanner and assert a hard-tier finding naming the TPA id. **Acceptance Scenarios**: @@ -95,7 +96,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **Signature key rotation**: the trust anchor is a SET of publisher keys embedded per binary release. Rotation is a binary-release event: a release ships old+new keys, bundles are signed with the new key, the old key is removed in a later release. Keys are NEVER delivered over the network, and rotation is not done casually — every shipped binary pins its key set, so a hasty rotation strands older installs on unsigned-refresh only (same philosophy as the app-update signing keys). Key compromise → remove the key in the next release AND publish a higher-sequence bundle signed by the surviving key. - **Bundle signed by a formerly-trusted, now-removed key**: verification fails on binaries that dropped the key (fail-closed to last-known-good); older binaries still trusting it keep working — the downgrade floor still blocks sequence replay. - **Tampered bundle / tampered sidecar / swapped pair** (valid signature belonging to a different bundle's bytes): all reduce to signature-verification failure before parse; never a crash, never partial load. -- **Downgrade attack via the embedded default**: a fresh binary whose embedded bundle sequence is LOWER than a previously-activated external bundle's persisted floor does not silently downgrade — the external candidate re-validates and wins by precedence; if only the embedded bundle is available, it serves (scanning must never stop) with the floor retained and the situation surfaced. +- **Downgrade attack via the embedded default**: a fresh binary whose embedded bundle sequence is LOWER than a previously-activated external bundle's persisted floor does not silently downgrade — the external candidate re-validates and wins by precedence; if only the embedded bundle is available, it serves (scanning must never stop) with the floor retained and the situation surfaced, but per FR-009a as DEGRADED coverage, so an attacker who merely deletes the external bundle cannot use the weaker embedded corpus to obtain `scan`-mode auto-approvals. This is the one explicit exemption to SC-002. - **Anti-downgrade state lost** (data directory wiped/reset): the floor resets; this is accepted — the floor is best-effort hardening on top of signing, not the sole defense, and a wiped data dir already resets approvals. - **Equal sequence, different bytes** (two distinct bundles claiming the same sequence): refused as suspicious unless it is the currently-active fingerprint (idempotent re-drop stays a no-op per Spec 087 FR-011). - **Clock skew**: no validity windows or expiry in v1 — sequence ordering, not wall-clock time, is the trust signal; `generated_at` is advisory freshness only. @@ -110,19 +111,21 @@ The publisher cuts a new database release (new signature for a fresh campaign), **Database format & signing** - **FR-001**: The publishable database artifact MUST be the existing compiled bundle (`scanner-bundle.json`, Scanner Bundle Contract) plus a detached signature sidecar over the exact bundle bytes. Bundle bytes MUST remain deterministic for a given corpus (byte-identical rebuilds), so the signature and the fingerprint are stable. -- **FR-002**: The bundle manifest MUST gain additive metadata within the supported schema line: `generated_at` (RFC3339 build stamp — the loader already surfaces it), `sequence` (a strictly-increasing integer publish counter), and a publisher key identifier. These keys MUST be additive so existing v0.1 loaders ignore them (contract §4 forward-compat preserved). -- **FR-003**: The signature scheme MUST be Ed25519 detached signatures (or an equivalent modern EdDSA scheme selected in design); the sidecar MUST carry the algorithm identifier and signing key id so the scheme can evolve without breaking verification of existing artifacts. +- **FR-002**: The bundle manifest MUST gain additive metadata within the supported schema line: `generated_at` (RFC3339 build stamp — the loader already surfaces it), `sequence` (a strictly-increasing integer publish counter), and a publisher key identifier. These keys MUST be additive so existing v0.1 loaders ignore them (contract §4 forward-compat preserved). `generated_at` MUST be derived from deterministic release metadata (the release commit's timestamp or `SOURCE_DATE_EPOCH`), NEVER from build wall-clock time — a wall-clock stamp would change the bundle bytes, fingerprint, and signature on every rebuild of an identical corpus and so would directly violate FR-001's byte-identical-rebuild guarantee. +- **FR-003**: The signature scheme MUST be Ed25519 detached signatures (or an equivalent modern EdDSA scheme selected in design); the sidecar MUST carry the algorithm identifier and signing key id so the scheme can evolve without breaking verification of existing artifacts. A key id MUST be a canonical full-length fingerprint of the public key itself (not an operator-chosen label), the trust set MUST reject duplicate ids at build/config time, and verification MUST succeed only when the manifest key id, the sidecar key id, and the id of the trust-set key whose signature actually verified are all identical — a manifest naming one publisher while the bytes are signed by another MUST be a verification failure, not a cosmetic mismatch. - **FR-004**: The set of trusted verification keys MUST be the union of (a) publisher public keys embedded in the binary at build time and (b) operator-configured keys (the Spec 087 FR-013 key). Keys MUST NOT be retrieved over the network, and verification MUST work fully offline. - **FR-005**: Key rotation MUST be supported by trusting multiple keys simultaneously: a binary release MAY add a new key while retaining the old for at least one release cycle, and key removal is also a binary-release event. The design MUST document the compromise procedure (drop key next release + publish higher-sequence bundle under surviving key) and the "do not rotate casually" operational stance. **Loader & activation** -- **FR-006**: For any candidate carrying a signature, verification MUST run BEFORE JSON parsing or pattern compilation; a candidate failing verification MUST be rejected without further processing, keeping last-known-good (extends the Spec 086 fail-closed pipeline with a new first stage). -- **FR-007**: Signing policy MUST be: network-fetched candidates ALWAYS require a valid signature (Spec 087 FR-013/FR-014 unchanged); file-drop candidates are verified whenever a sidecar is present; a new `require_signed_bundle` setting (default off for back-compat with existing unsigned drops) makes the sidecar mandatory for file drops too; the embedded default needs no sidecar (its integrity rides on binary distribution integrity). Bundle status MUST always report whether the active bundle was signature-verified. -- **FR-008**: The full validate-before-activate order MUST be: signature (per policy) → sequence/anti-downgrade → version compatibility → parse → compile-all-or-reject → runnable-rules > 0 → activation self-check (Spec 087 FR-008) → atomic activation. Any failure at any stage MUST keep last-known-good and record a machine-readable reason and timestamp (surfaced via the existing bundle status), never an empty or partial rule set. -- **FR-009**: The system MUST persist the highest sequence ever activated (per data directory) and refuse any candidate with a lower sequence — and any candidate with an equal sequence but different content fingerprint — as a downgrade/replay, unless an explicit operator rollback (FR-010) is in effect. The embedded default competes under the same rule; when no higher-sequence valid candidate exists, the embedded default still serves (scanning never stops) with the retained floor surfaced. -- **FR-010**: An explicit rollback action MUST let an operator revert to the embedded default or a named bundle file: the target is re-validated (signature policy and self-check still apply), activated despite a lower sequence, the persisted floor is reset to the rolled-back sequence, and the override is loudly logged and visible in bundle status. Rollback MUST be the ONLY path that lowers the floor. +- **FR-006**: For any candidate carrying a signature, verification MUST run over the exact raw candidate bytes BEFORE those bytes are parsed as JSON or any pattern is compiled; a candidate failing verification MUST be rejected without further processing, keeping last-known-good (extends the Spec 086 fail-closed pipeline with a new first stage). The bytes that are verified MUST be the same bytes that are subsequently parsed (read once into memory, verify, then parse that buffer) — the file MUST NOT be re-read from disk after verification, or a writer racing the loader could swap content between verify and parse. +- **FR-007**: Signing policy MUST be: network-fetched candidates ALWAYS require a valid signature; file-drop candidates are verified whenever a sidecar is present; a new `require_signed_bundle` setting (default off for back-compat with existing unsigned drops) makes the sidecar mandatory for file drops too; the embedded default needs no sidecar (its integrity rides on binary distribution integrity). Bundle status MUST always report whether the active bundle was signature-verified. While an UNSIGNED (or signature-unverified) bundle is active, the scan coverage the approval gates consume MUST be marked degraded so `scan`-mode auto-approval is suspended and changes route to human review — today `internal/security/scanner/inprocess.go` sets `coverageOK` from mere bundle *presence*, so without this an attacker-supplied unsigned corpus would keep silently auto-approving under weakened rules. This requirement supersedes Spec 087 FR-014's precondition: because FR-004 makes publisher keys embedded in the binary part of the trust set, a fetch is no longer refused merely because the operator configured no key of their own; the "never activate an unverified fetched bundle" guarantee of Spec 087 FR-013/FR-014 is preserved unchanged. +- **FR-008**: The full validate-before-activate order MUST be: size/rule-count ceiling (FR-011) → signature over the raw bytes (per policy, FR-006) → parse manifest → sequence/fingerprint anti-downgrade (FR-009) → version compatibility → compile-all-or-reject → runnable-rules > 0 → activation self-check (Spec 087 FR-008) → atomic activation. Sequence is a manifest field, so it is necessarily read after parsing; what MUST precede parsing is signature verification, not the sequence check — an unverified candidate's manifest is never trusted for any decision. Any failure at any stage MUST keep last-known-good and record a machine-readable reason and timestamp (surfaced via the existing bundle status), never an empty or partial rule set. +- **FR-009**: The system MUST persist the highest sequence ever activated (per data directory) and refuse any candidate with a lower sequence — and any candidate with an equal sequence but different content fingerprint — as a downgrade/replay, unless an explicit operator rollback (FR-010) is in effect. **The floor is a property of SIGNED artifacts only**: `sequence` lives in the manifest, so an unsigned candidate's sequence is attacker-chosen. An unsigned/unverified candidate MUST NOT advance the floor (otherwise a single unsigned drop claiming a huge sequence permanently locks out every legitimate signed release), and once an install has ever activated a signature-verified bundle, unsigned candidates MUST be refused thereafter regardless of `require_signed_bundle` — a signature ratchet, so signing cannot be stripped back off an install by whoever can write the bundle path. The embedded default is exempt from the floor as an availability fallback (see FR-009a). +- **FR-009a**: When no valid external candidate is available, the embedded default MUST still serve so scanning never stops, even if its sequence is below the floor. Because it may be an older, weaker corpus than the floor implies, this state MUST be treated as degraded coverage (auto-approval suspended, same as FR-007's unsigned case), the retained floor and the degraded reason MUST be surfaced, and it MUST NOT count as an anti-downgrade violation. This is the single explicit exemption to SC-002. +- **FR-010**: An explicit rollback action MUST let an operator revert to the embedded default or a named bundle file: the target is re-validated (signature policy and self-check still apply), activated despite a lower sequence, and the override is loudly logged and visible in bundle status. Rollback MUST NOT lower the historical high-watermark: the watermark is monotonic and normal activation MUST still require exceeding it. Rollback is instead expressed as a **pinned exception** — the operator pins the rolled-back artifact by fingerprint, and the (sequence, fingerprint) being rolled away from is deny-listed so the very release the operator just rejected cannot be re-activated by the next refresh cycle (and neither can any intermediate release between the pin and the watermark). Clearing the pin returns the install to normal watermark-governed activation. - **FR-011**: The loader MUST enforce load ceilings (maximum bundle byte size and maximum rule count, fixed in design) and reject a candidate exceeding them as a whole; all patterns remain RE2-only. A database at the ceilings MUST NOT measurably regress the scan hot path (see SC-008). +- **FR-011a**: Activation MUST be crash-consistent, not merely atomic in memory. Spec 087 FR-010 makes the rule-set swap atomic with respect to concurrent scans; that says nothing about persisted state, and a process death between "activated" and "floor written" reopens exactly the replay window FR-009 exists to close (while writing the floor first can leave the install with only an older recoverable bundle). The activation record — active fingerprint, sequence, watermark, signature-verified state, any FR-010 pin — MUST be persisted as one transaction, and the recoverable last-known-good bundle MUST be durable before the in-memory pointer is published. Startup, refresh, hot-reload, and rollback MUST all serialize through that same transaction, so any crash leaves the install on a consistent (bundle, watermark) pair. **Seed corpus** @@ -131,6 +134,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **FR-014**: Gating signatures MUST emit hard-tier signals (the eval gate and the `scan`-mode approval gates score hard tier only); any soft-tier-only signature MUST be explicitly marked non-gating in its metadata and MUST NOT be counted toward gate coverage. - **FR-015**: Signatures MUST be authored in the corpus source-of-truth pipeline (signature sources compiled to the bundle); mcpproxy MUST continue to consume only the compiled bundle and never parse signature sources (Spec 087 FR-002 parity). The contribution pipeline MUST run the same validation the loader runs (compile, schema, license, eval gate) before a bundle can be signed. - **FR-016**: The CI eval gate (`cmd/scan-eval --gate`, recall ≥ 0.90 over gated categories, hard-negative FP rate ≤ 0.05, existing vacuity guard) MUST pass for: every change to the embedded default corpus (merge-blocking, existing `eval.yml` D2 job) and every published bundle BEFORE it is signed — a bundle that fails the gate is never signed or published. Thresholds are reused verbatim, not redefined. +- **FR-016a**: The gate harness MUST first be taught to see the database at all — today it does not, so FR-016 and SC-005 would otherwise pass vacuously. `cmd/scan-eval/gate.go`'s `gateChecks()` returns only the four built-in `detect.Check`s (`unicode.hidden`, `shadowing.cross_server`, `payload.decoded`, `phrase.injection`); `scanner.BundleCheck` — which production DOES append in `internal/security/scanner/inprocess.go` — is absent, and `scan-eval` never loads a bundle. Therefore: (a) `scan-eval` MUST accept a candidate bundle path, load it through the production loader (never a parallel parser), and register the resulting bundle check, defaulting to the embedded default so the existing `eval.yml` D2 invocation starts scoring the shipped corpus; (b) `gate.go`'s `categoryCheck` map MUST gain an entry for every new TPA campaign category, since `gatedCategory()` enforces a category ONLY when its mapped check id is registered — an unmapped category is measured but can never fail the build, which is precisely the vacuous-pass failure mode; (c) the same candidate-loading path MUST be what the Spec 087 FR-008 activation self-check runs, so the CI layer and the activation layer cannot drift apart while SC-005 claims they are independent. **Publication & refresh channel** @@ -141,7 +145,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), **Observability, telemetry & docs** - **FR-020**: Bundle status on all existing surfaces (CLI security commands, REST security overview, Web UI) MUST additionally report sequence, signature-verified state, and staleness (age since `generated_at`), alongside the existing version/fingerprint/source/load-error fields, and MUST show the last rejection reason (tamper, downgrade, gate failure) so an operator can tell "current and verified" from "stuck on last-known-good and why". -- **FR-021**: The existing anonymous TPA-scanner telemetry MUST gain the active bundle version, sequence, and source (and signature-verified state) so database adoption and freshness are measurable fleet-wide; the existing telemetry opt-out MUST fully cover these fields, and no new identifying data is introduced. +- **FR-021**: The existing anonymous TPA-scanner telemetry MUST gain the active bundle version, sequence, and source (and signature-verified state) so database adoption and freshness are measurable fleet-wide; the existing telemetry opt-out MUST fully cover these fields, and no new identifying data is introduced. This is NOT a drop-in addition: `internal/telemetry/anonymity.go`'s schema-v8 backstop hard-whitelists the `tpa_scanner` sub-object to `{scans_completed, scans_failed, scans_with_findings, findings}` and requires every scalar to be a non-negative integer, so a string `bundle_version`/`source` or a boolean `signature_verified` would be rejected before transmit as "carries a key outside the whitelist". The design MUST therefore extend that backstop with explicitly TYPED entries (integer `sequence`, enum-bounded `source` ∈ {embedded, file, fetch}, boolean signature-verified) rather than relaxing it to accept arbitrary keys or free strings. `bundle_version` MUST be constrained to publisher-issued values (unrecognized/operator-built corpus versions reported as a single `other` bucket): the backstop exists precisely to stop a bespoke, near-unique string from acting as a fleet identifier, and a self-built corpus version would be exactly that. - **FR-022**: Documentation MUST cover the database lifecycle end to end for operators (verify, drop, fetch, roll back, read status) and for contributors (author a signature, provenance/license rules, eval-sample requirements, how the gate blocks a bad corpus), updating the existing security-quarantine/tool-scanner docs. ### Key Entities *(include if feature involves data)* @@ -161,10 +165,10 @@ The publisher cuts a new database release (new signature for a fresh campaign), ### Measurable Outcomes - **SC-001**: 100% of tampered candidates (any byte of bundle or sidecar altered, or mismatched pair) are rejected before parsing, with the last-known-good database still serving and the reason surfaced — demonstrated by fixture in CI. -- **SC-002**: A validly-signed but lower-sequence bundle is never activated by any non-rollback path (drop, fetch, restart, embedded fallback race), and an explicit rollback both succeeds and leaves an audit trail — demonstrated by fixture in CI. +- **SC-002**: A validly-signed but lower-sequence bundle is never activated by any non-rollback path (drop, fetch, restart), and an unsigned candidate never advances the floor nor survives the signature ratchet — demonstrated by fixture in CI. The sole exemption is the FR-009a embedded-default availability fallback, which serves as degraded coverage rather than counting as a downgrade. An explicit rollback succeeds, leaves an audit trail, does not lower the historical watermark, and the deny-listed release it rolled away from is not re-activated by the following refresh cycle. - **SC-003**: An air-gapped install (zero network, ever) reaches full P1 capability: verified activation of a file-dropped signed database, downgrade refusal, and rollback, with behavior byte-identical to a connected install's file-drop path. - **SC-004**: The shipped seed corpus contains at least 25 signatures spanning at least 8 distinct public campaign/technique classes, every one carrying provenance + redistributable license, and every gating signature backed by its eval sample pair; the canonical fixture of each class produces a hard-tier finding naming its TPA id. -- **SC-005**: The scan-eval CI gate (recall ≥ 0.90 gated categories, hard-negative FP ≤ 0.05) passes with the full seed corpus embedded, and a deliberately-regressing corpus change is blocked twice independently: at CI (cannot merge) and at activation (self-check rejects it) — both demonstrated by fixture. +- **SC-005**: The scan-eval CI gate (recall ≥ 0.90 gated categories, hard-negative FP ≤ 0.05) passes with the full seed corpus embedded AND actually scored — i.e. the gate run registers the bundle check and maps every new campaign category (FR-016a), proven by a negative control: removing a gating signature from the bundle must turn the gate red. Without that control the criterion is vacuous, since today's gate would pass with the bundle entirely absent. A deliberately-regressing corpus change is then blocked twice independently: at CI (cannot merge) and at activation (self-check rejects it) — both demonstrated by fixture. - **SC-006**: Fleet freshness is answerable from telemetry: for opted-in installs, the distribution of active database version/sequence/source is reportable, enabling the metric "% of active installs on a database ≤ 30 days old" — a metric that is impossible to compute today. - **SC-007**: A newly published signature reaches detection within one daily refresh cycle on a fetch-enabled install (and immediately on manual drop), measured end to end: publish → active → the new signature's fixture fires. - **SC-008**: With a database at the design load ceilings (≥ 500 regex rules), p95 scan latency over a 100-tool server changes by less than 10% versus the 6-signature baseline, and bundle load-plus-verify completes under one second on commodity hardware. From bbc1b795ef12d6c876fd6af10a640920990e824c Mon Sep 17 00:00:00 2001 From: Algis Dumbris Date: Sun, 23 Aug 2026 21:23:21 +0300 Subject: [PATCH 03/15] docs(specs): fix watermark lockout + embedded-default regression in 101-tpa-db MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round-2 cross-model review (opencode/gpt-5.6-sol) on the round-1 revisions. Six findings, all verified genuine; two of them were regressions the round-1 fixes introduced. ## Changes Watermark could be weaponized into a permanent update lockout - FR-005a (new): the trust set carries a monotonic trust EPOCH that only a binary release can advance, and anti-downgrade is scoped (authority, epoch, sequence). Without it, key-compromise recovery was impossible rather than merely painful: an attacker holding a compromised publisher key signs at the maximum representable sequence, and since round-1 made the watermark monotonic, no surviving key could ever outbid it — a one-shot permanent denial of updates. FR-005 and the key-rotation edge case now route compromise recovery through the epoch bump. - FR-004a (new): anti-downgrade state is tracked per SIGNING AUTHORITY, not in one global namespace. An operator with a self-built signed corpus (an explicitly supported persona) would otherwise raise the single global watermark past the publisher's release line and permanently block official updates. All publisher keys group into one authority so per-authority watermarks stay compatible with key rotation. - SC-002a (new): both lockout scenarios must be proven by fixture. Regressions introduced by the round-1 fixes - FR-007/FR-009a: round-1 made "any unsigned active bundle" degrade coverage, but the embedded default has no sidecar — so every fresh install would have shipped with scan-mode auto-approval silently disabled. Degradation now applies to unsigned EXTERNAL candidates only; the embedded default is binary-trusted and degrades solely when serving BELOW a retained signed watermark. - FR-008: round-1 put the rule-count ceiling before signature verification and parsing, which is impossible — rule count is not knowable from raw bytes. The two ceilings are now split: byte size before verification, rule count after parse. Consistency - FR-011a: the crash-consistent transaction now explicitly includes the watermark fingerprint and the FR-010 deny-list. Omitting them let a crash preserve a rollback while losing the state that rejects the rolled-away release, silently undoing the rollback on the next refresh. - Key Entities: "Sequence Floor" still said the floor is "lowered only by explicit rollback", directly contradicting revised FR-010. Renamed to Sequence Watermark, defined as monotonic, with rollback as a pinned exception. Terminology normalized from floor to watermark throughout so the two do not read as separate mechanisms. - Added a threat-model boundary note: an attacker with config write is out of scope (they can already disable quarantine), which is why the ratchet keys on install state and require_signed_bundle is config-resident. Related #1028 --- specs/101-tpa-db/spec.md | 45 ++++++++++++++++++++++------------------ 1 file changed, 25 insertions(+), 20 deletions(-) diff --git a/specs/101-tpa-db/spec.md b/specs/101-tpa-db/spec.md index 9ae31e27..c652e590 100644 --- a/specs/101-tpa-db/spec.md +++ b/specs/101-tpa-db/spec.md @@ -40,15 +40,15 @@ An operator receives a TPA signature database out of band — a `scanner-bundle. **Why this priority**: The database is a code-adjacent security input: whoever controls its bytes controls what the scanner ignores. Without signing and anti-rollback, the "refreshable signature DB" pillar creates a new supply-chain surface (swap the file → blind the scanner; replay an old bundle → resurrect a fixed blind spot). Signing + sequence + fail-closed loading is what makes every other story safe to ship, so it goes first. -**Independent Test**: Fully offline, no live upstream. Sign a fixture bundle with a test key configured as a trust anchor; drop bundle+signature and assert activation with `signature_verified=true` and the new sequence recorded. Then assert each of the following is rejected with the active bundle unchanged and a machine-readable reason: (a) one flipped byte in the bundle, (b) a signature from a non-trusted key, (c) a missing sidecar with `require_signed_bundle=true`, (d) a validly-signed bundle whose sequence is lower than the active one, (e) a validly-signed bundle failing the activation self-check, (f) an unsigned bundle dropped after a signed one was active (ratchet), and (g) an unsigned bundle claiming a huge sequence — assert the floor is unchanged so a later legitimate signed release still activates. Finally, invoke the explicit rollback action and assert the embedded default is restored, the override is logged, the high-watermark is unchanged, and a subsequent refresh does not re-activate the deny-listed release. +**Independent Test**: Fully offline, no live upstream. Sign a fixture bundle with a test key configured as a trust anchor; drop bundle+signature and assert activation with `signature_verified=true` and the new sequence recorded. Then assert each of the following is rejected with the active bundle unchanged and a machine-readable reason: (a) one flipped byte in the bundle, (b) a signature from a non-trusted key, (c) a missing sidecar with `require_signed_bundle=true`, (d) a validly-signed bundle whose sequence is lower than the active one, (e) a validly-signed bundle failing the activation self-check, (f) an unsigned bundle dropped after a signed one was active (ratchet), and (g) an unsigned bundle claiming a huge sequence — assert the watermark is unchanged so a later legitimate signed release still activates. Finally, invoke the explicit rollback action and assert the embedded default is restored, the override is logged, the high-watermark is unchanged, and a subsequent refresh does not re-activate the deny-listed release. **Acceptance Scenarios**: 1. **Given** a candidate bundle whose detached signature verifies against a configured trust anchor and whose sequence is greater than the active bundle's, **When** refresh runs, **Then** the candidate proceeds through the existing validation pipeline and, on success, becomes active with its sequence persisted and `signature_verified=true` surfaced in bundle status. 2. **Given** a candidate whose bytes were altered after signing (signature mismatch), **When** refresh runs, **Then** the candidate is rejected BEFORE its JSON is parsed or any pattern compiled, the active bundle is unchanged, and a "signature verification failed" reason is recorded. 3. **Given** a validly-signed candidate whose sequence is lower than the highest sequence ever activated on this install, **When** refresh runs, **Then** it is refused as a downgrade, the active bundle is unchanged, and a "downgrade refused" reason names both sequences. -4. **Given** `require_signed_bundle` is enabled and a file-drop candidate has no signature sidecar, **When** refresh runs, **Then** the candidate is refused (fail-closed) with a "signature required" reason; with the setting disabled (default) and no signature-verified bundle ever having been active on this install, an unsigned drop is accepted through the existing Spec 086 validation path but surfaced as `signature_verified=false`, does NOT advance the sequence floor, and marks coverage degraded so `scan`-mode auto-approval is suspended while it is active. -4a. **Given** an install that has already activated a signature-verified bundle, **When** an unsigned candidate is dropped (with `require_signed_bundle` still off), **Then** it is refused by the signature ratchet — signing cannot be stripped back off an install by whoever can write the bundle path — and an unsigned candidate claiming an arbitrarily high sequence never becomes the floor, so it cannot lock out subsequent legitimate signed releases. +4. **Given** `require_signed_bundle` is enabled and a file-drop candidate has no signature sidecar, **When** refresh runs, **Then** the candidate is refused (fail-closed) with a "signature required" reason; with the setting disabled (default) and no signature-verified bundle ever having been active on this install, an unsigned drop is accepted through the existing Spec 086 validation path but surfaced as `signature_verified=false`, does NOT advance the sequence watermark, and marks coverage degraded so `scan`-mode auto-approval is suspended while it is active. +4a. **Given** an install that has already activated a signature-verified bundle, **When** an unsigned candidate is dropped (with `require_signed_bundle` still off), **Then** it is refused by the signature ratchet — signing cannot be stripped back off an install by whoever can write the bundle path — and an unsigned candidate claiming an arbitrarily high sequence never becomes the watermark, so it cannot lock out subsequent legitimate signed releases. 5. **Given** a network-fetched candidate (Spec 087 opt-in path), **When** it arrives, **Then** signature verification is ALWAYS mandatory regardless of `require_signed_bundle` — an unsigned or wrongly-signed fetched bundle is never activated (parity with Spec 087 FR-013/FR-014). 6. **Given** an operator invokes the explicit rollback action naming the embedded default (or a prior bundle file), **When** it runs, **Then** the target is re-validated and activated even though its sequence is lower, the historical high-watermark is NOT lowered, the artifact being rolled away from is deny-listed so the next refresh cycle cannot silently re-activate the very release the operator just rejected, and the override is loudly logged and visible in bundle status. 7. **Given** an air-gapped host with no network at any point, **When** any of the above scenarios run via file drop, **Then** behavior is identical — signature verification, sequence checks, and rollback require no network. @@ -93,11 +93,12 @@ The publisher cuts a new database release (new signature for a fresh campaign), ### Edge Cases - **Air-gapped host, forever**: every P1 behavior (verification, sequence check, rollback, activation self-check) is network-free; the trust anchors ship inside the binary; staleness is surfaced but never blocks scanning — an old database is degraded coverage, not an outage. -- **Signature key rotation**: the trust anchor is a SET of publisher keys embedded per binary release. Rotation is a binary-release event: a release ships old+new keys, bundles are signed with the new key, the old key is removed in a later release. Keys are NEVER delivered over the network, and rotation is not done casually — every shipped binary pins its key set, so a hasty rotation strands older installs on unsigned-refresh only (same philosophy as the app-update signing keys). Key compromise → remove the key in the next release AND publish a higher-sequence bundle signed by the surviving key. -- **Bundle signed by a formerly-trusted, now-removed key**: verification fails on binaries that dropped the key (fail-closed to last-known-good); older binaries still trusting it keep working — the downgrade floor still blocks sequence replay. +- **Threat-model boundary — attacker with config write**: FR-004 lets operator-configured keys join the trust set, so anyone who can write `mcp_config.json` can trust their own key and sign anything. This is explicitly OUT of the threat model and is not a regression: an attacker with config write can already disable quarantine outright or set a server's trust mode to `auto`. The attacker this design defends against is the one who can write the BUNDLE PATH or intercept the FETCH but not the config — which is precisely why the signature ratchet (FR-009) is keyed to install state rather than to the bundle path, and why `require_signed_bundle` is config-resident (it must not be defeatable from the data dir alone). +- **Signature key rotation**: the trust anchor is a SET of publisher keys embedded per binary release. Rotation is a binary-release event: a release ships old+new keys, bundles are signed with the new key, the old key is removed in a later release. Keys are NEVER delivered over the network, and rotation is not done casually — every shipped binary pins its key set, so a hasty rotation strands older installs on unsigned-refresh only (same philosophy as the app-update signing keys). Key compromise → remove the key in the next release, **bump the trust epoch (FR-005a)**, and publish a bundle in the new epoch signed by the surviving key. The epoch bump is what makes recovery possible at all: a compromised key can sign at the maximum representable sequence, and against a monotonic watermark no surviving key could ever outbid that within the same epoch, so revocation alone would leave installs permanently unable to accept any future release. +- **Bundle signed by a formerly-trusted, now-removed key**: verification fails on binaries that dropped the key (fail-closed to last-known-good); older binaries still trusting it keep working — the downgrade watermark still blocks sequence replay. - **Tampered bundle / tampered sidecar / swapped pair** (valid signature belonging to a different bundle's bytes): all reduce to signature-verification failure before parse; never a crash, never partial load. -- **Downgrade attack via the embedded default**: a fresh binary whose embedded bundle sequence is LOWER than a previously-activated external bundle's persisted floor does not silently downgrade — the external candidate re-validates and wins by precedence; if only the embedded bundle is available, it serves (scanning must never stop) with the floor retained and the situation surfaced, but per FR-009a as DEGRADED coverage, so an attacker who merely deletes the external bundle cannot use the weaker embedded corpus to obtain `scan`-mode auto-approvals. This is the one explicit exemption to SC-002. -- **Anti-downgrade state lost** (data directory wiped/reset): the floor resets; this is accepted — the floor is best-effort hardening on top of signing, not the sole defense, and a wiped data dir already resets approvals. +- **Downgrade attack via the embedded default**: a fresh binary whose embedded bundle sequence is LOWER than a previously-activated external bundle's persisted watermark does not silently downgrade — the external candidate re-validates and wins by precedence; if only the embedded bundle is available, it serves (scanning must never stop) with the watermark retained and the situation surfaced, but per FR-009a as DEGRADED coverage, so an attacker who merely deletes the external bundle cannot use the weaker embedded corpus to obtain `scan`-mode auto-approvals. This is the one explicit exemption to SC-002. +- **Anti-downgrade state lost** (data directory wiped/reset): the watermark resets, and so does the FR-009 signature-ratchet state (both live in the data dir), so a wiped install accepts an unsigned drop again. This is accepted — the watermark and ratchet are best-effort hardening on top of signing, not the sole defense, and a wiped data dir already resets approvals. Operators who need this closed absolutely set `require_signed_bundle`, which is config-resident rather than data-dir-resident and therefore survives the wipe. - **Equal sequence, different bytes** (two distinct bundles claiming the same sequence): refused as suspicious unless it is the currently-active fingerprint (idempotent re-drop stays a no-op per Spec 087 FR-011). - **Clock skew**: no validity windows or expiry in v1 — sequence ordering, not wall-clock time, is the trust signal; `generated_at` is advisory freshness only. - **Huge or adversarial database** (thousands of rules, pathological regex): size and rule-count ceilings are enforced at load; every pattern remains RE2 (linear-time, no catastrophic backtracking by construction); a candidate exceeding ceilings is rejected whole, keeping last-known-good. @@ -113,19 +114,21 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **FR-001**: The publishable database artifact MUST be the existing compiled bundle (`scanner-bundle.json`, Scanner Bundle Contract) plus a detached signature sidecar over the exact bundle bytes. Bundle bytes MUST remain deterministic for a given corpus (byte-identical rebuilds), so the signature and the fingerprint are stable. - **FR-002**: The bundle manifest MUST gain additive metadata within the supported schema line: `generated_at` (RFC3339 build stamp — the loader already surfaces it), `sequence` (a strictly-increasing integer publish counter), and a publisher key identifier. These keys MUST be additive so existing v0.1 loaders ignore them (contract §4 forward-compat preserved). `generated_at` MUST be derived from deterministic release metadata (the release commit's timestamp or `SOURCE_DATE_EPOCH`), NEVER from build wall-clock time — a wall-clock stamp would change the bundle bytes, fingerprint, and signature on every rebuild of an identical corpus and so would directly violate FR-001's byte-identical-rebuild guarantee. - **FR-003**: The signature scheme MUST be Ed25519 detached signatures (or an equivalent modern EdDSA scheme selected in design); the sidecar MUST carry the algorithm identifier and signing key id so the scheme can evolve without breaking verification of existing artifacts. A key id MUST be a canonical full-length fingerprint of the public key itself (not an operator-chosen label), the trust set MUST reject duplicate ids at build/config time, and verification MUST succeed only when the manifest key id, the sidecar key id, and the id of the trust-set key whose signature actually verified are all identical — a manifest naming one publisher while the bytes are signed by another MUST be a verification failure, not a cosmetic mismatch. -- **FR-004**: The set of trusted verification keys MUST be the union of (a) publisher public keys embedded in the binary at build time and (b) operator-configured keys (the Spec 087 FR-013 key). Keys MUST NOT be retrieved over the network, and verification MUST work fully offline. -- **FR-005**: Key rotation MUST be supported by trusting multiple keys simultaneously: a binary release MAY add a new key while retaining the old for at least one release cycle, and key removal is also a binary-release event. The design MUST document the compromise procedure (drop key next release + publish higher-sequence bundle under surviving key) and the "do not rotate casually" operational stance. +- **FR-004**: The set of trusted verification keys MUST be the union of (a) publisher public keys embedded in the binary at build time and (b) operator-configured keys (the Spec 087 FR-013 key). Keys MUST NOT be retrieved over the network, and verification MUST work fully offline. These are two distinct **signing authorities**, and the design MUST keep them so: all publisher keys (including keys being rotated in or out, FR-005) belong to ONE publisher authority, while operator-configured keys form a separate operator authority. +- **FR-004a**: Anti-downgrade state MUST be tracked per signing authority, not in one global namespace. An operator maintaining a self-built corpus is an explicitly supported persona (air-gapped installs, private signature sets), and with a single shared counter their locally-signed bundle at sequence N would raise the one global watermark above the publisher's release line and permanently block official updates — or, symmetrically, a publisher release would block the operator's own corpus. Each authority MUST therefore carry its own watermark, and a candidate MUST be compared only against the watermark of the authority whose key verified it. Grouping all publisher keys into one authority is what keeps per-authority watermarks compatible with key rotation: a rotated-in key inherits the publisher authority's watermark rather than starting a fresh one. +- **FR-005**: Key rotation MUST be supported by trusting multiple keys simultaneously: a binary release MAY add a new key while retaining the old for at least one release cycle, and key removal is also a binary-release event. The design MUST document the compromise procedure and the "do not rotate casually" operational stance. +- **FR-005a**: The trust set MUST carry a **trust epoch** — a monotonic integer that advances ONLY via a binary release — and every anti-downgrade comparison (FR-009) MUST be scoped `(authority, epoch, sequence)`, ordered by epoch first. Without this, key-compromise recovery is impossible rather than merely painful: an attacker holding a compromised publisher key signs a bundle at the maximum representable sequence, and because the watermark is monotonic and no surviving key can ever exceed that sequence, the install is permanently unable to accept any future legitimate release — a one-shot permanent denial of updates, downgraded only to the FR-009a fallback. A binary release that revokes the compromised key MUST therefore also bump the epoch, and a validly-signed bundle from a higher epoch MUST supersede the retained watermark and re-baseline it. Epoch advancement is safe to treat as authoritative because it arrives only inside a release binary, whose integrity is already the root of trust for the embedded key set (FR-004) — it is not attacker-reachable over the bundle or fetch path. **Loader & activation** - **FR-006**: For any candidate carrying a signature, verification MUST run over the exact raw candidate bytes BEFORE those bytes are parsed as JSON or any pattern is compiled; a candidate failing verification MUST be rejected without further processing, keeping last-known-good (extends the Spec 086 fail-closed pipeline with a new first stage). The bytes that are verified MUST be the same bytes that are subsequently parsed (read once into memory, verify, then parse that buffer) — the file MUST NOT be re-read from disk after verification, or a writer racing the loader could swap content between verify and parse. -- **FR-007**: Signing policy MUST be: network-fetched candidates ALWAYS require a valid signature; file-drop candidates are verified whenever a sidecar is present; a new `require_signed_bundle` setting (default off for back-compat with existing unsigned drops) makes the sidecar mandatory for file drops too; the embedded default needs no sidecar (its integrity rides on binary distribution integrity). Bundle status MUST always report whether the active bundle was signature-verified. While an UNSIGNED (or signature-unverified) bundle is active, the scan coverage the approval gates consume MUST be marked degraded so `scan`-mode auto-approval is suspended and changes route to human review — today `internal/security/scanner/inprocess.go` sets `coverageOK` from mere bundle *presence*, so without this an attacker-supplied unsigned corpus would keep silently auto-approving under weakened rules. This requirement supersedes Spec 087 FR-014's precondition: because FR-004 makes publisher keys embedded in the binary part of the trust set, a fetch is no longer refused merely because the operator configured no key of their own; the "never activate an unverified fetched bundle" guarantee of Spec 087 FR-013/FR-014 is preserved unchanged. -- **FR-008**: The full validate-before-activate order MUST be: size/rule-count ceiling (FR-011) → signature over the raw bytes (per policy, FR-006) → parse manifest → sequence/fingerprint anti-downgrade (FR-009) → version compatibility → compile-all-or-reject → runnable-rules > 0 → activation self-check (Spec 087 FR-008) → atomic activation. Sequence is a manifest field, so it is necessarily read after parsing; what MUST precede parsing is signature verification, not the sequence check — an unverified candidate's manifest is never trusted for any decision. Any failure at any stage MUST keep last-known-good and record a machine-readable reason and timestamp (surfaced via the existing bundle status), never an empty or partial rule set. -- **FR-009**: The system MUST persist the highest sequence ever activated (per data directory) and refuse any candidate with a lower sequence — and any candidate with an equal sequence but different content fingerprint — as a downgrade/replay, unless an explicit operator rollback (FR-010) is in effect. **The floor is a property of SIGNED artifacts only**: `sequence` lives in the manifest, so an unsigned candidate's sequence is attacker-chosen. An unsigned/unverified candidate MUST NOT advance the floor (otherwise a single unsigned drop claiming a huge sequence permanently locks out every legitimate signed release), and once an install has ever activated a signature-verified bundle, unsigned candidates MUST be refused thereafter regardless of `require_signed_bundle` — a signature ratchet, so signing cannot be stripped back off an install by whoever can write the bundle path. The embedded default is exempt from the floor as an availability fallback (see FR-009a). -- **FR-009a**: When no valid external candidate is available, the embedded default MUST still serve so scanning never stops, even if its sequence is below the floor. Because it may be an older, weaker corpus than the floor implies, this state MUST be treated as degraded coverage (auto-approval suspended, same as FR-007's unsigned case), the retained floor and the degraded reason MUST be surfaced, and it MUST NOT count as an anti-downgrade violation. This is the single explicit exemption to SC-002. +- **FR-007**: Signing policy MUST be: network-fetched candidates ALWAYS require a valid signature; file-drop candidates are verified whenever a sidecar is present; a new `require_signed_bundle` setting (default off for back-compat with existing unsigned drops) makes the sidecar mandatory for file drops too; the embedded default needs no sidecar (its integrity rides on binary distribution integrity). Bundle status MUST always report whether the active bundle was signature-verified. While an unsigned EXTERNAL (file-dropped or fetched) bundle is active, the scan coverage the approval gates consume MUST be marked degraded so `scan`-mode auto-approval is suspended and changes route to human review — today `internal/security/scanner/inprocess.go` sets `coverageOK` from mere bundle *presence*, so without this an attacker-supplied unsigned corpus would keep silently auto-approving under weakened rules. This degradation rule applies to EXTERNAL candidates ONLY: the embedded default is binary-trusted (its integrity rides on the signed binary distribution, not on a sidecar) and MUST be full-coverage on an ordinary install, or every fresh install would ship with `scan` mode silently disabled. The embedded default is degraded only in the FR-009a case — when it is serving as a fallback BELOW a retained signed watermark, i.e. when the install has evidence a newer signed corpus should be active. This requirement supersedes Spec 087 FR-014's precondition: because FR-004 makes publisher keys embedded in the binary part of the trust set, a fetch is no longer refused merely because the operator configured no key of their own; the "never activate an unverified fetched bundle" guarantee of Spec 087 FR-013/FR-014 is preserved unchanged. +- **FR-008**: The full validate-before-activate order MUST be: byte-size ceiling → signature over the raw bytes (per policy, FR-006) → parse manifest → rule-count ceiling → sequence/fingerprint anti-downgrade (FR-009) → version compatibility → compile-all-or-reject → runnable-rules > 0 → activation self-check (Spec 087 FR-008) → atomic activation. FR-011's two ceilings MUST therefore be split across the pipeline rather than applied as one stage: byte size is measurable on raw bytes and so guards the verification step itself, while rule count is only knowable after parsing and so cannot precede it. The same reasoning applies to sequence, a manifest field necessarily read after parsing: what MUST precede parsing is signature verification, not the sequence or rule-count check — an unverified candidate's manifest is never trusted for any decision. Any failure at any stage MUST keep last-known-good and record a machine-readable reason and timestamp (surfaced via the existing bundle status), never an empty or partial rule set. +- **FR-009**: The system MUST persist the highest sequence ever activated — the **watermark**, scoped per signing authority and trust epoch per FR-004a/FR-005a — and refuse any candidate with a lower sequence, or an equal sequence with a different content fingerprint, as a downgrade/replay, unless an explicit operator rollback (FR-010) is in effect. **The watermark is a property of SIGNED artifacts only**: `sequence` lives in the manifest, so an unsigned candidate's sequence is attacker-chosen. An unsigned/unverified candidate MUST NOT advance the watermark (otherwise a single unsigned drop claiming a huge sequence permanently locks out every legitimate signed release), and once an install has ever activated a signature-verified bundle, unsigned candidates MUST be refused thereafter regardless of `require_signed_bundle` — a signature ratchet, so signing cannot be stripped back off an install by whoever can write the bundle path. The embedded default is exempt from BOTH the watermark and the ratchet — it is the availability fallback of last resort and must never be locked out (scanning must never stop), which is exactly why FR-009a demotes it to degraded coverage instead of refusing it. +- **FR-009a**: When no valid external candidate is available, the embedded default MUST still serve so scanning never stops, even if its sequence is below the watermark. Where the install has NO retained signed watermark (the ordinary fresh install), this is simply the normal full-coverage state. Where a signed watermark IS retained and the embedded default is below it, the embedded corpus may be older and weaker than the watermark implies, so THAT state MUST be treated as degraded coverage (auto-approval suspended, as in FR-007's unsigned-external case), the retained watermark and the degraded reason MUST be surfaced, and it MUST NOT count as an anti-downgrade violation. This below-watermark fallback is the single explicit exemption to SC-002. - **FR-010**: An explicit rollback action MUST let an operator revert to the embedded default or a named bundle file: the target is re-validated (signature policy and self-check still apply), activated despite a lower sequence, and the override is loudly logged and visible in bundle status. Rollback MUST NOT lower the historical high-watermark: the watermark is monotonic and normal activation MUST still require exceeding it. Rollback is instead expressed as a **pinned exception** — the operator pins the rolled-back artifact by fingerprint, and the (sequence, fingerprint) being rolled away from is deny-listed so the very release the operator just rejected cannot be re-activated by the next refresh cycle (and neither can any intermediate release between the pin and the watermark). Clearing the pin returns the install to normal watermark-governed activation. - **FR-011**: The loader MUST enforce load ceilings (maximum bundle byte size and maximum rule count, fixed in design) and reject a candidate exceeding them as a whole; all patterns remain RE2-only. A database at the ceilings MUST NOT measurably regress the scan hot path (see SC-008). -- **FR-011a**: Activation MUST be crash-consistent, not merely atomic in memory. Spec 087 FR-010 makes the rule-set swap atomic with respect to concurrent scans; that says nothing about persisted state, and a process death between "activated" and "floor written" reopens exactly the replay window FR-009 exists to close (while writing the floor first can leave the install with only an older recoverable bundle). The activation record — active fingerprint, sequence, watermark, signature-verified state, any FR-010 pin — MUST be persisted as one transaction, and the recoverable last-known-good bundle MUST be durable before the in-memory pointer is published. Startup, refresh, hot-reload, and rollback MUST all serialize through that same transaction, so any crash leaves the install on a consistent (bundle, watermark) pair. +- **FR-011a**: Activation MUST be crash-consistent, not merely atomic in memory. Spec 087 FR-010 makes the rule-set swap atomic with respect to concurrent scans; that says nothing about persisted state, and a process death between "activated" and "watermark written" reopens exactly the replay window FR-009 exists to close (while writing the watermark first can leave the install with only an older recoverable bundle). The activation record — active fingerprint, sequence, per-authority watermarks and their epochs, the fingerprint each watermark was set by, signature-verified state, any FR-010 pin, and the FR-010 deny-list — MUST be persisted as ONE transaction, and the recoverable last-known-good bundle MUST be durable before the in-memory pointer is published. The deny-list and watermark fingerprint are load-bearing members of that transaction, not bookkeeping: a crash that preserved a rollback while losing the deny-list would let the next refresh immediately re-activate the exact release the operator had just rejected, silently undoing the rollback. Startup, refresh, hot-reload, and rollback MUST all serialize through that same transaction, so any crash leaves the install on a consistent (bundle, watermark) pair. **Seed corpus** @@ -144,7 +147,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), **Observability, telemetry & docs** -- **FR-020**: Bundle status on all existing surfaces (CLI security commands, REST security overview, Web UI) MUST additionally report sequence, signature-verified state, and staleness (age since `generated_at`), alongside the existing version/fingerprint/source/load-error fields, and MUST show the last rejection reason (tamper, downgrade, gate failure) so an operator can tell "current and verified" from "stuck on last-known-good and why". +- **FR-020**: Bundle status on all existing surfaces (CLI security commands, REST security overview, Web UI) MUST additionally report sequence, signature-verified state, and staleness (age since `generated_at`), alongside the existing version/fingerprint/source/load-error fields, and MUST show the last rejection reason (tamper, downgrade, unsigned-refused-by-ratchet, gate failure) so an operator can tell "current and verified" from "stuck on last-known-good and why". Where the active database has demoted the install to DEGRADED coverage (FR-007 unsigned-active, FR-009a embedded fallback), that state and its reason MUST be surfaced too — a silently suspended `scan`-mode auto-approval is otherwise indistinguishable from a quiet install. - **FR-021**: The existing anonymous TPA-scanner telemetry MUST gain the active bundle version, sequence, and source (and signature-verified state) so database adoption and freshness are measurable fleet-wide; the existing telemetry opt-out MUST fully cover these fields, and no new identifying data is introduced. This is NOT a drop-in addition: `internal/telemetry/anonymity.go`'s schema-v8 backstop hard-whitelists the `tpa_scanner` sub-object to `{scans_completed, scans_failed, scans_with_findings, findings}` and requires every scalar to be a non-negative integer, so a string `bundle_version`/`source` or a boolean `signature_verified` would be rejected before transmit as "carries a key outside the whitelist". The design MUST therefore extend that backstop with explicitly TYPED entries (integer `sequence`, enum-bounded `source` ∈ {embedded, file, fetch}, boolean signature-verified) rather than relaxing it to accept arbitrary keys or free strings. `bundle_version` MUST be constrained to publisher-issued values (unrecognized/operator-built corpus versions reported as a single `other` bucket): the backstop exists precisely to stop a bespoke, near-unique string from acting as a fleet identifier, and a self-built corpus version would be exactly that. - **FR-022**: Documentation MUST cover the database lifecycle end to end for operators (verify, drop, fetch, roll back, read status) and for contributors (author a signature, provenance/license rules, eval-sample requirements, how the gate blocks a bad corpus), updating the existing security-quarantine/tool-scanner docs. @@ -152,10 +155,11 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **Signature Database (bundle + sidecar)**: the publishable unit — the deterministic compiled `scanner-bundle.json` plus a detached signature over its exact bytes. Identified by (bundle_version, sequence, fingerprint). - **Bundle Manifest metadata**: `bundle_version`/`schema_version` (existing), plus additive `generated_at`, `sequence`, publisher key id. -- **Publisher Trust Anchor**: the set of publisher public keys embedded in a binary release, unioned with operator-configured keys; the only roots of trust for verification, never network-delivered. +- **Publisher Trust Anchor**: the set of publisher public keys embedded in a binary release, unioned with operator-configured keys; the only roots of trust for verification, never network-delivered. Carries a **trust epoch** (FR-005a), a monotonic integer that only a binary release can advance. +- **Signing Authority**: the grouping anti-downgrade state is keyed by (FR-004a) — one publisher authority spanning all publisher keys including those mid-rotation, and a separate operator authority for operator-configured keys. Each holds its own watermark so neither can lock the other out. - **Signature (TPA record)**: one cataloged campaign/technique — `TPA-YYYY-NNNN` id, detectors, category, level, confidence, tier intent (gating vs non-gating), provenance reference, license. - **Eval Sample Pair**: the labeled gated-malicious sample(s) plus category-matched hard-negative(s) a gating signature must contribute to the frozen eval dataset. -- **Sequence Floor**: the persisted highest-activated sequence per install; the anti-downgrade/anti-replay state, lowered only by explicit rollback. +- **Sequence Watermark**: the persisted highest-activated sequence, tracked per signing authority and scoped by trust epoch (FR-004a, FR-005a); the anti-downgrade/anti-replay state. It is MONOTONIC — rollback does not lower it (FR-010 expresses rollback as a pinned exception plus a deny-list instead), and the only thing that re-baselines it is a higher trust epoch arriving in a binary release. - **Publication Channel**: the stable public location of versioned signed artifact pairs with a "latest" reference; consumed by the Spec 087 fetch or by manual download. - **Post-Activation Re-scan**: the one-shot, off-hot-path evaluation of cached approved tool metadata against a newly-activated database, emitting review findings only. - **Active Database / Last-Known-Good**: unchanged from Spec 086/087 — the serving bundle and the fail-closed fallback target. @@ -165,7 +169,8 @@ The publisher cuts a new database release (new signature for a fresh campaign), ### Measurable Outcomes - **SC-001**: 100% of tampered candidates (any byte of bundle or sidecar altered, or mismatched pair) are rejected before parsing, with the last-known-good database still serving and the reason surfaced — demonstrated by fixture in CI. -- **SC-002**: A validly-signed but lower-sequence bundle is never activated by any non-rollback path (drop, fetch, restart), and an unsigned candidate never advances the floor nor survives the signature ratchet — demonstrated by fixture in CI. The sole exemption is the FR-009a embedded-default availability fallback, which serves as degraded coverage rather than counting as a downgrade. An explicit rollback succeeds, leaves an audit trail, does not lower the historical watermark, and the deny-listed release it rolled away from is not re-activated by the following refresh cycle. +- **SC-002**: A validly-signed but lower-sequence bundle is never activated by any non-rollback path (drop, fetch, restart), and an unsigned candidate never advances the watermark nor survives the signature ratchet — demonstrated by fixture in CI. The sole exemption is the FR-009a embedded-default availability fallback, which serves as degraded coverage rather than counting as a downgrade. An explicit rollback succeeds, leaves an audit trail, does not lower the historical watermark, and the deny-listed release it rolled away from is not re-activated by the following refresh cycle. +- **SC-002a**: The watermark cannot be weaponized into a permanent update lockout, demonstrated by two fixtures: (a) a bundle signed by a to-be-revoked key at the MAXIMUM representable sequence, after which a binary carrying a bumped trust epoch and the surviving key successfully activates a new release — proving key-compromise recovery is possible rather than merely documented; and (b) an operator-authority-signed bundle at a high sequence that leaves the publisher authority's watermark untouched, so the next official publisher release still activates — proving the two signing authorities cannot lock each other out. - **SC-003**: An air-gapped install (zero network, ever) reaches full P1 capability: verified activation of a file-dropped signed database, downgrade refusal, and rollback, with behavior byte-identical to a connected install's file-drop path. - **SC-004**: The shipped seed corpus contains at least 25 signatures spanning at least 8 distinct public campaign/technique classes, every one carrying provenance + redistributable license, and every gating signature backed by its eval sample pair; the canonical fixture of each class produces a hard-tier finding naming its TPA id. - **SC-005**: The scan-eval CI gate (recall ≥ 0.90 gated categories, hard-negative FP ≤ 0.05) passes with the full seed corpus embedded AND actually scored — i.e. the gate run registers the bundle check and maps every new campaign category (FR-016a), proven by a negative control: removing a gating signature from the bundle must turn the gate red. Without that control the criterion is vacuous, since today's gate would pass with the bundle entirely absent. A deliberately-regressing corpus change is then blocked twice independently: at CI (cannot merge) and at activation (self-check rejects it) — both demonstrated by fixture. @@ -182,7 +187,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), - The eval gate's thresholds, dataset conventions (including hard-negative naming), vacuity guard, and hard-tier-only scoring are reused verbatim; growing the corpus means growing the dataset alongside it. - The offline tier's runnable surface remains `engine: regex` × `target: tool_description` for v1; `structural_diff`, `resource_content`, and `server_manifest` rules stay declared-not-runnable (skipped, never clean coverage) exactly as today. - Cached tool metadata already held by mcpproxy (approval baselines, index) is sufficient for the post-activation re-scan; no upstream connection is initiated by refresh. -- [NEEDS CLARIFICATION: back-compat window for unsigned file drops — should `require_signed_bundle` flip to default-on after one or two release cycles once signed publishing is live, or remain opt-in indefinitely for air-gapped/self-built-corpus operators?] +- [NEEDS CLARIFICATION: back-compat window for unsigned file drops — should `require_signed_bundle` flip to default-on after one or two release cycles once signed publishing is live, or remain opt-in indefinitely for air-gapped/self-built-corpus operators? Note this is now a smaller question than it looks: FR-009's signature ratchet already means any install that has ever run a signed bundle refuses unsigned drops, and FR-007 suspends auto-approval while an unsigned bundle is active, so the default-on flip would only change behavior for installs that have never seen a signed bundle at all.] ## Out of Scope @@ -197,7 +202,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), ## Constitution Check *(note)* -Principle IV (Security by Default) governs this spec: the database is treated as a hostile input until proven otherwise (verify-before-parse, fail-closed to last-known-good, anti-downgrade floor, mandatory signatures on the network path), and refresh can never silently widen approvals. Principle III (Configuration-Driven Architecture): `require_signed_bundle`, operator keys, and the existing bundle path/fetch settings live in `mcp_config.json` with env override and hot-reload; no hardcoded URLs or paths. Principle V (TDD): every rejection class (tamper, downgrade, unsigned, gate regression, ceiling breach) is built against failing fixtures first, and the corpus itself is test-gated by scan-eval. Principle I (Performance at Scale): verification, activation, and the post-activation re-scan stay off the scan hot path with atomic swaps (SC-008). +Principle IV (Security by Default) governs this spec: the database is treated as a hostile input until proven otherwise (verify-before-parse, fail-closed to last-known-good, anti-downgrade watermark, mandatory signatures on the network path), and refresh can never silently widen approvals. Principle III (Configuration-Driven Architecture): `require_signed_bundle`, operator keys, and the existing bundle path/fetch settings live in `mcp_config.json` with env override and hot-reload; no hardcoded URLs or paths. Principle V (TDD): every rejection class (tamper, downgrade, unsigned, gate regression, ceiling breach) is built against failing fixtures first, and the corpus itself is test-gated by scan-eval. Principle I (Performance at Scale): verification, activation, and the post-activation re-scan stay off the scan hot path with atomic swaps (SC-008). ## Commit Message Conventions *(mandatory)* From f2ce0fec764ce2bdf43c7d432cd481f9f0345cf0 Mon Sep 17 00:00:00 2001 From: Algis Dumbris Date: Sun, 23 Aug 2026 21:26:20 +0300 Subject: [PATCH 04/15] docs(specs): bind the trust epoch into signed bundle bytes in 101-tpa-db MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round-3 cross-model review (opencode/gpt-5.6-sol) found one CRITICAL defect, verified genuine: the trust epoch introduced in round 2 was itself a replay window. ## The defect FR-005a made the epoch a property of the binary's trust set only; FR-002's manifest carried sequence and key id but no epoch. After a compromise recovery bumped the epoch and re-baselined the watermark, an OLD, weaker bundle legitimately signed by a still-trusted surviving key would be reinterpreted as belonging to the NEW epoch, clear the fresh (near-zero) watermark, and activate with signature_verified=true and full coverage — reinstating precisely the blind spots the recovery release was published to close. SC-002a tested recovery but not this replay. ## Changes - FR-002: the trust epoch is now carried inside the SIGNED manifest, so it is cryptographically bound to the bundle and cannot be reinterpreted. - FR-005a: epochs are per-authority, and a candidate is accepted only when its manifest epoch EQUALS the current binary-trusted epoch of the verifying authority — never merely less-than-or-equal. A below-epoch bundle is refused as stale-epoch replay even when its signature verifies against a still trusted key. Documents the accepted consequence: a binary on the old epoch stops receiving database updates until updated, which is correct because it still trusts the compromised key, and must surface as a stale-epoch degraded state rather than failing silently. - FR-008: epoch match added to the pipeline, before the anti-downgrade stage. - SC-002a: new fixture (a2) — a pre-bump bundle signed by the surviving key is refused after the bump rather than clearing the re-baselined watermark. - FR-020: stale-epoch replay added to the surfaced rejection reasons; bundle status also reports signing authority and trust epoch. Also, from a self-audit of the round-2 per-authority model: - FR-009a: the embedded default degrades when below a watermark retained by ANY authority (previously ambiguous which authority was meant). - FR-004a: choosing an operator-signed corpus over a higher-sequence publisher one is a deliberate operator decision, not a downgrade and not degraded coverage — but the active authority must be visible in bundle status. Related #1028 --- specs/101-tpa-db/spec.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/specs/101-tpa-db/spec.md b/specs/101-tpa-db/spec.md index c652e590..6f5c8f21 100644 --- a/specs/101-tpa-db/spec.md +++ b/specs/101-tpa-db/spec.md @@ -112,20 +112,20 @@ The publisher cuts a new database release (new signature for a fresh campaign), **Database format & signing** - **FR-001**: The publishable database artifact MUST be the existing compiled bundle (`scanner-bundle.json`, Scanner Bundle Contract) plus a detached signature sidecar over the exact bundle bytes. Bundle bytes MUST remain deterministic for a given corpus (byte-identical rebuilds), so the signature and the fingerprint are stable. -- **FR-002**: The bundle manifest MUST gain additive metadata within the supported schema line: `generated_at` (RFC3339 build stamp — the loader already surfaces it), `sequence` (a strictly-increasing integer publish counter), and a publisher key identifier. These keys MUST be additive so existing v0.1 loaders ignore them (contract §4 forward-compat preserved). `generated_at` MUST be derived from deterministic release metadata (the release commit's timestamp or `SOURCE_DATE_EPOCH`), NEVER from build wall-clock time — a wall-clock stamp would change the bundle bytes, fingerprint, and signature on every rebuild of an identical corpus and so would directly violate FR-001's byte-identical-rebuild guarantee. +- **FR-002**: The bundle manifest MUST gain additive metadata within the supported schema line: `generated_at` (RFC3339 build stamp — the loader already surfaces it), `sequence` (a strictly-increasing integer publish counter), a publisher key identifier, and the **trust epoch** the artifact was published under (FR-005a). Because these keys live inside the signed bytes, the epoch is cryptographically bound to the bundle and cannot be reinterpreted after the fact. These keys MUST be additive so existing v0.1 loaders ignore them (contract §4 forward-compat preserved). `generated_at` MUST be derived from deterministic release metadata (the release commit's timestamp or `SOURCE_DATE_EPOCH`), NEVER from build wall-clock time — a wall-clock stamp would change the bundle bytes, fingerprint, and signature on every rebuild of an identical corpus and so would directly violate FR-001's byte-identical-rebuild guarantee. - **FR-003**: The signature scheme MUST be Ed25519 detached signatures (or an equivalent modern EdDSA scheme selected in design); the sidecar MUST carry the algorithm identifier and signing key id so the scheme can evolve without breaking verification of existing artifacts. A key id MUST be a canonical full-length fingerprint of the public key itself (not an operator-chosen label), the trust set MUST reject duplicate ids at build/config time, and verification MUST succeed only when the manifest key id, the sidecar key id, and the id of the trust-set key whose signature actually verified are all identical — a manifest naming one publisher while the bytes are signed by another MUST be a verification failure, not a cosmetic mismatch. - **FR-004**: The set of trusted verification keys MUST be the union of (a) publisher public keys embedded in the binary at build time and (b) operator-configured keys (the Spec 087 FR-013 key). Keys MUST NOT be retrieved over the network, and verification MUST work fully offline. These are two distinct **signing authorities**, and the design MUST keep them so: all publisher keys (including keys being rotated in or out, FR-005) belong to ONE publisher authority, while operator-configured keys form a separate operator authority. -- **FR-004a**: Anti-downgrade state MUST be tracked per signing authority, not in one global namespace. An operator maintaining a self-built corpus is an explicitly supported persona (air-gapped installs, private signature sets), and with a single shared counter their locally-signed bundle at sequence N would raise the one global watermark above the publisher's release line and permanently block official updates — or, symmetrically, a publisher release would block the operator's own corpus. Each authority MUST therefore carry its own watermark, and a candidate MUST be compared only against the watermark of the authority whose key verified it. Grouping all publisher keys into one authority is what keeps per-authority watermarks compatible with key rotation: a rotated-in key inherits the publisher authority's watermark rather than starting a fresh one. +- **FR-004a**: Anti-downgrade state MUST be tracked per signing authority, not in one global namespace. An operator maintaining a self-built corpus is an explicitly supported persona (air-gapped installs, private signature sets), and with a single shared counter their locally-signed bundle at sequence N would raise the one global watermark above the publisher's release line and permanently block official updates — or, symmetrically, a publisher release would block the operator's own corpus. Each authority MUST therefore carry its own watermark, and a candidate MUST be compared only against the watermark of the authority whose key verified it. Choosing an operator-signed corpus over a higher-sequence publisher one is a deliberate operator decision and MUST NOT by itself be treated as a downgrade or as degraded coverage — the operator configured that key precisely to assert their own corpus — but the active authority MUST be visible in bundle status (FR-020) so the choice is never invisible. Grouping all publisher keys into one authority is what keeps per-authority watermarks compatible with key rotation: a rotated-in key inherits the publisher authority's watermark rather than starting a fresh one. - **FR-005**: Key rotation MUST be supported by trusting multiple keys simultaneously: a binary release MAY add a new key while retaining the old for at least one release cycle, and key removal is also a binary-release event. The design MUST document the compromise procedure and the "do not rotate casually" operational stance. -- **FR-005a**: The trust set MUST carry a **trust epoch** — a monotonic integer that advances ONLY via a binary release — and every anti-downgrade comparison (FR-009) MUST be scoped `(authority, epoch, sequence)`, ordered by epoch first. Without this, key-compromise recovery is impossible rather than merely painful: an attacker holding a compromised publisher key signs a bundle at the maximum representable sequence, and because the watermark is monotonic and no surviving key can ever exceed that sequence, the install is permanently unable to accept any future legitimate release — a one-shot permanent denial of updates, downgraded only to the FR-009a fallback. A binary release that revokes the compromised key MUST therefore also bump the epoch, and a validly-signed bundle from a higher epoch MUST supersede the retained watermark and re-baseline it. Epoch advancement is safe to treat as authoritative because it arrives only inside a release binary, whose integrity is already the root of trust for the embedded key set (FR-004) — it is not attacker-reachable over the bundle or fetch path. +- **FR-005a**: Each signing authority's trust set MUST carry a **trust epoch** — a monotonic per-authority integer that advances ONLY via a binary release — and every anti-downgrade comparison (FR-009) MUST be scoped `(authority, epoch, sequence)`, ordered by epoch first. The epoch MUST be carried inside the SIGNED manifest (FR-002) and a candidate MUST be accepted only when its manifest epoch EQUALS the current binary-trusted epoch of the authority whose key verified it — never merely less-than-or-equal. Binding the epoch to the signed bytes and requiring exact equality is what stops the epoch bump from becoming a replay window of its own: if the epoch lived only in the binary, then after a bump-and-revoke the re-baselined watermark would start from zero, and an OLD, weaker bundle legitimately signed by a still-trusted surviving key would be silently reinterpreted as belonging to the new epoch, clear the fresh watermark, and activate with `signature_verified=true` and full coverage — reinstating exactly the blind spots the recovery release was published to close. Without this, key-compromise recovery is impossible rather than merely painful: an attacker holding a compromised publisher key signs a bundle at the maximum representable sequence, and because the watermark is monotonic and no surviving key can ever exceed that sequence, the install is permanently unable to accept any future legitimate release — a one-shot permanent denial of updates, downgraded only to the FR-009a fallback. A binary release that revokes the compromised key MUST therefore also bump the epoch, and a validly-signed bundle whose manifest epoch matches a newly-bumped binary epoch MUST supersede the retained watermark and re-baseline it. A bundle whose manifest epoch is BELOW the current binary-trusted epoch MUST be refused as stale-epoch replay even when its signature verifies against a still-trusted key. Epoch advancement is safe to treat as authoritative because it arrives only inside a release binary, whose integrity is already the root of trust for the embedded key set (FR-004) — it is not attacker-reachable over the bundle or fetch path. The accepted consequence of exact-equality matching is that a binary still on the OLD epoch will refuse the new epoch's bundles and stop receiving database updates until it is itself updated. That is deliberate, not incidental: an epoch bump means a key was compromised, and an un-updated binary still trusts that compromised key, so freezing its database is safer than feeding it artifacts whose trust basis it cannot evaluate. It also matches the existing "a hasty rotation strands older installs" stance, and it MUST be surfaced as a stale-epoch degraded state (FR-020) rather than failing silently. **Loader & activation** - **FR-006**: For any candidate carrying a signature, verification MUST run over the exact raw candidate bytes BEFORE those bytes are parsed as JSON or any pattern is compiled; a candidate failing verification MUST be rejected without further processing, keeping last-known-good (extends the Spec 086 fail-closed pipeline with a new first stage). The bytes that are verified MUST be the same bytes that are subsequently parsed (read once into memory, verify, then parse that buffer) — the file MUST NOT be re-read from disk after verification, or a writer racing the loader could swap content between verify and parse. - **FR-007**: Signing policy MUST be: network-fetched candidates ALWAYS require a valid signature; file-drop candidates are verified whenever a sidecar is present; a new `require_signed_bundle` setting (default off for back-compat with existing unsigned drops) makes the sidecar mandatory for file drops too; the embedded default needs no sidecar (its integrity rides on binary distribution integrity). Bundle status MUST always report whether the active bundle was signature-verified. While an unsigned EXTERNAL (file-dropped or fetched) bundle is active, the scan coverage the approval gates consume MUST be marked degraded so `scan`-mode auto-approval is suspended and changes route to human review — today `internal/security/scanner/inprocess.go` sets `coverageOK` from mere bundle *presence*, so without this an attacker-supplied unsigned corpus would keep silently auto-approving under weakened rules. This degradation rule applies to EXTERNAL candidates ONLY: the embedded default is binary-trusted (its integrity rides on the signed binary distribution, not on a sidecar) and MUST be full-coverage on an ordinary install, or every fresh install would ship with `scan` mode silently disabled. The embedded default is degraded only in the FR-009a case — when it is serving as a fallback BELOW a retained signed watermark, i.e. when the install has evidence a newer signed corpus should be active. This requirement supersedes Spec 087 FR-014's precondition: because FR-004 makes publisher keys embedded in the binary part of the trust set, a fetch is no longer refused merely because the operator configured no key of their own; the "never activate an unverified fetched bundle" guarantee of Spec 087 FR-013/FR-014 is preserved unchanged. -- **FR-008**: The full validate-before-activate order MUST be: byte-size ceiling → signature over the raw bytes (per policy, FR-006) → parse manifest → rule-count ceiling → sequence/fingerprint anti-downgrade (FR-009) → version compatibility → compile-all-or-reject → runnable-rules > 0 → activation self-check (Spec 087 FR-008) → atomic activation. FR-011's two ceilings MUST therefore be split across the pipeline rather than applied as one stage: byte size is measurable on raw bytes and so guards the verification step itself, while rule count is only knowable after parsing and so cannot precede it. The same reasoning applies to sequence, a manifest field necessarily read after parsing: what MUST precede parsing is signature verification, not the sequence or rule-count check — an unverified candidate's manifest is never trusted for any decision. Any failure at any stage MUST keep last-known-good and record a machine-readable reason and timestamp (surfaced via the existing bundle status), never an empty or partial rule set. +- **FR-008**: The full validate-before-activate order MUST be: byte-size ceiling → signature over the raw bytes (per policy, FR-006) → parse manifest → rule-count ceiling → epoch match (FR-005a) → sequence/fingerprint anti-downgrade (FR-009) → version compatibility → compile-all-or-reject → runnable-rules > 0 → activation self-check (Spec 087 FR-008) → atomic activation. FR-011's two ceilings MUST therefore be split across the pipeline rather than applied as one stage: byte size is measurable on raw bytes and so guards the verification step itself, while rule count is only knowable after parsing and so cannot precede it. The same reasoning applies to sequence, a manifest field necessarily read after parsing: what MUST precede parsing is signature verification, not the sequence or rule-count check — an unverified candidate's manifest is never trusted for any decision. Any failure at any stage MUST keep last-known-good and record a machine-readable reason and timestamp (surfaced via the existing bundle status), never an empty or partial rule set. - **FR-009**: The system MUST persist the highest sequence ever activated — the **watermark**, scoped per signing authority and trust epoch per FR-004a/FR-005a — and refuse any candidate with a lower sequence, or an equal sequence with a different content fingerprint, as a downgrade/replay, unless an explicit operator rollback (FR-010) is in effect. **The watermark is a property of SIGNED artifacts only**: `sequence` lives in the manifest, so an unsigned candidate's sequence is attacker-chosen. An unsigned/unverified candidate MUST NOT advance the watermark (otherwise a single unsigned drop claiming a huge sequence permanently locks out every legitimate signed release), and once an install has ever activated a signature-verified bundle, unsigned candidates MUST be refused thereafter regardless of `require_signed_bundle` — a signature ratchet, so signing cannot be stripped back off an install by whoever can write the bundle path. The embedded default is exempt from BOTH the watermark and the ratchet — it is the availability fallback of last resort and must never be locked out (scanning must never stop), which is exactly why FR-009a demotes it to degraded coverage instead of refusing it. -- **FR-009a**: When no valid external candidate is available, the embedded default MUST still serve so scanning never stops, even if its sequence is below the watermark. Where the install has NO retained signed watermark (the ordinary fresh install), this is simply the normal full-coverage state. Where a signed watermark IS retained and the embedded default is below it, the embedded corpus may be older and weaker than the watermark implies, so THAT state MUST be treated as degraded coverage (auto-approval suspended, as in FR-007's unsigned-external case), the retained watermark and the degraded reason MUST be surfaced, and it MUST NOT count as an anti-downgrade violation. This below-watermark fallback is the single explicit exemption to SC-002. +- **FR-009a**: When no valid external candidate is available, the embedded default MUST still serve so scanning never stops, even if its sequence is below the watermark. Where the install has NO retained signed watermark (the ordinary fresh install), this is simply the normal full-coverage state. Where a signed watermark IS retained by ANY authority (FR-004a) and the embedded default is below it, the embedded corpus may be older and weaker than that watermark implies, so THAT state MUST be treated as degraded coverage (auto-approval suspended, as in FR-007's unsigned-external case), the retained watermark and the degraded reason MUST be surfaced, and it MUST NOT count as an anti-downgrade violation. This below-watermark fallback is the single explicit exemption to SC-002. - **FR-010**: An explicit rollback action MUST let an operator revert to the embedded default or a named bundle file: the target is re-validated (signature policy and self-check still apply), activated despite a lower sequence, and the override is loudly logged and visible in bundle status. Rollback MUST NOT lower the historical high-watermark: the watermark is monotonic and normal activation MUST still require exceeding it. Rollback is instead expressed as a **pinned exception** — the operator pins the rolled-back artifact by fingerprint, and the (sequence, fingerprint) being rolled away from is deny-listed so the very release the operator just rejected cannot be re-activated by the next refresh cycle (and neither can any intermediate release between the pin and the watermark). Clearing the pin returns the install to normal watermark-governed activation. - **FR-011**: The loader MUST enforce load ceilings (maximum bundle byte size and maximum rule count, fixed in design) and reject a candidate exceeding them as a whole; all patterns remain RE2-only. A database at the ceilings MUST NOT measurably regress the scan hot path (see SC-008). - **FR-011a**: Activation MUST be crash-consistent, not merely atomic in memory. Spec 087 FR-010 makes the rule-set swap atomic with respect to concurrent scans; that says nothing about persisted state, and a process death between "activated" and "watermark written" reopens exactly the replay window FR-009 exists to close (while writing the watermark first can leave the install with only an older recoverable bundle). The activation record — active fingerprint, sequence, per-authority watermarks and their epochs, the fingerprint each watermark was set by, signature-verified state, any FR-010 pin, and the FR-010 deny-list — MUST be persisted as ONE transaction, and the recoverable last-known-good bundle MUST be durable before the in-memory pointer is published. The deny-list and watermark fingerprint are load-bearing members of that transaction, not bookkeeping: a crash that preserved a rollback while losing the deny-list would let the next refresh immediately re-activate the exact release the operator had just rejected, silently undoing the rollback. Startup, refresh, hot-reload, and rollback MUST all serialize through that same transaction, so any crash leaves the install on a consistent (bundle, watermark) pair. @@ -147,7 +147,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), **Observability, telemetry & docs** -- **FR-020**: Bundle status on all existing surfaces (CLI security commands, REST security overview, Web UI) MUST additionally report sequence, signature-verified state, and staleness (age since `generated_at`), alongside the existing version/fingerprint/source/load-error fields, and MUST show the last rejection reason (tamper, downgrade, unsigned-refused-by-ratchet, gate failure) so an operator can tell "current and verified" from "stuck on last-known-good and why". Where the active database has demoted the install to DEGRADED coverage (FR-007 unsigned-active, FR-009a embedded fallback), that state and its reason MUST be surfaced too — a silently suspended `scan`-mode auto-approval is otherwise indistinguishable from a quiet install. +- **FR-020**: Bundle status on all existing surfaces (CLI security commands, REST security overview, Web UI) MUST additionally report sequence, signing authority and trust epoch, signature-verified state, and staleness (age since `generated_at`), alongside the existing version/fingerprint/source/load-error fields, and MUST show the last rejection reason (tamper, downgrade, stale-epoch replay, unsigned-refused-by-ratchet, gate failure) so an operator can tell "current and verified" from "stuck on last-known-good and why". Where the active database has demoted the install to DEGRADED coverage (FR-007 unsigned-active, FR-009a embedded fallback), that state and its reason MUST be surfaced too — a silently suspended `scan`-mode auto-approval is otherwise indistinguishable from a quiet install. - **FR-021**: The existing anonymous TPA-scanner telemetry MUST gain the active bundle version, sequence, and source (and signature-verified state) so database adoption and freshness are measurable fleet-wide; the existing telemetry opt-out MUST fully cover these fields, and no new identifying data is introduced. This is NOT a drop-in addition: `internal/telemetry/anonymity.go`'s schema-v8 backstop hard-whitelists the `tpa_scanner` sub-object to `{scans_completed, scans_failed, scans_with_findings, findings}` and requires every scalar to be a non-negative integer, so a string `bundle_version`/`source` or a boolean `signature_verified` would be rejected before transmit as "carries a key outside the whitelist". The design MUST therefore extend that backstop with explicitly TYPED entries (integer `sequence`, enum-bounded `source` ∈ {embedded, file, fetch}, boolean signature-verified) rather than relaxing it to accept arbitrary keys or free strings. `bundle_version` MUST be constrained to publisher-issued values (unrecognized/operator-built corpus versions reported as a single `other` bucket): the backstop exists precisely to stop a bespoke, near-unique string from acting as a fleet identifier, and a self-built corpus version would be exactly that. - **FR-022**: Documentation MUST cover the database lifecycle end to end for operators (verify, drop, fetch, roll back, read status) and for contributors (author a signature, provenance/license rules, eval-sample requirements, how the gate blocks a bad corpus), updating the existing security-quarantine/tool-scanner docs. @@ -155,7 +155,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **Signature Database (bundle + sidecar)**: the publishable unit — the deterministic compiled `scanner-bundle.json` plus a detached signature over its exact bytes. Identified by (bundle_version, sequence, fingerprint). - **Bundle Manifest metadata**: `bundle_version`/`schema_version` (existing), plus additive `generated_at`, `sequence`, publisher key id. -- **Publisher Trust Anchor**: the set of publisher public keys embedded in a binary release, unioned with operator-configured keys; the only roots of trust for verification, never network-delivered. Carries a **trust epoch** (FR-005a), a monotonic integer that only a binary release can advance. +- **Publisher Trust Anchor**: the set of publisher public keys embedded in a binary release, unioned with operator-configured keys; the only roots of trust for verification, never network-delivered. Each authority carries its own **trust epoch** (FR-005a), a monotonic integer that only a binary release can advance and that a candidate's signed manifest must match exactly. - **Signing Authority**: the grouping anti-downgrade state is keyed by (FR-004a) — one publisher authority spanning all publisher keys including those mid-rotation, and a separate operator authority for operator-configured keys. Each holds its own watermark so neither can lock the other out. - **Signature (TPA record)**: one cataloged campaign/technique — `TPA-YYYY-NNNN` id, detectors, category, level, confidence, tier intent (gating vs non-gating), provenance reference, license. - **Eval Sample Pair**: the labeled gated-malicious sample(s) plus category-matched hard-negative(s) a gating signature must contribute to the frozen eval dataset. @@ -170,7 +170,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **SC-001**: 100% of tampered candidates (any byte of bundle or sidecar altered, or mismatched pair) are rejected before parsing, with the last-known-good database still serving and the reason surfaced — demonstrated by fixture in CI. - **SC-002**: A validly-signed but lower-sequence bundle is never activated by any non-rollback path (drop, fetch, restart), and an unsigned candidate never advances the watermark nor survives the signature ratchet — demonstrated by fixture in CI. The sole exemption is the FR-009a embedded-default availability fallback, which serves as degraded coverage rather than counting as a downgrade. An explicit rollback succeeds, leaves an audit trail, does not lower the historical watermark, and the deny-listed release it rolled away from is not re-activated by the following refresh cycle. -- **SC-002a**: The watermark cannot be weaponized into a permanent update lockout, demonstrated by two fixtures: (a) a bundle signed by a to-be-revoked key at the MAXIMUM representable sequence, after which a binary carrying a bumped trust epoch and the surviving key successfully activates a new release — proving key-compromise recovery is possible rather than merely documented; and (b) an operator-authority-signed bundle at a high sequence that leaves the publisher authority's watermark untouched, so the next official publisher release still activates — proving the two signing authorities cannot lock each other out. +- **SC-002a**: The watermark cannot be weaponized into a permanent update lockout, demonstrated by two fixtures: (a) a bundle signed by a to-be-revoked key at the MAXIMUM representable sequence, after which a binary carrying a bumped trust epoch and the surviving key successfully activates a new release — proving key-compromise recovery is possible rather than merely documented; (a2) immediately after that bump, an OLD pre-bump bundle carrying the previous epoch and signed by the still-trusted surviving key is REFUSED as stale-epoch replay rather than clearing the re-baselined watermark — the epoch bump must not itself become a downgrade window; and (b) an operator-authority-signed bundle at a high sequence that leaves the publisher authority's watermark untouched, so the next official publisher release still activates — proving the two signing authorities cannot lock each other out. - **SC-003**: An air-gapped install (zero network, ever) reaches full P1 capability: verified activation of a file-dropped signed database, downgrade refusal, and rollback, with behavior byte-identical to a connected install's file-drop path. - **SC-004**: The shipped seed corpus contains at least 25 signatures spanning at least 8 distinct public campaign/technique classes, every one carrying provenance + redistributable license, and every gating signature backed by its eval sample pair; the canonical fixture of each class produces a hard-tier finding naming its TPA id. - **SC-005**: The scan-eval CI gate (recall ≥ 0.90 gated categories, hard-negative FP ≤ 0.05) passes with the full seed corpus embedded AND actually scored — i.e. the gate run registers the bundle check and maps every new campaign category (FR-016a), proven by a negative control: removing a gating signature from the bundle must turn the gate red. Without that control the criterion is vacuous, since today's gate would pass with the bundle entirely absent. A deliberately-regressing corpus change is then blocked twice independently: at CI (cannot merge) and at activation (self-check rejects it) — both demonstrated by fixture. From 0527bfe0df0ea76dfe0ca1adc70c4e03c12ac564 Mon Sep 17 00:00:00 2001 From: Algis Dumbris Date: Sun, 23 Aug 2026 21:29:45 +0300 Subject: [PATCH 05/15] docs(specs): seed the anti-downgrade watermark from a trusted baseline (101-tpa-db) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round-4 cross-model review (opencode/gpt-5.6-sol) found one HIGH defect, verified genuine: the anti-downgrade guarantee did not exist at first use. ## The defect Every anti-downgrade mechanism in the spec binds only once persisted state exists. On a fresh install — or after any data-directory reset — there is no watermark, so an attacker who can write the bundle path drops an OLD but genuinely publisher-signed, current-epoch bundle (sequence 5 while the shipping release is at 100). Signature verifies, epoch matches, nothing to compare the sequence against: it activates as a fully signature-verified bundle at FULL coverage, silently reinstating every blind spot fixed between 5 and 100. The FR-009 ratchet and the FR-010 pin/deny-list do not help, since both also require pre-existing state. Confirmed no new contradiction was introduced by the round-3 epoch fix. ## Changes - FR-009b (new): watermarks are SEEDED from a trusted baseline rather than starting empty. The publisher authority seeds from the sequence of the bundle embedded in the running binary — a candidate older than what the binary already ships is strictly worse than the fallback and is refused — and the baseline is re-applied on every startup, so wiping the data directory cannot lower it below the binary's own corpus. The operator authority, which has no binary-trusted baseline, gets a minimum sequence configurable alongside the operator key. This also makes the FR-005a epoch re-baseline safe: a bumped epoch starts at the embedded sequence, not zero. - SC-002 / US1 Independent Test: fresh-install and post-wipe downgrade are now explicitly in scope, with fixture (h) covering the old-but-validly-signed bundle on an install with no persisted state. - Edge case "Anti-downgrade state lost": corrected — a wipe resets the watermark to the seeded baseline, not to zero. What a wipe genuinely loses is the ratchet state and any pin/deny-list. - Key Entities: Sequence Watermark is documented as never empty. Related #1028 --- specs/101-tpa-db/spec.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/specs/101-tpa-db/spec.md b/specs/101-tpa-db/spec.md index 6f5c8f21..ea1deadd 100644 --- a/specs/101-tpa-db/spec.md +++ b/specs/101-tpa-db/spec.md @@ -40,7 +40,7 @@ An operator receives a TPA signature database out of band — a `scanner-bundle. **Why this priority**: The database is a code-adjacent security input: whoever controls its bytes controls what the scanner ignores. Without signing and anti-rollback, the "refreshable signature DB" pillar creates a new supply-chain surface (swap the file → blind the scanner; replay an old bundle → resurrect a fixed blind spot). Signing + sequence + fail-closed loading is what makes every other story safe to ship, so it goes first. -**Independent Test**: Fully offline, no live upstream. Sign a fixture bundle with a test key configured as a trust anchor; drop bundle+signature and assert activation with `signature_verified=true` and the new sequence recorded. Then assert each of the following is rejected with the active bundle unchanged and a machine-readable reason: (a) one flipped byte in the bundle, (b) a signature from a non-trusted key, (c) a missing sidecar with `require_signed_bundle=true`, (d) a validly-signed bundle whose sequence is lower than the active one, (e) a validly-signed bundle failing the activation self-check, (f) an unsigned bundle dropped after a signed one was active (ratchet), and (g) an unsigned bundle claiming a huge sequence — assert the watermark is unchanged so a later legitimate signed release still activates. Finally, invoke the explicit rollback action and assert the embedded default is restored, the override is logged, the high-watermark is unchanged, and a subsequent refresh does not re-activate the deny-listed release. +**Independent Test**: Fully offline, no live upstream. Sign a fixture bundle with a test key configured as a trust anchor; drop bundle+signature and assert activation with `signature_verified=true` and the new sequence recorded. Then assert each of the following is rejected with the active bundle unchanged and a machine-readable reason: (a) one flipped byte in the bundle, (b) a signature from a non-trusted key, (c) a missing sidecar with `require_signed_bundle=true`, (d) a validly-signed bundle whose sequence is lower than the active one, (e) a validly-signed bundle failing the activation self-check, (f) an unsigned bundle dropped after a signed one was active (ratchet), (g) an unsigned bundle claiming a huge sequence — assert the watermark is unchanged so a later legitimate signed release still activates, and (h) on a FRESH install with no persisted state, a genuinely publisher-signed current-epoch bundle whose sequence is older than the binary's embedded corpus — assert it is refused by the FR-009b seeded baseline rather than activating at full coverage. Finally, invoke the explicit rollback action and assert the embedded default is restored, the override is logged, the high-watermark is unchanged, and a subsequent refresh does not re-activate the deny-listed release. **Acceptance Scenarios**: @@ -98,7 +98,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **Bundle signed by a formerly-trusted, now-removed key**: verification fails on binaries that dropped the key (fail-closed to last-known-good); older binaries still trusting it keep working — the downgrade watermark still blocks sequence replay. - **Tampered bundle / tampered sidecar / swapped pair** (valid signature belonging to a different bundle's bytes): all reduce to signature-verification failure before parse; never a crash, never partial load. - **Downgrade attack via the embedded default**: a fresh binary whose embedded bundle sequence is LOWER than a previously-activated external bundle's persisted watermark does not silently downgrade — the external candidate re-validates and wins by precedence; if only the embedded bundle is available, it serves (scanning must never stop) with the watermark retained and the situation surfaced, but per FR-009a as DEGRADED coverage, so an attacker who merely deletes the external bundle cannot use the weaker embedded corpus to obtain `scan`-mode auto-approvals. This is the one explicit exemption to SC-002. -- **Anti-downgrade state lost** (data directory wiped/reset): the watermark resets, and so does the FR-009 signature-ratchet state (both live in the data dir), so a wiped install accepts an unsigned drop again. This is accepted — the watermark and ratchet are best-effort hardening on top of signing, not the sole defense, and a wiped data dir already resets approvals. Operators who need this closed absolutely set `require_signed_bundle`, which is config-resident rather than data-dir-resident and therefore survives the wipe. +- **Anti-downgrade state lost** (data directory wiped/reset): the persisted watermark resets to the FR-009b seeded baseline — NOT to zero, so a wiped install still refuses any publisher-signed bundle older than the corpus its own binary ships. What a wipe does lose is the FR-009 signature-ratchet state and any FR-010 pin/deny-list (both data-dir resident), so a wiped install accepts an unsigned drop again. This is accepted — the ratchet is best-effort hardening on top of signing, not the sole defense, and a wiped data dir already resets approvals. Operators who need this closed absolutely set `require_signed_bundle`, which is config-resident rather than data-dir-resident and therefore survives the wipe. - **Equal sequence, different bytes** (two distinct bundles claiming the same sequence): refused as suspicious unless it is the currently-active fingerprint (idempotent re-drop stays a no-op per Spec 087 FR-011). - **Clock skew**: no validity windows or expiry in v1 — sequence ordering, not wall-clock time, is the trust signal; `generated_at` is advisory freshness only. - **Huge or adversarial database** (thousands of rules, pathological regex): size and rule-count ceilings are enforced at load; every pattern remains RE2 (linear-time, no catastrophic backtracking by construction); a candidate exceeding ceilings is rejected whole, keeping last-known-good. @@ -125,6 +125,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **FR-007**: Signing policy MUST be: network-fetched candidates ALWAYS require a valid signature; file-drop candidates are verified whenever a sidecar is present; a new `require_signed_bundle` setting (default off for back-compat with existing unsigned drops) makes the sidecar mandatory for file drops too; the embedded default needs no sidecar (its integrity rides on binary distribution integrity). Bundle status MUST always report whether the active bundle was signature-verified. While an unsigned EXTERNAL (file-dropped or fetched) bundle is active, the scan coverage the approval gates consume MUST be marked degraded so `scan`-mode auto-approval is suspended and changes route to human review — today `internal/security/scanner/inprocess.go` sets `coverageOK` from mere bundle *presence*, so without this an attacker-supplied unsigned corpus would keep silently auto-approving under weakened rules. This degradation rule applies to EXTERNAL candidates ONLY: the embedded default is binary-trusted (its integrity rides on the signed binary distribution, not on a sidecar) and MUST be full-coverage on an ordinary install, or every fresh install would ship with `scan` mode silently disabled. The embedded default is degraded only in the FR-009a case — when it is serving as a fallback BELOW a retained signed watermark, i.e. when the install has evidence a newer signed corpus should be active. This requirement supersedes Spec 087 FR-014's precondition: because FR-004 makes publisher keys embedded in the binary part of the trust set, a fetch is no longer refused merely because the operator configured no key of their own; the "never activate an unverified fetched bundle" guarantee of Spec 087 FR-013/FR-014 is preserved unchanged. - **FR-008**: The full validate-before-activate order MUST be: byte-size ceiling → signature over the raw bytes (per policy, FR-006) → parse manifest → rule-count ceiling → epoch match (FR-005a) → sequence/fingerprint anti-downgrade (FR-009) → version compatibility → compile-all-or-reject → runnable-rules > 0 → activation self-check (Spec 087 FR-008) → atomic activation. FR-011's two ceilings MUST therefore be split across the pipeline rather than applied as one stage: byte size is measurable on raw bytes and so guards the verification step itself, while rule count is only knowable after parsing and so cannot precede it. The same reasoning applies to sequence, a manifest field necessarily read after parsing: what MUST precede parsing is signature verification, not the sequence or rule-count check — an unverified candidate's manifest is never trusted for any decision. Any failure at any stage MUST keep last-known-good and record a machine-readable reason and timestamp (surfaced via the existing bundle status), never an empty or partial rule set. - **FR-009**: The system MUST persist the highest sequence ever activated — the **watermark**, scoped per signing authority and trust epoch per FR-004a/FR-005a — and refuse any candidate with a lower sequence, or an equal sequence with a different content fingerprint, as a downgrade/replay, unless an explicit operator rollback (FR-010) is in effect. **The watermark is a property of SIGNED artifacts only**: `sequence` lives in the manifest, so an unsigned candidate's sequence is attacker-chosen. An unsigned/unverified candidate MUST NOT advance the watermark (otherwise a single unsigned drop claiming a huge sequence permanently locks out every legitimate signed release), and once an install has ever activated a signature-verified bundle, unsigned candidates MUST be refused thereafter regardless of `require_signed_bundle` — a signature ratchet, so signing cannot be stripped back off an install by whoever can write the bundle path. The embedded default is exempt from BOTH the watermark and the ratchet — it is the availability fallback of last resort and must never be locked out (scanning must never stop), which is exactly why FR-009a demotes it to degraded coverage instead of refusing it. +- **FR-009b**: Each authority's watermark MUST be SEEDED from a trusted baseline rather than starting empty, because an absent watermark accepts anything. Otherwise the anti-downgrade guarantee simply does not exist at first use or after the data directory is reset: an attacker who can write the bundle path drops an OLD but genuinely publisher-signed, current-epoch bundle — say sequence 5 while the shipping release is at 100 — and with no watermark to compare against it activates as a fully signature-verified bundle at FULL coverage, silently reinstating every blind spot fixed between 5 and 100. Neither the FR-009 ratchet nor the FR-010 pin/deny-list helps here; both only bind once persisted state exists. Therefore: the **publisher** authority's watermark MUST be seeded from the sequence of the bundle embedded in the running binary — a candidate older than what the binary already ships is strictly worse than the fallback and MUST be refused — and this baseline MUST be re-applied on every startup, so wiping the data directory cannot lower it below the binary's own corpus. The **operator** authority has no binary-trusted baseline available, so its minimum sequence MUST be configurable alongside the operator key (defaulting to zero, with the residual exposure being the already-out-of-scope config-write attacker). The seeded baseline also makes the epoch re-baseline of FR-005a safe: after an epoch bump the new epoch's watermark starts at the embedded bundle's sequence, not at zero. - **FR-009a**: When no valid external candidate is available, the embedded default MUST still serve so scanning never stops, even if its sequence is below the watermark. Where the install has NO retained signed watermark (the ordinary fresh install), this is simply the normal full-coverage state. Where a signed watermark IS retained by ANY authority (FR-004a) and the embedded default is below it, the embedded corpus may be older and weaker than that watermark implies, so THAT state MUST be treated as degraded coverage (auto-approval suspended, as in FR-007's unsigned-external case), the retained watermark and the degraded reason MUST be surfaced, and it MUST NOT count as an anti-downgrade violation. This below-watermark fallback is the single explicit exemption to SC-002. - **FR-010**: An explicit rollback action MUST let an operator revert to the embedded default or a named bundle file: the target is re-validated (signature policy and self-check still apply), activated despite a lower sequence, and the override is loudly logged and visible in bundle status. Rollback MUST NOT lower the historical high-watermark: the watermark is monotonic and normal activation MUST still require exceeding it. Rollback is instead expressed as a **pinned exception** — the operator pins the rolled-back artifact by fingerprint, and the (sequence, fingerprint) being rolled away from is deny-listed so the very release the operator just rejected cannot be re-activated by the next refresh cycle (and neither can any intermediate release between the pin and the watermark). Clearing the pin returns the install to normal watermark-governed activation. - **FR-011**: The loader MUST enforce load ceilings (maximum bundle byte size and maximum rule count, fixed in design) and reject a candidate exceeding them as a whole; all patterns remain RE2-only. A database at the ceilings MUST NOT measurably regress the scan hot path (see SC-008). @@ -159,7 +160,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **Signing Authority**: the grouping anti-downgrade state is keyed by (FR-004a) — one publisher authority spanning all publisher keys including those mid-rotation, and a separate operator authority for operator-configured keys. Each holds its own watermark so neither can lock the other out. - **Signature (TPA record)**: one cataloged campaign/technique — `TPA-YYYY-NNNN` id, detectors, category, level, confidence, tier intent (gating vs non-gating), provenance reference, license. - **Eval Sample Pair**: the labeled gated-malicious sample(s) plus category-matched hard-negative(s) a gating signature must contribute to the frozen eval dataset. -- **Sequence Watermark**: the persisted highest-activated sequence, tracked per signing authority and scoped by trust epoch (FR-004a, FR-005a); the anti-downgrade/anti-replay state. It is MONOTONIC — rollback does not lower it (FR-010 expresses rollback as a pinned exception plus a deny-list instead), and the only thing that re-baselines it is a higher trust epoch arriving in a binary release. +- **Sequence Watermark**: the persisted highest-activated sequence, tracked per signing authority and scoped by trust epoch (FR-004a, FR-005a); the anti-downgrade/anti-replay state. Never empty — it is seeded from a trusted baseline (the embedded bundle's sequence, for the publisher authority) per FR-009b, so "no state yet" is not an accept-anything state. It is MONOTONIC — rollback does not lower it (FR-010 expresses rollback as a pinned exception plus a deny-list instead), and the only thing that re-baselines it is a higher trust epoch arriving in a binary release. - **Publication Channel**: the stable public location of versioned signed artifact pairs with a "latest" reference; consumed by the Spec 087 fetch or by manual download. - **Post-Activation Re-scan**: the one-shot, off-hot-path evaluation of cached approved tool metadata against a newly-activated database, emitting review findings only. - **Active Database / Last-Known-Good**: unchanged from Spec 086/087 — the serving bundle and the fail-closed fallback target. @@ -169,7 +170,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), ### Measurable Outcomes - **SC-001**: 100% of tampered candidates (any byte of bundle or sidecar altered, or mismatched pair) are rejected before parsing, with the last-known-good database still serving and the reason surfaced — demonstrated by fixture in CI. -- **SC-002**: A validly-signed but lower-sequence bundle is never activated by any non-rollback path (drop, fetch, restart), and an unsigned candidate never advances the watermark nor survives the signature ratchet — demonstrated by fixture in CI. The sole exemption is the FR-009a embedded-default availability fallback, which serves as degraded coverage rather than counting as a downgrade. An explicit rollback succeeds, leaves an audit trail, does not lower the historical watermark, and the deny-listed release it rolled away from is not re-activated by the following refresh cycle. +- **SC-002**: A validly-signed but lower-sequence bundle is never activated by any non-rollback path (drop, fetch, restart) — including on a fresh install or after a data-directory wipe, where the FR-009b seeded baseline stands in for the absent persisted watermark — and an unsigned candidate never advances the watermark nor survives the signature ratchet — demonstrated by fixture in CI. The sole exemption is the FR-009a embedded-default availability fallback, which serves as degraded coverage rather than counting as a downgrade. An explicit rollback succeeds, leaves an audit trail, does not lower the historical watermark, and the deny-listed release it rolled away from is not re-activated by the following refresh cycle. - **SC-002a**: The watermark cannot be weaponized into a permanent update lockout, demonstrated by two fixtures: (a) a bundle signed by a to-be-revoked key at the MAXIMUM representable sequence, after which a binary carrying a bumped trust epoch and the surviving key successfully activates a new release — proving key-compromise recovery is possible rather than merely documented; (a2) immediately after that bump, an OLD pre-bump bundle carrying the previous epoch and signed by the still-trusted surviving key is REFUSED as stale-epoch replay rather than clearing the re-baselined watermark — the epoch bump must not itself become a downgrade window; and (b) an operator-authority-signed bundle at a high sequence that leaves the publisher authority's watermark untouched, so the next official publisher release still activates — proving the two signing authorities cannot lock each other out. - **SC-003**: An air-gapped install (zero network, ever) reaches full P1 capability: verified activation of a file-dropped signed database, downgrade refusal, and rollback, with behavior byte-identical to a connected install's file-drop path. - **SC-004**: The shipped seed corpus contains at least 25 signatures spanning at least 8 distinct public campaign/technique classes, every one carrying provenance + redistributable license, and every gating signature backed by its eval sample pair; the canonical fixture of each class produces a hard-tier finding naming its TPA id. From c178144722a5022e7b8c6760c5855f49cd71daf1 Mon Sep 17 00:00:00 2001 From: Algis Dumbris Date: Sun, 23 Aug 2026 21:32:16 +0300 Subject: [PATCH 06/15] docs(specs): make the operator-authority baseline explicit and durable (101-tpa-db) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Round-5 cross-model review (opencode/gpt-5.6-sol) found one HIGH defect, verified genuine. It confirmed FR-009b closes the publisher-side replay without contradicting FR-005a, FR-009, FR-009a, or FR-010, but caught that the operator authority was left with the same hole. ## The defect FR-009b gave the operator authority a minimum sequence "defaulting to zero, with the residual exposure being the already-out-of-scope config-write attacker". That parenthetical was wrong. With a zero default, no config write is needed: after a data-directory wipe (or on a fresh install), a bundle-path attacker — squarely inside the threat model — can replay any older genuinely operator-signed bundle at full coverage. The operator authority therefore had no post-wipe anti-downgrade at all by default, while the spec read as though it did. ## Changes - FR-009b: the operator baseline must be declared explicitly in mcp_config.json alongside the operator key, not defaulted to zero and not stored in the data directory — a baseline held in resettable state gives no protection exactly when it is needed. If an operator key is configured without a baseline, operator-authority anti-downgrade is treated as unsupported and coverage is marked degraded while an operator-signed bundle is active, instead of presenting it as fully trusted. The data-directory watermark may only raise the effective minimum above the configured baseline, never lower it. Epoch re-baseline now starts at the authority's baseline rather than zero. - Edge case "Anti-downgrade state lost": states plainly that the operator-side guarantee survives a wipe only with a config-resident baseline. - SC-002a: fixture (c) covers the operator authority after a wipe, both with and without a declared baseline. Note: per the repo's per-PR cross-model review cap, this is the fifth round. This fix has NOT itself been re-reviewed. Related #1028 --- specs/101-tpa-db/spec.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/specs/101-tpa-db/spec.md b/specs/101-tpa-db/spec.md index ea1deadd..7233ebbe 100644 --- a/specs/101-tpa-db/spec.md +++ b/specs/101-tpa-db/spec.md @@ -98,7 +98,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **Bundle signed by a formerly-trusted, now-removed key**: verification fails on binaries that dropped the key (fail-closed to last-known-good); older binaries still trusting it keep working — the downgrade watermark still blocks sequence replay. - **Tampered bundle / tampered sidecar / swapped pair** (valid signature belonging to a different bundle's bytes): all reduce to signature-verification failure before parse; never a crash, never partial load. - **Downgrade attack via the embedded default**: a fresh binary whose embedded bundle sequence is LOWER than a previously-activated external bundle's persisted watermark does not silently downgrade — the external candidate re-validates and wins by precedence; if only the embedded bundle is available, it serves (scanning must never stop) with the watermark retained and the situation surfaced, but per FR-009a as DEGRADED coverage, so an attacker who merely deletes the external bundle cannot use the weaker embedded corpus to obtain `scan`-mode auto-approvals. This is the one explicit exemption to SC-002. -- **Anti-downgrade state lost** (data directory wiped/reset): the persisted watermark resets to the FR-009b seeded baseline — NOT to zero, so a wiped install still refuses any publisher-signed bundle older than the corpus its own binary ships. What a wipe does lose is the FR-009 signature-ratchet state and any FR-010 pin/deny-list (both data-dir resident), so a wiped install accepts an unsigned drop again. This is accepted — the ratchet is best-effort hardening on top of signing, not the sole defense, and a wiped data dir already resets approvals. Operators who need this closed absolutely set `require_signed_bundle`, which is config-resident rather than data-dir-resident and therefore survives the wipe. +- **Anti-downgrade state lost** (data directory wiped/reset): the persisted watermark resets to the FR-009b seeded baseline — NOT to zero, so a wiped install still refuses any publisher-signed bundle older than the corpus its own binary ships. For the operator authority the same protection holds only if the operator declared a config-resident baseline; without one, operator-authority anti-downgrade does not survive the wipe and the install runs degraded while an operator-signed bundle is active, by FR-009b, rather than pretending to a guarantee it does not have. What a wipe does lose is the FR-009 signature-ratchet state and any FR-010 pin/deny-list (both data-dir resident), so a wiped install accepts an unsigned drop again. This is accepted — the ratchet is best-effort hardening on top of signing, not the sole defense, and a wiped data dir already resets approvals. Operators who need this closed absolutely set `require_signed_bundle`, which is config-resident rather than data-dir-resident and therefore survives the wipe. - **Equal sequence, different bytes** (two distinct bundles claiming the same sequence): refused as suspicious unless it is the currently-active fingerprint (idempotent re-drop stays a no-op per Spec 087 FR-011). - **Clock skew**: no validity windows or expiry in v1 — sequence ordering, not wall-clock time, is the trust signal; `generated_at` is advisory freshness only. - **Huge or adversarial database** (thousands of rules, pathological regex): size and rule-count ceilings are enforced at load; every pattern remains RE2 (linear-time, no catastrophic backtracking by construction); a candidate exceeding ceilings is rejected whole, keeping last-known-good. @@ -125,7 +125,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **FR-007**: Signing policy MUST be: network-fetched candidates ALWAYS require a valid signature; file-drop candidates are verified whenever a sidecar is present; a new `require_signed_bundle` setting (default off for back-compat with existing unsigned drops) makes the sidecar mandatory for file drops too; the embedded default needs no sidecar (its integrity rides on binary distribution integrity). Bundle status MUST always report whether the active bundle was signature-verified. While an unsigned EXTERNAL (file-dropped or fetched) bundle is active, the scan coverage the approval gates consume MUST be marked degraded so `scan`-mode auto-approval is suspended and changes route to human review — today `internal/security/scanner/inprocess.go` sets `coverageOK` from mere bundle *presence*, so without this an attacker-supplied unsigned corpus would keep silently auto-approving under weakened rules. This degradation rule applies to EXTERNAL candidates ONLY: the embedded default is binary-trusted (its integrity rides on the signed binary distribution, not on a sidecar) and MUST be full-coverage on an ordinary install, or every fresh install would ship with `scan` mode silently disabled. The embedded default is degraded only in the FR-009a case — when it is serving as a fallback BELOW a retained signed watermark, i.e. when the install has evidence a newer signed corpus should be active. This requirement supersedes Spec 087 FR-014's precondition: because FR-004 makes publisher keys embedded in the binary part of the trust set, a fetch is no longer refused merely because the operator configured no key of their own; the "never activate an unverified fetched bundle" guarantee of Spec 087 FR-013/FR-014 is preserved unchanged. - **FR-008**: The full validate-before-activate order MUST be: byte-size ceiling → signature over the raw bytes (per policy, FR-006) → parse manifest → rule-count ceiling → epoch match (FR-005a) → sequence/fingerprint anti-downgrade (FR-009) → version compatibility → compile-all-or-reject → runnable-rules > 0 → activation self-check (Spec 087 FR-008) → atomic activation. FR-011's two ceilings MUST therefore be split across the pipeline rather than applied as one stage: byte size is measurable on raw bytes and so guards the verification step itself, while rule count is only knowable after parsing and so cannot precede it. The same reasoning applies to sequence, a manifest field necessarily read after parsing: what MUST precede parsing is signature verification, not the sequence or rule-count check — an unverified candidate's manifest is never trusted for any decision. Any failure at any stage MUST keep last-known-good and record a machine-readable reason and timestamp (surfaced via the existing bundle status), never an empty or partial rule set. - **FR-009**: The system MUST persist the highest sequence ever activated — the **watermark**, scoped per signing authority and trust epoch per FR-004a/FR-005a — and refuse any candidate with a lower sequence, or an equal sequence with a different content fingerprint, as a downgrade/replay, unless an explicit operator rollback (FR-010) is in effect. **The watermark is a property of SIGNED artifacts only**: `sequence` lives in the manifest, so an unsigned candidate's sequence is attacker-chosen. An unsigned/unverified candidate MUST NOT advance the watermark (otherwise a single unsigned drop claiming a huge sequence permanently locks out every legitimate signed release), and once an install has ever activated a signature-verified bundle, unsigned candidates MUST be refused thereafter regardless of `require_signed_bundle` — a signature ratchet, so signing cannot be stripped back off an install by whoever can write the bundle path. The embedded default is exempt from BOTH the watermark and the ratchet — it is the availability fallback of last resort and must never be locked out (scanning must never stop), which is exactly why FR-009a demotes it to degraded coverage instead of refusing it. -- **FR-009b**: Each authority's watermark MUST be SEEDED from a trusted baseline rather than starting empty, because an absent watermark accepts anything. Otherwise the anti-downgrade guarantee simply does not exist at first use or after the data directory is reset: an attacker who can write the bundle path drops an OLD but genuinely publisher-signed, current-epoch bundle — say sequence 5 while the shipping release is at 100 — and with no watermark to compare against it activates as a fully signature-verified bundle at FULL coverage, silently reinstating every blind spot fixed between 5 and 100. Neither the FR-009 ratchet nor the FR-010 pin/deny-list helps here; both only bind once persisted state exists. Therefore: the **publisher** authority's watermark MUST be seeded from the sequence of the bundle embedded in the running binary — a candidate older than what the binary already ships is strictly worse than the fallback and MUST be refused — and this baseline MUST be re-applied on every startup, so wiping the data directory cannot lower it below the binary's own corpus. The **operator** authority has no binary-trusted baseline available, so its minimum sequence MUST be configurable alongside the operator key (defaulting to zero, with the residual exposure being the already-out-of-scope config-write attacker). The seeded baseline also makes the epoch re-baseline of FR-005a safe: after an epoch bump the new epoch's watermark starts at the embedded bundle's sequence, not at zero. +- **FR-009b**: Each authority's watermark MUST be SEEDED from a trusted baseline rather than starting empty, because an absent watermark accepts anything. Otherwise the anti-downgrade guarantee simply does not exist at first use or after the data directory is reset: an attacker who can write the bundle path drops an OLD but genuinely publisher-signed, current-epoch bundle — say sequence 5 while the shipping release is at 100 — and with no watermark to compare against it activates as a fully signature-verified bundle at FULL coverage, silently reinstating every blind spot fixed between 5 and 100. Neither the FR-009 ratchet nor the FR-010 pin/deny-list helps here; both only bind once persisted state exists. Therefore: the **publisher** authority's watermark MUST be seeded from the sequence of the bundle embedded in the running binary — a candidate older than what the binary already ships is strictly worse than the fallback and MUST be refused — and this baseline MUST be re-applied on every startup, so wiping the data directory cannot lower it below the binary's own corpus. The **operator** authority has no binary-trusted baseline available, so its baseline MUST be declared explicitly in configuration alongside the operator key, and it MUST live in `mcp_config.json` rather than the data directory — that is what makes it durable across the wipe, since a baseline stored in resettable state provides no protection precisely when it is needed. A silent zero default MUST NOT be used: with a zero baseline the operator authority has NO anti-downgrade after state loss at all, and a bundle-path attacker — who is squarely inside the threat model, unlike the config-write attacker — could replay any older genuinely operator-signed bundle at full coverage on a fresh or wiped install. Therefore: if an operator key is configured WITHOUT a baseline, the system MUST treat operator-authority anti-downgrade as unsupported and mark coverage degraded (per FR-007) while an operator-signed bundle is active, rather than silently presenting it as fully trusted. The data-directory watermark then tracks activations above the configured baseline and may only raise the effective minimum, never lower it. The seeded baseline also makes the epoch re-baseline of FR-005a safe: after an epoch bump the new epoch's watermark starts at the authority's baseline, not at zero. - **FR-009a**: When no valid external candidate is available, the embedded default MUST still serve so scanning never stops, even if its sequence is below the watermark. Where the install has NO retained signed watermark (the ordinary fresh install), this is simply the normal full-coverage state. Where a signed watermark IS retained by ANY authority (FR-004a) and the embedded default is below it, the embedded corpus may be older and weaker than that watermark implies, so THAT state MUST be treated as degraded coverage (auto-approval suspended, as in FR-007's unsigned-external case), the retained watermark and the degraded reason MUST be surfaced, and it MUST NOT count as an anti-downgrade violation. This below-watermark fallback is the single explicit exemption to SC-002. - **FR-010**: An explicit rollback action MUST let an operator revert to the embedded default or a named bundle file: the target is re-validated (signature policy and self-check still apply), activated despite a lower sequence, and the override is loudly logged and visible in bundle status. Rollback MUST NOT lower the historical high-watermark: the watermark is monotonic and normal activation MUST still require exceeding it. Rollback is instead expressed as a **pinned exception** — the operator pins the rolled-back artifact by fingerprint, and the (sequence, fingerprint) being rolled away from is deny-listed so the very release the operator just rejected cannot be re-activated by the next refresh cycle (and neither can any intermediate release between the pin and the watermark). Clearing the pin returns the install to normal watermark-governed activation. - **FR-011**: The loader MUST enforce load ceilings (maximum bundle byte size and maximum rule count, fixed in design) and reject a candidate exceeding them as a whole; all patterns remain RE2-only. A database at the ceilings MUST NOT measurably regress the scan hot path (see SC-008). @@ -171,7 +171,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **SC-001**: 100% of tampered candidates (any byte of bundle or sidecar altered, or mismatched pair) are rejected before parsing, with the last-known-good database still serving and the reason surfaced — demonstrated by fixture in CI. - **SC-002**: A validly-signed but lower-sequence bundle is never activated by any non-rollback path (drop, fetch, restart) — including on a fresh install or after a data-directory wipe, where the FR-009b seeded baseline stands in for the absent persisted watermark — and an unsigned candidate never advances the watermark nor survives the signature ratchet — demonstrated by fixture in CI. The sole exemption is the FR-009a embedded-default availability fallback, which serves as degraded coverage rather than counting as a downgrade. An explicit rollback succeeds, leaves an audit trail, does not lower the historical watermark, and the deny-listed release it rolled away from is not re-activated by the following refresh cycle. -- **SC-002a**: The watermark cannot be weaponized into a permanent update lockout, demonstrated by two fixtures: (a) a bundle signed by a to-be-revoked key at the MAXIMUM representable sequence, after which a binary carrying a bumped trust epoch and the surviving key successfully activates a new release — proving key-compromise recovery is possible rather than merely documented; (a2) immediately after that bump, an OLD pre-bump bundle carrying the previous epoch and signed by the still-trusted surviving key is REFUSED as stale-epoch replay rather than clearing the re-baselined watermark — the epoch bump must not itself become a downgrade window; and (b) an operator-authority-signed bundle at a high sequence that leaves the publisher authority's watermark untouched, so the next official publisher release still activates — proving the two signing authorities cannot lock each other out. +- **SC-002a**: The watermark cannot be weaponized into a permanent update lockout, demonstrated by two fixtures: (a) a bundle signed by a to-be-revoked key at the MAXIMUM representable sequence, after which a binary carrying a bumped trust epoch and the surviving key successfully activates a new release — proving key-compromise recovery is possible rather than merely documented; (a2) immediately after that bump, an OLD pre-bump bundle carrying the previous epoch and signed by the still-trusted surviving key is REFUSED as stale-epoch replay rather than clearing the re-baselined watermark — the epoch bump must not itself become a downgrade window; and (b) an operator-authority-signed bundle at a high sequence that leaves the publisher authority's watermark untouched, so the next official publisher release still activates — proving the two signing authorities cannot lock each other out. A third fixture (c) covers the operator authority after a data-directory wipe: with a config-resident baseline declared, an older operator-signed bundle is refused; with no baseline declared, it may activate but the install reports degraded coverage rather than full trust. - **SC-003**: An air-gapped install (zero network, ever) reaches full P1 capability: verified activation of a file-dropped signed database, downgrade refusal, and rollback, with behavior byte-identical to a connected install's file-drop path. - **SC-004**: The shipped seed corpus contains at least 25 signatures spanning at least 8 distinct public campaign/technique classes, every one carrying provenance + redistributable license, and every gating signature backed by its eval sample pair; the canonical fixture of each class produces a hard-tier finding naming its TPA id. - **SC-005**: The scan-eval CI gate (recall ≥ 0.90 gated categories, hard-negative FP ≤ 0.05) passes with the full seed corpus embedded AND actually scored — i.e. the gate run registers the bundle check and maps every new campaign category (FR-016a), proven by a negative control: removing a gating signature from the bundle must turn the gate red. Without that control the criterion is vacuous, since today's gate would pass with the bundle entirely absent. A deliberately-regressing corpus change is then blocked twice independently: at CI (cannot merge) and at activation (self-check rejects it) — both demonstrated by fixture. From 49c4e7edf70697deddd288927e72adbda5c40d18 Mon Sep 17 00:00:00 2001 From: Algis Dumbris Date: Mon, 24 Aug 2026 06:40:02 +0300 Subject: [PATCH 07/15] docs(specs): close the authority-swap hole and de-vacuate the gate control (101-tpa-db) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cross-model review (opencode / gpt-5.6-sol) round 1 on specs/101-tpa-db/spec.md. All five findings verified against the current code before changing anything. ## Changes - FR-004b (new): the active signing authority is a config-resident declaration, default publisher; a candidate signed by a non-active authority is refused ("authority not active"). Per-authority watermarks alone let a bundle-path attacker — in the threat model — swap a publisher bundle at sequence 100 for any operator-signed artifact clearing the operator baseline, silently narrowing coverage while still reporting signature_verified=true at full coverage, because the comparison never crosses namespaces. - FR-009a: the embedded default's sequence is compared ONLY against the publisher watermark (same namespace); the "any authority" rule made an operator corpus at sequence 900 permanently degrade a current publisher fallback at 100. The intent behind it is restated directly as condition (b): a previously-active external bundle of the active authority having gone missing. - FR-016a: corrected the claim about unmapped eval categories — they are excluded from OverallRecall so their MISSES cannot fail the gate, but their false positives still count, since FPRate is computed over the whole hard-negative set regardless of category mapping (gate.go:193-197, 287-289). - FR-016b (new) + SC-005 + US2 independent test: the anti-vacuity negative control is made deterministic. Aggregate recall over 25+ signatures absorbs a single removal within the 0.10 threshold headroom, and built-in checks (phrase.injection especially) often flag the dropped signature's sample anyway, so "remove a gating signature and the gate must go red" could pass vacuously. A designated control signature with no built-in or sibling overlap, an asserted gate-breach exit code, and a CI assertion of the no-overlap property replace it. - SC-001: tampering is defined over the fields the detached signature actually covers (bundle bytes, signature value, sidecar identity fields, mismatched pair) instead of "any byte of ... sidecar", which no detached scheme can guarantee against whitespace/key-order edits. - SC-002a gains fixture (b2) and US1's independent test gains case (i) for the authority-not-active refusal; FR-020, the threat-model edge case, Key Entities, and the Constitution Check note the new config-resident field. ## Testing - Docs-only change; no Go sources touched. - Every codebase claim re-verified before editing: cmd/scan-eval/gate.go (gateChecks/categoryCheck/OverallRecall/FPRate, no --bundle flag today), internal/security/scanner/inprocess.go (coverageOK), and internal/telemetry/anonymity.go (schema-v8 tpa_scanner whitelist). --- specs/101-tpa-db/spec.md | 26 ++++++++++++++------------ 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/specs/101-tpa-db/spec.md b/specs/101-tpa-db/spec.md index 7233ebbe..d42ea97a 100644 --- a/specs/101-tpa-db/spec.md +++ b/specs/101-tpa-db/spec.md @@ -40,7 +40,7 @@ An operator receives a TPA signature database out of band — a `scanner-bundle. **Why this priority**: The database is a code-adjacent security input: whoever controls its bytes controls what the scanner ignores. Without signing and anti-rollback, the "refreshable signature DB" pillar creates a new supply-chain surface (swap the file → blind the scanner; replay an old bundle → resurrect a fixed blind spot). Signing + sequence + fail-closed loading is what makes every other story safe to ship, so it goes first. -**Independent Test**: Fully offline, no live upstream. Sign a fixture bundle with a test key configured as a trust anchor; drop bundle+signature and assert activation with `signature_verified=true` and the new sequence recorded. Then assert each of the following is rejected with the active bundle unchanged and a machine-readable reason: (a) one flipped byte in the bundle, (b) a signature from a non-trusted key, (c) a missing sidecar with `require_signed_bundle=true`, (d) a validly-signed bundle whose sequence is lower than the active one, (e) a validly-signed bundle failing the activation self-check, (f) an unsigned bundle dropped after a signed one was active (ratchet), (g) an unsigned bundle claiming a huge sequence — assert the watermark is unchanged so a later legitimate signed release still activates, and (h) on a FRESH install with no persisted state, a genuinely publisher-signed current-epoch bundle whose sequence is older than the binary's embedded corpus — assert it is refused by the FR-009b seeded baseline rather than activating at full coverage. Finally, invoke the explicit rollback action and assert the embedded default is restored, the override is logged, the high-watermark is unchanged, and a subsequent refresh does not re-activate the deny-listed release. +**Independent Test**: Fully offline, no live upstream. Sign a fixture bundle with a test key configured as a trust anchor; drop bundle+signature and assert activation with `signature_verified=true` and the new sequence recorded. Then assert each of the following is rejected with the active bundle unchanged and a machine-readable reason: (a) one flipped byte in the bundle, (b) a signature from a non-trusted key, (c) a missing sidecar with `require_signed_bundle=true`, (d) a validly-signed bundle whose sequence is lower than the active one, (e) a validly-signed bundle failing the activation self-check, (f) an unsigned bundle dropped after a signed one was active (ratchet), (g) an unsigned bundle claiming a huge sequence — assert the watermark is unchanged so a later legitimate signed release still activates, and (h) on a FRESH install with no persisted state, a genuinely publisher-signed current-epoch bundle whose sequence is older than the binary's embedded corpus — assert it is refused by the FR-009b seeded baseline rather than activating at full coverage, and (i) with an operator key configured but the publisher authority declared active, a valid operator-signed bundle clearing the operator baseline — assert it is refused as "authority not active" (FR-004b) and the publisher bundle keeps serving. Finally, invoke the explicit rollback action and assert the embedded default is restored, the override is logged, the high-watermark is unchanged, and a subsequent refresh does not re-activate the deny-listed release. **Acceptance Scenarios**: @@ -61,7 +61,7 @@ A security-conscious developer installs mcpproxy and gets, out of the box, a sig **Why this priority**: The format (US1) is worthless empty, and the refresh channel (US3) has nothing to carry without a corpus. This is the "database" in tpa-db. It is P2 only because the P1 trust machinery must exist before a larger corpus becomes an attractive distribution target. -**Independent Test**: Run the corpus build and assert every signature carries id, category, detector(s), provenance source, and license; assert every gated signature has ≥1 labeled malicious eval sample and ≥1 category-matched hard-negative in the eval dataset; run `go run ./cmd/scan-eval --corpus specs/065-evaluation-foundation/datasets/detect_corpus_v1.json --gate --min-recall 0.90 --max-fp 0.05` with the new embedded default and assert PASS — and, because that command today scores built-in checks only, assert the accompanying NEGATIVE CONTROL that proves the bundle is being scored at all: with a gating signature removed from the bundle the same command must FAIL. Feed each campaign class's canonical fixture through the offline scanner and assert a hard-tier finding naming the TPA id. +**Independent Test**: Run the corpus build and assert every signature carries id, category, detector(s), provenance source, and license; assert every gated signature has ≥1 labeled malicious eval sample and ≥1 category-matched hard-negative in the eval dataset; run `go run ./cmd/scan-eval --corpus specs/065-evaluation-foundation/datasets/detect_corpus_v1.json --gate --min-recall 0.90 --max-fp 0.05` with the new embedded default and assert PASS — and, because that command today scores built-in checks only, assert the accompanying NEGATIVE CONTROL that proves the bundle is being scored at all: with the designated FR-016b control signature removed from the bundle, the same command must FAIL with the gate-breach exit code (the control signature is the one whose sample no built-in check and no other signature detects, so its removal necessarily crosses the threshold rather than being absorbed by aggregate recall headroom). Feed each campaign class's canonical fixture through the offline scanner and assert a hard-tier finding naming the TPA id. **Acceptance Scenarios**: @@ -93,7 +93,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), ### Edge Cases - **Air-gapped host, forever**: every P1 behavior (verification, sequence check, rollback, activation self-check) is network-free; the trust anchors ship inside the binary; staleness is surfaced but never blocks scanning — an old database is degraded coverage, not an outage. -- **Threat-model boundary — attacker with config write**: FR-004 lets operator-configured keys join the trust set, so anyone who can write `mcp_config.json` can trust their own key and sign anything. This is explicitly OUT of the threat model and is not a regression: an attacker with config write can already disable quarantine outright or set a server's trust mode to `auto`. The attacker this design defends against is the one who can write the BUNDLE PATH or intercept the FETCH but not the config — which is precisely why the signature ratchet (FR-009) is keyed to install state rather than to the bundle path, and why `require_signed_bundle` is config-resident (it must not be defeatable from the data dir alone). +- **Threat-model boundary — attacker with config write**: FR-004 lets operator-configured keys join the trust set, so anyone who can write `mcp_config.json` can trust their own key and sign anything. This is explicitly OUT of the threat model and is not a regression: an attacker with config write can already disable quarantine outright or set a server's trust mode to `auto`. The attacker this design defends against is the one who can write the BUNDLE PATH or intercept the FETCH but not the config — which is precisely why the signature ratchet (FR-009) is keyed to install state rather than to the bundle path, why the active-authority declaration (FR-004b) and the operator baseline (FR-009b) are config-resident, and why `require_signed_bundle` is config-resident (none of them must be defeatable from the data dir alone). - **Signature key rotation**: the trust anchor is a SET of publisher keys embedded per binary release. Rotation is a binary-release event: a release ships old+new keys, bundles are signed with the new key, the old key is removed in a later release. Keys are NEVER delivered over the network, and rotation is not done casually — every shipped binary pins its key set, so a hasty rotation strands older installs on unsigned-refresh only (same philosophy as the app-update signing keys). Key compromise → remove the key in the next release, **bump the trust epoch (FR-005a)**, and publish a bundle in the new epoch signed by the surviving key. The epoch bump is what makes recovery possible at all: a compromised key can sign at the maximum representable sequence, and against a monotonic watermark no surviving key could ever outbid that within the same epoch, so revocation alone would leave installs permanently unable to accept any future release. - **Bundle signed by a formerly-trusted, now-removed key**: verification fails on binaries that dropped the key (fail-closed to last-known-good); older binaries still trusting it keep working — the downgrade watermark still blocks sequence replay. - **Tampered bundle / tampered sidecar / swapped pair** (valid signature belonging to a different bundle's bytes): all reduce to signature-verification failure before parse; never a crash, never partial load. @@ -115,7 +115,8 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **FR-002**: The bundle manifest MUST gain additive metadata within the supported schema line: `generated_at` (RFC3339 build stamp — the loader already surfaces it), `sequence` (a strictly-increasing integer publish counter), a publisher key identifier, and the **trust epoch** the artifact was published under (FR-005a). Because these keys live inside the signed bytes, the epoch is cryptographically bound to the bundle and cannot be reinterpreted after the fact. These keys MUST be additive so existing v0.1 loaders ignore them (contract §4 forward-compat preserved). `generated_at` MUST be derived from deterministic release metadata (the release commit's timestamp or `SOURCE_DATE_EPOCH`), NEVER from build wall-clock time — a wall-clock stamp would change the bundle bytes, fingerprint, and signature on every rebuild of an identical corpus and so would directly violate FR-001's byte-identical-rebuild guarantee. - **FR-003**: The signature scheme MUST be Ed25519 detached signatures (or an equivalent modern EdDSA scheme selected in design); the sidecar MUST carry the algorithm identifier and signing key id so the scheme can evolve without breaking verification of existing artifacts. A key id MUST be a canonical full-length fingerprint of the public key itself (not an operator-chosen label), the trust set MUST reject duplicate ids at build/config time, and verification MUST succeed only when the manifest key id, the sidecar key id, and the id of the trust-set key whose signature actually verified are all identical — a manifest naming one publisher while the bytes are signed by another MUST be a verification failure, not a cosmetic mismatch. - **FR-004**: The set of trusted verification keys MUST be the union of (a) publisher public keys embedded in the binary at build time and (b) operator-configured keys (the Spec 087 FR-013 key). Keys MUST NOT be retrieved over the network, and verification MUST work fully offline. These are two distinct **signing authorities**, and the design MUST keep them so: all publisher keys (including keys being rotated in or out, FR-005) belong to ONE publisher authority, while operator-configured keys form a separate operator authority. -- **FR-004a**: Anti-downgrade state MUST be tracked per signing authority, not in one global namespace. An operator maintaining a self-built corpus is an explicitly supported persona (air-gapped installs, private signature sets), and with a single shared counter their locally-signed bundle at sequence N would raise the one global watermark above the publisher's release line and permanently block official updates — or, symmetrically, a publisher release would block the operator's own corpus. Each authority MUST therefore carry its own watermark, and a candidate MUST be compared only against the watermark of the authority whose key verified it. Choosing an operator-signed corpus over a higher-sequence publisher one is a deliberate operator decision and MUST NOT by itself be treated as a downgrade or as degraded coverage — the operator configured that key precisely to assert their own corpus — but the active authority MUST be visible in bundle status (FR-020) so the choice is never invisible. Grouping all publisher keys into one authority is what keeps per-authority watermarks compatible with key rotation: a rotated-in key inherits the publisher authority's watermark rather than starting a fresh one. +- **FR-004a**: Anti-downgrade state MUST be tracked per signing authority, not in one global namespace. An operator maintaining a self-built corpus is an explicitly supported persona (air-gapped installs, private signature sets), and with a single shared counter their locally-signed bundle at sequence N would raise the one global watermark above the publisher's release line and permanently block official updates — or, symmetrically, a publisher release would block the operator's own corpus. Each authority MUST therefore carry its own watermark, and a candidate MUST be compared only against the watermark of the authority whose key verified it. Choosing an operator-signed corpus over a higher-sequence publisher one is a deliberate operator decision (declared per FR-004b) and MUST NOT by itself be treated as a downgrade or as degraded coverage — the operator configured that key precisely to assert their own corpus — but the active authority MUST be visible in bundle status (FR-020) so the choice is never invisible. Grouping all publisher keys into one authority is what keeps per-authority watermarks compatible with key rotation: a rotated-in key inherits the publisher authority's watermark rather than starting a fresh one. +- **FR-004b**: The **active signing authority MUST be a config-resident declaration**, defaulting to the publisher authority, and MUST NOT be inferred solely from whichever key happens to verify a candidate. A candidate whose verifying key belongs to a non-active authority MUST be refused with an "authority not active" reason, exactly as any other failed candidate (last-known-good keeps serving). Without this, per-authority watermarks (FR-004a) open the very hole they were introduced to avoid: because each authority is compared only against its own counter, an attacker who can write the bundle path — squarely inside the threat model, unlike the config-write attacker — can replace a current publisher bundle at sequence 100 with ANY operator-signed artifact that merely clears the operator authority's FR-009b baseline, silently swapping the install onto an older or deliberately narrower corpus while it still reports `signature_verified=true` at FULL coverage. The publisher watermark cannot detect this, since the comparison never crosses namespaces, and no operator decision is involved — whoever writes the path picks the authority. Config-residency is what puts that choice out of the attacker's reach, matching `require_signed_bundle` (FR-007) and the operator baseline (FR-009b), which are config-resident for the same reason. Both key sets MAY remain configured simultaneously; only artifacts of the declared active authority activate, and switching authorities is a config edit, not a file drop. - **FR-005**: Key rotation MUST be supported by trusting multiple keys simultaneously: a binary release MAY add a new key while retaining the old for at least one release cycle, and key removal is also a binary-release event. The design MUST document the compromise procedure and the "do not rotate casually" operational stance. - **FR-005a**: Each signing authority's trust set MUST carry a **trust epoch** — a monotonic per-authority integer that advances ONLY via a binary release — and every anti-downgrade comparison (FR-009) MUST be scoped `(authority, epoch, sequence)`, ordered by epoch first. The epoch MUST be carried inside the SIGNED manifest (FR-002) and a candidate MUST be accepted only when its manifest epoch EQUALS the current binary-trusted epoch of the authority whose key verified it — never merely less-than-or-equal. Binding the epoch to the signed bytes and requiring exact equality is what stops the epoch bump from becoming a replay window of its own: if the epoch lived only in the binary, then after a bump-and-revoke the re-baselined watermark would start from zero, and an OLD, weaker bundle legitimately signed by a still-trusted surviving key would be silently reinterpreted as belonging to the new epoch, clear the fresh watermark, and activate with `signature_verified=true` and full coverage — reinstating exactly the blind spots the recovery release was published to close. Without this, key-compromise recovery is impossible rather than merely painful: an attacker holding a compromised publisher key signs a bundle at the maximum representable sequence, and because the watermark is monotonic and no surviving key can ever exceed that sequence, the install is permanently unable to accept any future legitimate release — a one-shot permanent denial of updates, downgraded only to the FR-009a fallback. A binary release that revokes the compromised key MUST therefore also bump the epoch, and a validly-signed bundle whose manifest epoch matches a newly-bumped binary epoch MUST supersede the retained watermark and re-baseline it. A bundle whose manifest epoch is BELOW the current binary-trusted epoch MUST be refused as stale-epoch replay even when its signature verifies against a still-trusted key. Epoch advancement is safe to treat as authoritative because it arrives only inside a release binary, whose integrity is already the root of trust for the embedded key set (FR-004) — it is not attacker-reachable over the bundle or fetch path. The accepted consequence of exact-equality matching is that a binary still on the OLD epoch will refuse the new epoch's bundles and stop receiving database updates until it is itself updated. That is deliberate, not incidental: an epoch bump means a key was compromised, and an un-updated binary still trusts that compromised key, so freezing its database is safer than feeding it artifacts whose trust basis it cannot evaluate. It also matches the existing "a hasty rotation strands older installs" stance, and it MUST be surfaced as a stale-epoch degraded state (FR-020) rather than failing silently. @@ -126,7 +127,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **FR-008**: The full validate-before-activate order MUST be: byte-size ceiling → signature over the raw bytes (per policy, FR-006) → parse manifest → rule-count ceiling → epoch match (FR-005a) → sequence/fingerprint anti-downgrade (FR-009) → version compatibility → compile-all-or-reject → runnable-rules > 0 → activation self-check (Spec 087 FR-008) → atomic activation. FR-011's two ceilings MUST therefore be split across the pipeline rather than applied as one stage: byte size is measurable on raw bytes and so guards the verification step itself, while rule count is only knowable after parsing and so cannot precede it. The same reasoning applies to sequence, a manifest field necessarily read after parsing: what MUST precede parsing is signature verification, not the sequence or rule-count check — an unverified candidate's manifest is never trusted for any decision. Any failure at any stage MUST keep last-known-good and record a machine-readable reason and timestamp (surfaced via the existing bundle status), never an empty or partial rule set. - **FR-009**: The system MUST persist the highest sequence ever activated — the **watermark**, scoped per signing authority and trust epoch per FR-004a/FR-005a — and refuse any candidate with a lower sequence, or an equal sequence with a different content fingerprint, as a downgrade/replay, unless an explicit operator rollback (FR-010) is in effect. **The watermark is a property of SIGNED artifacts only**: `sequence` lives in the manifest, so an unsigned candidate's sequence is attacker-chosen. An unsigned/unverified candidate MUST NOT advance the watermark (otherwise a single unsigned drop claiming a huge sequence permanently locks out every legitimate signed release), and once an install has ever activated a signature-verified bundle, unsigned candidates MUST be refused thereafter regardless of `require_signed_bundle` — a signature ratchet, so signing cannot be stripped back off an install by whoever can write the bundle path. The embedded default is exempt from BOTH the watermark and the ratchet — it is the availability fallback of last resort and must never be locked out (scanning must never stop), which is exactly why FR-009a demotes it to degraded coverage instead of refusing it. - **FR-009b**: Each authority's watermark MUST be SEEDED from a trusted baseline rather than starting empty, because an absent watermark accepts anything. Otherwise the anti-downgrade guarantee simply does not exist at first use or after the data directory is reset: an attacker who can write the bundle path drops an OLD but genuinely publisher-signed, current-epoch bundle — say sequence 5 while the shipping release is at 100 — and with no watermark to compare against it activates as a fully signature-verified bundle at FULL coverage, silently reinstating every blind spot fixed between 5 and 100. Neither the FR-009 ratchet nor the FR-010 pin/deny-list helps here; both only bind once persisted state exists. Therefore: the **publisher** authority's watermark MUST be seeded from the sequence of the bundle embedded in the running binary — a candidate older than what the binary already ships is strictly worse than the fallback and MUST be refused — and this baseline MUST be re-applied on every startup, so wiping the data directory cannot lower it below the binary's own corpus. The **operator** authority has no binary-trusted baseline available, so its baseline MUST be declared explicitly in configuration alongside the operator key, and it MUST live in `mcp_config.json` rather than the data directory — that is what makes it durable across the wipe, since a baseline stored in resettable state provides no protection precisely when it is needed. A silent zero default MUST NOT be used: with a zero baseline the operator authority has NO anti-downgrade after state loss at all, and a bundle-path attacker — who is squarely inside the threat model, unlike the config-write attacker — could replay any older genuinely operator-signed bundle at full coverage on a fresh or wiped install. Therefore: if an operator key is configured WITHOUT a baseline, the system MUST treat operator-authority anti-downgrade as unsupported and mark coverage degraded (per FR-007) while an operator-signed bundle is active, rather than silently presenting it as fully trusted. The data-directory watermark then tracks activations above the configured baseline and may only raise the effective minimum, never lower it. The seeded baseline also makes the epoch re-baseline of FR-005a safe: after an epoch bump the new epoch's watermark starts at the authority's baseline, not at zero. -- **FR-009a**: When no valid external candidate is available, the embedded default MUST still serve so scanning never stops, even if its sequence is below the watermark. Where the install has NO retained signed watermark (the ordinary fresh install), this is simply the normal full-coverage state. Where a signed watermark IS retained by ANY authority (FR-004a) and the embedded default is below it, the embedded corpus may be older and weaker than that watermark implies, so THAT state MUST be treated as degraded coverage (auto-approval suspended, as in FR-007's unsigned-external case), the retained watermark and the degraded reason MUST be surfaced, and it MUST NOT count as an anti-downgrade violation. This below-watermark fallback is the single explicit exemption to SC-002. +- **FR-009a**: When no valid external candidate is available, the embedded default MUST still serve so scanning never stops, even if its sequence is below the watermark. Where the install has NO retained signed watermark and no missing previously-active external bundle (the ordinary fresh install), this is simply the normal full-coverage state. Two — and only two — conditions demote that fallback to degraded coverage (auto-approval suspended, as in FR-007's unsigned-external case): (a) the embedded default's sequence is below the **publisher** authority's retained watermark, or (b) an external bundle of the currently active authority (FR-004b) was previously activated and is no longer available or no longer valid. The comparison in (a) MUST be same-namespace: the embedded default is a publisher artifact, and sequences of different authorities are independent counters, so it MUST be compared ONLY against the publisher watermark and never against an operator watermark — a cross-authority numeric comparison is meaningless and would, for an operator corpus at sequence 900 against an embedded publisher corpus at 100, permanently suspend auto-approval on an install whose publisher fallback is perfectly current. Condition (b) is what a cross-authority comparison was reaching for and states it directly: the trigger is evidence that a corpus the install was actually running has gone missing, not an arithmetic accident. In either case the retained watermark, the active authority, and the degraded reason MUST be surfaced, and it MUST NOT count as an anti-downgrade violation. This below-watermark fallback is the single explicit exemption to SC-002. - **FR-010**: An explicit rollback action MUST let an operator revert to the embedded default or a named bundle file: the target is re-validated (signature policy and self-check still apply), activated despite a lower sequence, and the override is loudly logged and visible in bundle status. Rollback MUST NOT lower the historical high-watermark: the watermark is monotonic and normal activation MUST still require exceeding it. Rollback is instead expressed as a **pinned exception** — the operator pins the rolled-back artifact by fingerprint, and the (sequence, fingerprint) being rolled away from is deny-listed so the very release the operator just rejected cannot be re-activated by the next refresh cycle (and neither can any intermediate release between the pin and the watermark). Clearing the pin returns the install to normal watermark-governed activation. - **FR-011**: The loader MUST enforce load ceilings (maximum bundle byte size and maximum rule count, fixed in design) and reject a candidate exceeding them as a whole; all patterns remain RE2-only. A database at the ceilings MUST NOT measurably regress the scan hot path (see SC-008). - **FR-011a**: Activation MUST be crash-consistent, not merely atomic in memory. Spec 087 FR-010 makes the rule-set swap atomic with respect to concurrent scans; that says nothing about persisted state, and a process death between "activated" and "watermark written" reopens exactly the replay window FR-009 exists to close (while writing the watermark first can leave the install with only an older recoverable bundle). The activation record — active fingerprint, sequence, per-authority watermarks and their epochs, the fingerprint each watermark was set by, signature-verified state, any FR-010 pin, and the FR-010 deny-list — MUST be persisted as ONE transaction, and the recoverable last-known-good bundle MUST be durable before the in-memory pointer is published. The deny-list and watermark fingerprint are load-bearing members of that transaction, not bookkeeping: a crash that preserved a rollback while losing the deny-list would let the next refresh immediately re-activate the exact release the operator had just rejected, silently undoing the rollback. Startup, refresh, hot-reload, and rollback MUST all serialize through that same transaction, so any crash leaves the install on a consistent (bundle, watermark) pair. @@ -138,7 +139,8 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **FR-014**: Gating signatures MUST emit hard-tier signals (the eval gate and the `scan`-mode approval gates score hard tier only); any soft-tier-only signature MUST be explicitly marked non-gating in its metadata and MUST NOT be counted toward gate coverage. - **FR-015**: Signatures MUST be authored in the corpus source-of-truth pipeline (signature sources compiled to the bundle); mcpproxy MUST continue to consume only the compiled bundle and never parse signature sources (Spec 087 FR-002 parity). The contribution pipeline MUST run the same validation the loader runs (compile, schema, license, eval gate) before a bundle can be signed. - **FR-016**: The CI eval gate (`cmd/scan-eval --gate`, recall ≥ 0.90 over gated categories, hard-negative FP rate ≤ 0.05, existing vacuity guard) MUST pass for: every change to the embedded default corpus (merge-blocking, existing `eval.yml` D2 job) and every published bundle BEFORE it is signed — a bundle that fails the gate is never signed or published. Thresholds are reused verbatim, not redefined. -- **FR-016a**: The gate harness MUST first be taught to see the database at all — today it does not, so FR-016 and SC-005 would otherwise pass vacuously. `cmd/scan-eval/gate.go`'s `gateChecks()` returns only the four built-in `detect.Check`s (`unicode.hidden`, `shadowing.cross_server`, `payload.decoded`, `phrase.injection`); `scanner.BundleCheck` — which production DOES append in `internal/security/scanner/inprocess.go` — is absent, and `scan-eval` never loads a bundle. Therefore: (a) `scan-eval` MUST accept a candidate bundle path, load it through the production loader (never a parallel parser), and register the resulting bundle check, defaulting to the embedded default so the existing `eval.yml` D2 invocation starts scoring the shipped corpus; (b) `gate.go`'s `categoryCheck` map MUST gain an entry for every new TPA campaign category, since `gatedCategory()` enforces a category ONLY when its mapped check id is registered — an unmapped category is measured but can never fail the build, which is precisely the vacuous-pass failure mode; (c) the same candidate-loading path MUST be what the Spec 087 FR-008 activation self-check runs, so the CI layer and the activation layer cannot drift apart while SC-005 claims they are independent. +- **FR-016a**: The gate harness MUST first be taught to see the database at all — today it does not, so FR-016 and SC-005 would otherwise pass vacuously. `cmd/scan-eval/gate.go`'s `gateChecks()` returns only the four built-in `detect.Check`s (`unicode.hidden`, `shadowing.cross_server`, `payload.decoded`, `phrase.injection`); `scanner.BundleCheck` — which production DOES append in `internal/security/scanner/inprocess.go` — is absent, and `scan-eval` never loads a bundle. Therefore: (a) `scan-eval` MUST accept a candidate bundle path, load it through the production loader (never a parallel parser), and register the resulting bundle check, defaulting to the embedded default so the existing `eval.yml` D2 invocation starts scoring the shipped corpus; (b) `gate.go`'s `categoryCheck` map MUST gain an entry for every new TPA campaign category, since `gatedCategory()` enforces a category ONLY when its mapped check id is registered — an unmapped category's malicious samples are tallied and reported but excluded from `gatedMalicious`/`OverallRecall`, so its MISSES can never fail the gate (its false positives still count, because `FPRate` is computed over the whole hard-negative set regardless of category mapping). A category whose recall is unenforceable is precisely the vacuous-pass failure mode; (c) the same candidate-loading path MUST be what the Spec 087 FR-008 activation self-check runs, so the CI layer and the activation layer cannot drift apart while SC-005 claims they are independent. +- **FR-016b**: The FR-016a/SC-005 negative control MUST be constructed to fail DETERMINISTICALLY, not left as "remove any signature and hope the gate turns red". As written it can pass vacuously twice over: `OverallRecall` is an aggregate `gatedDetected/gatedMalicious` over ALL gated categories, so with a 25-signature corpus dropping one signature typically moves recall by well under the 0.10 headroom the 0.90 threshold allows; and the dropped signature's malicious sample may still be flagged at hard tier by a built-in check (`phrase.injection` in particular overlaps most instruction-injection payloads), in which case recall does not move at all. Therefore the corpus MUST designate at least one **control signature** whose canonical malicious sample is verified to be hard-flagged by NO built-in `detect.Check` and by no other bundle signature, and the control MUST assert the gate's exit code is `exitGateBreach` when that signature alone is removed. The control MUST be evaluated over a scope in which the removal necessarily crosses the threshold (a control corpus restricted to the control signature's category, or an equivalent per-category assertion), and the no-overlap property of the control sample MUST itself be asserted in CI so that a later built-in check gaining coverage of it turns the control red rather than silently re-vacuating it. **Publication & refresh channel** @@ -148,7 +150,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), **Observability, telemetry & docs** -- **FR-020**: Bundle status on all existing surfaces (CLI security commands, REST security overview, Web UI) MUST additionally report sequence, signing authority and trust epoch, signature-verified state, and staleness (age since `generated_at`), alongside the existing version/fingerprint/source/load-error fields, and MUST show the last rejection reason (tamper, downgrade, stale-epoch replay, unsigned-refused-by-ratchet, gate failure) so an operator can tell "current and verified" from "stuck on last-known-good and why". Where the active database has demoted the install to DEGRADED coverage (FR-007 unsigned-active, FR-009a embedded fallback), that state and its reason MUST be surfaced too — a silently suspended `scan`-mode auto-approval is otherwise indistinguishable from a quiet install. +- **FR-020**: Bundle status on all existing surfaces (CLI security commands, REST security overview, Web UI) MUST additionally report sequence, signing authority and trust epoch, signature-verified state, and staleness (age since `generated_at`), alongside the existing version/fingerprint/source/load-error fields, and MUST show the last rejection reason (tamper, downgrade, stale-epoch replay, unsigned-refused-by-ratchet, authority-not-active, gate failure) so an operator can tell "current and verified" from "stuck on last-known-good and why". Where the active database has demoted the install to DEGRADED coverage (FR-007 unsigned-active, FR-009a embedded fallback), that state and its reason MUST be surfaced too — a silently suspended `scan`-mode auto-approval is otherwise indistinguishable from a quiet install. - **FR-021**: The existing anonymous TPA-scanner telemetry MUST gain the active bundle version, sequence, and source (and signature-verified state) so database adoption and freshness are measurable fleet-wide; the existing telemetry opt-out MUST fully cover these fields, and no new identifying data is introduced. This is NOT a drop-in addition: `internal/telemetry/anonymity.go`'s schema-v8 backstop hard-whitelists the `tpa_scanner` sub-object to `{scans_completed, scans_failed, scans_with_findings, findings}` and requires every scalar to be a non-negative integer, so a string `bundle_version`/`source` or a boolean `signature_verified` would be rejected before transmit as "carries a key outside the whitelist". The design MUST therefore extend that backstop with explicitly TYPED entries (integer `sequence`, enum-bounded `source` ∈ {embedded, file, fetch}, boolean signature-verified) rather than relaxing it to accept arbitrary keys or free strings. `bundle_version` MUST be constrained to publisher-issued values (unrecognized/operator-built corpus versions reported as a single `other` bucket): the backstop exists precisely to stop a bespoke, near-unique string from acting as a fleet identifier, and a self-built corpus version would be exactly that. - **FR-022**: Documentation MUST cover the database lifecycle end to end for operators (verify, drop, fetch, roll back, read status) and for contributors (author a signature, provenance/license rules, eval-sample requirements, how the gate blocks a bad corpus), updating the existing security-quarantine/tool-scanner docs. @@ -157,7 +159,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **Signature Database (bundle + sidecar)**: the publishable unit — the deterministic compiled `scanner-bundle.json` plus a detached signature over its exact bytes. Identified by (bundle_version, sequence, fingerprint). - **Bundle Manifest metadata**: `bundle_version`/`schema_version` (existing), plus additive `generated_at`, `sequence`, publisher key id. - **Publisher Trust Anchor**: the set of publisher public keys embedded in a binary release, unioned with operator-configured keys; the only roots of trust for verification, never network-delivered. Each authority carries its own **trust epoch** (FR-005a), a monotonic integer that only a binary release can advance and that a candidate's signed manifest must match exactly. -- **Signing Authority**: the grouping anti-downgrade state is keyed by (FR-004a) — one publisher authority spanning all publisher keys including those mid-rotation, and a separate operator authority for operator-configured keys. Each holds its own watermark so neither can lock the other out. +- **Signing Authority**: the grouping anti-downgrade state is keyed by (FR-004a) — one publisher authority spanning all publisher keys including those mid-rotation, and a separate operator authority for operator-configured keys. Each holds its own watermark so neither can lock the other out. Exactly one authority is **active** at a time, declared in config (FR-004b, default publisher); artifacts of a non-active authority are refused, so per-authority isolation cannot be turned into an authority-swap by whoever can write the bundle path. - **Signature (TPA record)**: one cataloged campaign/technique — `TPA-YYYY-NNNN` id, detectors, category, level, confidence, tier intent (gating vs non-gating), provenance reference, license. - **Eval Sample Pair**: the labeled gated-malicious sample(s) plus category-matched hard-negative(s) a gating signature must contribute to the frozen eval dataset. - **Sequence Watermark**: the persisted highest-activated sequence, tracked per signing authority and scoped by trust epoch (FR-004a, FR-005a); the anti-downgrade/anti-replay state. Never empty — it is seeded from a trusted baseline (the embedded bundle's sequence, for the publisher authority) per FR-009b, so "no state yet" is not an accept-anything state. It is MONOTONIC — rollback does not lower it (FR-010 expresses rollback as a pinned exception plus a deny-list instead), and the only thing that re-baselines it is a higher trust epoch arriving in a binary release. @@ -169,12 +171,12 @@ The publisher cuts a new database release (new signature for a fresh campaign), ### Measurable Outcomes -- **SC-001**: 100% of tampered candidates (any byte of bundle or sidecar altered, or mismatched pair) are rejected before parsing, with the last-known-good database still serving and the reason surfaced — demonstrated by fixture in CI. +- **SC-001**: 100% of tampered candidates are rejected before parsing, with the last-known-good database still serving and the reason surfaced — demonstrated by fixture in CI. "Tampered" is defined precisely, because the signature covers the bundle bytes and NOT the sidecar's own encoding: any altered bundle byte, any altered signature value, any altered sidecar identity field (algorithm or key id, whose agreement with the manifest FR-003 already requires), and any mismatched bundle/sidecar pair. Byte changes to the sidecar that alter neither the signature value nor an identity field (whitespace, key order) are explicitly NOT required to be rejected — a detached signature cannot detect them — so the criterion is stated in terms of the fields that carry meaning rather than an unachievable "every byte of the sidecar". - **SC-002**: A validly-signed but lower-sequence bundle is never activated by any non-rollback path (drop, fetch, restart) — including on a fresh install or after a data-directory wipe, where the FR-009b seeded baseline stands in for the absent persisted watermark — and an unsigned candidate never advances the watermark nor survives the signature ratchet — demonstrated by fixture in CI. The sole exemption is the FR-009a embedded-default availability fallback, which serves as degraded coverage rather than counting as a downgrade. An explicit rollback succeeds, leaves an audit trail, does not lower the historical watermark, and the deny-listed release it rolled away from is not re-activated by the following refresh cycle. -- **SC-002a**: The watermark cannot be weaponized into a permanent update lockout, demonstrated by two fixtures: (a) a bundle signed by a to-be-revoked key at the MAXIMUM representable sequence, after which a binary carrying a bumped trust epoch and the surviving key successfully activates a new release — proving key-compromise recovery is possible rather than merely documented; (a2) immediately after that bump, an OLD pre-bump bundle carrying the previous epoch and signed by the still-trusted surviving key is REFUSED as stale-epoch replay rather than clearing the re-baselined watermark — the epoch bump must not itself become a downgrade window; and (b) an operator-authority-signed bundle at a high sequence that leaves the publisher authority's watermark untouched, so the next official publisher release still activates — proving the two signing authorities cannot lock each other out. A third fixture (c) covers the operator authority after a data-directory wipe: with a config-resident baseline declared, an older operator-signed bundle is refused; with no baseline declared, it may activate but the install reports degraded coverage rather than full trust. +- **SC-002a**: The watermark cannot be weaponized into a permanent update lockout, demonstrated by two fixtures: (a) a bundle signed by a to-be-revoked key at the MAXIMUM representable sequence, after which a binary carrying a bumped trust epoch and the surviving key successfully activates a new release — proving key-compromise recovery is possible rather than merely documented; (a2) immediately after that bump, an OLD pre-bump bundle carrying the previous epoch and signed by the still-trusted surviving key is REFUSED as stale-epoch replay rather than clearing the re-baselined watermark — the epoch bump must not itself become a downgrade window; and (b) with the operator authority declared active (FR-004b), an operator-authority-signed bundle at a high sequence that leaves the publisher authority's watermark untouched, so after the declaration is switched back the next official publisher release still activates — proving the two signing authorities cannot lock each other out. A fixture (b2) covers the declaration itself: the same operator-signed bundle dropped while the PUBLISHER authority is declared active is refused as "authority not active" with the publisher bundle still serving, so the choice of authority cannot be made by whoever writes the bundle path. A third fixture (c) covers the operator authority after a data-directory wipe: with a config-resident baseline declared, an older operator-signed bundle is refused; with no baseline declared, it may activate but the install reports degraded coverage rather than full trust. - **SC-003**: An air-gapped install (zero network, ever) reaches full P1 capability: verified activation of a file-dropped signed database, downgrade refusal, and rollback, with behavior byte-identical to a connected install's file-drop path. - **SC-004**: The shipped seed corpus contains at least 25 signatures spanning at least 8 distinct public campaign/technique classes, every one carrying provenance + redistributable license, and every gating signature backed by its eval sample pair; the canonical fixture of each class produces a hard-tier finding naming its TPA id. -- **SC-005**: The scan-eval CI gate (recall ≥ 0.90 gated categories, hard-negative FP ≤ 0.05) passes with the full seed corpus embedded AND actually scored — i.e. the gate run registers the bundle check and maps every new campaign category (FR-016a), proven by a negative control: removing a gating signature from the bundle must turn the gate red. Without that control the criterion is vacuous, since today's gate would pass with the bundle entirely absent. A deliberately-regressing corpus change is then blocked twice independently: at CI (cannot merge) and at activation (self-check rejects it) — both demonstrated by fixture. +- **SC-005**: The scan-eval CI gate (recall ≥ 0.90 gated categories, hard-negative FP ≤ 0.05) passes with the full seed corpus embedded AND actually scored — i.e. the gate run registers the bundle check and maps every new campaign category (FR-016a), proven by a **deterministic** negative control (FR-016b): removing a designated control signature from the bundle must turn the gate red. Without that control the criterion is vacuous, since today's gate would pass with the bundle entirely absent. A deliberately-regressing corpus change is then blocked twice independently: at CI (cannot merge) and at activation (self-check rejects it) — both demonstrated by fixture. - **SC-006**: Fleet freshness is answerable from telemetry: for opted-in installs, the distribution of active database version/sequence/source is reportable, enabling the metric "% of active installs on a database ≤ 30 days old" — a metric that is impossible to compute today. - **SC-007**: A newly published signature reaches detection within one daily refresh cycle on a fetch-enabled install (and immediately on manual drop), measured end to end: publish → active → the new signature's fixture fires. - **SC-008**: With a database at the design load ceilings (≥ 500 regex rules), p95 scan latency over a 100-tool server changes by less than 10% versus the 6-signature baseline, and bundle load-plus-verify completes under one second on commodity hardware. @@ -203,7 +205,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), ## Constitution Check *(note)* -Principle IV (Security by Default) governs this spec: the database is treated as a hostile input until proven otherwise (verify-before-parse, fail-closed to last-known-good, anti-downgrade watermark, mandatory signatures on the network path), and refresh can never silently widen approvals. Principle III (Configuration-Driven Architecture): `require_signed_bundle`, operator keys, and the existing bundle path/fetch settings live in `mcp_config.json` with env override and hot-reload; no hardcoded URLs or paths. Principle V (TDD): every rejection class (tamper, downgrade, unsigned, gate regression, ceiling breach) is built against failing fixtures first, and the corpus itself is test-gated by scan-eval. Principle I (Performance at Scale): verification, activation, and the post-activation re-scan stay off the scan hot path with atomic swaps (SC-008). +Principle IV (Security by Default) governs this spec: the database is treated as a hostile input until proven otherwise (verify-before-parse, fail-closed to last-known-good, anti-downgrade watermark, mandatory signatures on the network path), and refresh can never silently widen approvals. Principle III (Configuration-Driven Architecture): `require_signed_bundle`, operator keys, the active-authority declaration and its baseline, and the existing bundle path/fetch settings live in `mcp_config.json` with env override and hot-reload; no hardcoded URLs or paths. Principle V (TDD): every rejection class (tamper, downgrade, unsigned, gate regression, ceiling breach) is built against failing fixtures first, and the corpus itself is test-gated by scan-eval. Principle I (Performance at Scale): verification, activation, and the post-activation re-scan stay off the scan hot path with atomic swaps (SC-008). ## Commit Message Conventions *(mandatory)* From eea76d30f751972c4e0b6f528dcc56a1817946b3 Mon Sep 17 00:00:00 2001 From: Algis Dumbris Date: Mon, 24 Aug 2026 06:44:41 +0300 Subject: [PATCH 08/15] docs(specs): bind rollback to a fingerprint and narrow the degraded fallback (101-tpa-db) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cross-model review (opencode / gpt-5.6-sol) round 2 on specs/101-tpa-db/spec.md. All three findings verified against the document and gate.go before changing anything. ## Changes - FR-010: the rollback target must be identified by content fingerprint, not by pathname alone. Rollback is the one path that deliberately bypasses the watermark, so naming its target by an attacker-writable path lets a bundle-path attacker choose what the bypass lands on — substituting a genuinely publisher-signed, current-epoch bundle at sequence 5 for the sequence-90 artifact the operator intended, which then gets PINNED. The operator now supplies the expected fingerprint, the path is only a hint, a mismatch is refused, and a rollback target must sit at or above the FR-009b seeded baseline (the embedded default excepted, since it is that baseline). - FR-009a condition (b): narrowed to a missing OPERATOR-authority bundle. As written it degraded coverage whenever any previously-activated external bundle went missing, including the ordinary case of a publisher external file removed after a binary upgrade shipped a strictly newer embedded corpus — where condition (a) already answers the question on the same axis and says full coverage. - US2 independent test: the negative control now runs over the FR-016b control scope instead of re-running the full-corpus command. One miss out of a 25-signature corpus leaves aggregate recall near 0.96, above the 0.90 threshold, so the full-corpus form contradicted FR-016b and would have proven nothing. - US1 independent test, SC-002, FR-020: assertions and rejection reason for the rollback-target mismatch. ## Testing - Docs-only change; no Go sources touched. - Recall/FP semantics re-checked in cmd/scan-eval/gate.go (OverallRecall is gatedDetected/gatedMalicious aggregated across gated categories). --- specs/101-tpa-db/spec.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/specs/101-tpa-db/spec.md b/specs/101-tpa-db/spec.md index d42ea97a..1910b133 100644 --- a/specs/101-tpa-db/spec.md +++ b/specs/101-tpa-db/spec.md @@ -40,7 +40,7 @@ An operator receives a TPA signature database out of band — a `scanner-bundle. **Why this priority**: The database is a code-adjacent security input: whoever controls its bytes controls what the scanner ignores. Without signing and anti-rollback, the "refreshable signature DB" pillar creates a new supply-chain surface (swap the file → blind the scanner; replay an old bundle → resurrect a fixed blind spot). Signing + sequence + fail-closed loading is what makes every other story safe to ship, so it goes first. -**Independent Test**: Fully offline, no live upstream. Sign a fixture bundle with a test key configured as a trust anchor; drop bundle+signature and assert activation with `signature_verified=true` and the new sequence recorded. Then assert each of the following is rejected with the active bundle unchanged and a machine-readable reason: (a) one flipped byte in the bundle, (b) a signature from a non-trusted key, (c) a missing sidecar with `require_signed_bundle=true`, (d) a validly-signed bundle whose sequence is lower than the active one, (e) a validly-signed bundle failing the activation self-check, (f) an unsigned bundle dropped after a signed one was active (ratchet), (g) an unsigned bundle claiming a huge sequence — assert the watermark is unchanged so a later legitimate signed release still activates, and (h) on a FRESH install with no persisted state, a genuinely publisher-signed current-epoch bundle whose sequence is older than the binary's embedded corpus — assert it is refused by the FR-009b seeded baseline rather than activating at full coverage, and (i) with an operator key configured but the publisher authority declared active, a valid operator-signed bundle clearing the operator baseline — assert it is refused as "authority not active" (FR-004b) and the publisher bundle keeps serving. Finally, invoke the explicit rollback action and assert the embedded default is restored, the override is logged, the high-watermark is unchanged, and a subsequent refresh does not re-activate the deny-listed release. +**Independent Test**: Fully offline, no live upstream. Sign a fixture bundle with a test key configured as a trust anchor; drop bundle+signature and assert activation with `signature_verified=true` and the new sequence recorded. Then assert each of the following is rejected with the active bundle unchanged and a machine-readable reason: (a) one flipped byte in the bundle, (b) a signature from a non-trusted key, (c) a missing sidecar with `require_signed_bundle=true`, (d) a validly-signed bundle whose sequence is lower than the active one, (e) a validly-signed bundle failing the activation self-check, (f) an unsigned bundle dropped after a signed one was active (ratchet), (g) an unsigned bundle claiming a huge sequence — assert the watermark is unchanged so a later legitimate signed release still activates, and (h) on a FRESH install with no persisted state, a genuinely publisher-signed current-epoch bundle whose sequence is older than the binary's embedded corpus — assert it is refused by the FR-009b seeded baseline rather than activating at full coverage, and (i) with an operator key configured but the publisher authority declared active, a valid operator-signed bundle clearing the operator baseline — assert it is refused as "authority not active" (FR-004b) and the publisher bundle keeps serving. Finally, invoke the explicit rollback action and assert the embedded default is restored, the override is logged, the high-watermark is unchanged, and a subsequent refresh does not re-activate the deny-listed release; and assert that a rollback whose named file has been substituted (valid signature, current epoch, but a fingerprint other than the one the operator supplied) is REFUSED as a target mismatch rather than activated through the watermark bypass (FR-010). **Acceptance Scenarios**: @@ -61,7 +61,7 @@ A security-conscious developer installs mcpproxy and gets, out of the box, a sig **Why this priority**: The format (US1) is worthless empty, and the refresh channel (US3) has nothing to carry without a corpus. This is the "database" in tpa-db. It is P2 only because the P1 trust machinery must exist before a larger corpus becomes an attractive distribution target. -**Independent Test**: Run the corpus build and assert every signature carries id, category, detector(s), provenance source, and license; assert every gated signature has ≥1 labeled malicious eval sample and ≥1 category-matched hard-negative in the eval dataset; run `go run ./cmd/scan-eval --corpus specs/065-evaluation-foundation/datasets/detect_corpus_v1.json --gate --min-recall 0.90 --max-fp 0.05` with the new embedded default and assert PASS — and, because that command today scores built-in checks only, assert the accompanying NEGATIVE CONTROL that proves the bundle is being scored at all: with the designated FR-016b control signature removed from the bundle, the same command must FAIL with the gate-breach exit code (the control signature is the one whose sample no built-in check and no other signature detects, so its removal necessarily crosses the threshold rather than being absorbed by aggregate recall headroom). Feed each campaign class's canonical fixture through the offline scanner and assert a hard-tier finding naming the TPA id. +**Independent Test**: Run the corpus build and assert every signature carries id, category, detector(s), provenance source, and license; assert every gated signature has ≥1 labeled malicious eval sample and ≥1 category-matched hard-negative in the eval dataset; run `go run ./cmd/scan-eval --corpus specs/065-evaluation-foundation/datasets/detect_corpus_v1.json --gate --min-recall 0.90 --max-fp 0.05` with the new embedded default and assert PASS — and, because that command today scores built-in checks only, assert the accompanying NEGATIVE CONTROL that proves the bundle is being scored at all: with the designated FR-016b control signature removed from the bundle, the gate must exit with the gate-breach code when run over the FR-016b control scope (the corpus restricted to that signature's category, or the equivalent per-category recall assertion). The control MUST NOT be expressed as "remove the signature and re-run the full-corpus command" — one miss out of a 25-signature corpus leaves aggregate recall at roughly 0.96, comfortably above the 0.90 threshold, so the full-corpus run would stay green and the control would prove nothing. Feed each campaign class's canonical fixture through the offline scanner and assert a hard-tier finding naming the TPA id. **Acceptance Scenarios**: @@ -127,8 +127,8 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **FR-008**: The full validate-before-activate order MUST be: byte-size ceiling → signature over the raw bytes (per policy, FR-006) → parse manifest → rule-count ceiling → epoch match (FR-005a) → sequence/fingerprint anti-downgrade (FR-009) → version compatibility → compile-all-or-reject → runnable-rules > 0 → activation self-check (Spec 087 FR-008) → atomic activation. FR-011's two ceilings MUST therefore be split across the pipeline rather than applied as one stage: byte size is measurable on raw bytes and so guards the verification step itself, while rule count is only knowable after parsing and so cannot precede it. The same reasoning applies to sequence, a manifest field necessarily read after parsing: what MUST precede parsing is signature verification, not the sequence or rule-count check — an unverified candidate's manifest is never trusted for any decision. Any failure at any stage MUST keep last-known-good and record a machine-readable reason and timestamp (surfaced via the existing bundle status), never an empty or partial rule set. - **FR-009**: The system MUST persist the highest sequence ever activated — the **watermark**, scoped per signing authority and trust epoch per FR-004a/FR-005a — and refuse any candidate with a lower sequence, or an equal sequence with a different content fingerprint, as a downgrade/replay, unless an explicit operator rollback (FR-010) is in effect. **The watermark is a property of SIGNED artifacts only**: `sequence` lives in the manifest, so an unsigned candidate's sequence is attacker-chosen. An unsigned/unverified candidate MUST NOT advance the watermark (otherwise a single unsigned drop claiming a huge sequence permanently locks out every legitimate signed release), and once an install has ever activated a signature-verified bundle, unsigned candidates MUST be refused thereafter regardless of `require_signed_bundle` — a signature ratchet, so signing cannot be stripped back off an install by whoever can write the bundle path. The embedded default is exempt from BOTH the watermark and the ratchet — it is the availability fallback of last resort and must never be locked out (scanning must never stop), which is exactly why FR-009a demotes it to degraded coverage instead of refusing it. - **FR-009b**: Each authority's watermark MUST be SEEDED from a trusted baseline rather than starting empty, because an absent watermark accepts anything. Otherwise the anti-downgrade guarantee simply does not exist at first use or after the data directory is reset: an attacker who can write the bundle path drops an OLD but genuinely publisher-signed, current-epoch bundle — say sequence 5 while the shipping release is at 100 — and with no watermark to compare against it activates as a fully signature-verified bundle at FULL coverage, silently reinstating every blind spot fixed between 5 and 100. Neither the FR-009 ratchet nor the FR-010 pin/deny-list helps here; both only bind once persisted state exists. Therefore: the **publisher** authority's watermark MUST be seeded from the sequence of the bundle embedded in the running binary — a candidate older than what the binary already ships is strictly worse than the fallback and MUST be refused — and this baseline MUST be re-applied on every startup, so wiping the data directory cannot lower it below the binary's own corpus. The **operator** authority has no binary-trusted baseline available, so its baseline MUST be declared explicitly in configuration alongside the operator key, and it MUST live in `mcp_config.json` rather than the data directory — that is what makes it durable across the wipe, since a baseline stored in resettable state provides no protection precisely when it is needed. A silent zero default MUST NOT be used: with a zero baseline the operator authority has NO anti-downgrade after state loss at all, and a bundle-path attacker — who is squarely inside the threat model, unlike the config-write attacker — could replay any older genuinely operator-signed bundle at full coverage on a fresh or wiped install. Therefore: if an operator key is configured WITHOUT a baseline, the system MUST treat operator-authority anti-downgrade as unsupported and mark coverage degraded (per FR-007) while an operator-signed bundle is active, rather than silently presenting it as fully trusted. The data-directory watermark then tracks activations above the configured baseline and may only raise the effective minimum, never lower it. The seeded baseline also makes the epoch re-baseline of FR-005a safe: after an epoch bump the new epoch's watermark starts at the authority's baseline, not at zero. -- **FR-009a**: When no valid external candidate is available, the embedded default MUST still serve so scanning never stops, even if its sequence is below the watermark. Where the install has NO retained signed watermark and no missing previously-active external bundle (the ordinary fresh install), this is simply the normal full-coverage state. Two — and only two — conditions demote that fallback to degraded coverage (auto-approval suspended, as in FR-007's unsigned-external case): (a) the embedded default's sequence is below the **publisher** authority's retained watermark, or (b) an external bundle of the currently active authority (FR-004b) was previously activated and is no longer available or no longer valid. The comparison in (a) MUST be same-namespace: the embedded default is a publisher artifact, and sequences of different authorities are independent counters, so it MUST be compared ONLY against the publisher watermark and never against an operator watermark — a cross-authority numeric comparison is meaningless and would, for an operator corpus at sequence 900 against an embedded publisher corpus at 100, permanently suspend auto-approval on an install whose publisher fallback is perfectly current. Condition (b) is what a cross-authority comparison was reaching for and states it directly: the trigger is evidence that a corpus the install was actually running has gone missing, not an arithmetic accident. In either case the retained watermark, the active authority, and the degraded reason MUST be surfaced, and it MUST NOT count as an anti-downgrade violation. This below-watermark fallback is the single explicit exemption to SC-002. -- **FR-010**: An explicit rollback action MUST let an operator revert to the embedded default or a named bundle file: the target is re-validated (signature policy and self-check still apply), activated despite a lower sequence, and the override is loudly logged and visible in bundle status. Rollback MUST NOT lower the historical high-watermark: the watermark is monotonic and normal activation MUST still require exceeding it. Rollback is instead expressed as a **pinned exception** — the operator pins the rolled-back artifact by fingerprint, and the (sequence, fingerprint) being rolled away from is deny-listed so the very release the operator just rejected cannot be re-activated by the next refresh cycle (and neither can any intermediate release between the pin and the watermark). Clearing the pin returns the install to normal watermark-governed activation. +- **FR-009a**: When no valid external candidate is available, the embedded default MUST still serve so scanning never stops, even if its sequence is below the watermark. Where the install has NO retained signed watermark and no missing previously-active external bundle (the ordinary fresh install), this is simply the normal full-coverage state. Two — and only two — conditions demote that fallback to degraded coverage (auto-approval suspended, as in FR-007's unsigned-external case): (a) the embedded default's sequence is below the **publisher** authority's retained watermark, or (b) the **operator** authority is the active one (FR-004b) and its previously-activated external bundle is no longer available or no longer valid. The comparison in (a) MUST be same-namespace: the embedded default is a publisher artifact, and sequences of different authorities are independent counters, so it MUST be compared ONLY against the publisher watermark and never against an operator watermark — a cross-authority numeric comparison is meaningless and would, for an operator corpus at sequence 900 against an embedded publisher corpus at 100, permanently suspend auto-approval on an install whose publisher fallback is perfectly current. Condition (b) is what a cross-authority comparison was reaching for, restricted to the case where the fallback cannot be shown to be at least as fresh as what was lost: an operator corpus is not comparable to the publisher's, so a missing one leaves the install with no evidence the embedded default covers it. It MUST NOT extend to a missing PUBLISHER external bundle, because there condition (a) already decides the question on the same axis — an embedded default at or above the publisher watermark is demonstrably no older than the bundle that went missing (the ordinary "external file removed after a binary upgrade shipped a newer corpus" case) and MUST stay full coverage rather than being degraded merely because an older external file once existed. In either case the retained watermark, the active authority, and the degraded reason MUST be surfaced, and it MUST NOT count as an anti-downgrade violation. This below-watermark fallback is the single explicit exemption to SC-002. +- **FR-010**: An explicit rollback action MUST let an operator revert to the embedded default or a named bundle file: the target is re-validated (signature policy and self-check still apply), activated despite a lower sequence, and the override is loudly logged and visible in bundle status. **The rollback target MUST be identified by content fingerprint, not by pathname alone.** Rollback is the one path that deliberately bypasses the watermark, so identifying its target by an attacker-writable path hands the adversary the choice of what the bypass lands on: with a legitimate bundle at sequence 100 active and the operator intending to revert to a known-good sequence 90, an attacker who can write the bundle path — in scope, unlike the config-write attacker — swaps the named file for a genuinely publisher-signed, current-epoch bundle at sequence 5 before the rollback reads it. It passes signature, epoch, compilation, and plausibly the self-check; the watermark that would normally refuse it is switched off by the rollback itself; and the result is then PINNED, so the downgrade persists across refresh cycles. FR-006's read-once rule closes the verify/parse race but not substitution before the read. Therefore the operator MUST supply the expected fingerprint (as offered by bundle status history), the path is only a hint for where to find those bytes, and a loaded artifact whose fingerprint differs MUST be refused as a rollback-target mismatch rather than activated. A rollback target MUST additionally still be at or above the FR-009b seeded baseline (the running binary's own embedded corpus) — the embedded default itself being the one target always permitted, since it IS that baseline and is the availability fallback of last resort. Rollback MUST NOT lower the historical high-watermark: the watermark is monotonic and normal activation MUST still require exceeding it. Rollback is instead expressed as a **pinned exception** — the operator pins the rolled-back artifact by fingerprint, and the (sequence, fingerprint) being rolled away from is deny-listed so the very release the operator just rejected cannot be re-activated by the next refresh cycle (and neither can any intermediate release between the pin and the watermark). Clearing the pin returns the install to normal watermark-governed activation. - **FR-011**: The loader MUST enforce load ceilings (maximum bundle byte size and maximum rule count, fixed in design) and reject a candidate exceeding them as a whole; all patterns remain RE2-only. A database at the ceilings MUST NOT measurably regress the scan hot path (see SC-008). - **FR-011a**: Activation MUST be crash-consistent, not merely atomic in memory. Spec 087 FR-010 makes the rule-set swap atomic with respect to concurrent scans; that says nothing about persisted state, and a process death between "activated" and "watermark written" reopens exactly the replay window FR-009 exists to close (while writing the watermark first can leave the install with only an older recoverable bundle). The activation record — active fingerprint, sequence, per-authority watermarks and their epochs, the fingerprint each watermark was set by, signature-verified state, any FR-010 pin, and the FR-010 deny-list — MUST be persisted as ONE transaction, and the recoverable last-known-good bundle MUST be durable before the in-memory pointer is published. The deny-list and watermark fingerprint are load-bearing members of that transaction, not bookkeeping: a crash that preserved a rollback while losing the deny-list would let the next refresh immediately re-activate the exact release the operator had just rejected, silently undoing the rollback. Startup, refresh, hot-reload, and rollback MUST all serialize through that same transaction, so any crash leaves the install on a consistent (bundle, watermark) pair. @@ -150,7 +150,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), **Observability, telemetry & docs** -- **FR-020**: Bundle status on all existing surfaces (CLI security commands, REST security overview, Web UI) MUST additionally report sequence, signing authority and trust epoch, signature-verified state, and staleness (age since `generated_at`), alongside the existing version/fingerprint/source/load-error fields, and MUST show the last rejection reason (tamper, downgrade, stale-epoch replay, unsigned-refused-by-ratchet, authority-not-active, gate failure) so an operator can tell "current and verified" from "stuck on last-known-good and why". Where the active database has demoted the install to DEGRADED coverage (FR-007 unsigned-active, FR-009a embedded fallback), that state and its reason MUST be surfaced too — a silently suspended `scan`-mode auto-approval is otherwise indistinguishable from a quiet install. +- **FR-020**: Bundle status on all existing surfaces (CLI security commands, REST security overview, Web UI) MUST additionally report sequence, signing authority and trust epoch, signature-verified state, and staleness (age since `generated_at`), alongside the existing version/fingerprint/source/load-error fields, and MUST show the last rejection reason (tamper, downgrade, stale-epoch replay, unsigned-refused-by-ratchet, authority-not-active, rollback-target mismatch, gate failure) so an operator can tell "current and verified" from "stuck on last-known-good and why". Where the active database has demoted the install to DEGRADED coverage (FR-007 unsigned-active, FR-009a embedded fallback), that state and its reason MUST be surfaced too — a silently suspended `scan`-mode auto-approval is otherwise indistinguishable from a quiet install. - **FR-021**: The existing anonymous TPA-scanner telemetry MUST gain the active bundle version, sequence, and source (and signature-verified state) so database adoption and freshness are measurable fleet-wide; the existing telemetry opt-out MUST fully cover these fields, and no new identifying data is introduced. This is NOT a drop-in addition: `internal/telemetry/anonymity.go`'s schema-v8 backstop hard-whitelists the `tpa_scanner` sub-object to `{scans_completed, scans_failed, scans_with_findings, findings}` and requires every scalar to be a non-negative integer, so a string `bundle_version`/`source` or a boolean `signature_verified` would be rejected before transmit as "carries a key outside the whitelist". The design MUST therefore extend that backstop with explicitly TYPED entries (integer `sequence`, enum-bounded `source` ∈ {embedded, file, fetch}, boolean signature-verified) rather than relaxing it to accept arbitrary keys or free strings. `bundle_version` MUST be constrained to publisher-issued values (unrecognized/operator-built corpus versions reported as a single `other` bucket): the backstop exists precisely to stop a bespoke, near-unique string from acting as a fleet identifier, and a self-built corpus version would be exactly that. - **FR-022**: Documentation MUST cover the database lifecycle end to end for operators (verify, drop, fetch, roll back, read status) and for contributors (author a signature, provenance/license rules, eval-sample requirements, how the gate blocks a bad corpus), updating the existing security-quarantine/tool-scanner docs. @@ -172,7 +172,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), ### Measurable Outcomes - **SC-001**: 100% of tampered candidates are rejected before parsing, with the last-known-good database still serving and the reason surfaced — demonstrated by fixture in CI. "Tampered" is defined precisely, because the signature covers the bundle bytes and NOT the sidecar's own encoding: any altered bundle byte, any altered signature value, any altered sidecar identity field (algorithm or key id, whose agreement with the manifest FR-003 already requires), and any mismatched bundle/sidecar pair. Byte changes to the sidecar that alter neither the signature value nor an identity field (whitespace, key order) are explicitly NOT required to be rejected — a detached signature cannot detect them — so the criterion is stated in terms of the fields that carry meaning rather than an unachievable "every byte of the sidecar". -- **SC-002**: A validly-signed but lower-sequence bundle is never activated by any non-rollback path (drop, fetch, restart) — including on a fresh install or after a data-directory wipe, where the FR-009b seeded baseline stands in for the absent persisted watermark — and an unsigned candidate never advances the watermark nor survives the signature ratchet — demonstrated by fixture in CI. The sole exemption is the FR-009a embedded-default availability fallback, which serves as degraded coverage rather than counting as a downgrade. An explicit rollback succeeds, leaves an audit trail, does not lower the historical watermark, and the deny-listed release it rolled away from is not re-activated by the following refresh cycle. +- **SC-002**: A validly-signed but lower-sequence bundle is never activated by any non-rollback path (drop, fetch, restart) — including on a fresh install or after a data-directory wipe, where the FR-009b seeded baseline stands in for the absent persisted watermark — and an unsigned candidate never advances the watermark nor survives the signature ratchet — demonstrated by fixture in CI. The sole exemption is the FR-009a embedded-default availability fallback, which serves as degraded coverage rather than counting as a downgrade. An explicit rollback succeeds, leaves an audit trail, does not lower the historical watermark, activates only the fingerprint the operator named (a substituted file at the same path is refused), and the deny-listed release it rolled away from is not re-activated by the following refresh cycle. - **SC-002a**: The watermark cannot be weaponized into a permanent update lockout, demonstrated by two fixtures: (a) a bundle signed by a to-be-revoked key at the MAXIMUM representable sequence, after which a binary carrying a bumped trust epoch and the surviving key successfully activates a new release — proving key-compromise recovery is possible rather than merely documented; (a2) immediately after that bump, an OLD pre-bump bundle carrying the previous epoch and signed by the still-trusted surviving key is REFUSED as stale-epoch replay rather than clearing the re-baselined watermark — the epoch bump must not itself become a downgrade window; and (b) with the operator authority declared active (FR-004b), an operator-authority-signed bundle at a high sequence that leaves the publisher authority's watermark untouched, so after the declaration is switched back the next official publisher release still activates — proving the two signing authorities cannot lock each other out. A fixture (b2) covers the declaration itself: the same operator-signed bundle dropped while the PUBLISHER authority is declared active is refused as "authority not active" with the publisher bundle still serving, so the choice of authority cannot be made by whoever writes the bundle path. A third fixture (c) covers the operator authority after a data-directory wipe: with a config-resident baseline declared, an older operator-signed bundle is refused; with no baseline declared, it may activate but the install reports degraded coverage rather than full trust. - **SC-003**: An air-gapped install (zero network, ever) reaches full P1 capability: verified activation of a file-dropped signed database, downgrade refusal, and rollback, with behavior byte-identical to a connected install's file-drop path. - **SC-004**: The shipped seed corpus contains at least 25 signatures spanning at least 8 distinct public campaign/technique classes, every one carrying provenance + redistributable license, and every gating signature backed by its eval sample pair; the canonical fixture of each class produces a hard-tier finding naming its TPA id. From 9feda6ea4a8b68802bae2071fb66e1df1a10923b Mon Sep 17 00:00:00 2001 From: Algis Dumbris Date: Mon, 24 Aug 2026 06:49:28 +0300 Subject: [PATCH 09/15] docs(specs): cap the sidecar, scope the rollback floor, pin the gate control (101-tpa-db) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cross-model review (opencode / gpt-5.6-sol) round 3 on specs/101-tpa-db/spec.md. All three findings verified against the document and gate.go before changing anything. ## Changes - FR-011 + FR-008 + edge cases: the sidecar gets its own small byte cap and a bounded read, applied before it is parsed, on both the file-drop and fetch paths. The sidecar is the one artifact that must be read before any signature is verified — its algorithm id, key id, and signature value are verification's own inputs — so FR-006's verify-before-parse protection does not cover it, and a bundle-path attacker could drop a multi-gigabyte sidecar next to an otherwise valid bundle and stall or exhaust the refresh path. The pipeline order and the "two ceilings" note are updated accordingly. - FR-010: the rollback floor is now its OWN authority's FR-009b baseline, not the publisher one in both cases. As written it compared an operator-signed rollback target at sequence 9 against the publisher embedded corpus at 100 and would have refused every legitimate operator rollback, contradicting the per-authority isolation of FR-004a. - FR-016b: "restrict the control to its category" is not by itself provably failing — decide() breaches only on OverallRecall < minRecall, so a control category with ten gated malicious samples lands on exactly 0.90 after one miss and stays green on the boundary. The canonical construction is now a control corpus whose gated malicious set is the control sample alone; any other scope must compute and assert the recall drop rather than assume it. ## Testing - Docs-only change; no Go sources touched. - Boundary semantics re-checked in cmd/scan-eval/gate.go decide() (strict `<` against min-recall) and the per-authority baseline rules in FR-009b. --- specs/101-tpa-db/spec.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/specs/101-tpa-db/spec.md b/specs/101-tpa-db/spec.md index 1910b133..6228caab 100644 --- a/specs/101-tpa-db/spec.md +++ b/specs/101-tpa-db/spec.md @@ -101,7 +101,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **Anti-downgrade state lost** (data directory wiped/reset): the persisted watermark resets to the FR-009b seeded baseline — NOT to zero, so a wiped install still refuses any publisher-signed bundle older than the corpus its own binary ships. For the operator authority the same protection holds only if the operator declared a config-resident baseline; without one, operator-authority anti-downgrade does not survive the wipe and the install runs degraded while an operator-signed bundle is active, by FR-009b, rather than pretending to a guarantee it does not have. What a wipe does lose is the FR-009 signature-ratchet state and any FR-010 pin/deny-list (both data-dir resident), so a wiped install accepts an unsigned drop again. This is accepted — the ratchet is best-effort hardening on top of signing, not the sole defense, and a wiped data dir already resets approvals. Operators who need this closed absolutely set `require_signed_bundle`, which is config-resident rather than data-dir-resident and therefore survives the wipe. - **Equal sequence, different bytes** (two distinct bundles claiming the same sequence): refused as suspicious unless it is the currently-active fingerprint (idempotent re-drop stays a no-op per Spec 087 FR-011). - **Clock skew**: no validity windows or expiry in v1 — sequence ordering, not wall-clock time, is the trust signal; `generated_at` is advisory freshness only. -- **Huge or adversarial database** (thousands of rules, pathological regex): size and rule-count ceilings are enforced at load; every pattern remains RE2 (linear-time, no catastrophic backtracking by construction); a candidate exceeding ceilings is rejected whole, keeping last-known-good. +- **Huge or adversarial database** (thousands of rules, pathological regex) **or an adversarial sidecar**: the sidecar byte cap (read before it is parsed, since verification's own inputs come from it), plus the bundle size and rule-count ceilings, are enforced at load; every pattern remains RE2 (linear-time, no catastrophic backtracking by construction); a candidate exceeding ceilings is rejected whole, keeping last-known-good. - **Post-activation re-scan storms**: the re-scan is off the hot path, rate-limited, runs once per activation over cached metadata only (no upstream reconnects), and produces at most one review finding per tool per bundle activation — no notification storm. - **Signature removed from a newer bundle** (false-positive retired): tools previously held by it are NOT auto-approved by refresh; the held state persists for human review (state changes only flow toward review, never silently toward approval). @@ -124,12 +124,12 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **FR-006**: For any candidate carrying a signature, verification MUST run over the exact raw candidate bytes BEFORE those bytes are parsed as JSON or any pattern is compiled; a candidate failing verification MUST be rejected without further processing, keeping last-known-good (extends the Spec 086 fail-closed pipeline with a new first stage). The bytes that are verified MUST be the same bytes that are subsequently parsed (read once into memory, verify, then parse that buffer) — the file MUST NOT be re-read from disk after verification, or a writer racing the loader could swap content between verify and parse. - **FR-007**: Signing policy MUST be: network-fetched candidates ALWAYS require a valid signature; file-drop candidates are verified whenever a sidecar is present; a new `require_signed_bundle` setting (default off for back-compat with existing unsigned drops) makes the sidecar mandatory for file drops too; the embedded default needs no sidecar (its integrity rides on binary distribution integrity). Bundle status MUST always report whether the active bundle was signature-verified. While an unsigned EXTERNAL (file-dropped or fetched) bundle is active, the scan coverage the approval gates consume MUST be marked degraded so `scan`-mode auto-approval is suspended and changes route to human review — today `internal/security/scanner/inprocess.go` sets `coverageOK` from mere bundle *presence*, so without this an attacker-supplied unsigned corpus would keep silently auto-approving under weakened rules. This degradation rule applies to EXTERNAL candidates ONLY: the embedded default is binary-trusted (its integrity rides on the signed binary distribution, not on a sidecar) and MUST be full-coverage on an ordinary install, or every fresh install would ship with `scan` mode silently disabled. The embedded default is degraded only in the FR-009a case — when it is serving as a fallback BELOW a retained signed watermark, i.e. when the install has evidence a newer signed corpus should be active. This requirement supersedes Spec 087 FR-014's precondition: because FR-004 makes publisher keys embedded in the binary part of the trust set, a fetch is no longer refused merely because the operator configured no key of their own; the "never activate an unverified fetched bundle" guarantee of Spec 087 FR-013/FR-014 is preserved unchanged. -- **FR-008**: The full validate-before-activate order MUST be: byte-size ceiling → signature over the raw bytes (per policy, FR-006) → parse manifest → rule-count ceiling → epoch match (FR-005a) → sequence/fingerprint anti-downgrade (FR-009) → version compatibility → compile-all-or-reject → runnable-rules > 0 → activation self-check (Spec 087 FR-008) → atomic activation. FR-011's two ceilings MUST therefore be split across the pipeline rather than applied as one stage: byte size is measurable on raw bytes and so guards the verification step itself, while rule count is only knowable after parsing and so cannot precede it. The same reasoning applies to sequence, a manifest field necessarily read after parsing: what MUST precede parsing is signature verification, not the sequence or rule-count check — an unverified candidate's manifest is never trusted for any decision. Any failure at any stage MUST keep last-known-good and record a machine-readable reason and timestamp (surfaced via the existing bundle status), never an empty or partial rule set. +- **FR-008**: The full validate-before-activate order MUST be: sidecar byte cap → parse sidecar → bundle byte-size ceiling → signature over the raw bytes (per policy, FR-006) → parse manifest → rule-count ceiling → epoch match (FR-005a) → sequence/fingerprint anti-downgrade (FR-009) → version compatibility → compile-all-or-reject → runnable-rules > 0 → activation self-check (Spec 087 FR-008) → atomic activation. FR-011's ceilings MUST therefore be split across the pipeline rather than applied as one stage: the sidecar cap comes first because the sidecar is read before any signature exists to check, bundle byte size is measurable on raw bytes and so guards the verification step itself, while rule count is only knowable after parsing and so cannot precede it. The same reasoning applies to sequence, a manifest field necessarily read after parsing: what MUST precede parsing is signature verification, not the sequence or rule-count check — an unverified candidate's manifest is never trusted for any decision. Any failure at any stage MUST keep last-known-good and record a machine-readable reason and timestamp (surfaced via the existing bundle status), never an empty or partial rule set. - **FR-009**: The system MUST persist the highest sequence ever activated — the **watermark**, scoped per signing authority and trust epoch per FR-004a/FR-005a — and refuse any candidate with a lower sequence, or an equal sequence with a different content fingerprint, as a downgrade/replay, unless an explicit operator rollback (FR-010) is in effect. **The watermark is a property of SIGNED artifacts only**: `sequence` lives in the manifest, so an unsigned candidate's sequence is attacker-chosen. An unsigned/unverified candidate MUST NOT advance the watermark (otherwise a single unsigned drop claiming a huge sequence permanently locks out every legitimate signed release), and once an install has ever activated a signature-verified bundle, unsigned candidates MUST be refused thereafter regardless of `require_signed_bundle` — a signature ratchet, so signing cannot be stripped back off an install by whoever can write the bundle path. The embedded default is exempt from BOTH the watermark and the ratchet — it is the availability fallback of last resort and must never be locked out (scanning must never stop), which is exactly why FR-009a demotes it to degraded coverage instead of refusing it. - **FR-009b**: Each authority's watermark MUST be SEEDED from a trusted baseline rather than starting empty, because an absent watermark accepts anything. Otherwise the anti-downgrade guarantee simply does not exist at first use or after the data directory is reset: an attacker who can write the bundle path drops an OLD but genuinely publisher-signed, current-epoch bundle — say sequence 5 while the shipping release is at 100 — and with no watermark to compare against it activates as a fully signature-verified bundle at FULL coverage, silently reinstating every blind spot fixed between 5 and 100. Neither the FR-009 ratchet nor the FR-010 pin/deny-list helps here; both only bind once persisted state exists. Therefore: the **publisher** authority's watermark MUST be seeded from the sequence of the bundle embedded in the running binary — a candidate older than what the binary already ships is strictly worse than the fallback and MUST be refused — and this baseline MUST be re-applied on every startup, so wiping the data directory cannot lower it below the binary's own corpus. The **operator** authority has no binary-trusted baseline available, so its baseline MUST be declared explicitly in configuration alongside the operator key, and it MUST live in `mcp_config.json` rather than the data directory — that is what makes it durable across the wipe, since a baseline stored in resettable state provides no protection precisely when it is needed. A silent zero default MUST NOT be used: with a zero baseline the operator authority has NO anti-downgrade after state loss at all, and a bundle-path attacker — who is squarely inside the threat model, unlike the config-write attacker — could replay any older genuinely operator-signed bundle at full coverage on a fresh or wiped install. Therefore: if an operator key is configured WITHOUT a baseline, the system MUST treat operator-authority anti-downgrade as unsupported and mark coverage degraded (per FR-007) while an operator-signed bundle is active, rather than silently presenting it as fully trusted. The data-directory watermark then tracks activations above the configured baseline and may only raise the effective minimum, never lower it. The seeded baseline also makes the epoch re-baseline of FR-005a safe: after an epoch bump the new epoch's watermark starts at the authority's baseline, not at zero. - **FR-009a**: When no valid external candidate is available, the embedded default MUST still serve so scanning never stops, even if its sequence is below the watermark. Where the install has NO retained signed watermark and no missing previously-active external bundle (the ordinary fresh install), this is simply the normal full-coverage state. Two — and only two — conditions demote that fallback to degraded coverage (auto-approval suspended, as in FR-007's unsigned-external case): (a) the embedded default's sequence is below the **publisher** authority's retained watermark, or (b) the **operator** authority is the active one (FR-004b) and its previously-activated external bundle is no longer available or no longer valid. The comparison in (a) MUST be same-namespace: the embedded default is a publisher artifact, and sequences of different authorities are independent counters, so it MUST be compared ONLY against the publisher watermark and never against an operator watermark — a cross-authority numeric comparison is meaningless and would, for an operator corpus at sequence 900 against an embedded publisher corpus at 100, permanently suspend auto-approval on an install whose publisher fallback is perfectly current. Condition (b) is what a cross-authority comparison was reaching for, restricted to the case where the fallback cannot be shown to be at least as fresh as what was lost: an operator corpus is not comparable to the publisher's, so a missing one leaves the install with no evidence the embedded default covers it. It MUST NOT extend to a missing PUBLISHER external bundle, because there condition (a) already decides the question on the same axis — an embedded default at or above the publisher watermark is demonstrably no older than the bundle that went missing (the ordinary "external file removed after a binary upgrade shipped a newer corpus" case) and MUST stay full coverage rather than being degraded merely because an older external file once existed. In either case the retained watermark, the active authority, and the degraded reason MUST be surfaced, and it MUST NOT count as an anti-downgrade violation. This below-watermark fallback is the single explicit exemption to SC-002. -- **FR-010**: An explicit rollback action MUST let an operator revert to the embedded default or a named bundle file: the target is re-validated (signature policy and self-check still apply), activated despite a lower sequence, and the override is loudly logged and visible in bundle status. **The rollback target MUST be identified by content fingerprint, not by pathname alone.** Rollback is the one path that deliberately bypasses the watermark, so identifying its target by an attacker-writable path hands the adversary the choice of what the bypass lands on: with a legitimate bundle at sequence 100 active and the operator intending to revert to a known-good sequence 90, an attacker who can write the bundle path — in scope, unlike the config-write attacker — swaps the named file for a genuinely publisher-signed, current-epoch bundle at sequence 5 before the rollback reads it. It passes signature, epoch, compilation, and plausibly the self-check; the watermark that would normally refuse it is switched off by the rollback itself; and the result is then PINNED, so the downgrade persists across refresh cycles. FR-006's read-once rule closes the verify/parse race but not substitution before the read. Therefore the operator MUST supply the expected fingerprint (as offered by bundle status history), the path is only a hint for where to find those bytes, and a loaded artifact whose fingerprint differs MUST be refused as a rollback-target mismatch rather than activated. A rollback target MUST additionally still be at or above the FR-009b seeded baseline (the running binary's own embedded corpus) — the embedded default itself being the one target always permitted, since it IS that baseline and is the availability fallback of last resort. Rollback MUST NOT lower the historical high-watermark: the watermark is monotonic and normal activation MUST still require exceeding it. Rollback is instead expressed as a **pinned exception** — the operator pins the rolled-back artifact by fingerprint, and the (sequence, fingerprint) being rolled away from is deny-listed so the very release the operator just rejected cannot be re-activated by the next refresh cycle (and neither can any intermediate release between the pin and the watermark). Clearing the pin returns the install to normal watermark-governed activation. -- **FR-011**: The loader MUST enforce load ceilings (maximum bundle byte size and maximum rule count, fixed in design) and reject a candidate exceeding them as a whole; all patterns remain RE2-only. A database at the ceilings MUST NOT measurably regress the scan hot path (see SC-008). +- **FR-010**: An explicit rollback action MUST let an operator revert to the embedded default or a named bundle file: the target is re-validated (signature policy and self-check still apply), activated despite a lower sequence, and the override is loudly logged and visible in bundle status. **The rollback target MUST be identified by content fingerprint, not by pathname alone.** Rollback is the one path that deliberately bypasses the watermark, so identifying its target by an attacker-writable path hands the adversary the choice of what the bypass lands on: with a legitimate bundle at sequence 100 active and the operator intending to revert to a known-good sequence 90, an attacker who can write the bundle path — in scope, unlike the config-write attacker — swaps the named file for a genuinely publisher-signed, current-epoch bundle at sequence 5 before the rollback reads it. It passes signature, epoch, compilation, and plausibly the self-check; the watermark that would normally refuse it is switched off by the rollback itself; and the result is then PINNED, so the downgrade persists across refresh cycles. FR-006's read-once rule closes the verify/parse race but not substitution before the read. Therefore the operator MUST supply the expected fingerprint (as offered by bundle status history), the path is only a hint for where to find those bytes, and a loaded artifact whose fingerprint differs MUST be refused as a rollback-target mismatch rather than activated. A rollback target MUST additionally still be at or above **its own authority's** FR-009b baseline — the publisher baseline for a publisher-signed target, the config-declared operator baseline for an operator-signed one. The comparison MUST NOT be made against the publisher baseline in both cases: sequences of different authorities are independent counters (FR-004a), so measuring an operator target at sequence 9 against a publisher embedded corpus at 100 would refuse every legitimate operator rollback. The embedded default is the one target always permitted, since it IS the publisher baseline and is the availability fallback of last resort. Rollback MUST NOT lower the historical high-watermark: the watermark is monotonic and normal activation MUST still require exceeding it. Rollback is instead expressed as a **pinned exception** — the operator pins the rolled-back artifact by fingerprint, and the (sequence, fingerprint) being rolled away from is deny-listed so the very release the operator just rejected cannot be re-activated by the next refresh cycle (and neither can any intermediate release between the pin and the watermark). Clearing the pin returns the install to normal watermark-governed activation. +- **FR-011**: The loader MUST enforce load ceilings (maximum bundle byte size and maximum rule count, fixed in design) and reject a candidate exceeding them as a whole; all patterns remain RE2-only. **The sidecar needs its own, much smaller ceiling and a bounded read**, because it is the one artifact that must be read and parsed BEFORE any signature has been verified (its algorithm identifier, key id, and signature value are the inputs to verification), which puts it outside the protection FR-006 gives the bundle. Without it the in-scope bundle-path attacker simply writes a multi-gigabyte or never-terminating sidecar next to an otherwise acceptable bundle and exhausts memory or stalls the refresh before anything can be rejected — a denial of the scanner's own update path. The sidecar MUST therefore be read through a hard byte cap sized to its fixed small structure and rejected on overrun, before it is parsed, on both the file-drop and fetch paths. A database at the ceilings MUST NOT measurably regress the scan hot path (see SC-008). - **FR-011a**: Activation MUST be crash-consistent, not merely atomic in memory. Spec 087 FR-010 makes the rule-set swap atomic with respect to concurrent scans; that says nothing about persisted state, and a process death between "activated" and "watermark written" reopens exactly the replay window FR-009 exists to close (while writing the watermark first can leave the install with only an older recoverable bundle). The activation record — active fingerprint, sequence, per-authority watermarks and their epochs, the fingerprint each watermark was set by, signature-verified state, any FR-010 pin, and the FR-010 deny-list — MUST be persisted as ONE transaction, and the recoverable last-known-good bundle MUST be durable before the in-memory pointer is published. The deny-list and watermark fingerprint are load-bearing members of that transaction, not bookkeeping: a crash that preserved a rollback while losing the deny-list would let the next refresh immediately re-activate the exact release the operator had just rejected, silently undoing the rollback. Startup, refresh, hot-reload, and rollback MUST all serialize through that same transaction, so any crash leaves the install on a consistent (bundle, watermark) pair. **Seed corpus** @@ -140,7 +140,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **FR-015**: Signatures MUST be authored in the corpus source-of-truth pipeline (signature sources compiled to the bundle); mcpproxy MUST continue to consume only the compiled bundle and never parse signature sources (Spec 087 FR-002 parity). The contribution pipeline MUST run the same validation the loader runs (compile, schema, license, eval gate) before a bundle can be signed. - **FR-016**: The CI eval gate (`cmd/scan-eval --gate`, recall ≥ 0.90 over gated categories, hard-negative FP rate ≤ 0.05, existing vacuity guard) MUST pass for: every change to the embedded default corpus (merge-blocking, existing `eval.yml` D2 job) and every published bundle BEFORE it is signed — a bundle that fails the gate is never signed or published. Thresholds are reused verbatim, not redefined. - **FR-016a**: The gate harness MUST first be taught to see the database at all — today it does not, so FR-016 and SC-005 would otherwise pass vacuously. `cmd/scan-eval/gate.go`'s `gateChecks()` returns only the four built-in `detect.Check`s (`unicode.hidden`, `shadowing.cross_server`, `payload.decoded`, `phrase.injection`); `scanner.BundleCheck` — which production DOES append in `internal/security/scanner/inprocess.go` — is absent, and `scan-eval` never loads a bundle. Therefore: (a) `scan-eval` MUST accept a candidate bundle path, load it through the production loader (never a parallel parser), and register the resulting bundle check, defaulting to the embedded default so the existing `eval.yml` D2 invocation starts scoring the shipped corpus; (b) `gate.go`'s `categoryCheck` map MUST gain an entry for every new TPA campaign category, since `gatedCategory()` enforces a category ONLY when its mapped check id is registered — an unmapped category's malicious samples are tallied and reported but excluded from `gatedMalicious`/`OverallRecall`, so its MISSES can never fail the gate (its false positives still count, because `FPRate` is computed over the whole hard-negative set regardless of category mapping). A category whose recall is unenforceable is precisely the vacuous-pass failure mode; (c) the same candidate-loading path MUST be what the Spec 087 FR-008 activation self-check runs, so the CI layer and the activation layer cannot drift apart while SC-005 claims they are independent. -- **FR-016b**: The FR-016a/SC-005 negative control MUST be constructed to fail DETERMINISTICALLY, not left as "remove any signature and hope the gate turns red". As written it can pass vacuously twice over: `OverallRecall` is an aggregate `gatedDetected/gatedMalicious` over ALL gated categories, so with a 25-signature corpus dropping one signature typically moves recall by well under the 0.10 headroom the 0.90 threshold allows; and the dropped signature's malicious sample may still be flagged at hard tier by a built-in check (`phrase.injection` in particular overlaps most instruction-injection payloads), in which case recall does not move at all. Therefore the corpus MUST designate at least one **control signature** whose canonical malicious sample is verified to be hard-flagged by NO built-in `detect.Check` and by no other bundle signature, and the control MUST assert the gate's exit code is `exitGateBreach` when that signature alone is removed. The control MUST be evaluated over a scope in which the removal necessarily crosses the threshold (a control corpus restricted to the control signature's category, or an equivalent per-category assertion), and the no-overlap property of the control sample MUST itself be asserted in CI so that a later built-in check gaining coverage of it turns the control red rather than silently re-vacuating it. +- **FR-016b**: The FR-016a/SC-005 negative control MUST be constructed to fail DETERMINISTICALLY, not left as "remove any signature and hope the gate turns red". As written it can pass vacuously twice over: `OverallRecall` is an aggregate `gatedDetected/gatedMalicious` over ALL gated categories, so with a 25-signature corpus dropping one signature typically moves recall by well under the 0.10 headroom the 0.90 threshold allows; and the dropped signature's malicious sample may still be flagged at hard tier by a built-in check (`phrase.injection` in particular overlaps most instruction-injection payloads), in which case recall does not move at all. Therefore the corpus MUST designate at least one **control signature** whose canonical malicious sample is verified to be hard-flagged by NO built-in `detect.Check` and by no other bundle signature, and the control MUST assert the gate's exit code is `exitGateBreach` when that signature alone is removed. The control MUST be evaluated over a scope in which the removal PROVABLY crosses the threshold, and merely "restricting to the control signature's category" does not establish that: `decide()` fails only on `OverallRecall < minRecall`, so a control category holding ten gated malicious samples lands on exactly 0.90 after one miss and the gate stays green on the boundary. The canonical construction is therefore a control corpus whose gated malicious set is the control signature's sample ALONE, so removal drives recall to 0 — any other scope MUST be accompanied by an explicit assertion that the control category's recall is 1.0 with the signature present and strictly below the threshold without it, computed rather than assumed. The no-overlap property of the control sample MUST itself be asserted in CI so that a later built-in check gaining coverage of it turns the control red rather than silently re-vacuating it. **Publication & refresh channel** From c2fbc7fe6a000848baad01114b93bc44b451a148 Mon Sep 17 00:00:00 2001 From: Algis Dumbris Date: Mon, 24 Aug 2026 06:53:06 +0300 Subject: [PATCH 10/15] docs(specs): refuse non-regular candidates and scope the freshness metric (101-tpa-db) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cross-model review (opencode / gpt-5.6-sol) round 4 on specs/101-tpa-db/spec.md. Findings verified against the document before changing anything; the telemetry finding was accepted in substance but not in its proposed remedy (see below). ## Changes - FR-011 + Edge Cases: a byte cap bounds bytes, not time. Local candidate artifacts (bundle and sidecar) must be REGULAR files — a FIFO, device, socket, or symlink resolving to one is refused outright rather than read — and every candidate read, local or fetched, runs under an enforceable I/O deadline whose expiry releases Spec 087's single-flight slot. Otherwise the bundle-path attacker leaves a FIFO with no writer and pins the refresh path indefinitely without ever approaching the cap. - FR-010: an authority with no declared baseline — possible only for the operator authority, which FR-009b explicitly permits — has no rollback floor to enforce, and none is invented. The rollback proceeds and the install is marked degraded while that bundle is active, which is already FR-009b's treatment. Refusing the rollback would break FR-010 for a configuration FR-009b allows; assuming a zero floor would claim a guarantee that does not exist. - SC-006: the freshness metric is scoped to publisher-issued corpora and its derivation is stated — age comes from a server-side join of the reported version/sequence to the publisher's own release dates. Installs on an operator-built corpus (the `other` bucket) are not ageable and form their own bucket rather than being folded into the ratio. ## Notes on the rejected remedy The reviewer's suggested fix for SC-006 was to report `generated_at`/age in the heartbeat. That is exactly the near-unique value FR-021's schema-v8 backstop exists to keep out of the payload, and it would buy nothing the version/sequence join does not already give, so the derivation was specified instead. ## Testing - Docs-only change; no Go sources touched. --- specs/101-tpa-db/spec.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/specs/101-tpa-db/spec.md b/specs/101-tpa-db/spec.md index 6228caab..f9c07fb5 100644 --- a/specs/101-tpa-db/spec.md +++ b/specs/101-tpa-db/spec.md @@ -101,7 +101,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **Anti-downgrade state lost** (data directory wiped/reset): the persisted watermark resets to the FR-009b seeded baseline — NOT to zero, so a wiped install still refuses any publisher-signed bundle older than the corpus its own binary ships. For the operator authority the same protection holds only if the operator declared a config-resident baseline; without one, operator-authority anti-downgrade does not survive the wipe and the install runs degraded while an operator-signed bundle is active, by FR-009b, rather than pretending to a guarantee it does not have. What a wipe does lose is the FR-009 signature-ratchet state and any FR-010 pin/deny-list (both data-dir resident), so a wiped install accepts an unsigned drop again. This is accepted — the ratchet is best-effort hardening on top of signing, not the sole defense, and a wiped data dir already resets approvals. Operators who need this closed absolutely set `require_signed_bundle`, which is config-resident rather than data-dir-resident and therefore survives the wipe. - **Equal sequence, different bytes** (two distinct bundles claiming the same sequence): refused as suspicious unless it is the currently-active fingerprint (idempotent re-drop stays a no-op per Spec 087 FR-011). - **Clock skew**: no validity windows or expiry in v1 — sequence ordering, not wall-clock time, is the trust signal; `generated_at` is advisory freshness only. -- **Huge or adversarial database** (thousands of rules, pathological regex) **or an adversarial sidecar**: the sidecar byte cap (read before it is parsed, since verification's own inputs come from it), plus the bundle size and rule-count ceilings, are enforced at load; every pattern remains RE2 (linear-time, no catastrophic backtracking by construction); a candidate exceeding ceilings is rejected whole, keeping last-known-good. +- **Huge or adversarial database** (thousands of rules, pathological regex) **or an adversarial sidecar**: the sidecar byte cap (read before it is parsed, since verification's own inputs come from it), the regular-file requirement and read deadline that stop a FIFO or stalled stream from pinning the refresh path, plus the bundle size and rule-count ceilings, are enforced at load; every pattern remains RE2 (linear-time, no catastrophic backtracking by construction); a candidate exceeding ceilings is rejected whole, keeping last-known-good. - **Post-activation re-scan storms**: the re-scan is off the hot path, rate-limited, runs once per activation over cached metadata only (no upstream reconnects), and produces at most one review finding per tool per bundle activation — no notification storm. - **Signature removed from a newer bundle** (false-positive retired): tools previously held by it are NOT auto-approved by refresh; the held state persists for human review (state changes only flow toward review, never silently toward approval). @@ -128,8 +128,8 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **FR-009**: The system MUST persist the highest sequence ever activated — the **watermark**, scoped per signing authority and trust epoch per FR-004a/FR-005a — and refuse any candidate with a lower sequence, or an equal sequence with a different content fingerprint, as a downgrade/replay, unless an explicit operator rollback (FR-010) is in effect. **The watermark is a property of SIGNED artifacts only**: `sequence` lives in the manifest, so an unsigned candidate's sequence is attacker-chosen. An unsigned/unverified candidate MUST NOT advance the watermark (otherwise a single unsigned drop claiming a huge sequence permanently locks out every legitimate signed release), and once an install has ever activated a signature-verified bundle, unsigned candidates MUST be refused thereafter regardless of `require_signed_bundle` — a signature ratchet, so signing cannot be stripped back off an install by whoever can write the bundle path. The embedded default is exempt from BOTH the watermark and the ratchet — it is the availability fallback of last resort and must never be locked out (scanning must never stop), which is exactly why FR-009a demotes it to degraded coverage instead of refusing it. - **FR-009b**: Each authority's watermark MUST be SEEDED from a trusted baseline rather than starting empty, because an absent watermark accepts anything. Otherwise the anti-downgrade guarantee simply does not exist at first use or after the data directory is reset: an attacker who can write the bundle path drops an OLD but genuinely publisher-signed, current-epoch bundle — say sequence 5 while the shipping release is at 100 — and with no watermark to compare against it activates as a fully signature-verified bundle at FULL coverage, silently reinstating every blind spot fixed between 5 and 100. Neither the FR-009 ratchet nor the FR-010 pin/deny-list helps here; both only bind once persisted state exists. Therefore: the **publisher** authority's watermark MUST be seeded from the sequence of the bundle embedded in the running binary — a candidate older than what the binary already ships is strictly worse than the fallback and MUST be refused — and this baseline MUST be re-applied on every startup, so wiping the data directory cannot lower it below the binary's own corpus. The **operator** authority has no binary-trusted baseline available, so its baseline MUST be declared explicitly in configuration alongside the operator key, and it MUST live in `mcp_config.json` rather than the data directory — that is what makes it durable across the wipe, since a baseline stored in resettable state provides no protection precisely when it is needed. A silent zero default MUST NOT be used: with a zero baseline the operator authority has NO anti-downgrade after state loss at all, and a bundle-path attacker — who is squarely inside the threat model, unlike the config-write attacker — could replay any older genuinely operator-signed bundle at full coverage on a fresh or wiped install. Therefore: if an operator key is configured WITHOUT a baseline, the system MUST treat operator-authority anti-downgrade as unsupported and mark coverage degraded (per FR-007) while an operator-signed bundle is active, rather than silently presenting it as fully trusted. The data-directory watermark then tracks activations above the configured baseline and may only raise the effective minimum, never lower it. The seeded baseline also makes the epoch re-baseline of FR-005a safe: after an epoch bump the new epoch's watermark starts at the authority's baseline, not at zero. - **FR-009a**: When no valid external candidate is available, the embedded default MUST still serve so scanning never stops, even if its sequence is below the watermark. Where the install has NO retained signed watermark and no missing previously-active external bundle (the ordinary fresh install), this is simply the normal full-coverage state. Two — and only two — conditions demote that fallback to degraded coverage (auto-approval suspended, as in FR-007's unsigned-external case): (a) the embedded default's sequence is below the **publisher** authority's retained watermark, or (b) the **operator** authority is the active one (FR-004b) and its previously-activated external bundle is no longer available or no longer valid. The comparison in (a) MUST be same-namespace: the embedded default is a publisher artifact, and sequences of different authorities are independent counters, so it MUST be compared ONLY against the publisher watermark and never against an operator watermark — a cross-authority numeric comparison is meaningless and would, for an operator corpus at sequence 900 against an embedded publisher corpus at 100, permanently suspend auto-approval on an install whose publisher fallback is perfectly current. Condition (b) is what a cross-authority comparison was reaching for, restricted to the case where the fallback cannot be shown to be at least as fresh as what was lost: an operator corpus is not comparable to the publisher's, so a missing one leaves the install with no evidence the embedded default covers it. It MUST NOT extend to a missing PUBLISHER external bundle, because there condition (a) already decides the question on the same axis — an embedded default at or above the publisher watermark is demonstrably no older than the bundle that went missing (the ordinary "external file removed after a binary upgrade shipped a newer corpus" case) and MUST stay full coverage rather than being degraded merely because an older external file once existed. In either case the retained watermark, the active authority, and the degraded reason MUST be surfaced, and it MUST NOT count as an anti-downgrade violation. This below-watermark fallback is the single explicit exemption to SC-002. -- **FR-010**: An explicit rollback action MUST let an operator revert to the embedded default or a named bundle file: the target is re-validated (signature policy and self-check still apply), activated despite a lower sequence, and the override is loudly logged and visible in bundle status. **The rollback target MUST be identified by content fingerprint, not by pathname alone.** Rollback is the one path that deliberately bypasses the watermark, so identifying its target by an attacker-writable path hands the adversary the choice of what the bypass lands on: with a legitimate bundle at sequence 100 active and the operator intending to revert to a known-good sequence 90, an attacker who can write the bundle path — in scope, unlike the config-write attacker — swaps the named file for a genuinely publisher-signed, current-epoch bundle at sequence 5 before the rollback reads it. It passes signature, epoch, compilation, and plausibly the self-check; the watermark that would normally refuse it is switched off by the rollback itself; and the result is then PINNED, so the downgrade persists across refresh cycles. FR-006's read-once rule closes the verify/parse race but not substitution before the read. Therefore the operator MUST supply the expected fingerprint (as offered by bundle status history), the path is only a hint for where to find those bytes, and a loaded artifact whose fingerprint differs MUST be refused as a rollback-target mismatch rather than activated. A rollback target MUST additionally still be at or above **its own authority's** FR-009b baseline — the publisher baseline for a publisher-signed target, the config-declared operator baseline for an operator-signed one. The comparison MUST NOT be made against the publisher baseline in both cases: sequences of different authorities are independent counters (FR-004a), so measuring an operator target at sequence 9 against a publisher embedded corpus at 100 would refuse every legitimate operator rollback. The embedded default is the one target always permitted, since it IS the publisher baseline and is the availability fallback of last resort. Rollback MUST NOT lower the historical high-watermark: the watermark is monotonic and normal activation MUST still require exceeding it. Rollback is instead expressed as a **pinned exception** — the operator pins the rolled-back artifact by fingerprint, and the (sequence, fingerprint) being rolled away from is deny-listed so the very release the operator just rejected cannot be re-activated by the next refresh cycle (and neither can any intermediate release between the pin and the watermark). Clearing the pin returns the install to normal watermark-governed activation. -- **FR-011**: The loader MUST enforce load ceilings (maximum bundle byte size and maximum rule count, fixed in design) and reject a candidate exceeding them as a whole; all patterns remain RE2-only. **The sidecar needs its own, much smaller ceiling and a bounded read**, because it is the one artifact that must be read and parsed BEFORE any signature has been verified (its algorithm identifier, key id, and signature value are the inputs to verification), which puts it outside the protection FR-006 gives the bundle. Without it the in-scope bundle-path attacker simply writes a multi-gigabyte or never-terminating sidecar next to an otherwise acceptable bundle and exhausts memory or stalls the refresh before anything can be rejected — a denial of the scanner's own update path. The sidecar MUST therefore be read through a hard byte cap sized to its fixed small structure and rejected on overrun, before it is parsed, on both the file-drop and fetch paths. A database at the ceilings MUST NOT measurably regress the scan hot path (see SC-008). +- **FR-010**: An explicit rollback action MUST let an operator revert to the embedded default or a named bundle file: the target is re-validated (signature policy and self-check still apply), activated despite a lower sequence, and the override is loudly logged and visible in bundle status. **The rollback target MUST be identified by content fingerprint, not by pathname alone.** Rollback is the one path that deliberately bypasses the watermark, so identifying its target by an attacker-writable path hands the adversary the choice of what the bypass lands on: with a legitimate bundle at sequence 100 active and the operator intending to revert to a known-good sequence 90, an attacker who can write the bundle path — in scope, unlike the config-write attacker — swaps the named file for a genuinely publisher-signed, current-epoch bundle at sequence 5 before the rollback reads it. It passes signature, epoch, compilation, and plausibly the self-check; the watermark that would normally refuse it is switched off by the rollback itself; and the result is then PINNED, so the downgrade persists across refresh cycles. FR-006's read-once rule closes the verify/parse race but not substitution before the read. Therefore the operator MUST supply the expected fingerprint (as offered by bundle status history), the path is only a hint for where to find those bytes, and a loaded artifact whose fingerprint differs MUST be refused as a rollback-target mismatch rather than activated. A rollback target MUST additionally still be at or above **its own authority's** FR-009b baseline — the publisher baseline for a publisher-signed target, the config-declared operator baseline for an operator-signed one. The comparison MUST NOT be made against the publisher baseline in both cases: sequences of different authorities are independent counters (FR-004a), so measuring an operator target at sequence 9 against a publisher embedded corpus at 100 would refuse every legitimate operator rollback. Where an authority has NO declared baseline — possible only for the operator authority, which FR-009b explicitly permits to run without one — there is no floor to enforce and none is invented: the rollback proceeds and the install is marked degraded for exactly as long as that operator-signed bundle is active, which is already FR-009b's treatment of a baseline-less operator authority. Neither refusing the rollback (which would break FR-010's guarantee for a configuration FR-009b allows) nor assuming a zero floor (which would claim an anti-downgrade guarantee that does not exist) is acceptable. The embedded default is the one target always permitted, since it IS the publisher baseline and is the availability fallback of last resort. Rollback MUST NOT lower the historical high-watermark: the watermark is monotonic and normal activation MUST still require exceeding it. Rollback is instead expressed as a **pinned exception** — the operator pins the rolled-back artifact by fingerprint, and the (sequence, fingerprint) being rolled away from is deny-listed so the very release the operator just rejected cannot be re-activated by the next refresh cycle (and neither can any intermediate release between the pin and the watermark). Clearing the pin returns the install to normal watermark-governed activation. +- **FR-011**: The loader MUST enforce load ceilings (maximum bundle byte size and maximum rule count, fixed in design) and reject a candidate exceeding them as a whole; all patterns remain RE2-only. **The sidecar needs its own, much smaller ceiling and a bounded read**, because it is the one artifact that must be read and parsed BEFORE any signature has been verified (its algorithm identifier, key id, and signature value are the inputs to verification), which puts it outside the protection FR-006 gives the bundle. Without it the in-scope bundle-path attacker simply writes a multi-gigabyte or never-terminating sidecar next to an otherwise acceptable bundle and exhausts memory or stalls the refresh before anything can be rejected — a denial of the scanner's own update path. The sidecar MUST therefore be read through a hard byte cap sized to its fixed small structure and rejected on overrun, before it is parsed, on both the file-drop and fetch paths. A byte cap alone is not sufficient, because it bounds bytes consumed but not time spent: the same attacker replaces the sidecar (or bundle) path with a FIFO and never writes to it, so the read blocks forever without ever approaching the cap and Spec 087's single-flight refresh stays occupied, blocking every legitimate database update while the loader waits. Both local candidate artifacts MUST therefore be required to be REGULAR files — a FIFO, device, socket, or a symlink resolving to one is refused outright, not read — and every candidate read (local files and fetch response bodies alike) MUST run under an enforceable I/O deadline whose expiry releases the single-flight slot and records a rejection reason, so no candidate can pin the refresh path. A database at the ceilings MUST NOT measurably regress the scan hot path (see SC-008). - **FR-011a**: Activation MUST be crash-consistent, not merely atomic in memory. Spec 087 FR-010 makes the rule-set swap atomic with respect to concurrent scans; that says nothing about persisted state, and a process death between "activated" and "watermark written" reopens exactly the replay window FR-009 exists to close (while writing the watermark first can leave the install with only an older recoverable bundle). The activation record — active fingerprint, sequence, per-authority watermarks and their epochs, the fingerprint each watermark was set by, signature-verified state, any FR-010 pin, and the FR-010 deny-list — MUST be persisted as ONE transaction, and the recoverable last-known-good bundle MUST be durable before the in-memory pointer is published. The deny-list and watermark fingerprint are load-bearing members of that transaction, not bookkeeping: a crash that preserved a rollback while losing the deny-list would let the next refresh immediately re-activate the exact release the operator had just rejected, silently undoing the rollback. Startup, refresh, hot-reload, and rollback MUST all serialize through that same transaction, so any crash leaves the install on a consistent (bundle, watermark) pair. **Seed corpus** @@ -177,7 +177,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **SC-003**: An air-gapped install (zero network, ever) reaches full P1 capability: verified activation of a file-dropped signed database, downgrade refusal, and rollback, with behavior byte-identical to a connected install's file-drop path. - **SC-004**: The shipped seed corpus contains at least 25 signatures spanning at least 8 distinct public campaign/technique classes, every one carrying provenance + redistributable license, and every gating signature backed by its eval sample pair; the canonical fixture of each class produces a hard-tier finding naming its TPA id. - **SC-005**: The scan-eval CI gate (recall ≥ 0.90 gated categories, hard-negative FP ≤ 0.05) passes with the full seed corpus embedded AND actually scored — i.e. the gate run registers the bundle check and maps every new campaign category (FR-016a), proven by a **deterministic** negative control (FR-016b): removing a designated control signature from the bundle must turn the gate red. Without that control the criterion is vacuous, since today's gate would pass with the bundle entirely absent. A deliberately-regressing corpus change is then blocked twice independently: at CI (cannot merge) and at activation (self-check rejects it) — both demonstrated by fixture. -- **SC-006**: Fleet freshness is answerable from telemetry: for opted-in installs, the distribution of active database version/sequence/source is reportable, enabling the metric "% of active installs on a database ≤ 30 days old" — a metric that is impossible to compute today. +- **SC-006**: Fleet freshness is answerable from telemetry: for opted-in installs, the distribution of active database version/sequence/source is reportable, enabling the metric "% of active installs on a **publisher-issued** database ≤ 30 days old" — a metric that is impossible to compute today. Age MUST be derived server-side by joining the reported publisher-issued version/sequence to the publication date the publisher already knows for its own releases, NOT by adding a `generated_at` or age field to the heartbeat: a build timestamp is precisely the kind of near-unique value FR-021's backstop exists to keep out of the payload, and it would undo that protection to buy nothing the join does not already give. Installs on an operator-built corpus (reported in the `other` bucket per FR-021) are consequently NOT ageable and MUST be reported as their own unaged bucket rather than silently folded into either side of the ratio — the metric's denominator is publisher-corpus installs. - **SC-007**: A newly published signature reaches detection within one daily refresh cycle on a fetch-enabled install (and immediately on manual drop), measured end to end: publish → active → the new signature's fixture fires. - **SC-008**: With a database at the design load ceilings (≥ 500 regex rules), p95 scan latency over a 100-tool server changes by less than 10% versus the 6-signature baseline, and bundle load-plus-verify completes under one second on commodity hardware. - **SC-009**: After activating a database containing a new signature, an already-approved tool whose cached description matches it is surfaced for review within the same refresh cycle, with zero approvals auto-revoked and zero servers auto-quarantined by the refresh itself. From 3979f29f79fe1d16381a575cfb01bd915e0a7a44 Mon Sep 17 00:00:00 2001 From: Algis Dumbris Date: Mon, 24 Aug 2026 06:57:16 +0300 Subject: [PATCH 11/15] docs(specs): name the baseline-less exemption and bucket operator sequences (101-tpa-db) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cross-model review (opencode / gpt-5.6-sol) round 5 on specs/101-tpa-db/spec.md. Round 5 reported no remaining adversary-reachable security hole; both findings were internal contradictions, verified against the document before editing. ## Changes - FR-009b, the Sequence Watermark entity, and SC-002: the baseline-less operator authority is now named as an explicit exemption everywhere the document made a universal claim. FR-009b permits an operator authority with no declared baseline (marking the install degraded), while the same requirement opened with "every watermark MUST be SEEDED", the entity said "Never empty", and SC-002 called FR-009a the "sole exemption" — an implementation could not satisfy all four at once. There are exactly two exemptions and both are degraded-coverage states. - FR-021: `sequence` is reported only when `bundle_version` resolves to a publisher-issued value; in the `other` bucket it is omitted. Bucketing the version string while transmitting a raw operator-chosen sequence defeats the purpose of the bucket, since an operator-built corpus can number itself with a unix timestamp or private build number and be just as fleet-identifying. SC-006 already scopes freshness to publisher corpora, so no analysis is lost. ## Testing - Docs-only change; no Go sources touched. - Round 5 is the per-PR cross-model review cap (CLAUDE.md escalation trigger 4); these two fixes are not themselves re-reviewed. --- specs/101-tpa-db/spec.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/specs/101-tpa-db/spec.md b/specs/101-tpa-db/spec.md index f9c07fb5..3bc60040 100644 --- a/specs/101-tpa-db/spec.md +++ b/specs/101-tpa-db/spec.md @@ -126,7 +126,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **FR-007**: Signing policy MUST be: network-fetched candidates ALWAYS require a valid signature; file-drop candidates are verified whenever a sidecar is present; a new `require_signed_bundle` setting (default off for back-compat with existing unsigned drops) makes the sidecar mandatory for file drops too; the embedded default needs no sidecar (its integrity rides on binary distribution integrity). Bundle status MUST always report whether the active bundle was signature-verified. While an unsigned EXTERNAL (file-dropped or fetched) bundle is active, the scan coverage the approval gates consume MUST be marked degraded so `scan`-mode auto-approval is suspended and changes route to human review — today `internal/security/scanner/inprocess.go` sets `coverageOK` from mere bundle *presence*, so without this an attacker-supplied unsigned corpus would keep silently auto-approving under weakened rules. This degradation rule applies to EXTERNAL candidates ONLY: the embedded default is binary-trusted (its integrity rides on the signed binary distribution, not on a sidecar) and MUST be full-coverage on an ordinary install, or every fresh install would ship with `scan` mode silently disabled. The embedded default is degraded only in the FR-009a case — when it is serving as a fallback BELOW a retained signed watermark, i.e. when the install has evidence a newer signed corpus should be active. This requirement supersedes Spec 087 FR-014's precondition: because FR-004 makes publisher keys embedded in the binary part of the trust set, a fetch is no longer refused merely because the operator configured no key of their own; the "never activate an unverified fetched bundle" guarantee of Spec 087 FR-013/FR-014 is preserved unchanged. - **FR-008**: The full validate-before-activate order MUST be: sidecar byte cap → parse sidecar → bundle byte-size ceiling → signature over the raw bytes (per policy, FR-006) → parse manifest → rule-count ceiling → epoch match (FR-005a) → sequence/fingerprint anti-downgrade (FR-009) → version compatibility → compile-all-or-reject → runnable-rules > 0 → activation self-check (Spec 087 FR-008) → atomic activation. FR-011's ceilings MUST therefore be split across the pipeline rather than applied as one stage: the sidecar cap comes first because the sidecar is read before any signature exists to check, bundle byte size is measurable on raw bytes and so guards the verification step itself, while rule count is only knowable after parsing and so cannot precede it. The same reasoning applies to sequence, a manifest field necessarily read after parsing: what MUST precede parsing is signature verification, not the sequence or rule-count check — an unverified candidate's manifest is never trusted for any decision. Any failure at any stage MUST keep last-known-good and record a machine-readable reason and timestamp (surfaced via the existing bundle status), never an empty or partial rule set. - **FR-009**: The system MUST persist the highest sequence ever activated — the **watermark**, scoped per signing authority and trust epoch per FR-004a/FR-005a — and refuse any candidate with a lower sequence, or an equal sequence with a different content fingerprint, as a downgrade/replay, unless an explicit operator rollback (FR-010) is in effect. **The watermark is a property of SIGNED artifacts only**: `sequence` lives in the manifest, so an unsigned candidate's sequence is attacker-chosen. An unsigned/unverified candidate MUST NOT advance the watermark (otherwise a single unsigned drop claiming a huge sequence permanently locks out every legitimate signed release), and once an install has ever activated a signature-verified bundle, unsigned candidates MUST be refused thereafter regardless of `require_signed_bundle` — a signature ratchet, so signing cannot be stripped back off an install by whoever can write the bundle path. The embedded default is exempt from BOTH the watermark and the ratchet — it is the availability fallback of last resort and must never be locked out (scanning must never stop), which is exactly why FR-009a demotes it to degraded coverage instead of refusing it. -- **FR-009b**: Each authority's watermark MUST be SEEDED from a trusted baseline rather than starting empty, because an absent watermark accepts anything. Otherwise the anti-downgrade guarantee simply does not exist at first use or after the data directory is reset: an attacker who can write the bundle path drops an OLD but genuinely publisher-signed, current-epoch bundle — say sequence 5 while the shipping release is at 100 — and with no watermark to compare against it activates as a fully signature-verified bundle at FULL coverage, silently reinstating every blind spot fixed between 5 and 100. Neither the FR-009 ratchet nor the FR-010 pin/deny-list helps here; both only bind once persisted state exists. Therefore: the **publisher** authority's watermark MUST be seeded from the sequence of the bundle embedded in the running binary — a candidate older than what the binary already ships is strictly worse than the fallback and MUST be refused — and this baseline MUST be re-applied on every startup, so wiping the data directory cannot lower it below the binary's own corpus. The **operator** authority has no binary-trusted baseline available, so its baseline MUST be declared explicitly in configuration alongside the operator key, and it MUST live in `mcp_config.json` rather than the data directory — that is what makes it durable across the wipe, since a baseline stored in resettable state provides no protection precisely when it is needed. A silent zero default MUST NOT be used: with a zero baseline the operator authority has NO anti-downgrade after state loss at all, and a bundle-path attacker — who is squarely inside the threat model, unlike the config-write attacker — could replay any older genuinely operator-signed bundle at full coverage on a fresh or wiped install. Therefore: if an operator key is configured WITHOUT a baseline, the system MUST treat operator-authority anti-downgrade as unsupported and mark coverage degraded (per FR-007) while an operator-signed bundle is active, rather than silently presenting it as fully trusted. The data-directory watermark then tracks activations above the configured baseline and may only raise the effective minimum, never lower it. The seeded baseline also makes the epoch re-baseline of FR-005a safe: after an epoch bump the new epoch's watermark starts at the authority's baseline, not at zero. +- **FR-009b**: Each authority's watermark MUST be SEEDED from a trusted baseline rather than starting empty, because an absent watermark accepts anything — with exactly one exception, carved out at the end of this requirement: an operator authority configured without a baseline, which has no seed available and is therefore run as explicitly degraded rather than as a silent zero. Otherwise the anti-downgrade guarantee simply does not exist at first use or after the data directory is reset: an attacker who can write the bundle path drops an OLD but genuinely publisher-signed, current-epoch bundle — say sequence 5 while the shipping release is at 100 — and with no watermark to compare against it activates as a fully signature-verified bundle at FULL coverage, silently reinstating every blind spot fixed between 5 and 100. Neither the FR-009 ratchet nor the FR-010 pin/deny-list helps here; both only bind once persisted state exists. Therefore: the **publisher** authority's watermark MUST be seeded from the sequence of the bundle embedded in the running binary — a candidate older than what the binary already ships is strictly worse than the fallback and MUST be refused — and this baseline MUST be re-applied on every startup, so wiping the data directory cannot lower it below the binary's own corpus. The **operator** authority has no binary-trusted baseline available, so its baseline MUST be declared explicitly in configuration alongside the operator key, and it MUST live in `mcp_config.json` rather than the data directory — that is what makes it durable across the wipe, since a baseline stored in resettable state provides no protection precisely when it is needed. A silent zero default MUST NOT be used: with a zero baseline the operator authority has NO anti-downgrade after state loss at all, and a bundle-path attacker — who is squarely inside the threat model, unlike the config-write attacker — could replay any older genuinely operator-signed bundle at full coverage on a fresh or wiped install. Therefore: if an operator key is configured WITHOUT a baseline, the system MUST treat operator-authority anti-downgrade as unsupported and mark coverage degraded (per FR-007) while an operator-signed bundle is active, rather than silently presenting it as fully trusted. The data-directory watermark then tracks activations above the configured baseline and may only raise the effective minimum, never lower it. The seeded baseline also makes the epoch re-baseline of FR-005a safe: after an epoch bump the new epoch's watermark starts at the authority's baseline, not at zero. - **FR-009a**: When no valid external candidate is available, the embedded default MUST still serve so scanning never stops, even if its sequence is below the watermark. Where the install has NO retained signed watermark and no missing previously-active external bundle (the ordinary fresh install), this is simply the normal full-coverage state. Two — and only two — conditions demote that fallback to degraded coverage (auto-approval suspended, as in FR-007's unsigned-external case): (a) the embedded default's sequence is below the **publisher** authority's retained watermark, or (b) the **operator** authority is the active one (FR-004b) and its previously-activated external bundle is no longer available or no longer valid. The comparison in (a) MUST be same-namespace: the embedded default is a publisher artifact, and sequences of different authorities are independent counters, so it MUST be compared ONLY against the publisher watermark and never against an operator watermark — a cross-authority numeric comparison is meaningless and would, for an operator corpus at sequence 900 against an embedded publisher corpus at 100, permanently suspend auto-approval on an install whose publisher fallback is perfectly current. Condition (b) is what a cross-authority comparison was reaching for, restricted to the case where the fallback cannot be shown to be at least as fresh as what was lost: an operator corpus is not comparable to the publisher's, so a missing one leaves the install with no evidence the embedded default covers it. It MUST NOT extend to a missing PUBLISHER external bundle, because there condition (a) already decides the question on the same axis — an embedded default at or above the publisher watermark is demonstrably no older than the bundle that went missing (the ordinary "external file removed after a binary upgrade shipped a newer corpus" case) and MUST stay full coverage rather than being degraded merely because an older external file once existed. In either case the retained watermark, the active authority, and the degraded reason MUST be surfaced, and it MUST NOT count as an anti-downgrade violation. This below-watermark fallback is the single explicit exemption to SC-002. - **FR-010**: An explicit rollback action MUST let an operator revert to the embedded default or a named bundle file: the target is re-validated (signature policy and self-check still apply), activated despite a lower sequence, and the override is loudly logged and visible in bundle status. **The rollback target MUST be identified by content fingerprint, not by pathname alone.** Rollback is the one path that deliberately bypasses the watermark, so identifying its target by an attacker-writable path hands the adversary the choice of what the bypass lands on: with a legitimate bundle at sequence 100 active and the operator intending to revert to a known-good sequence 90, an attacker who can write the bundle path — in scope, unlike the config-write attacker — swaps the named file for a genuinely publisher-signed, current-epoch bundle at sequence 5 before the rollback reads it. It passes signature, epoch, compilation, and plausibly the self-check; the watermark that would normally refuse it is switched off by the rollback itself; and the result is then PINNED, so the downgrade persists across refresh cycles. FR-006's read-once rule closes the verify/parse race but not substitution before the read. Therefore the operator MUST supply the expected fingerprint (as offered by bundle status history), the path is only a hint for where to find those bytes, and a loaded artifact whose fingerprint differs MUST be refused as a rollback-target mismatch rather than activated. A rollback target MUST additionally still be at or above **its own authority's** FR-009b baseline — the publisher baseline for a publisher-signed target, the config-declared operator baseline for an operator-signed one. The comparison MUST NOT be made against the publisher baseline in both cases: sequences of different authorities are independent counters (FR-004a), so measuring an operator target at sequence 9 against a publisher embedded corpus at 100 would refuse every legitimate operator rollback. Where an authority has NO declared baseline — possible only for the operator authority, which FR-009b explicitly permits to run without one — there is no floor to enforce and none is invented: the rollback proceeds and the install is marked degraded for exactly as long as that operator-signed bundle is active, which is already FR-009b's treatment of a baseline-less operator authority. Neither refusing the rollback (which would break FR-010's guarantee for a configuration FR-009b allows) nor assuming a zero floor (which would claim an anti-downgrade guarantee that does not exist) is acceptable. The embedded default is the one target always permitted, since it IS the publisher baseline and is the availability fallback of last resort. Rollback MUST NOT lower the historical high-watermark: the watermark is monotonic and normal activation MUST still require exceeding it. Rollback is instead expressed as a **pinned exception** — the operator pins the rolled-back artifact by fingerprint, and the (sequence, fingerprint) being rolled away from is deny-listed so the very release the operator just rejected cannot be re-activated by the next refresh cycle (and neither can any intermediate release between the pin and the watermark). Clearing the pin returns the install to normal watermark-governed activation. - **FR-011**: The loader MUST enforce load ceilings (maximum bundle byte size and maximum rule count, fixed in design) and reject a candidate exceeding them as a whole; all patterns remain RE2-only. **The sidecar needs its own, much smaller ceiling and a bounded read**, because it is the one artifact that must be read and parsed BEFORE any signature has been verified (its algorithm identifier, key id, and signature value are the inputs to verification), which puts it outside the protection FR-006 gives the bundle. Without it the in-scope bundle-path attacker simply writes a multi-gigabyte or never-terminating sidecar next to an otherwise acceptable bundle and exhausts memory or stalls the refresh before anything can be rejected — a denial of the scanner's own update path. The sidecar MUST therefore be read through a hard byte cap sized to its fixed small structure and rejected on overrun, before it is parsed, on both the file-drop and fetch paths. A byte cap alone is not sufficient, because it bounds bytes consumed but not time spent: the same attacker replaces the sidecar (or bundle) path with a FIFO and never writes to it, so the read blocks forever without ever approaching the cap and Spec 087's single-flight refresh stays occupied, blocking every legitimate database update while the loader waits. Both local candidate artifacts MUST therefore be required to be REGULAR files — a FIFO, device, socket, or a symlink resolving to one is refused outright, not read — and every candidate read (local files and fetch response bodies alike) MUST run under an enforceable I/O deadline whose expiry releases the single-flight slot and records a rejection reason, so no candidate can pin the refresh path. A database at the ceilings MUST NOT measurably regress the scan hot path (see SC-008). @@ -151,7 +151,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), **Observability, telemetry & docs** - **FR-020**: Bundle status on all existing surfaces (CLI security commands, REST security overview, Web UI) MUST additionally report sequence, signing authority and trust epoch, signature-verified state, and staleness (age since `generated_at`), alongside the existing version/fingerprint/source/load-error fields, and MUST show the last rejection reason (tamper, downgrade, stale-epoch replay, unsigned-refused-by-ratchet, authority-not-active, rollback-target mismatch, gate failure) so an operator can tell "current and verified" from "stuck on last-known-good and why". Where the active database has demoted the install to DEGRADED coverage (FR-007 unsigned-active, FR-009a embedded fallback), that state and its reason MUST be surfaced too — a silently suspended `scan`-mode auto-approval is otherwise indistinguishable from a quiet install. -- **FR-021**: The existing anonymous TPA-scanner telemetry MUST gain the active bundle version, sequence, and source (and signature-verified state) so database adoption and freshness are measurable fleet-wide; the existing telemetry opt-out MUST fully cover these fields, and no new identifying data is introduced. This is NOT a drop-in addition: `internal/telemetry/anonymity.go`'s schema-v8 backstop hard-whitelists the `tpa_scanner` sub-object to `{scans_completed, scans_failed, scans_with_findings, findings}` and requires every scalar to be a non-negative integer, so a string `bundle_version`/`source` or a boolean `signature_verified` would be rejected before transmit as "carries a key outside the whitelist". The design MUST therefore extend that backstop with explicitly TYPED entries (integer `sequence`, enum-bounded `source` ∈ {embedded, file, fetch}, boolean signature-verified) rather than relaxing it to accept arbitrary keys or free strings. `bundle_version` MUST be constrained to publisher-issued values (unrecognized/operator-built corpus versions reported as a single `other` bucket): the backstop exists precisely to stop a bespoke, near-unique string from acting as a fleet identifier, and a self-built corpus version would be exactly that. +- **FR-021**: The existing anonymous TPA-scanner telemetry MUST gain the active bundle version, sequence, and source (and signature-verified state) so database adoption and freshness are measurable fleet-wide; the existing telemetry opt-out MUST fully cover these fields, and no new identifying data is introduced. This is NOT a drop-in addition: `internal/telemetry/anonymity.go`'s schema-v8 backstop hard-whitelists the `tpa_scanner` sub-object to `{scans_completed, scans_failed, scans_with_findings, findings}` and requires every scalar to be a non-negative integer, so a string `bundle_version`/`source` or a boolean `signature_verified` would be rejected before transmit as "carries a key outside the whitelist". The design MUST therefore extend that backstop with explicitly TYPED entries (integer `sequence`, enum-bounded `source` ∈ {embedded, file, fetch}, boolean signature-verified) rather than relaxing it to accept arbitrary keys or free strings. `bundle_version` MUST be constrained to publisher-issued values (unrecognized/operator-built corpus versions reported as a single `other` bucket): the backstop exists precisely to stop a bespoke, near-unique string from acting as a fleet identifier, and a self-built corpus version would be exactly that. **The same reasoning MUST be applied to `sequence`, not only to `bundle_version`**: an operator picks their own corpus's sequence values freely — a unix timestamp or a private build number is the obvious choice — so a raw integer sequence from an operator-built corpus is every bit the near-unique fleet identifier the `other` bucket was introduced to suppress, and bucketing the version while transmitting the sequence verbatim would defeat it. `sequence` MUST therefore be reported only when `bundle_version` resolves to a publisher-issued value (a known, low-cardinality release line); when the version falls into the `other` bucket the sequence MUST be omitted. Nothing analytic is lost: SC-006 already scopes fleet-freshness to publisher-issued corpora and reports operator-built ones as an unaged bucket. - **FR-022**: Documentation MUST cover the database lifecycle end to end for operators (verify, drop, fetch, roll back, read status) and for contributors (author a signature, provenance/license rules, eval-sample requirements, how the gate blocks a bad corpus), updating the existing security-quarantine/tool-scanner docs. ### Key Entities *(include if feature involves data)* @@ -162,7 +162,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **Signing Authority**: the grouping anti-downgrade state is keyed by (FR-004a) — one publisher authority spanning all publisher keys including those mid-rotation, and a separate operator authority for operator-configured keys. Each holds its own watermark so neither can lock the other out. Exactly one authority is **active** at a time, declared in config (FR-004b, default publisher); artifacts of a non-active authority are refused, so per-authority isolation cannot be turned into an authority-swap by whoever can write the bundle path. - **Signature (TPA record)**: one cataloged campaign/technique — `TPA-YYYY-NNNN` id, detectors, category, level, confidence, tier intent (gating vs non-gating), provenance reference, license. - **Eval Sample Pair**: the labeled gated-malicious sample(s) plus category-matched hard-negative(s) a gating signature must contribute to the frozen eval dataset. -- **Sequence Watermark**: the persisted highest-activated sequence, tracked per signing authority and scoped by trust epoch (FR-004a, FR-005a); the anti-downgrade/anti-replay state. Never empty — it is seeded from a trusted baseline (the embedded bundle's sequence, for the publisher authority) per FR-009b, so "no state yet" is not an accept-anything state. It is MONOTONIC — rollback does not lower it (FR-010 expresses rollback as a pinned exception plus a deny-list instead), and the only thing that re-baselines it is a higher trust epoch arriving in a binary release. +- **Sequence Watermark**: the persisted highest-activated sequence, tracked per signing authority and scoped by trust epoch (FR-004a, FR-005a); the anti-downgrade/anti-replay state. Never empty for any authority that has a baseline — it is seeded from a trusted one (the embedded bundle's sequence, for the publisher authority) per FR-009b, so "no state yet" is not an accept-anything state. The single exception is an operator authority configured without a baseline, which FR-009b permits precisely because no binary-trusted seed exists for it; there the install carries no anti-downgrade guarantee and is marked degraded while such a bundle is active rather than being presented as fully trusted. It is MONOTONIC — rollback does not lower it (FR-010 expresses rollback as a pinned exception plus a deny-list instead), and the only thing that re-baselines it is a higher trust epoch arriving in a binary release. - **Publication Channel**: the stable public location of versioned signed artifact pairs with a "latest" reference; consumed by the Spec 087 fetch or by manual download. - **Post-Activation Re-scan**: the one-shot, off-hot-path evaluation of cached approved tool metadata against a newly-activated database, emitting review findings only. - **Active Database / Last-Known-Good**: unchanged from Spec 086/087 — the serving bundle and the fail-closed fallback target. @@ -172,7 +172,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), ### Measurable Outcomes - **SC-001**: 100% of tampered candidates are rejected before parsing, with the last-known-good database still serving and the reason surfaced — demonstrated by fixture in CI. "Tampered" is defined precisely, because the signature covers the bundle bytes and NOT the sidecar's own encoding: any altered bundle byte, any altered signature value, any altered sidecar identity field (algorithm or key id, whose agreement with the manifest FR-003 already requires), and any mismatched bundle/sidecar pair. Byte changes to the sidecar that alter neither the signature value nor an identity field (whitespace, key order) are explicitly NOT required to be rejected — a detached signature cannot detect them — so the criterion is stated in terms of the fields that carry meaning rather than an unachievable "every byte of the sidecar". -- **SC-002**: A validly-signed but lower-sequence bundle is never activated by any non-rollback path (drop, fetch, restart) — including on a fresh install or after a data-directory wipe, where the FR-009b seeded baseline stands in for the absent persisted watermark — and an unsigned candidate never advances the watermark nor survives the signature ratchet — demonstrated by fixture in CI. The sole exemption is the FR-009a embedded-default availability fallback, which serves as degraded coverage rather than counting as a downgrade. An explicit rollback succeeds, leaves an audit trail, does not lower the historical watermark, activates only the fingerprint the operator named (a substituted file at the same path is refused), and the deny-listed release it rolled away from is not re-activated by the following refresh cycle. +- **SC-002**: A validly-signed but lower-sequence bundle is never activated by any non-rollback path (drop, fetch, restart) — including on a fresh install or after a data-directory wipe, where the FR-009b seeded baseline stands in for the absent persisted watermark — and an unsigned candidate never advances the watermark nor survives the signature ratchet — demonstrated by fixture in CI. There are exactly two exemptions, and both are degraded-coverage states rather than silent ones: the FR-009a embedded-default availability fallback, and the baseline-less operator authority of FR-009b (SC-002a fixture (c)), which has no seed to compare against and so carries no anti-downgrade guarantee to claim. Neither counts as a downgrade; both MUST surface as degraded. An explicit rollback succeeds, leaves an audit trail, does not lower the historical watermark, activates only the fingerprint the operator named (a substituted file at the same path is refused), and the deny-listed release it rolled away from is not re-activated by the following refresh cycle. - **SC-002a**: The watermark cannot be weaponized into a permanent update lockout, demonstrated by two fixtures: (a) a bundle signed by a to-be-revoked key at the MAXIMUM representable sequence, after which a binary carrying a bumped trust epoch and the surviving key successfully activates a new release — proving key-compromise recovery is possible rather than merely documented; (a2) immediately after that bump, an OLD pre-bump bundle carrying the previous epoch and signed by the still-trusted surviving key is REFUSED as stale-epoch replay rather than clearing the re-baselined watermark — the epoch bump must not itself become a downgrade window; and (b) with the operator authority declared active (FR-004b), an operator-authority-signed bundle at a high sequence that leaves the publisher authority's watermark untouched, so after the declaration is switched back the next official publisher release still activates — proving the two signing authorities cannot lock each other out. A fixture (b2) covers the declaration itself: the same operator-signed bundle dropped while the PUBLISHER authority is declared active is refused as "authority not active" with the publisher bundle still serving, so the choice of authority cannot be made by whoever writes the bundle path. A third fixture (c) covers the operator authority after a data-directory wipe: with a config-resident baseline declared, an older operator-signed bundle is refused; with no baseline declared, it may activate but the install reports degraded coverage rather than full trust. - **SC-003**: An air-gapped install (zero network, ever) reaches full P1 capability: verified activation of a file-dropped signed database, downgrade refusal, and rollback, with behavior byte-identical to a connected install's file-drop path. - **SC-004**: The shipped seed corpus contains at least 25 signatures spanning at least 8 distinct public campaign/technique classes, every one carrying provenance + redistributable license, and every gating signature backed by its eval sample pair; the canonical fixture of each class produces a hard-tier finding naming its TPA id. From 544d7135c72f87629c2c24ed151d81806a06525f Mon Sep 17 00:00:00 2001 From: Algis Dumbris Date: Mon, 24 Aug 2026 07:09:21 +0300 Subject: [PATCH 12/15] docs(specs): correct the SC-002a fixture count after adding (b2) (101-tpa-db) --- specs/101-tpa-db/spec.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/specs/101-tpa-db/spec.md b/specs/101-tpa-db/spec.md index 3bc60040..51ddbefd 100644 --- a/specs/101-tpa-db/spec.md +++ b/specs/101-tpa-db/spec.md @@ -173,7 +173,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **SC-001**: 100% of tampered candidates are rejected before parsing, with the last-known-good database still serving and the reason surfaced — demonstrated by fixture in CI. "Tampered" is defined precisely, because the signature covers the bundle bytes and NOT the sidecar's own encoding: any altered bundle byte, any altered signature value, any altered sidecar identity field (algorithm or key id, whose agreement with the manifest FR-003 already requires), and any mismatched bundle/sidecar pair. Byte changes to the sidecar that alter neither the signature value nor an identity field (whitespace, key order) are explicitly NOT required to be rejected — a detached signature cannot detect them — so the criterion is stated in terms of the fields that carry meaning rather than an unachievable "every byte of the sidecar". - **SC-002**: A validly-signed but lower-sequence bundle is never activated by any non-rollback path (drop, fetch, restart) — including on a fresh install or after a data-directory wipe, where the FR-009b seeded baseline stands in for the absent persisted watermark — and an unsigned candidate never advances the watermark nor survives the signature ratchet — demonstrated by fixture in CI. There are exactly two exemptions, and both are degraded-coverage states rather than silent ones: the FR-009a embedded-default availability fallback, and the baseline-less operator authority of FR-009b (SC-002a fixture (c)), which has no seed to compare against and so carries no anti-downgrade guarantee to claim. Neither counts as a downgrade; both MUST surface as degraded. An explicit rollback succeeds, leaves an audit trail, does not lower the historical watermark, activates only the fingerprint the operator named (a substituted file at the same path is refused), and the deny-listed release it rolled away from is not re-activated by the following refresh cycle. -- **SC-002a**: The watermark cannot be weaponized into a permanent update lockout, demonstrated by two fixtures: (a) a bundle signed by a to-be-revoked key at the MAXIMUM representable sequence, after which a binary carrying a bumped trust epoch and the surviving key successfully activates a new release — proving key-compromise recovery is possible rather than merely documented; (a2) immediately after that bump, an OLD pre-bump bundle carrying the previous epoch and signed by the still-trusted surviving key is REFUSED as stale-epoch replay rather than clearing the re-baselined watermark — the epoch bump must not itself become a downgrade window; and (b) with the operator authority declared active (FR-004b), an operator-authority-signed bundle at a high sequence that leaves the publisher authority's watermark untouched, so after the declaration is switched back the next official publisher release still activates — proving the two signing authorities cannot lock each other out. A fixture (b2) covers the declaration itself: the same operator-signed bundle dropped while the PUBLISHER authority is declared active is refused as "authority not active" with the publisher bundle still serving, so the choice of authority cannot be made by whoever writes the bundle path. A third fixture (c) covers the operator authority after a data-directory wipe: with a config-resident baseline declared, an older operator-signed bundle is refused; with no baseline declared, it may activate but the install reports degraded coverage rather than full trust. +- **SC-002a**: The watermark cannot be weaponized into a permanent update lockout, demonstrated by the following fixtures: (a) a bundle signed by a to-be-revoked key at the MAXIMUM representable sequence, after which a binary carrying a bumped trust epoch and the surviving key successfully activates a new release — proving key-compromise recovery is possible rather than merely documented; (a2) immediately after that bump, an OLD pre-bump bundle carrying the previous epoch and signed by the still-trusted surviving key is REFUSED as stale-epoch replay rather than clearing the re-baselined watermark — the epoch bump must not itself become a downgrade window; and (b) with the operator authority declared active (FR-004b), an operator-authority-signed bundle at a high sequence that leaves the publisher authority's watermark untouched, so after the declaration is switched back the next official publisher release still activates — proving the two signing authorities cannot lock each other out. A fixture (b2) covers the declaration itself: the same operator-signed bundle dropped while the PUBLISHER authority is declared active is refused as "authority not active" with the publisher bundle still serving, so the choice of authority cannot be made by whoever writes the bundle path. A third fixture (c) covers the operator authority after a data-directory wipe: with a config-resident baseline declared, an older operator-signed bundle is refused; with no baseline declared, it may activate but the install reports degraded coverage rather than full trust. - **SC-003**: An air-gapped install (zero network, ever) reaches full P1 capability: verified activation of a file-dropped signed database, downgrade refusal, and rollback, with behavior byte-identical to a connected install's file-drop path. - **SC-004**: The shipped seed corpus contains at least 25 signatures spanning at least 8 distinct public campaign/technique classes, every one carrying provenance + redistributable license, and every gating signature backed by its eval sample pair; the canonical fixture of each class produces a hard-tier finding naming its TPA id. - **SC-005**: The scan-eval CI gate (recall ≥ 0.90 gated categories, hard-negative FP ≤ 0.05) passes with the full seed corpus embedded AND actually scored — i.e. the gate run registers the bundle check and maps every new campaign category (FR-016a), proven by a **deterministic** negative control (FR-016b): removing a designated control signature from the bundle must turn the gate red. Without that control the criterion is vacuous, since today's gate would pass with the bundle entirely absent. A deliberately-regressing corpus change is then blocked twice independently: at CI (cannot merge) and at activation (self-check rejects it) — both demonstrated by fixture. From 82d544ddf49888d80509259499b37e44e3409c50 Mon Sep 17 00:00:00 2001 From: Algis Dumbris Date: Mon, 24 Aug 2026 07:21:07 +0300 Subject: [PATCH 13/15] docs(specs): align FR-007 and FR-009a with the two-exemption rule (101-tpa-db) --- specs/101-tpa-db/spec.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/specs/101-tpa-db/spec.md b/specs/101-tpa-db/spec.md index 51ddbefd..05cc70b3 100644 --- a/specs/101-tpa-db/spec.md +++ b/specs/101-tpa-db/spec.md @@ -123,11 +123,11 @@ The publisher cuts a new database release (new signature for a fresh campaign), **Loader & activation** - **FR-006**: For any candidate carrying a signature, verification MUST run over the exact raw candidate bytes BEFORE those bytes are parsed as JSON or any pattern is compiled; a candidate failing verification MUST be rejected without further processing, keeping last-known-good (extends the Spec 086 fail-closed pipeline with a new first stage). The bytes that are verified MUST be the same bytes that are subsequently parsed (read once into memory, verify, then parse that buffer) — the file MUST NOT be re-read from disk after verification, or a writer racing the loader could swap content between verify and parse. -- **FR-007**: Signing policy MUST be: network-fetched candidates ALWAYS require a valid signature; file-drop candidates are verified whenever a sidecar is present; a new `require_signed_bundle` setting (default off for back-compat with existing unsigned drops) makes the sidecar mandatory for file drops too; the embedded default needs no sidecar (its integrity rides on binary distribution integrity). Bundle status MUST always report whether the active bundle was signature-verified. While an unsigned EXTERNAL (file-dropped or fetched) bundle is active, the scan coverage the approval gates consume MUST be marked degraded so `scan`-mode auto-approval is suspended and changes route to human review — today `internal/security/scanner/inprocess.go` sets `coverageOK` from mere bundle *presence*, so without this an attacker-supplied unsigned corpus would keep silently auto-approving under weakened rules. This degradation rule applies to EXTERNAL candidates ONLY: the embedded default is binary-trusted (its integrity rides on the signed binary distribution, not on a sidecar) and MUST be full-coverage on an ordinary install, or every fresh install would ship with `scan` mode silently disabled. The embedded default is degraded only in the FR-009a case — when it is serving as a fallback BELOW a retained signed watermark, i.e. when the install has evidence a newer signed corpus should be active. This requirement supersedes Spec 087 FR-014's precondition: because FR-004 makes publisher keys embedded in the binary part of the trust set, a fetch is no longer refused merely because the operator configured no key of their own; the "never activate an unverified fetched bundle" guarantee of Spec 087 FR-013/FR-014 is preserved unchanged. +- **FR-007**: Signing policy MUST be: network-fetched candidates ALWAYS require a valid signature; file-drop candidates are verified whenever a sidecar is present; a new `require_signed_bundle` setting (default off for back-compat with existing unsigned drops) makes the sidecar mandatory for file drops too; the embedded default needs no sidecar (its integrity rides on binary distribution integrity). Bundle status MUST always report whether the active bundle was signature-verified. While an unsigned EXTERNAL (file-dropped or fetched) bundle is active, the scan coverage the approval gates consume MUST be marked degraded so `scan`-mode auto-approval is suspended and changes route to human review — today `internal/security/scanner/inprocess.go` sets `coverageOK` from mere bundle *presence*, so without this an attacker-supplied unsigned corpus would keep silently auto-approving under weakened rules. This degradation rule applies to EXTERNAL candidates ONLY: the embedded default is binary-trusted (its integrity rides on the signed binary distribution, not on a sidecar) and MUST be full-coverage on an ordinary install, or every fresh install would ship with `scan` mode silently disabled. The embedded default is degraded only in the FR-009a cases — when it is serving as a fallback BELOW the retained publisher watermark, or when the active operator authority's previously-activated bundle has gone missing, i.e. when the install has evidence that a corpus it should be running is newer than, or not covered by, the embedded one. This requirement supersedes Spec 087 FR-014's precondition: because FR-004 makes publisher keys embedded in the binary part of the trust set, a fetch is no longer refused merely because the operator configured no key of their own; the "never activate an unverified fetched bundle" guarantee of Spec 087 FR-013/FR-014 is preserved unchanged. - **FR-008**: The full validate-before-activate order MUST be: sidecar byte cap → parse sidecar → bundle byte-size ceiling → signature over the raw bytes (per policy, FR-006) → parse manifest → rule-count ceiling → epoch match (FR-005a) → sequence/fingerprint anti-downgrade (FR-009) → version compatibility → compile-all-or-reject → runnable-rules > 0 → activation self-check (Spec 087 FR-008) → atomic activation. FR-011's ceilings MUST therefore be split across the pipeline rather than applied as one stage: the sidecar cap comes first because the sidecar is read before any signature exists to check, bundle byte size is measurable on raw bytes and so guards the verification step itself, while rule count is only knowable after parsing and so cannot precede it. The same reasoning applies to sequence, a manifest field necessarily read after parsing: what MUST precede parsing is signature verification, not the sequence or rule-count check — an unverified candidate's manifest is never trusted for any decision. Any failure at any stage MUST keep last-known-good and record a machine-readable reason and timestamp (surfaced via the existing bundle status), never an empty or partial rule set. - **FR-009**: The system MUST persist the highest sequence ever activated — the **watermark**, scoped per signing authority and trust epoch per FR-004a/FR-005a — and refuse any candidate with a lower sequence, or an equal sequence with a different content fingerprint, as a downgrade/replay, unless an explicit operator rollback (FR-010) is in effect. **The watermark is a property of SIGNED artifacts only**: `sequence` lives in the manifest, so an unsigned candidate's sequence is attacker-chosen. An unsigned/unverified candidate MUST NOT advance the watermark (otherwise a single unsigned drop claiming a huge sequence permanently locks out every legitimate signed release), and once an install has ever activated a signature-verified bundle, unsigned candidates MUST be refused thereafter regardless of `require_signed_bundle` — a signature ratchet, so signing cannot be stripped back off an install by whoever can write the bundle path. The embedded default is exempt from BOTH the watermark and the ratchet — it is the availability fallback of last resort and must never be locked out (scanning must never stop), which is exactly why FR-009a demotes it to degraded coverage instead of refusing it. - **FR-009b**: Each authority's watermark MUST be SEEDED from a trusted baseline rather than starting empty, because an absent watermark accepts anything — with exactly one exception, carved out at the end of this requirement: an operator authority configured without a baseline, which has no seed available and is therefore run as explicitly degraded rather than as a silent zero. Otherwise the anti-downgrade guarantee simply does not exist at first use or after the data directory is reset: an attacker who can write the bundle path drops an OLD but genuinely publisher-signed, current-epoch bundle — say sequence 5 while the shipping release is at 100 — and with no watermark to compare against it activates as a fully signature-verified bundle at FULL coverage, silently reinstating every blind spot fixed between 5 and 100. Neither the FR-009 ratchet nor the FR-010 pin/deny-list helps here; both only bind once persisted state exists. Therefore: the **publisher** authority's watermark MUST be seeded from the sequence of the bundle embedded in the running binary — a candidate older than what the binary already ships is strictly worse than the fallback and MUST be refused — and this baseline MUST be re-applied on every startup, so wiping the data directory cannot lower it below the binary's own corpus. The **operator** authority has no binary-trusted baseline available, so its baseline MUST be declared explicitly in configuration alongside the operator key, and it MUST live in `mcp_config.json` rather than the data directory — that is what makes it durable across the wipe, since a baseline stored in resettable state provides no protection precisely when it is needed. A silent zero default MUST NOT be used: with a zero baseline the operator authority has NO anti-downgrade after state loss at all, and a bundle-path attacker — who is squarely inside the threat model, unlike the config-write attacker — could replay any older genuinely operator-signed bundle at full coverage on a fresh or wiped install. Therefore: if an operator key is configured WITHOUT a baseline, the system MUST treat operator-authority anti-downgrade as unsupported and mark coverage degraded (per FR-007) while an operator-signed bundle is active, rather than silently presenting it as fully trusted. The data-directory watermark then tracks activations above the configured baseline and may only raise the effective minimum, never lower it. The seeded baseline also makes the epoch re-baseline of FR-005a safe: after an epoch bump the new epoch's watermark starts at the authority's baseline, not at zero. -- **FR-009a**: When no valid external candidate is available, the embedded default MUST still serve so scanning never stops, even if its sequence is below the watermark. Where the install has NO retained signed watermark and no missing previously-active external bundle (the ordinary fresh install), this is simply the normal full-coverage state. Two — and only two — conditions demote that fallback to degraded coverage (auto-approval suspended, as in FR-007's unsigned-external case): (a) the embedded default's sequence is below the **publisher** authority's retained watermark, or (b) the **operator** authority is the active one (FR-004b) and its previously-activated external bundle is no longer available or no longer valid. The comparison in (a) MUST be same-namespace: the embedded default is a publisher artifact, and sequences of different authorities are independent counters, so it MUST be compared ONLY against the publisher watermark and never against an operator watermark — a cross-authority numeric comparison is meaningless and would, for an operator corpus at sequence 900 against an embedded publisher corpus at 100, permanently suspend auto-approval on an install whose publisher fallback is perfectly current. Condition (b) is what a cross-authority comparison was reaching for, restricted to the case where the fallback cannot be shown to be at least as fresh as what was lost: an operator corpus is not comparable to the publisher's, so a missing one leaves the install with no evidence the embedded default covers it. It MUST NOT extend to a missing PUBLISHER external bundle, because there condition (a) already decides the question on the same axis — an embedded default at or above the publisher watermark is demonstrably no older than the bundle that went missing (the ordinary "external file removed after a binary upgrade shipped a newer corpus" case) and MUST stay full coverage rather than being degraded merely because an older external file once existed. In either case the retained watermark, the active authority, and the degraded reason MUST be surfaced, and it MUST NOT count as an anti-downgrade violation. This below-watermark fallback is the single explicit exemption to SC-002. +- **FR-009a**: When no valid external candidate is available, the embedded default MUST still serve so scanning never stops, even if its sequence is below the watermark. Where the install has NO retained signed watermark and no missing previously-active external bundle (the ordinary fresh install), this is simply the normal full-coverage state. Two — and only two — conditions demote that fallback to degraded coverage (auto-approval suspended, as in FR-007's unsigned-external case): (a) the embedded default's sequence is below the **publisher** authority's retained watermark, or (b) the **operator** authority is the active one (FR-004b) and its previously-activated external bundle is no longer available or no longer valid. The comparison in (a) MUST be same-namespace: the embedded default is a publisher artifact, and sequences of different authorities are independent counters, so it MUST be compared ONLY against the publisher watermark and never against an operator watermark — a cross-authority numeric comparison is meaningless and would, for an operator corpus at sequence 900 against an embedded publisher corpus at 100, permanently suspend auto-approval on an install whose publisher fallback is perfectly current. Condition (b) is what a cross-authority comparison was reaching for, restricted to the case where the fallback cannot be shown to be at least as fresh as what was lost: an operator corpus is not comparable to the publisher's, so a missing one leaves the install with no evidence the embedded default covers it. It MUST NOT extend to a missing PUBLISHER external bundle, because there condition (a) already decides the question on the same axis — an embedded default at or above the publisher watermark is demonstrably no older than the bundle that went missing (the ordinary "external file removed after a binary upgrade shipped a newer corpus" case) and MUST stay full coverage rather than being degraded merely because an older external file once existed. In either case the retained watermark, the active authority, and the degraded reason MUST be surfaced, and it MUST NOT count as an anti-downgrade violation. This fallback is one of SC-002's two explicit exemptions; the other is FR-009b's baseline-less operator authority. - **FR-010**: An explicit rollback action MUST let an operator revert to the embedded default or a named bundle file: the target is re-validated (signature policy and self-check still apply), activated despite a lower sequence, and the override is loudly logged and visible in bundle status. **The rollback target MUST be identified by content fingerprint, not by pathname alone.** Rollback is the one path that deliberately bypasses the watermark, so identifying its target by an attacker-writable path hands the adversary the choice of what the bypass lands on: with a legitimate bundle at sequence 100 active and the operator intending to revert to a known-good sequence 90, an attacker who can write the bundle path — in scope, unlike the config-write attacker — swaps the named file for a genuinely publisher-signed, current-epoch bundle at sequence 5 before the rollback reads it. It passes signature, epoch, compilation, and plausibly the self-check; the watermark that would normally refuse it is switched off by the rollback itself; and the result is then PINNED, so the downgrade persists across refresh cycles. FR-006's read-once rule closes the verify/parse race but not substitution before the read. Therefore the operator MUST supply the expected fingerprint (as offered by bundle status history), the path is only a hint for where to find those bytes, and a loaded artifact whose fingerprint differs MUST be refused as a rollback-target mismatch rather than activated. A rollback target MUST additionally still be at or above **its own authority's** FR-009b baseline — the publisher baseline for a publisher-signed target, the config-declared operator baseline for an operator-signed one. The comparison MUST NOT be made against the publisher baseline in both cases: sequences of different authorities are independent counters (FR-004a), so measuring an operator target at sequence 9 against a publisher embedded corpus at 100 would refuse every legitimate operator rollback. Where an authority has NO declared baseline — possible only for the operator authority, which FR-009b explicitly permits to run without one — there is no floor to enforce and none is invented: the rollback proceeds and the install is marked degraded for exactly as long as that operator-signed bundle is active, which is already FR-009b's treatment of a baseline-less operator authority. Neither refusing the rollback (which would break FR-010's guarantee for a configuration FR-009b allows) nor assuming a zero floor (which would claim an anti-downgrade guarantee that does not exist) is acceptable. The embedded default is the one target always permitted, since it IS the publisher baseline and is the availability fallback of last resort. Rollback MUST NOT lower the historical high-watermark: the watermark is monotonic and normal activation MUST still require exceeding it. Rollback is instead expressed as a **pinned exception** — the operator pins the rolled-back artifact by fingerprint, and the (sequence, fingerprint) being rolled away from is deny-listed so the very release the operator just rejected cannot be re-activated by the next refresh cycle (and neither can any intermediate release between the pin and the watermark). Clearing the pin returns the install to normal watermark-governed activation. - **FR-011**: The loader MUST enforce load ceilings (maximum bundle byte size and maximum rule count, fixed in design) and reject a candidate exceeding them as a whole; all patterns remain RE2-only. **The sidecar needs its own, much smaller ceiling and a bounded read**, because it is the one artifact that must be read and parsed BEFORE any signature has been verified (its algorithm identifier, key id, and signature value are the inputs to verification), which puts it outside the protection FR-006 gives the bundle. Without it the in-scope bundle-path attacker simply writes a multi-gigabyte or never-terminating sidecar next to an otherwise acceptable bundle and exhausts memory or stalls the refresh before anything can be rejected — a denial of the scanner's own update path. The sidecar MUST therefore be read through a hard byte cap sized to its fixed small structure and rejected on overrun, before it is parsed, on both the file-drop and fetch paths. A byte cap alone is not sufficient, because it bounds bytes consumed but not time spent: the same attacker replaces the sidecar (or bundle) path with a FIFO and never writes to it, so the read blocks forever without ever approaching the cap and Spec 087's single-flight refresh stays occupied, blocking every legitimate database update while the loader waits. Both local candidate artifacts MUST therefore be required to be REGULAR files — a FIFO, device, socket, or a symlink resolving to one is refused outright, not read — and every candidate read (local files and fetch response bodies alike) MUST run under an enforceable I/O deadline whose expiry releases the single-flight slot and records a rejection reason, so no candidate can pin the refresh path. A database at the ceilings MUST NOT measurably regress the scan hot path (see SC-008). - **FR-011a**: Activation MUST be crash-consistent, not merely atomic in memory. Spec 087 FR-010 makes the rule-set swap atomic with respect to concurrent scans; that says nothing about persisted state, and a process death between "activated" and "watermark written" reopens exactly the replay window FR-009 exists to close (while writing the watermark first can leave the install with only an older recoverable bundle). The activation record — active fingerprint, sequence, per-authority watermarks and their epochs, the fingerprint each watermark was set by, signature-verified state, any FR-010 pin, and the FR-010 deny-list — MUST be persisted as ONE transaction, and the recoverable last-known-good bundle MUST be durable before the in-memory pointer is published. The deny-list and watermark fingerprint are load-bearing members of that transaction, not bookkeeping: a crash that preserved a rollback while losing the deny-list would let the next refresh immediately re-activate the exact release the operator had just rejected, silently undoing the rollback. Startup, refresh, hot-reload, and rollback MUST all serialize through that same transaction, so any crash leaves the install on a consistent (bundle, watermark) pair. From afff3cb7988c64284033d329fd9e61f5a83bd24f Mon Sep 17 00:00:00 2001 From: Algis Dumbris Date: Mon, 24 Aug 2026 13:37:59 +0300 Subject: [PATCH 14/15] docs(specs): scope pins per epoch, gate telemetry on provenance, fix exemption count (101-tpa-db) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cross-model review (opencode / gpt-5.6-sol) round 6 on specs/101-tpa-db/spec.md. All five findings verified against the document and the cited code before editing. - Edge Cases (threat model): the boundary is now explicit about the data directory — the bundle-path attacker may also be able to delete/reset the persisted security state (equivalent to the wipe), against which only the binary/config-anchored guarantees survive (seeded publisher baseline, config-resident operator baseline, require_signed_bundle, active-authority declaration); ratchet and pin/deny-list are best-effort. Chosen-value corruption of the state store is out of scope (that attacker is already past every mcpproxy guarantee). - Edge Cases + FR-009a: two stale 'single exemption to SC-002' statements now name both exemptions, matching SC-002's 'exactly two'. - US1 acceptance 3 + SC-002: 'lower sequence' is qualified everywhere as a same-authority, same-epoch comparison; a deliberate FR-004b authority switch is by definition not a downgrade and needs no exemption. - FR-010 + FR-005a: pin/deny-list entries are keyed (authority, epoch, sequence, fingerprint); an authority switch suspends (not deletes) the non-active authority's entries; an epoch bump retires prior-epoch entries so stale rollback state cannot block the key-compromise recovery bundle. - FR-021: 'resolves to a publisher-issued value' is a provenance test (embedded default or publisher-authority-verified), never a claimed-version-string match — an operator corpus can copy a publisher version string verbatim; the backstop also rejects sequence-present-with-version-other cross-field. - Docs-only change; no Go sources touched. - Round 6 of the raised (10-round) cross-model review cap. --- specs/101-tpa-db/spec.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/specs/101-tpa-db/spec.md b/specs/101-tpa-db/spec.md index 05cc70b3..c384a912 100644 --- a/specs/101-tpa-db/spec.md +++ b/specs/101-tpa-db/spec.md @@ -46,7 +46,7 @@ An operator receives a TPA signature database out of band — a `scanner-bundle. 1. **Given** a candidate bundle whose detached signature verifies against a configured trust anchor and whose sequence is greater than the active bundle's, **When** refresh runs, **Then** the candidate proceeds through the existing validation pipeline and, on success, becomes active with its sequence persisted and `signature_verified=true` surfaced in bundle status. 2. **Given** a candidate whose bytes were altered after signing (signature mismatch), **When** refresh runs, **Then** the candidate is rejected BEFORE its JSON is parsed or any pattern compiled, the active bundle is unchanged, and a "signature verification failed" reason is recorded. -3. **Given** a validly-signed candidate whose sequence is lower than the highest sequence ever activated on this install, **When** refresh runs, **Then** it is refused as a downgrade, the active bundle is unchanged, and a "downgrade refused" reason names both sequences. +3. **Given** a validly-signed candidate whose sequence is lower than its OWN signing authority's watermark — the highest sequence that authority has ever activated on this install within the current trust epoch (FR-004a/FR-005a), never a cross-authority comparison — **When** refresh runs, **Then** it is refused as a downgrade, the active bundle is unchanged, and a "downgrade refused" reason names both sequences. 4. **Given** `require_signed_bundle` is enabled and a file-drop candidate has no signature sidecar, **When** refresh runs, **Then** the candidate is refused (fail-closed) with a "signature required" reason; with the setting disabled (default) and no signature-verified bundle ever having been active on this install, an unsigned drop is accepted through the existing Spec 086 validation path but surfaced as `signature_verified=false`, does NOT advance the sequence watermark, and marks coverage degraded so `scan`-mode auto-approval is suspended while it is active. 4a. **Given** an install that has already activated a signature-verified bundle, **When** an unsigned candidate is dropped (with `require_signed_bundle` still off), **Then** it is refused by the signature ratchet — signing cannot be stripped back off an install by whoever can write the bundle path — and an unsigned candidate claiming an arbitrarily high sequence never becomes the watermark, so it cannot lock out subsequent legitimate signed releases. 5. **Given** a network-fetched candidate (Spec 087 opt-in path), **When** it arrives, **Then** signature verification is ALWAYS mandatory regardless of `require_signed_bundle` — an unsigned or wrongly-signed fetched bundle is never activated (parity with Spec 087 FR-013/FR-014). @@ -93,11 +93,11 @@ The publisher cuts a new database release (new signature for a fresh campaign), ### Edge Cases - **Air-gapped host, forever**: every P1 behavior (verification, sequence check, rollback, activation self-check) is network-free; the trust anchors ship inside the binary; staleness is surfaced but never blocks scanning — an old database is degraded coverage, not an outage. -- **Threat-model boundary — attacker with config write**: FR-004 lets operator-configured keys join the trust set, so anyone who can write `mcp_config.json` can trust their own key and sign anything. This is explicitly OUT of the threat model and is not a regression: an attacker with config write can already disable quarantine outright or set a server's trust mode to `auto`. The attacker this design defends against is the one who can write the BUNDLE PATH or intercept the FETCH but not the config — which is precisely why the signature ratchet (FR-009) is keyed to install state rather than to the bundle path, why the active-authority declaration (FR-004b) and the operator baseline (FR-009b) are config-resident, and why `require_signed_bundle` is config-resident (none of them must be defeatable from the data dir alone). +- **Threat-model boundary — attacker with config write**: FR-004 lets operator-configured keys join the trust set, so anyone who can write `mcp_config.json` can trust their own key and sign anything. This is explicitly OUT of the threat model and is not a regression: an attacker with config write can already disable quarantine outright or set a server's trust mode to `auto`. The attacker this design defends against is the one who can write the BUNDLE PATH or intercept the FETCH but not the config — which is precisely why the signature ratchet (FR-009) is keyed to install state rather than to the bundle path, why the active-authority declaration (FR-004b) and the operator baseline (FR-009b) are config-resident, and why `require_signed_bundle` is config-resident (none of them must be defeatable from the data dir alone). The boundary MUST be read conservatively: because the bundle path may be configured to live inside the data directory, the design MUST NOT assume the bundle-path attacker cannot also delete or reset the data-directory persisted security state (watermark record, ratchet flag, pin/deny-list) — the worst such an attacker can achieve is exactly the "anti-downgrade state lost" wipe below, against which the guarantees that survive are the ones anchored outside the data dir: the FR-009b publisher baseline (re-seeded from the binary at every startup), the config-resident operator baseline, `require_signed_bundle`, and the FR-004b active-authority declaration. The ratchet and the pin/deny-list are data-dir resident and are therefore explicitly best-effort hardening against this attacker, never the sole defense. What the design DOES assume is that the attacker cannot substitute or corrupt the persisted state with CHOSEN values (e.g. rewrite the watermark to a lower number without triggering the seeded-baseline floor): the state store is the same integrity-protected local database that already holds approvals and quarantine state, and an attacker who can write arbitrary records into it is already past every guarantee mcpproxy makes, not just this feature's. - **Signature key rotation**: the trust anchor is a SET of publisher keys embedded per binary release. Rotation is a binary-release event: a release ships old+new keys, bundles are signed with the new key, the old key is removed in a later release. Keys are NEVER delivered over the network, and rotation is not done casually — every shipped binary pins its key set, so a hasty rotation strands older installs on unsigned-refresh only (same philosophy as the app-update signing keys). Key compromise → remove the key in the next release, **bump the trust epoch (FR-005a)**, and publish a bundle in the new epoch signed by the surviving key. The epoch bump is what makes recovery possible at all: a compromised key can sign at the maximum representable sequence, and against a monotonic watermark no surviving key could ever outbid that within the same epoch, so revocation alone would leave installs permanently unable to accept any future release. - **Bundle signed by a formerly-trusted, now-removed key**: verification fails on binaries that dropped the key (fail-closed to last-known-good); older binaries still trusting it keep working — the downgrade watermark still blocks sequence replay. - **Tampered bundle / tampered sidecar / swapped pair** (valid signature belonging to a different bundle's bytes): all reduce to signature-verification failure before parse; never a crash, never partial load. -- **Downgrade attack via the embedded default**: a fresh binary whose embedded bundle sequence is LOWER than a previously-activated external bundle's persisted watermark does not silently downgrade — the external candidate re-validates and wins by precedence; if only the embedded bundle is available, it serves (scanning must never stop) with the watermark retained and the situation surfaced, but per FR-009a as DEGRADED coverage, so an attacker who merely deletes the external bundle cannot use the weaker embedded corpus to obtain `scan`-mode auto-approvals. This is the one explicit exemption to SC-002. +- **Downgrade attack via the embedded default**: a fresh binary whose embedded bundle sequence is LOWER than a previously-activated external bundle's persisted watermark does not silently downgrade — the external candidate re-validates and wins by precedence; if only the embedded bundle is available, it serves (scanning must never stop) with the watermark retained and the situation surfaced, but per FR-009a as DEGRADED coverage, so an attacker who merely deletes the external bundle cannot use the weaker embedded corpus to obtain `scan`-mode auto-approvals. This is one of SC-002's exactly two exemptions (the other is FR-009b's baseline-less operator authority). - **Anti-downgrade state lost** (data directory wiped/reset): the persisted watermark resets to the FR-009b seeded baseline — NOT to zero, so a wiped install still refuses any publisher-signed bundle older than the corpus its own binary ships. For the operator authority the same protection holds only if the operator declared a config-resident baseline; without one, operator-authority anti-downgrade does not survive the wipe and the install runs degraded while an operator-signed bundle is active, by FR-009b, rather than pretending to a guarantee it does not have. What a wipe does lose is the FR-009 signature-ratchet state and any FR-010 pin/deny-list (both data-dir resident), so a wiped install accepts an unsigned drop again. This is accepted — the ratchet is best-effort hardening on top of signing, not the sole defense, and a wiped data dir already resets approvals. Operators who need this closed absolutely set `require_signed_bundle`, which is config-resident rather than data-dir-resident and therefore survives the wipe. - **Equal sequence, different bytes** (two distinct bundles claiming the same sequence): refused as suspicious unless it is the currently-active fingerprint (idempotent re-drop stays a no-op per Spec 087 FR-011). - **Clock skew**: no validity windows or expiry in v1 — sequence ordering, not wall-clock time, is the trust signal; `generated_at` is advisory freshness only. @@ -118,7 +118,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **FR-004a**: Anti-downgrade state MUST be tracked per signing authority, not in one global namespace. An operator maintaining a self-built corpus is an explicitly supported persona (air-gapped installs, private signature sets), and with a single shared counter their locally-signed bundle at sequence N would raise the one global watermark above the publisher's release line and permanently block official updates — or, symmetrically, a publisher release would block the operator's own corpus. Each authority MUST therefore carry its own watermark, and a candidate MUST be compared only against the watermark of the authority whose key verified it. Choosing an operator-signed corpus over a higher-sequence publisher one is a deliberate operator decision (declared per FR-004b) and MUST NOT by itself be treated as a downgrade or as degraded coverage — the operator configured that key precisely to assert their own corpus — but the active authority MUST be visible in bundle status (FR-020) so the choice is never invisible. Grouping all publisher keys into one authority is what keeps per-authority watermarks compatible with key rotation: a rotated-in key inherits the publisher authority's watermark rather than starting a fresh one. - **FR-004b**: The **active signing authority MUST be a config-resident declaration**, defaulting to the publisher authority, and MUST NOT be inferred solely from whichever key happens to verify a candidate. A candidate whose verifying key belongs to a non-active authority MUST be refused with an "authority not active" reason, exactly as any other failed candidate (last-known-good keeps serving). Without this, per-authority watermarks (FR-004a) open the very hole they were introduced to avoid: because each authority is compared only against its own counter, an attacker who can write the bundle path — squarely inside the threat model, unlike the config-write attacker — can replace a current publisher bundle at sequence 100 with ANY operator-signed artifact that merely clears the operator authority's FR-009b baseline, silently swapping the install onto an older or deliberately narrower corpus while it still reports `signature_verified=true` at FULL coverage. The publisher watermark cannot detect this, since the comparison never crosses namespaces, and no operator decision is involved — whoever writes the path picks the authority. Config-residency is what puts that choice out of the attacker's reach, matching `require_signed_bundle` (FR-007) and the operator baseline (FR-009b), which are config-resident for the same reason. Both key sets MAY remain configured simultaneously; only artifacts of the declared active authority activate, and switching authorities is a config edit, not a file drop. - **FR-005**: Key rotation MUST be supported by trusting multiple keys simultaneously: a binary release MAY add a new key while retaining the old for at least one release cycle, and key removal is also a binary-release event. The design MUST document the compromise procedure and the "do not rotate casually" operational stance. -- **FR-005a**: Each signing authority's trust set MUST carry a **trust epoch** — a monotonic per-authority integer that advances ONLY via a binary release — and every anti-downgrade comparison (FR-009) MUST be scoped `(authority, epoch, sequence)`, ordered by epoch first. The epoch MUST be carried inside the SIGNED manifest (FR-002) and a candidate MUST be accepted only when its manifest epoch EQUALS the current binary-trusted epoch of the authority whose key verified it — never merely less-than-or-equal. Binding the epoch to the signed bytes and requiring exact equality is what stops the epoch bump from becoming a replay window of its own: if the epoch lived only in the binary, then after a bump-and-revoke the re-baselined watermark would start from zero, and an OLD, weaker bundle legitimately signed by a still-trusted surviving key would be silently reinterpreted as belonging to the new epoch, clear the fresh watermark, and activate with `signature_verified=true` and full coverage — reinstating exactly the blind spots the recovery release was published to close. Without this, key-compromise recovery is impossible rather than merely painful: an attacker holding a compromised publisher key signs a bundle at the maximum representable sequence, and because the watermark is monotonic and no surviving key can ever exceed that sequence, the install is permanently unable to accept any future legitimate release — a one-shot permanent denial of updates, downgraded only to the FR-009a fallback. A binary release that revokes the compromised key MUST therefore also bump the epoch, and a validly-signed bundle whose manifest epoch matches a newly-bumped binary epoch MUST supersede the retained watermark and re-baseline it. A bundle whose manifest epoch is BELOW the current binary-trusted epoch MUST be refused as stale-epoch replay even when its signature verifies against a still-trusted key. Epoch advancement is safe to treat as authoritative because it arrives only inside a release binary, whose integrity is already the root of trust for the embedded key set (FR-004) — it is not attacker-reachable over the bundle or fetch path. The accepted consequence of exact-equality matching is that a binary still on the OLD epoch will refuse the new epoch's bundles and stop receiving database updates until it is itself updated. That is deliberate, not incidental: an epoch bump means a key was compromised, and an un-updated binary still trusts that compromised key, so freezing its database is safer than feeding it artifacts whose trust basis it cannot evaluate. It also matches the existing "a hasty rotation strands older installs" stance, and it MUST be surfaced as a stale-epoch degraded state (FR-020) rather than failing silently. +- **FR-005a**: Each signing authority's trust set MUST carry a **trust epoch** — a monotonic per-authority integer that advances ONLY via a binary release — and every anti-downgrade comparison (FR-009) MUST be scoped `(authority, epoch, sequence)`, ordered by epoch first. The epoch MUST be carried inside the SIGNED manifest (FR-002) and a candidate MUST be accepted only when its manifest epoch EQUALS the current binary-trusted epoch of the authority whose key verified it — never merely less-than-or-equal. Binding the epoch to the signed bytes and requiring exact equality is what stops the epoch bump from becoming a replay window of its own: if the epoch lived only in the binary, then after a bump-and-revoke the re-baselined watermark would start from zero, and an OLD, weaker bundle legitimately signed by a still-trusted surviving key would be silently reinterpreted as belonging to the new epoch, clear the fresh watermark, and activate with `signature_verified=true` and full coverage — reinstating exactly the blind spots the recovery release was published to close. Without this, key-compromise recovery is impossible rather than merely painful: an attacker holding a compromised publisher key signs a bundle at the maximum representable sequence, and because the watermark is monotonic and no surviving key can ever exceed that sequence, the install is permanently unable to accept any future legitimate release — a one-shot permanent denial of updates, downgraded only to the FR-009a fallback. A binary release that revokes the compromised key MUST therefore also bump the epoch, and a validly-signed bundle whose manifest epoch matches a newly-bumped binary epoch MUST supersede the retained watermark and re-baseline it; the bump likewise retires the authority's prior-epoch FR-010 pins and deny-list entries, so stale rollback state cannot block the recovery bundle (FR-010). A bundle whose manifest epoch is BELOW the current binary-trusted epoch MUST be refused as stale-epoch replay even when its signature verifies against a still-trusted key. Epoch advancement is safe to treat as authoritative because it arrives only inside a release binary, whose integrity is already the root of trust for the embedded key set (FR-004) — it is not attacker-reachable over the bundle or fetch path. The accepted consequence of exact-equality matching is that a binary still on the OLD epoch will refuse the new epoch's bundles and stop receiving database updates until it is itself updated. That is deliberate, not incidental: an epoch bump means a key was compromised, and an un-updated binary still trusts that compromised key, so freezing its database is safer than feeding it artifacts whose trust basis it cannot evaluate. It also matches the existing "a hasty rotation strands older installs" stance, and it MUST be surfaced as a stale-epoch degraded state (FR-020) rather than failing silently. **Loader & activation** @@ -128,7 +128,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **FR-009**: The system MUST persist the highest sequence ever activated — the **watermark**, scoped per signing authority and trust epoch per FR-004a/FR-005a — and refuse any candidate with a lower sequence, or an equal sequence with a different content fingerprint, as a downgrade/replay, unless an explicit operator rollback (FR-010) is in effect. **The watermark is a property of SIGNED artifacts only**: `sequence` lives in the manifest, so an unsigned candidate's sequence is attacker-chosen. An unsigned/unverified candidate MUST NOT advance the watermark (otherwise a single unsigned drop claiming a huge sequence permanently locks out every legitimate signed release), and once an install has ever activated a signature-verified bundle, unsigned candidates MUST be refused thereafter regardless of `require_signed_bundle` — a signature ratchet, so signing cannot be stripped back off an install by whoever can write the bundle path. The embedded default is exempt from BOTH the watermark and the ratchet — it is the availability fallback of last resort and must never be locked out (scanning must never stop), which is exactly why FR-009a demotes it to degraded coverage instead of refusing it. - **FR-009b**: Each authority's watermark MUST be SEEDED from a trusted baseline rather than starting empty, because an absent watermark accepts anything — with exactly one exception, carved out at the end of this requirement: an operator authority configured without a baseline, which has no seed available and is therefore run as explicitly degraded rather than as a silent zero. Otherwise the anti-downgrade guarantee simply does not exist at first use or after the data directory is reset: an attacker who can write the bundle path drops an OLD but genuinely publisher-signed, current-epoch bundle — say sequence 5 while the shipping release is at 100 — and with no watermark to compare against it activates as a fully signature-verified bundle at FULL coverage, silently reinstating every blind spot fixed between 5 and 100. Neither the FR-009 ratchet nor the FR-010 pin/deny-list helps here; both only bind once persisted state exists. Therefore: the **publisher** authority's watermark MUST be seeded from the sequence of the bundle embedded in the running binary — a candidate older than what the binary already ships is strictly worse than the fallback and MUST be refused — and this baseline MUST be re-applied on every startup, so wiping the data directory cannot lower it below the binary's own corpus. The **operator** authority has no binary-trusted baseline available, so its baseline MUST be declared explicitly in configuration alongside the operator key, and it MUST live in `mcp_config.json` rather than the data directory — that is what makes it durable across the wipe, since a baseline stored in resettable state provides no protection precisely when it is needed. A silent zero default MUST NOT be used: with a zero baseline the operator authority has NO anti-downgrade after state loss at all, and a bundle-path attacker — who is squarely inside the threat model, unlike the config-write attacker — could replay any older genuinely operator-signed bundle at full coverage on a fresh or wiped install. Therefore: if an operator key is configured WITHOUT a baseline, the system MUST treat operator-authority anti-downgrade as unsupported and mark coverage degraded (per FR-007) while an operator-signed bundle is active, rather than silently presenting it as fully trusted. The data-directory watermark then tracks activations above the configured baseline and may only raise the effective minimum, never lower it. The seeded baseline also makes the epoch re-baseline of FR-005a safe: after an epoch bump the new epoch's watermark starts at the authority's baseline, not at zero. - **FR-009a**: When no valid external candidate is available, the embedded default MUST still serve so scanning never stops, even if its sequence is below the watermark. Where the install has NO retained signed watermark and no missing previously-active external bundle (the ordinary fresh install), this is simply the normal full-coverage state. Two — and only two — conditions demote that fallback to degraded coverage (auto-approval suspended, as in FR-007's unsigned-external case): (a) the embedded default's sequence is below the **publisher** authority's retained watermark, or (b) the **operator** authority is the active one (FR-004b) and its previously-activated external bundle is no longer available or no longer valid. The comparison in (a) MUST be same-namespace: the embedded default is a publisher artifact, and sequences of different authorities are independent counters, so it MUST be compared ONLY against the publisher watermark and never against an operator watermark — a cross-authority numeric comparison is meaningless and would, for an operator corpus at sequence 900 against an embedded publisher corpus at 100, permanently suspend auto-approval on an install whose publisher fallback is perfectly current. Condition (b) is what a cross-authority comparison was reaching for, restricted to the case where the fallback cannot be shown to be at least as fresh as what was lost: an operator corpus is not comparable to the publisher's, so a missing one leaves the install with no evidence the embedded default covers it. It MUST NOT extend to a missing PUBLISHER external bundle, because there condition (a) already decides the question on the same axis — an embedded default at or above the publisher watermark is demonstrably no older than the bundle that went missing (the ordinary "external file removed after a binary upgrade shipped a newer corpus" case) and MUST stay full coverage rather than being degraded merely because an older external file once existed. In either case the retained watermark, the active authority, and the degraded reason MUST be surfaced, and it MUST NOT count as an anti-downgrade violation. This fallback is one of SC-002's two explicit exemptions; the other is FR-009b's baseline-less operator authority. -- **FR-010**: An explicit rollback action MUST let an operator revert to the embedded default or a named bundle file: the target is re-validated (signature policy and self-check still apply), activated despite a lower sequence, and the override is loudly logged and visible in bundle status. **The rollback target MUST be identified by content fingerprint, not by pathname alone.** Rollback is the one path that deliberately bypasses the watermark, so identifying its target by an attacker-writable path hands the adversary the choice of what the bypass lands on: with a legitimate bundle at sequence 100 active and the operator intending to revert to a known-good sequence 90, an attacker who can write the bundle path — in scope, unlike the config-write attacker — swaps the named file for a genuinely publisher-signed, current-epoch bundle at sequence 5 before the rollback reads it. It passes signature, epoch, compilation, and plausibly the self-check; the watermark that would normally refuse it is switched off by the rollback itself; and the result is then PINNED, so the downgrade persists across refresh cycles. FR-006's read-once rule closes the verify/parse race but not substitution before the read. Therefore the operator MUST supply the expected fingerprint (as offered by bundle status history), the path is only a hint for where to find those bytes, and a loaded artifact whose fingerprint differs MUST be refused as a rollback-target mismatch rather than activated. A rollback target MUST additionally still be at or above **its own authority's** FR-009b baseline — the publisher baseline for a publisher-signed target, the config-declared operator baseline for an operator-signed one. The comparison MUST NOT be made against the publisher baseline in both cases: sequences of different authorities are independent counters (FR-004a), so measuring an operator target at sequence 9 against a publisher embedded corpus at 100 would refuse every legitimate operator rollback. Where an authority has NO declared baseline — possible only for the operator authority, which FR-009b explicitly permits to run without one — there is no floor to enforce and none is invented: the rollback proceeds and the install is marked degraded for exactly as long as that operator-signed bundle is active, which is already FR-009b's treatment of a baseline-less operator authority. Neither refusing the rollback (which would break FR-010's guarantee for a configuration FR-009b allows) nor assuming a zero floor (which would claim an anti-downgrade guarantee that does not exist) is acceptable. The embedded default is the one target always permitted, since it IS the publisher baseline and is the availability fallback of last resort. Rollback MUST NOT lower the historical high-watermark: the watermark is monotonic and normal activation MUST still require exceeding it. Rollback is instead expressed as a **pinned exception** — the operator pins the rolled-back artifact by fingerprint, and the (sequence, fingerprint) being rolled away from is deny-listed so the very release the operator just rejected cannot be re-activated by the next refresh cycle (and neither can any intermediate release between the pin and the watermark). Clearing the pin returns the install to normal watermark-governed activation. +- **FR-010**: An explicit rollback action MUST let an operator revert to the embedded default or a named bundle file: the target is re-validated (signature policy and self-check still apply), activated despite a lower sequence, and the override is loudly logged and visible in bundle status. **The rollback target MUST be identified by content fingerprint, not by pathname alone.** Rollback is the one path that deliberately bypasses the watermark, so identifying its target by an attacker-writable path hands the adversary the choice of what the bypass lands on: with a legitimate bundle at sequence 100 active and the operator intending to revert to a known-good sequence 90, an attacker who can write the bundle path — in scope, unlike the config-write attacker — swaps the named file for a genuinely publisher-signed, current-epoch bundle at sequence 5 before the rollback reads it. It passes signature, epoch, compilation, and plausibly the self-check; the watermark that would normally refuse it is switched off by the rollback itself; and the result is then PINNED, so the downgrade persists across refresh cycles. FR-006's read-once rule closes the verify/parse race but not substitution before the read. Therefore the operator MUST supply the expected fingerprint (as offered by bundle status history), the path is only a hint for where to find those bytes, and a loaded artifact whose fingerprint differs MUST be refused as a rollback-target mismatch rather than activated. A rollback target MUST additionally still be at or above **its own authority's** FR-009b baseline — the publisher baseline for a publisher-signed target, the config-declared operator baseline for an operator-signed one. The comparison MUST NOT be made against the publisher baseline in both cases: sequences of different authorities are independent counters (FR-004a), so measuring an operator target at sequence 9 against a publisher embedded corpus at 100 would refuse every legitimate operator rollback. Where an authority has NO declared baseline — possible only for the operator authority, which FR-009b explicitly permits to run without one — there is no floor to enforce and none is invented: the rollback proceeds and the install is marked degraded for exactly as long as that operator-signed bundle is active, which is already FR-009b's treatment of a baseline-less operator authority. Neither refusing the rollback (which would break FR-010's guarantee for a configuration FR-009b allows) nor assuming a zero floor (which would claim an anti-downgrade guarantee that does not exist) is acceptable. The embedded default is the one target always permitted, since it IS the publisher baseline and is the availability fallback of last resort. Rollback MUST NOT lower the historical high-watermark: the watermark is monotonic and normal activation MUST still require exceeding it. Rollback is instead expressed as a **pinned exception** — the operator pins the rolled-back artifact by fingerprint, and the (sequence, fingerprint) being rolled away from is deny-listed so the very release the operator just rejected cannot be re-activated by the next refresh cycle (and neither can any intermediate release between the pin and the watermark). Clearing the pin returns the install to normal watermark-governed activation. **Pin and deny-list state MUST be scoped like the watermark it excepts**: each entry is keyed by (authority, trust epoch, sequence, fingerprint) and binds only while its authority is the active one (FR-004b) and its epoch equals the binary's current epoch for that authority. Switching the active authority SUSPENDS (does not delete) the non-active authority's pin and deny-list, so switching back restores them unchanged. A binary release that bumps an authority's trust epoch (FR-005a) RETIRES that authority's prior-epoch pins and deny-list entries, loudly logged: the epoch bump is the key-compromise recovery path and MUST NOT be blockable by stale pin state — a retained old-epoch pin could not re-activate its artifact anyway (the pinned manifest's epoch no longer matches, FR-005a), so honoring it could only strand the install on the embedded fallback while blocking the very recovery bundle the bump exists to deliver. - **FR-011**: The loader MUST enforce load ceilings (maximum bundle byte size and maximum rule count, fixed in design) and reject a candidate exceeding them as a whole; all patterns remain RE2-only. **The sidecar needs its own, much smaller ceiling and a bounded read**, because it is the one artifact that must be read and parsed BEFORE any signature has been verified (its algorithm identifier, key id, and signature value are the inputs to verification), which puts it outside the protection FR-006 gives the bundle. Without it the in-scope bundle-path attacker simply writes a multi-gigabyte or never-terminating sidecar next to an otherwise acceptable bundle and exhausts memory or stalls the refresh before anything can be rejected — a denial of the scanner's own update path. The sidecar MUST therefore be read through a hard byte cap sized to its fixed small structure and rejected on overrun, before it is parsed, on both the file-drop and fetch paths. A byte cap alone is not sufficient, because it bounds bytes consumed but not time spent: the same attacker replaces the sidecar (or bundle) path with a FIFO and never writes to it, so the read blocks forever without ever approaching the cap and Spec 087's single-flight refresh stays occupied, blocking every legitimate database update while the loader waits. Both local candidate artifacts MUST therefore be required to be REGULAR files — a FIFO, device, socket, or a symlink resolving to one is refused outright, not read — and every candidate read (local files and fetch response bodies alike) MUST run under an enforceable I/O deadline whose expiry releases the single-flight slot and records a rejection reason, so no candidate can pin the refresh path. A database at the ceilings MUST NOT measurably regress the scan hot path (see SC-008). - **FR-011a**: Activation MUST be crash-consistent, not merely atomic in memory. Spec 087 FR-010 makes the rule-set swap atomic with respect to concurrent scans; that says nothing about persisted state, and a process death between "activated" and "watermark written" reopens exactly the replay window FR-009 exists to close (while writing the watermark first can leave the install with only an older recoverable bundle). The activation record — active fingerprint, sequence, per-authority watermarks and their epochs, the fingerprint each watermark was set by, signature-verified state, any FR-010 pin, and the FR-010 deny-list — MUST be persisted as ONE transaction, and the recoverable last-known-good bundle MUST be durable before the in-memory pointer is published. The deny-list and watermark fingerprint are load-bearing members of that transaction, not bookkeeping: a crash that preserved a rollback while losing the deny-list would let the next refresh immediately re-activate the exact release the operator had just rejected, silently undoing the rollback. Startup, refresh, hot-reload, and rollback MUST all serialize through that same transaction, so any crash leaves the install on a consistent (bundle, watermark) pair. @@ -151,7 +151,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), **Observability, telemetry & docs** - **FR-020**: Bundle status on all existing surfaces (CLI security commands, REST security overview, Web UI) MUST additionally report sequence, signing authority and trust epoch, signature-verified state, and staleness (age since `generated_at`), alongside the existing version/fingerprint/source/load-error fields, and MUST show the last rejection reason (tamper, downgrade, stale-epoch replay, unsigned-refused-by-ratchet, authority-not-active, rollback-target mismatch, gate failure) so an operator can tell "current and verified" from "stuck on last-known-good and why". Where the active database has demoted the install to DEGRADED coverage (FR-007 unsigned-active, FR-009a embedded fallback), that state and its reason MUST be surfaced too — a silently suspended `scan`-mode auto-approval is otherwise indistinguishable from a quiet install. -- **FR-021**: The existing anonymous TPA-scanner telemetry MUST gain the active bundle version, sequence, and source (and signature-verified state) so database adoption and freshness are measurable fleet-wide; the existing telemetry opt-out MUST fully cover these fields, and no new identifying data is introduced. This is NOT a drop-in addition: `internal/telemetry/anonymity.go`'s schema-v8 backstop hard-whitelists the `tpa_scanner` sub-object to `{scans_completed, scans_failed, scans_with_findings, findings}` and requires every scalar to be a non-negative integer, so a string `bundle_version`/`source` or a boolean `signature_verified` would be rejected before transmit as "carries a key outside the whitelist". The design MUST therefore extend that backstop with explicitly TYPED entries (integer `sequence`, enum-bounded `source` ∈ {embedded, file, fetch}, boolean signature-verified) rather than relaxing it to accept arbitrary keys or free strings. `bundle_version` MUST be constrained to publisher-issued values (unrecognized/operator-built corpus versions reported as a single `other` bucket): the backstop exists precisely to stop a bespoke, near-unique string from acting as a fleet identifier, and a self-built corpus version would be exactly that. **The same reasoning MUST be applied to `sequence`, not only to `bundle_version`**: an operator picks their own corpus's sequence values freely — a unix timestamp or a private build number is the obvious choice — so a raw integer sequence from an operator-built corpus is every bit the near-unique fleet identifier the `other` bucket was introduced to suppress, and bucketing the version while transmitting the sequence verbatim would defeat it. `sequence` MUST therefore be reported only when `bundle_version` resolves to a publisher-issued value (a known, low-cardinality release line); when the version falls into the `other` bucket the sequence MUST be omitted. Nothing analytic is lost: SC-006 already scopes fleet-freshness to publisher-issued corpora and reports operator-built ones as an unaged bucket. +- **FR-021**: The existing anonymous TPA-scanner telemetry MUST gain the active bundle version, sequence, and source (and signature-verified state) so database adoption and freshness are measurable fleet-wide; the existing telemetry opt-out MUST fully cover these fields, and no new identifying data is introduced. This is NOT a drop-in addition: `internal/telemetry/anonymity.go`'s schema-v8 backstop hard-whitelists the `tpa_scanner` sub-object to `{scans_completed, scans_failed, scans_with_findings, findings}` and requires every scalar to be a non-negative integer, so a string `bundle_version`/`source` or a boolean `signature_verified` would be rejected before transmit as "carries a key outside the whitelist". The design MUST therefore extend that backstop with explicitly TYPED entries (integer `sequence`, enum-bounded `source` ∈ {embedded, file, fetch}, boolean signature-verified) rather than relaxing it to accept arbitrary keys or free strings. `bundle_version` MUST be constrained to publisher-issued values (unrecognized/operator-built corpus versions reported as a single `other` bucket): the backstop exists precisely to stop a bespoke, near-unique string from acting as a fleet identifier, and a self-built corpus version would be exactly that. **The same reasoning MUST be applied to `sequence`, not only to `bundle_version`**: an operator picks their own corpus's sequence values freely — a unix timestamp or a private build number is the obvious choice — so a raw integer sequence from an operator-built corpus is every bit the near-unique fleet identifier the `other` bucket was introduced to suppress, and bucketing the version while transmitting the sequence verbatim would defeat it. `sequence` MUST therefore be reported only when `bundle_version` resolves to a publisher-issued value (a known, low-cardinality release line); when the version falls into the `other` bucket the sequence MUST be omitted. **"Resolves to a publisher-issued value" MUST be a provenance test, never a match on the claimed version string**: an operator-signed or unsigned corpus controls its own manifest and can copy a genuine publisher version string verbatim while carrying a unique sequence, which would sail through a string allowlist and transmit the fingerprinting sequence anyway. The verbatim version and the sequence MUST therefore be reported only when the active bundle's provenance is the publisher authority — the embedded default, or an external bundle signature-verified against a publisher key (FR-004) — and every other active bundle reports `other` with sequence omitted REGARDLESS of the version string it claims. The reporting layer is where authority is known, so the gate lives there; the anonymity backstop additionally enforces the cross-field invariant it can see on the payload alone — a heartbeat carrying a `sequence` while `bundle_version` is `other` is rejected before transmit. Nothing analytic is lost: SC-006 already scopes fleet-freshness to publisher-issued corpora and reports operator-built ones as an unaged bucket. - **FR-022**: Documentation MUST cover the database lifecycle end to end for operators (verify, drop, fetch, roll back, read status) and for contributors (author a signature, provenance/license rules, eval-sample requirements, how the gate blocks a bad corpus), updating the existing security-quarantine/tool-scanner docs. ### Key Entities *(include if feature involves data)* @@ -172,7 +172,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), ### Measurable Outcomes - **SC-001**: 100% of tampered candidates are rejected before parsing, with the last-known-good database still serving and the reason surfaced — demonstrated by fixture in CI. "Tampered" is defined precisely, because the signature covers the bundle bytes and NOT the sidecar's own encoding: any altered bundle byte, any altered signature value, any altered sidecar identity field (algorithm or key id, whose agreement with the manifest FR-003 already requires), and any mismatched bundle/sidecar pair. Byte changes to the sidecar that alter neither the signature value nor an identity field (whitespace, key order) are explicitly NOT required to be rejected — a detached signature cannot detect them — so the criterion is stated in terms of the fields that carry meaning rather than an unachievable "every byte of the sidecar". -- **SC-002**: A validly-signed but lower-sequence bundle is never activated by any non-rollback path (drop, fetch, restart) — including on a fresh install or after a data-directory wipe, where the FR-009b seeded baseline stands in for the absent persisted watermark — and an unsigned candidate never advances the watermark nor survives the signature ratchet — demonstrated by fixture in CI. There are exactly two exemptions, and both are degraded-coverage states rather than silent ones: the FR-009a embedded-default availability fallback, and the baseline-less operator authority of FR-009b (SC-002a fixture (c)), which has no seed to compare against and so carries no anti-downgrade guarantee to claim. Neither counts as a downgrade; both MUST surface as degraded. An explicit rollback succeeds, leaves an audit trail, does not lower the historical watermark, activates only the fingerprint the operator named (a substituted file at the same path is refused), and the deny-listed release it rolled away from is not re-activated by the following refresh cycle. +- **SC-002**: A validly-signed bundle whose sequence is below its OWN signing authority's watermark for the current trust epoch is never activated by any non-rollback path (drop, fetch, restart) — "lower-sequence" is always a same-authority, same-epoch comparison per FR-004a/FR-005a, so a deliberate FR-004b authority switch followed by the newly-active authority's bundle clearing its own baseline is by definition not a downgrade and needs no exemption — including on a fresh install or after a data-directory wipe, where the FR-009b seeded baseline stands in for the absent persisted watermark — and an unsigned candidate never advances the watermark nor survives the signature ratchet — demonstrated by fixture in CI. There are exactly two exemptions, and both are degraded-coverage states rather than silent ones: the FR-009a embedded-default availability fallback, and the baseline-less operator authority of FR-009b (SC-002a fixture (c)), which has no seed to compare against and so carries no anti-downgrade guarantee to claim. Neither counts as a downgrade; both MUST surface as degraded. An explicit rollback succeeds, leaves an audit trail, does not lower the historical watermark, activates only the fingerprint the operator named (a substituted file at the same path is refused), and the deny-listed release it rolled away from is not re-activated by the following refresh cycle. - **SC-002a**: The watermark cannot be weaponized into a permanent update lockout, demonstrated by the following fixtures: (a) a bundle signed by a to-be-revoked key at the MAXIMUM representable sequence, after which a binary carrying a bumped trust epoch and the surviving key successfully activates a new release — proving key-compromise recovery is possible rather than merely documented; (a2) immediately after that bump, an OLD pre-bump bundle carrying the previous epoch and signed by the still-trusted surviving key is REFUSED as stale-epoch replay rather than clearing the re-baselined watermark — the epoch bump must not itself become a downgrade window; and (b) with the operator authority declared active (FR-004b), an operator-authority-signed bundle at a high sequence that leaves the publisher authority's watermark untouched, so after the declaration is switched back the next official publisher release still activates — proving the two signing authorities cannot lock each other out. A fixture (b2) covers the declaration itself: the same operator-signed bundle dropped while the PUBLISHER authority is declared active is refused as "authority not active" with the publisher bundle still serving, so the choice of authority cannot be made by whoever writes the bundle path. A third fixture (c) covers the operator authority after a data-directory wipe: with a config-resident baseline declared, an older operator-signed bundle is refused; with no baseline declared, it may activate but the install reports degraded coverage rather than full trust. - **SC-003**: An air-gapped install (zero network, ever) reaches full P1 capability: verified activation of a file-dropped signed database, downgrade refusal, and rollback, with behavior byte-identical to a connected install's file-drop path. - **SC-004**: The shipped seed corpus contains at least 25 signatures spanning at least 8 distinct public campaign/technique classes, every one carrying provenance + redistributable license, and every gating signature backed by its eval sample pair; the canonical fixture of each class produces a hard-tier finding naming its TPA id. From 10ae3cf583a6170b1205dd365a28358af6e35d75 Mon Sep 17 00:00:00 2001 From: Algis Dumbris Date: Mon, 24 Aug 2026 21:14:13 +0300 Subject: [PATCH 15/15] docs(specs): define authority-switch serving semantics + operator epoch recovery (101-tpa-db) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cross-model review (opencode / gpt-5.6-sol) round 7 on specs/101-tpa-db/spec.md. All three findings verified against the document before editing. ## Changes - FR-004b: the active-authority declaration governs the SERVING bundle, not only future candidates. On a declaration change, an active external bundle of the now-inactive authority is deactivated (loudly logged) and the install falls back to the embedded default per FR-009a until a bundle of the newly active authority activates. Otherwise keep-last-known-good makes the switch advisory: a bundle-path attacker keeping the new authority's candidates invalid could perpetuate the old corpus at full coverage. Declaring the operator authority active does not by itself degrade coverage. - FR-009 + SC-002: the highest-ever watermark and the signature ratchet are explicitly claims about retained data-directory state — after a reset only the FR-009b seeded/config-resident baselines survive and the ratchet re-arms on the next signed activation. Fixtures must test the post-reset guarantees the threat model actually supports, resolving the contradiction with the 'Anti-downgrade state lost' edge case. - FR-005a + Key Entities: epoch advancement is split per trust root — binary release for the publisher epoch, config edit for the operator epoch (declared alongside the operator keys/baseline). A binary-only rule made operator-key compromise an unrecoverable permanent lockout, since no publisher release can coordinate a private operator authority's epoch and a replacement key inherits the same-epoch watermark. A config bump re-baselines from the config-declared baseline and retires prior-epoch operator pins/deny-list. ## Testing - Docs-only change; no Go sources touched. - Round 7 of the raised (10-round) cross-model review cap. --- specs/101-tpa-db/spec.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/specs/101-tpa-db/spec.md b/specs/101-tpa-db/spec.md index c384a912..14047ac1 100644 --- a/specs/101-tpa-db/spec.md +++ b/specs/101-tpa-db/spec.md @@ -116,16 +116,16 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **FR-003**: The signature scheme MUST be Ed25519 detached signatures (or an equivalent modern EdDSA scheme selected in design); the sidecar MUST carry the algorithm identifier and signing key id so the scheme can evolve without breaking verification of existing artifacts. A key id MUST be a canonical full-length fingerprint of the public key itself (not an operator-chosen label), the trust set MUST reject duplicate ids at build/config time, and verification MUST succeed only when the manifest key id, the sidecar key id, and the id of the trust-set key whose signature actually verified are all identical — a manifest naming one publisher while the bytes are signed by another MUST be a verification failure, not a cosmetic mismatch. - **FR-004**: The set of trusted verification keys MUST be the union of (a) publisher public keys embedded in the binary at build time and (b) operator-configured keys (the Spec 087 FR-013 key). Keys MUST NOT be retrieved over the network, and verification MUST work fully offline. These are two distinct **signing authorities**, and the design MUST keep them so: all publisher keys (including keys being rotated in or out, FR-005) belong to ONE publisher authority, while operator-configured keys form a separate operator authority. - **FR-004a**: Anti-downgrade state MUST be tracked per signing authority, not in one global namespace. An operator maintaining a self-built corpus is an explicitly supported persona (air-gapped installs, private signature sets), and with a single shared counter their locally-signed bundle at sequence N would raise the one global watermark above the publisher's release line and permanently block official updates — or, symmetrically, a publisher release would block the operator's own corpus. Each authority MUST therefore carry its own watermark, and a candidate MUST be compared only against the watermark of the authority whose key verified it. Choosing an operator-signed corpus over a higher-sequence publisher one is a deliberate operator decision (declared per FR-004b) and MUST NOT by itself be treated as a downgrade or as degraded coverage — the operator configured that key precisely to assert their own corpus — but the active authority MUST be visible in bundle status (FR-020) so the choice is never invisible. Grouping all publisher keys into one authority is what keeps per-authority watermarks compatible with key rotation: a rotated-in key inherits the publisher authority's watermark rather than starting a fresh one. -- **FR-004b**: The **active signing authority MUST be a config-resident declaration**, defaulting to the publisher authority, and MUST NOT be inferred solely from whichever key happens to verify a candidate. A candidate whose verifying key belongs to a non-active authority MUST be refused with an "authority not active" reason, exactly as any other failed candidate (last-known-good keeps serving). Without this, per-authority watermarks (FR-004a) open the very hole they were introduced to avoid: because each authority is compared only against its own counter, an attacker who can write the bundle path — squarely inside the threat model, unlike the config-write attacker — can replace a current publisher bundle at sequence 100 with ANY operator-signed artifact that merely clears the operator authority's FR-009b baseline, silently swapping the install onto an older or deliberately narrower corpus while it still reports `signature_verified=true` at FULL coverage. The publisher watermark cannot detect this, since the comparison never crosses namespaces, and no operator decision is involved — whoever writes the path picks the authority. Config-residency is what puts that choice out of the attacker's reach, matching `require_signed_bundle` (FR-007) and the operator baseline (FR-009b), which are config-resident for the same reason. Both key sets MAY remain configured simultaneously; only artifacts of the declared active authority activate, and switching authorities is a config edit, not a file drop. +- **FR-004b**: The **active signing authority MUST be a config-resident declaration**, defaulting to the publisher authority, and MUST NOT be inferred solely from whichever key happens to verify a candidate. A candidate whose verifying key belongs to a non-active authority MUST be refused with an "authority not active" reason, exactly as any other failed candidate (last-known-good keeps serving). Without this, per-authority watermarks (FR-004a) open the very hole they were introduced to avoid: because each authority is compared only against its own counter, an attacker who can write the bundle path — squarely inside the threat model, unlike the config-write attacker — can replace a current publisher bundle at sequence 100 with ANY operator-signed artifact that merely clears the operator authority's FR-009b baseline, silently swapping the install onto an older or deliberately narrower corpus while it still reports `signature_verified=true` at FULL coverage. The publisher watermark cannot detect this, since the comparison never crosses namespaces, and no operator decision is involved — whoever writes the path picks the authority. Config-residency is what puts that choice out of the attacker's reach, matching `require_signed_bundle` (FR-007) and the operator baseline (FR-009b), which are config-resident for the same reason. Both key sets MAY remain configured simultaneously; only artifacts of the declared active authority activate, and switching authorities is a config edit, not a file drop. **The declaration governs the SERVING bundle, not only future candidates**: when the declaration changes, an active external bundle belonging to the now-inactive authority MUST NOT keep serving as full coverage — it is deactivated (loudly logged, surfaced per FR-020) and the install falls back to the embedded default under FR-009a's rules until a bundle of the newly-active authority activates. Without this the switch is advisory only: fail-safe keep-last-known-good would leave the old authority's corpus serving indefinitely, and an attacker who keeps the new authority's candidates invalid (bundle-path writes are in scope) could perpetuate that state at full coverage — the operator's explicit change of trust would simply never take effect. Merely declaring the operator authority active does not by itself degrade coverage: until an operator bundle activates, the embedded default serves exactly as on a fresh install (FR-009a condition (b) concerns a previously-activated operator bundle that was lost, not one that never existed). - **FR-005**: Key rotation MUST be supported by trusting multiple keys simultaneously: a binary release MAY add a new key while retaining the old for at least one release cycle, and key removal is also a binary-release event. The design MUST document the compromise procedure and the "do not rotate casually" operational stance. -- **FR-005a**: Each signing authority's trust set MUST carry a **trust epoch** — a monotonic per-authority integer that advances ONLY via a binary release — and every anti-downgrade comparison (FR-009) MUST be scoped `(authority, epoch, sequence)`, ordered by epoch first. The epoch MUST be carried inside the SIGNED manifest (FR-002) and a candidate MUST be accepted only when its manifest epoch EQUALS the current binary-trusted epoch of the authority whose key verified it — never merely less-than-or-equal. Binding the epoch to the signed bytes and requiring exact equality is what stops the epoch bump from becoming a replay window of its own: if the epoch lived only in the binary, then after a bump-and-revoke the re-baselined watermark would start from zero, and an OLD, weaker bundle legitimately signed by a still-trusted surviving key would be silently reinterpreted as belonging to the new epoch, clear the fresh watermark, and activate with `signature_verified=true` and full coverage — reinstating exactly the blind spots the recovery release was published to close. Without this, key-compromise recovery is impossible rather than merely painful: an attacker holding a compromised publisher key signs a bundle at the maximum representable sequence, and because the watermark is monotonic and no surviving key can ever exceed that sequence, the install is permanently unable to accept any future legitimate release — a one-shot permanent denial of updates, downgraded only to the FR-009a fallback. A binary release that revokes the compromised key MUST therefore also bump the epoch, and a validly-signed bundle whose manifest epoch matches a newly-bumped binary epoch MUST supersede the retained watermark and re-baseline it; the bump likewise retires the authority's prior-epoch FR-010 pins and deny-list entries, so stale rollback state cannot block the recovery bundle (FR-010). A bundle whose manifest epoch is BELOW the current binary-trusted epoch MUST be refused as stale-epoch replay even when its signature verifies against a still-trusted key. Epoch advancement is safe to treat as authoritative because it arrives only inside a release binary, whose integrity is already the root of trust for the embedded key set (FR-004) — it is not attacker-reachable over the bundle or fetch path. The accepted consequence of exact-equality matching is that a binary still on the OLD epoch will refuse the new epoch's bundles and stop receiving database updates until it is itself updated. That is deliberate, not incidental: an epoch bump means a key was compromised, and an un-updated binary still trusts that compromised key, so freezing its database is safer than feeding it artifacts whose trust basis it cannot evaluate. It also matches the existing "a hasty rotation strands older installs" stance, and it MUST be surfaced as a stale-epoch degraded state (FR-020) rather than failing silently. +- **FR-005a**: Each signing authority's trust set MUST carry a **trust epoch** — a monotonic per-authority integer that advances only through that authority's own trust root: the PUBLISHER epoch advances ONLY via a binary release, while the OPERATOR epoch is config-resident, declared alongside the operator keys and baseline, and advances only by a config edit. The split is forced by the same recovery argument that motivates the epoch itself: a compromised OPERATOR key can sign the maximum representable sequence exactly as a publisher key can, and no binary release can coordinate an epoch bump for a private operator authority the publisher has never heard of — a binary-only rule would make operator-key compromise an unrecoverable permanent lockout of the operator authority (replacing the key in config does not help, since the new key inherits the same-epoch watermark). A config-resident operator epoch is sound for the same reason the operator baseline is (FR-009b): the operator authority's entire trust root — its keys, baseline, and the active-authority declaration — is already config-resident, so the epoch adds no new attack surface for the config-write attacker, who is out of the threat model. Bumping the operator epoch in config behaves exactly as a publisher binary bump: it re-baselines the operator watermark from the config-declared baseline (or marks the authority baseline-less-degraded per FR-009b) and retires prior-epoch operator pins and deny-list entries (FR-010). Every anti-downgrade comparison (FR-009) MUST be scoped `(authority, epoch, sequence)`, ordered by epoch first. The epoch MUST be carried inside the SIGNED manifest (FR-002) and a candidate MUST be accepted only when its manifest epoch EQUALS the current binary-trusted epoch of the authority whose key verified it — never merely less-than-or-equal. Binding the epoch to the signed bytes and requiring exact equality is what stops the epoch bump from becoming a replay window of its own: if the epoch lived only in the binary, then after a bump-and-revoke the re-baselined watermark would start from zero, and an OLD, weaker bundle legitimately signed by a still-trusted surviving key would be silently reinterpreted as belonging to the new epoch, clear the fresh watermark, and activate with `signature_verified=true` and full coverage — reinstating exactly the blind spots the recovery release was published to close. Without this, key-compromise recovery is impossible rather than merely painful: an attacker holding a compromised publisher key signs a bundle at the maximum representable sequence, and because the watermark is monotonic and no surviving key can ever exceed that sequence, the install is permanently unable to accept any future legitimate release — a one-shot permanent denial of updates, downgraded only to the FR-009a fallback. A binary release that revokes the compromised key MUST therefore also bump the epoch, and a validly-signed bundle whose manifest epoch matches a newly-bumped binary epoch MUST supersede the retained watermark and re-baseline it; the bump likewise retires the authority's prior-epoch FR-010 pins and deny-list entries, so stale rollback state cannot block the recovery bundle (FR-010). A bundle whose manifest epoch is BELOW the current binary-trusted epoch MUST be refused as stale-epoch replay even when its signature verifies against a still-trusted key. Epoch advancement is safe to treat as authoritative because it arrives only through the authority's own trust root — a release binary for the publisher epoch (the binary's integrity is already the root of trust for the embedded key set, FR-004) and `mcp_config.json` for the operator epoch (already the root of trust for the operator keys themselves) — never over the bundle or fetch path, which is the attacker-reachable surface. The accepted consequence of exact-equality matching is that a binary still on the OLD epoch will refuse the new epoch's bundles and stop receiving database updates until it is itself updated. That is deliberate, not incidental: an epoch bump means a key was compromised, and an un-updated binary still trusts that compromised key, so freezing its database is safer than feeding it artifacts whose trust basis it cannot evaluate. It also matches the existing "a hasty rotation strands older installs" stance, and it MUST be surfaced as a stale-epoch degraded state (FR-020) rather than failing silently. **Loader & activation** - **FR-006**: For any candidate carrying a signature, verification MUST run over the exact raw candidate bytes BEFORE those bytes are parsed as JSON or any pattern is compiled; a candidate failing verification MUST be rejected without further processing, keeping last-known-good (extends the Spec 086 fail-closed pipeline with a new first stage). The bytes that are verified MUST be the same bytes that are subsequently parsed (read once into memory, verify, then parse that buffer) — the file MUST NOT be re-read from disk after verification, or a writer racing the loader could swap content between verify and parse. - **FR-007**: Signing policy MUST be: network-fetched candidates ALWAYS require a valid signature; file-drop candidates are verified whenever a sidecar is present; a new `require_signed_bundle` setting (default off for back-compat with existing unsigned drops) makes the sidecar mandatory for file drops too; the embedded default needs no sidecar (its integrity rides on binary distribution integrity). Bundle status MUST always report whether the active bundle was signature-verified. While an unsigned EXTERNAL (file-dropped or fetched) bundle is active, the scan coverage the approval gates consume MUST be marked degraded so `scan`-mode auto-approval is suspended and changes route to human review — today `internal/security/scanner/inprocess.go` sets `coverageOK` from mere bundle *presence*, so without this an attacker-supplied unsigned corpus would keep silently auto-approving under weakened rules. This degradation rule applies to EXTERNAL candidates ONLY: the embedded default is binary-trusted (its integrity rides on the signed binary distribution, not on a sidecar) and MUST be full-coverage on an ordinary install, or every fresh install would ship with `scan` mode silently disabled. The embedded default is degraded only in the FR-009a cases — when it is serving as a fallback BELOW the retained publisher watermark, or when the active operator authority's previously-activated bundle has gone missing, i.e. when the install has evidence that a corpus it should be running is newer than, or not covered by, the embedded one. This requirement supersedes Spec 087 FR-014's precondition: because FR-004 makes publisher keys embedded in the binary part of the trust set, a fetch is no longer refused merely because the operator configured no key of their own; the "never activate an unverified fetched bundle" guarantee of Spec 087 FR-013/FR-014 is preserved unchanged. - **FR-008**: The full validate-before-activate order MUST be: sidecar byte cap → parse sidecar → bundle byte-size ceiling → signature over the raw bytes (per policy, FR-006) → parse manifest → rule-count ceiling → epoch match (FR-005a) → sequence/fingerprint anti-downgrade (FR-009) → version compatibility → compile-all-or-reject → runnable-rules > 0 → activation self-check (Spec 087 FR-008) → atomic activation. FR-011's ceilings MUST therefore be split across the pipeline rather than applied as one stage: the sidecar cap comes first because the sidecar is read before any signature exists to check, bundle byte size is measurable on raw bytes and so guards the verification step itself, while rule count is only knowable after parsing and so cannot precede it. The same reasoning applies to sequence, a manifest field necessarily read after parsing: what MUST precede parsing is signature verification, not the sequence or rule-count check — an unverified candidate's manifest is never trusted for any decision. Any failure at any stage MUST keep last-known-good and record a machine-readable reason and timestamp (surfaced via the existing bundle status), never an empty or partial rule set. -- **FR-009**: The system MUST persist the highest sequence ever activated — the **watermark**, scoped per signing authority and trust epoch per FR-004a/FR-005a — and refuse any candidate with a lower sequence, or an equal sequence with a different content fingerprint, as a downgrade/replay, unless an explicit operator rollback (FR-010) is in effect. **The watermark is a property of SIGNED artifacts only**: `sequence` lives in the manifest, so an unsigned candidate's sequence is attacker-chosen. An unsigned/unverified candidate MUST NOT advance the watermark (otherwise a single unsigned drop claiming a huge sequence permanently locks out every legitimate signed release), and once an install has ever activated a signature-verified bundle, unsigned candidates MUST be refused thereafter regardless of `require_signed_bundle` — a signature ratchet, so signing cannot be stripped back off an install by whoever can write the bundle path. The embedded default is exempt from BOTH the watermark and the ratchet — it is the availability fallback of last resort and must never be locked out (scanning must never stop), which is exactly why FR-009a demotes it to degraded coverage instead of refusing it. +- **FR-009**: The system MUST persist the highest sequence ever activated — the **watermark**, scoped per signing authority and trust epoch per FR-004a/FR-005a — and refuse any candidate with a lower sequence, or an equal sequence with a different content fingerprint, as a downgrade/replay, unless an explicit operator rollback (FR-010) is in effect. **The watermark is a property of SIGNED artifacts only**: `sequence` lives in the manifest, so an unsigned candidate's sequence is attacker-chosen. An unsigned/unverified candidate MUST NOT advance the watermark (otherwise a single unsigned drop claiming a huge sequence permanently locks out every legitimate signed release), and once an install has ever activated a signature-verified bundle, unsigned candidates MUST be refused thereafter regardless of `require_signed_bundle` — a signature ratchet, so signing cannot be stripped back off an install by whoever can write the bundle path. Both "ever" clauses — the highest-ever watermark and the once-signed-always-signed ratchet — are claims about RETAINED data-directory state, not absolutes that survive its destruction: after a data-directory reset only the FR-009b seeded and config-resident baselines still hold, the ratchet re-arms on the next signed activation, and the accepted post-reset guarantees are exactly those of the "Anti-downgrade state lost" edge case — fixtures for FR-009 and SC-002 MUST test against those, not against a stronger guarantee the threat model disclaims. The embedded default is exempt from BOTH the watermark and the ratchet — it is the availability fallback of last resort and must never be locked out (scanning must never stop), which is exactly why FR-009a demotes it to degraded coverage instead of refusing it. - **FR-009b**: Each authority's watermark MUST be SEEDED from a trusted baseline rather than starting empty, because an absent watermark accepts anything — with exactly one exception, carved out at the end of this requirement: an operator authority configured without a baseline, which has no seed available and is therefore run as explicitly degraded rather than as a silent zero. Otherwise the anti-downgrade guarantee simply does not exist at first use or after the data directory is reset: an attacker who can write the bundle path drops an OLD but genuinely publisher-signed, current-epoch bundle — say sequence 5 while the shipping release is at 100 — and with no watermark to compare against it activates as a fully signature-verified bundle at FULL coverage, silently reinstating every blind spot fixed between 5 and 100. Neither the FR-009 ratchet nor the FR-010 pin/deny-list helps here; both only bind once persisted state exists. Therefore: the **publisher** authority's watermark MUST be seeded from the sequence of the bundle embedded in the running binary — a candidate older than what the binary already ships is strictly worse than the fallback and MUST be refused — and this baseline MUST be re-applied on every startup, so wiping the data directory cannot lower it below the binary's own corpus. The **operator** authority has no binary-trusted baseline available, so its baseline MUST be declared explicitly in configuration alongside the operator key, and it MUST live in `mcp_config.json` rather than the data directory — that is what makes it durable across the wipe, since a baseline stored in resettable state provides no protection precisely when it is needed. A silent zero default MUST NOT be used: with a zero baseline the operator authority has NO anti-downgrade after state loss at all, and a bundle-path attacker — who is squarely inside the threat model, unlike the config-write attacker — could replay any older genuinely operator-signed bundle at full coverage on a fresh or wiped install. Therefore: if an operator key is configured WITHOUT a baseline, the system MUST treat operator-authority anti-downgrade as unsupported and mark coverage degraded (per FR-007) while an operator-signed bundle is active, rather than silently presenting it as fully trusted. The data-directory watermark then tracks activations above the configured baseline and may only raise the effective minimum, never lower it. The seeded baseline also makes the epoch re-baseline of FR-005a safe: after an epoch bump the new epoch's watermark starts at the authority's baseline, not at zero. - **FR-009a**: When no valid external candidate is available, the embedded default MUST still serve so scanning never stops, even if its sequence is below the watermark. Where the install has NO retained signed watermark and no missing previously-active external bundle (the ordinary fresh install), this is simply the normal full-coverage state. Two — and only two — conditions demote that fallback to degraded coverage (auto-approval suspended, as in FR-007's unsigned-external case): (a) the embedded default's sequence is below the **publisher** authority's retained watermark, or (b) the **operator** authority is the active one (FR-004b) and its previously-activated external bundle is no longer available or no longer valid. The comparison in (a) MUST be same-namespace: the embedded default is a publisher artifact, and sequences of different authorities are independent counters, so it MUST be compared ONLY against the publisher watermark and never against an operator watermark — a cross-authority numeric comparison is meaningless and would, for an operator corpus at sequence 900 against an embedded publisher corpus at 100, permanently suspend auto-approval on an install whose publisher fallback is perfectly current. Condition (b) is what a cross-authority comparison was reaching for, restricted to the case where the fallback cannot be shown to be at least as fresh as what was lost: an operator corpus is not comparable to the publisher's, so a missing one leaves the install with no evidence the embedded default covers it. It MUST NOT extend to a missing PUBLISHER external bundle, because there condition (a) already decides the question on the same axis — an embedded default at or above the publisher watermark is demonstrably no older than the bundle that went missing (the ordinary "external file removed after a binary upgrade shipped a newer corpus" case) and MUST stay full coverage rather than being degraded merely because an older external file once existed. In either case the retained watermark, the active authority, and the degraded reason MUST be surfaced, and it MUST NOT count as an anti-downgrade violation. This fallback is one of SC-002's two explicit exemptions; the other is FR-009b's baseline-less operator authority. - **FR-010**: An explicit rollback action MUST let an operator revert to the embedded default or a named bundle file: the target is re-validated (signature policy and self-check still apply), activated despite a lower sequence, and the override is loudly logged and visible in bundle status. **The rollback target MUST be identified by content fingerprint, not by pathname alone.** Rollback is the one path that deliberately bypasses the watermark, so identifying its target by an attacker-writable path hands the adversary the choice of what the bypass lands on: with a legitimate bundle at sequence 100 active and the operator intending to revert to a known-good sequence 90, an attacker who can write the bundle path — in scope, unlike the config-write attacker — swaps the named file for a genuinely publisher-signed, current-epoch bundle at sequence 5 before the rollback reads it. It passes signature, epoch, compilation, and plausibly the self-check; the watermark that would normally refuse it is switched off by the rollback itself; and the result is then PINNED, so the downgrade persists across refresh cycles. FR-006's read-once rule closes the verify/parse race but not substitution before the read. Therefore the operator MUST supply the expected fingerprint (as offered by bundle status history), the path is only a hint for where to find those bytes, and a loaded artifact whose fingerprint differs MUST be refused as a rollback-target mismatch rather than activated. A rollback target MUST additionally still be at or above **its own authority's** FR-009b baseline — the publisher baseline for a publisher-signed target, the config-declared operator baseline for an operator-signed one. The comparison MUST NOT be made against the publisher baseline in both cases: sequences of different authorities are independent counters (FR-004a), so measuring an operator target at sequence 9 against a publisher embedded corpus at 100 would refuse every legitimate operator rollback. Where an authority has NO declared baseline — possible only for the operator authority, which FR-009b explicitly permits to run without one — there is no floor to enforce and none is invented: the rollback proceeds and the install is marked degraded for exactly as long as that operator-signed bundle is active, which is already FR-009b's treatment of a baseline-less operator authority. Neither refusing the rollback (which would break FR-010's guarantee for a configuration FR-009b allows) nor assuming a zero floor (which would claim an anti-downgrade guarantee that does not exist) is acceptable. The embedded default is the one target always permitted, since it IS the publisher baseline and is the availability fallback of last resort. Rollback MUST NOT lower the historical high-watermark: the watermark is monotonic and normal activation MUST still require exceeding it. Rollback is instead expressed as a **pinned exception** — the operator pins the rolled-back artifact by fingerprint, and the (sequence, fingerprint) being rolled away from is deny-listed so the very release the operator just rejected cannot be re-activated by the next refresh cycle (and neither can any intermediate release between the pin and the watermark). Clearing the pin returns the install to normal watermark-governed activation. **Pin and deny-list state MUST be scoped like the watermark it excepts**: each entry is keyed by (authority, trust epoch, sequence, fingerprint) and binds only while its authority is the active one (FR-004b) and its epoch equals the binary's current epoch for that authority. Switching the active authority SUSPENDS (does not delete) the non-active authority's pin and deny-list, so switching back restores them unchanged. A binary release that bumps an authority's trust epoch (FR-005a) RETIRES that authority's prior-epoch pins and deny-list entries, loudly logged: the epoch bump is the key-compromise recovery path and MUST NOT be blockable by stale pin state — a retained old-epoch pin could not re-activate its artifact anyway (the pinned manifest's epoch no longer matches, FR-005a), so honoring it could only strand the install on the embedded fallback while blocking the very recovery bundle the bump exists to deliver. @@ -158,11 +158,11 @@ The publisher cuts a new database release (new signature for a fresh campaign), - **Signature Database (bundle + sidecar)**: the publishable unit — the deterministic compiled `scanner-bundle.json` plus a detached signature over its exact bytes. Identified by (bundle_version, sequence, fingerprint). - **Bundle Manifest metadata**: `bundle_version`/`schema_version` (existing), plus additive `generated_at`, `sequence`, publisher key id. -- **Publisher Trust Anchor**: the set of publisher public keys embedded in a binary release, unioned with operator-configured keys; the only roots of trust for verification, never network-delivered. Each authority carries its own **trust epoch** (FR-005a), a monotonic integer that only a binary release can advance and that a candidate's signed manifest must match exactly. +- **Publisher Trust Anchor**: the set of publisher public keys embedded in a binary release, unioned with operator-configured keys; the only roots of trust for verification, never network-delivered. Each authority carries its own **trust epoch** (FR-005a), a monotonic integer advanced only through that authority's trust root (a binary release for the publisher, a config edit for the operator) and that a candidate's signed manifest must match exactly. - **Signing Authority**: the grouping anti-downgrade state is keyed by (FR-004a) — one publisher authority spanning all publisher keys including those mid-rotation, and a separate operator authority for operator-configured keys. Each holds its own watermark so neither can lock the other out. Exactly one authority is **active** at a time, declared in config (FR-004b, default publisher); artifacts of a non-active authority are refused, so per-authority isolation cannot be turned into an authority-swap by whoever can write the bundle path. - **Signature (TPA record)**: one cataloged campaign/technique — `TPA-YYYY-NNNN` id, detectors, category, level, confidence, tier intent (gating vs non-gating), provenance reference, license. - **Eval Sample Pair**: the labeled gated-malicious sample(s) plus category-matched hard-negative(s) a gating signature must contribute to the frozen eval dataset. -- **Sequence Watermark**: the persisted highest-activated sequence, tracked per signing authority and scoped by trust epoch (FR-004a, FR-005a); the anti-downgrade/anti-replay state. Never empty for any authority that has a baseline — it is seeded from a trusted one (the embedded bundle's sequence, for the publisher authority) per FR-009b, so "no state yet" is not an accept-anything state. The single exception is an operator authority configured without a baseline, which FR-009b permits precisely because no binary-trusted seed exists for it; there the install carries no anti-downgrade guarantee and is marked degraded while such a bundle is active rather than being presented as fully trusted. It is MONOTONIC — rollback does not lower it (FR-010 expresses rollback as a pinned exception plus a deny-list instead), and the only thing that re-baselines it is a higher trust epoch arriving in a binary release. +- **Sequence Watermark**: the persisted highest-activated sequence, tracked per signing authority and scoped by trust epoch (FR-004a, FR-005a); the anti-downgrade/anti-replay state. Never empty for any authority that has a baseline — it is seeded from a trusted one (the embedded bundle's sequence, for the publisher authority) per FR-009b, so "no state yet" is not an accept-anything state. The single exception is an operator authority configured without a baseline, which FR-009b permits precisely because no binary-trusted seed exists for it; there the install carries no anti-downgrade guarantee and is marked degraded while such a bundle is active rather than being presented as fully trusted. It is MONOTONIC — rollback does not lower it (FR-010 expresses rollback as a pinned exception plus a deny-list instead), and the only thing that re-baselines it is a higher trust epoch arriving through the authority's trust root (a binary release for the publisher authority, a config edit for the operator authority — FR-005a). - **Publication Channel**: the stable public location of versioned signed artifact pairs with a "latest" reference; consumed by the Spec 087 fetch or by manual download. - **Post-Activation Re-scan**: the one-shot, off-hot-path evaluation of cached approved tool metadata against a newly-activated database, emitting review findings only. - **Active Database / Last-Known-Good**: unchanged from Spec 086/087 — the serving bundle and the fail-closed fallback target. @@ -172,7 +172,7 @@ The publisher cuts a new database release (new signature for a fresh campaign), ### Measurable Outcomes - **SC-001**: 100% of tampered candidates are rejected before parsing, with the last-known-good database still serving and the reason surfaced — demonstrated by fixture in CI. "Tampered" is defined precisely, because the signature covers the bundle bytes and NOT the sidecar's own encoding: any altered bundle byte, any altered signature value, any altered sidecar identity field (algorithm or key id, whose agreement with the manifest FR-003 already requires), and any mismatched bundle/sidecar pair. Byte changes to the sidecar that alter neither the signature value nor an identity field (whitespace, key order) are explicitly NOT required to be rejected — a detached signature cannot detect them — so the criterion is stated in terms of the fields that carry meaning rather than an unachievable "every byte of the sidecar". -- **SC-002**: A validly-signed bundle whose sequence is below its OWN signing authority's watermark for the current trust epoch is never activated by any non-rollback path (drop, fetch, restart) — "lower-sequence" is always a same-authority, same-epoch comparison per FR-004a/FR-005a, so a deliberate FR-004b authority switch followed by the newly-active authority's bundle clearing its own baseline is by definition not a downgrade and needs no exemption — including on a fresh install or after a data-directory wipe, where the FR-009b seeded baseline stands in for the absent persisted watermark — and an unsigned candidate never advances the watermark nor survives the signature ratchet — demonstrated by fixture in CI. There are exactly two exemptions, and both are degraded-coverage states rather than silent ones: the FR-009a embedded-default availability fallback, and the baseline-less operator authority of FR-009b (SC-002a fixture (c)), which has no seed to compare against and so carries no anti-downgrade guarantee to claim. Neither counts as a downgrade; both MUST surface as degraded. An explicit rollback succeeds, leaves an audit trail, does not lower the historical watermark, activates only the fingerprint the operator named (a substituted file at the same path is refused), and the deny-listed release it rolled away from is not re-activated by the following refresh cycle. +- **SC-002**: A validly-signed bundle whose sequence is below its OWN signing authority's watermark for the current trust epoch is never activated by any non-rollback path (drop, fetch, restart) — "lower-sequence" is always a same-authority, same-epoch comparison per FR-004a/FR-005a, so a deliberate FR-004b authority switch followed by the newly-active authority's bundle clearing its own baseline is by definition not a downgrade and needs no exemption — including on a fresh install or after a data-directory wipe, where the FR-009b seeded baseline stands in for the absent persisted watermark — and an unsigned candidate never advances the watermark nor survives the signature ratchet (the ratchet claim holds while data-directory state is retained; after a wipe it re-arms on the next signed activation, per FR-009) — demonstrated by fixture in CI. There are exactly two exemptions, and both are degraded-coverage states rather than silent ones: the FR-009a embedded-default availability fallback, and the baseline-less operator authority of FR-009b (SC-002a fixture (c)), which has no seed to compare against and so carries no anti-downgrade guarantee to claim. Neither counts as a downgrade; both MUST surface as degraded. An explicit rollback succeeds, leaves an audit trail, does not lower the historical watermark, activates only the fingerprint the operator named (a substituted file at the same path is refused), and the deny-listed release it rolled away from is not re-activated by the following refresh cycle. - **SC-002a**: The watermark cannot be weaponized into a permanent update lockout, demonstrated by the following fixtures: (a) a bundle signed by a to-be-revoked key at the MAXIMUM representable sequence, after which a binary carrying a bumped trust epoch and the surviving key successfully activates a new release — proving key-compromise recovery is possible rather than merely documented; (a2) immediately after that bump, an OLD pre-bump bundle carrying the previous epoch and signed by the still-trusted surviving key is REFUSED as stale-epoch replay rather than clearing the re-baselined watermark — the epoch bump must not itself become a downgrade window; and (b) with the operator authority declared active (FR-004b), an operator-authority-signed bundle at a high sequence that leaves the publisher authority's watermark untouched, so after the declaration is switched back the next official publisher release still activates — proving the two signing authorities cannot lock each other out. A fixture (b2) covers the declaration itself: the same operator-signed bundle dropped while the PUBLISHER authority is declared active is refused as "authority not active" with the publisher bundle still serving, so the choice of authority cannot be made by whoever writes the bundle path. A third fixture (c) covers the operator authority after a data-directory wipe: with a config-resident baseline declared, an older operator-signed bundle is refused; with no baseline declared, it may activate but the install reports degraded coverage rather than full trust. - **SC-003**: An air-gapped install (zero network, ever) reaches full P1 capability: verified activation of a file-dropped signed database, downgrade refusal, and rollback, with behavior byte-identical to a connected install's file-drop path. - **SC-004**: The shipped seed corpus contains at least 25 signatures spanning at least 8 distinct public campaign/technique classes, every one carrying provenance + redistributable license, and every gating signature backed by its eval sample pair; the canonical fixture of each class produces a hard-tier finding naming its TPA id.