diff --git a/CLAUDE.md b/CLAUDE.md index a8311558..2bd9d46a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -158,6 +158,6 @@ tail -f ~/Library/Logs/mcpproxy/main.log # main log (macOS; Linux: ~/.mcpproxy/ - **Windows installer**: [docs/github-actions-windows-wix-research.md](docs/github-actions-windows-wix-research.md). **Prerelease** (`next` branch + `v*-rc.*` tags, opt-in, off stable channels): [docs/prerelease-builds.md](docs/prerelease-builds.md). ## Recent Changes +- 101-tpa-db: Go 1.25.5 module toolchain (`go.mod`), backend-only + stdlib only for the new work — `crypto/ed25519`, `crypto/sha256`, `syscall` (no ed25519 usage in the tree before this); existing bbolt (state), zap, Cobra. **No new dependencies.** - 098-tools-preflight: Added Go 1.24 module toolchain (repo builds with local Go 1.25) + existing only — chi (httpapi), bbolt (storage), Bleve (index), zap (logging), Cobra (CLI), swaggo/swag v2 (contract regen). **No new dependencies.** - 097-stored-scripts: Added Go 1.25 (os.Root/Root.ReadFile available — R1) + stdlib only (os.Root). **No new dependencies.** -- 096-batched-call-tools: Added Go 1.24 module toolchain (repo builds with local Go 1.25) + existing only — goja (sandbox), mark3labs/mcp-go (tool surface), zap. **No new dependencies.** diff --git a/ROADMAP.md b/ROADMAP.md index dce01646..d43a75db 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -881,4 +881,5 @@ Legend: `shipped` ≥95% checked · `in-flight` 1–94% · `drafted` 0% · `—` | [098-tools-preflight](./specs/098-tools-preflight/) | `in-flight` | 26/33 (79%) | | [099-describe-check-mode](./specs/099-describe-check-mode/) | `in-flight` | 9/10 (90%) | | [100-prompt-rugpull-baseline](./specs/100-prompt-rugpull-baseline/) | — | — | +| [101-tpa-db](./specs/101-tpa-db/) | — | — | | [102-schema-deferred](./specs/102-schema-deferred/) | — | — | diff --git a/specs/101-tpa-db/contracts/bundle-signatures-section.md b/specs/101-tpa-db/contracts/bundle-signatures-section.md new file mode 100644 index 00000000..89257343 --- /dev/null +++ b/specs/101-tpa-db/contracts/bundle-signatures-section.md @@ -0,0 +1,90 @@ +# Contract: Bundle `signatures[]` Section + +**Feature**: `101-tpa-db` · Resolves FR-001a. + +## 1. Why this exists + +FR-001 says the publishable artifact is "the existing compiled bundle". FR-012 says "each signature MUST carry provenance … and a redistributable license". **Both cannot hold today**, because the shipped `scanner-bundle.json` has nowhere to put either: + +```text +top level : bundle_version, generated_from, rules, schema_version, signature_count, skipped +rules[] : category, confidence, detector, engine, flags, id, indicators, + level, pattern, severity, target, type +``` + +No provenance. No licence. No campaign identity — `signature_count` counts *rules*, so SC-004's "at least 25 signatures spanning at least 8 distinct classes" is not even expressible. + +## 2. Shape + +A new **additive** top-level array. A v0.1 loader ignores unknown top-level keys, so contract §4 forward-compat is preserved (FR-001a). + +```json +{ + "signatures": [ + { + "id": "TPA-2026-0001", + "category": "instruction_injection", + "gating": true, + "provenance": { + "source": "https://invariantlabs.ai/blog/mcp-github-vulnerability", + "published": "2026-01-14", + "note": "hidden block coaxing ~/.ssh/id_rsa" + }, + "license": "CC-BY-4.0", + "rule_ids": ["r-hidden-important-block", "r-hidden-html-comment"] + } + ] +} +``` + +| Field | Type | Rules | +|---|---|---| +| `id` | string | `^TPA-[0-9]{4}-[0-9]{4}$`, unique across the bundle | +| `category` | string | the campaign/technique class; must have a `categoryCheck` entry in the gate (FR-016a) | +| `gating` | bool | `true` ⇒ must emit hard-tier signals and must ship eval evidence (FR-013/FR-014) | +| `provenance.source` | URL | the public disclosure that motivated the signature | +| `provenance.published` | `YYYY-MM-DD` | | +| `provenance.note` | string | optional, human context | +| `license` | SPDX id | from the redistributable allowlist below | +| `rule_ids` | array | ≥1, each naming an existing `rules[]` entry | + +**Licence allowlist** (redistributable, FR-012): `CC0-1.0`, `CC-BY-4.0`, `CC-BY-SA-4.0`, `MIT`, `Apache-2.0`. Anything else is refused at corpus build — including "no licence stated", which is the common case for a pattern lifted from a blog post and is precisely what FR-012 exists to stop shipping. + +## 3. Relationship to `rules[]` + +**Many-to-one, explicitly.** Several rules may implement one signature — a hidden-instruction class already needs distinct patterns for `` blocks, HTML comments and zero-width runs. One rule belongs to at most one signature. + +```text +signatures[] ──1─────*── rules[] + TPA id pattern +``` + +## 4. Referential integrity — enforced at build, in both directions + +A bundle failing any of these is **rejected whole**, exactly like any other contract violation: + +1. every `signatures[]` entry names ≥1 rule id that exists in `rules[]`; +2. every **hard-tier** rule is named by exactly one `gating: true` signature entry — not zero (an ungoverned gating rule has no provenance and no eval evidence) and not two (ambiguous campaign attribution). + + **"Gating" must be mechanical, because `rules[]` has no such field.** Its keys are `category, confidence, detector, engine, flags, id, indicators, level, pattern, severity, target, type` — nothing that says "this rule gates approvals". Left as prose about intent, the check is unenforceable and a hard-tier rule can evade attribution entirely, or be attached to a `gating: false` signature and thereby dodge the FR-013 eval-evidence requirement while still driving the approval gate at runtime. The build MUST therefore derive it from `level` — a rule is gating **iff** it emits at the hard tier, which is exactly what FR-014 says gating means and exactly what the eval gate and the `scan`-mode approval gates score — and MUST reject any bundle where a hard-tier rule's owning signature has `gating: false`. `signatures[].gating` is then a declaration that is *checked* against the rules it names, not an independent switch; +3. `id` values are unique; +4. `license` is in the allowlist; +5. every `gating: true` signature has its FR-013 eval pair in the frozen dataset — ≥1 labeled gated-malicious sample and ≥1 category-matched hard negative following the `hn__*` naming the dataset validator enforces; +6. every `gating: true` signature's `category` resolves to a **registered** `categoryCheck` entry in the gate. §5 below explains why an unmapped category is scored vacuously; that explanation is not enforcement, so this rule is what actually stops one shipping. A bundle naming a category the gate cannot enforce is rejected at build, not merely noted. + +Direction 2 is the one a naive implementation drops, and it is the one that matters: checking only "signatures point at real rules" lets a gating rule ship with no campaign record at all, which is the state the corpus is in today. Rule 6 is the same failure one level up — a campaign record that exists but that the gate silently declines to score. + +## 5. Interaction with the eval gate + +`gating: true` is not self-certifying. FR-016a's `categoryCheck` map decides whether a category is *enforced*: `gatedCategory()` enforces a category only when its mapped check id is registered, so **a new class whose category is missing from that map is tallied and reported but excluded from `gatedMalicious`/`OverallRecall`** — its misses can never fail the gate. Adding a signature class therefore means adding its `categoryCheck` entry in the same change, or the coverage is decorative. + +(Its false positives still count either way, since `FPRate` is computed over the whole hard-negative set regardless of category mapping. So an unmapped class can only ever hurt — never help — the gate outcome. That asymmetry is the tell.) + +## 6. Counting for SC-004 + +- **signatures** = `len(signatures)` — must be ≥ 25 +- **classes** = distinct `category` — must be ≥ 8 +- every signature carries provenance + licence (rule 4 above) +- every `gating: true` signature has its eval pair (rule 5) + +Note `signature_count` (existing, top-level) counts **rules** and keeps that meaning. Renaming it would break the v0.1 line for no gain; SC-004 counts `signatures[]`. diff --git a/specs/101-tpa-db/contracts/bundle-status-surface.md b/specs/101-tpa-db/contracts/bundle-status-surface.md new file mode 100644 index 00000000..64c08c8f --- /dev/null +++ b/specs/101-tpa-db/contracts/bundle-status-surface.md @@ -0,0 +1,80 @@ +# Contract: Bundle Status Surface + +**Feature**: `101-tpa-db` · Resolves FR-020. + +The operator question this surface must answer is not "what version am I on" but **"am I current and verified, or stuck on last-known-good — and why?"** Today's `BundleInfo` cannot answer the second half at all. + +## 1. Fields + +Existing fields keep their meaning and their names. `Fingerprint` stays the 12-hex short form (it is load-bearing in current operator output and in `logBundle`); the full digest is **added beside it**, not substituted. + +| Field | Existing | Purpose | +|---|---|---| +| `bundle_version`, `schema_version`, `source`, `generated_at`, `load_error` | ✅ | unchanged | +| `runnable_rules`, `skipped_rules`, `declared_skipped` | ✅ | unchanged — un-evaluated coverage, never clean coverage | +| `fingerprint` | ✅ | 12-hex short form, human-comparable | +| `full_digest` | **new** | complete SHA-256; the **only** value rollback accepts (FR-010) | +| `sequence` | **new** | decimal string, as published | +| `authority` | **new** | `publisher` \| `operator` — which authority is serving | +| `trust_epoch` | **new** | the epoch it was published under | +| `signature_verified` | **new** | bool | +| `staleness_days` | **new** | derived from `generated_at` | +| `coverage_degraded` | **new** | bool | +| `degraded_reason` | **new** | enum, §3 | +| `last_rejection` | **new** | `{reason, detail, at}`, §2 — retained across a successful load so "what did it refuse, and when" survives. `detail` is a short human string carrying the part the enum cannot (which ceiling was exceeded and by how much, which key id was untrusted, which epoch was expected); it is never parsed, and `reason` is the only field anything branches on | + +## 2. Rejection reasons + +The vocabulary an operator reads to tell a tamper attempt from a stale epoch from their own misconfiguration: + +| Reason | Means | +|---|---| +| `tamper` | signature did not verify over the raw bytes | +| `downgrade` | sequence below this authority's watermark for this epoch, or equal sequence with a different fingerprint | +| `stale_epoch_replay` | manifest epoch below the binary-trusted epoch — a validly-signed but superseded artifact | +| `unsigned_refused_by_ratchet` | this install has activated a signed bundle before; unsigned drops are refused thereafter | +| `authority_not_active` | verified against a real key of the **non-active** authority (FR-004b) | +| `rollback_target_mismatch` | the loaded artifact's full digest ≠ the digest the operator named | +| `gate_failure` | activation self-check (Spec 087 FR-008) rejected the candidate | +| `ceiling_exceeded` | over one of FR-011's four limits — which limit, and the observed value, in `detail` | +| `not_regular_file` | candidate was a FIFO/device/socket, or a symlink to one | +| `read_timeout` | the 5 s read budget expired; the single-flight slot was released | +| `sidecar_*`, `key_*`, `signature_invalid`, `sequence_mismatch` | the sidecar grammar — see [sidecar-format.md](sidecar-format.md) §3 | + +**Every one of these keeps last-known-good serving.** There is no rejection path that leaves the scanner with an empty or partial rule set. + +## 3. Degraded-coverage reasons + +`coverage_degraded` suspends `scan`-mode auto-approval, routing changes to human review. It **must** be visible: a silently suspended auto-approval is indistinguishable from a quiet install, which is the failure this field exists to prevent. + +| `degraded_reason` | Condition | +|---|---| +| `unsigned_external_active` | an unsigned file-dropped/fetched bundle is active (FR-007) | +| `embedded_below_watermark` | the embedded default is serving and its sequence is below the **publisher** watermark (FR-009a a) | +| `operator_bundle_missing` | the operator authority is active and its previously-activated external bundle is gone or invalid (FR-009a b) | +| `operator_authority_unbaselined` | an operator-signed bundle is active under an authority configured without a baseline (FR-009b) | + +**Not degraded** — the two cases a careless implementation gets wrong: + +- an ordinary fresh install serving the embedded default with no retained watermark and no missing external bundle. The embedded default is binary-trusted; degrading here would ship every fresh install with `scan` mode silently disabled. +- an embedded default **at or above** the publisher watermark after an external file was removed. It is demonstrably no older than what went missing — the ordinary "external file removed after a binary upgrade shipped a newer corpus" case. + +## 4. Surfaces + +Same field set everywhere; three renderings. + +| Surface | Rendering | +|---|---| +| **CLI** (`mcpproxy security …`) | human table + `-o json`/`yaml`; the full digest shown in full (it is the rollback input) | +| **REST** (security overview) | the JSON object above; `make swagger` regenerated | +| **Web UI** (Security page) | short fingerprint with the full digest on demand; degraded state as a banner with its reason, never a silent flag | + +The Web UI rule follows the house preference already established for status surfaces: **mark only the states that need action**, and always say *why* rather than showing a bare badge. + +## 5. Activation history + +`(authority, trust_epoch, sequence, full digest, generated_at, activated_at, source)` per activation, append-only, retained across restarts. + +Exposed read-only. It exists because rollback is authorized by full digest and there is otherwise **no trustworthy way to learn the digest of the known-good artifact you want to revert to** — reading it from the candidate file would defeat the purpose, since the file is what the attacker controls. + +Data-dir resident, so it carries the same caveat as the ratchet: it makes a legitimate rollback *possible*; it does not *authorize* one. diff --git a/specs/101-tpa-db/contracts/sidecar-format.md b/specs/101-tpa-db/contracts/sidecar-format.md new file mode 100644 index 00000000..3ecd085e --- /dev/null +++ b/specs/101-tpa-db/contracts/sidecar-format.md @@ -0,0 +1,83 @@ +# Contract: Signature Sidecar Format v1 + +**Feature**: `101-tpa-db` · Resolves FR-003's "the sidecar wire format MUST be specified in design, not left to the implementation". + +This is the one artifact an independent publisher must be able to produce and every loader must reject **identically**. It is also the only artifact read and parsed *before* any signature has been verified — its `algorithm`, `key_id` and `signature` are the inputs to verification — which is why the grammar below is total and why the byte cap (FR-011: **4 KiB**) applies before parsing. + +## 1. Encoding + +A single JSON object. UTF-8. No BOM. Trailing newline optional and ignored; **any other trailing bytes are a rejection**. + +File naming: `.sig` beside the bundle (e.g. `scanner-bundle.json.sig`). + +## 2. Fields — exactly these four, no others + +```json +{"sidecar_version":"1","algorithm":"ed25519","key_id":"3f0a…64hex","signature":"<88 canonical base64 chars>"} +``` + +| Field | Type | Accepted values | +|---|---|---| +| `sidecar_version` | string | exactly `"1"` | +| `algorithm` | string | exactly `"ed25519"` | +| `key_id` | string | exactly 64 lowercase hex chars — SHA-256 of the 32-byte raw Ed25519 public key | +| `signature` | string | **exactly 88 characters** of canonical RFC 4648 standard base64 ending `==`, decoding to exactly 64 bytes | + +`key_id` is a fingerprint **of the key itself**, never an operator-chosen label (FR-003). That is what makes "the manifest names one publisher while the bytes are signed by another" detectable rather than cosmetic. + +**`sequence` is deliberately NOT duplicated here.** An earlier draft carried it, reasoning that a manifest disagreement would be "cheap to detect before parsing". That rationale was wrong: detecting a disagreement requires parsing the manifest, which happens only *after* verification, so the duplicate bought no pre-parse check and added a whole inconsistency state — plus two rejection rows — for nothing. The sequence lives in the signed manifest, where it is authenticated, and nowhere else. + +**Canonical base64 is a requirement, not a formality.** Go's default `base64.StdEncoding` is lenient in two ways that both break the "two implementations reject identically" property this contract exists to provide: it silently accepts **embedded newlines** inside the value, and it accepts non-canonical trailing bits — an 88-character string that decodes without error yet does **not** round-trip to itself. Verified against Go 1.25: `StdEncoding.DecodeString` returns `err=nil` for both, while `StdEncoding.Strict().DecodeString` rejects the non-canonical case at the offending byte. Implementations MUST therefore use the strict decoder **and** assert that re-encoding the decoded bytes reproduces the input exactly. + +## 3. Rejection grammar — total, deterministic, and reason-bearing + +Every row is a hard rejection. There is no leniency, no fallback and no skip anywhere in this table. Each rejection records its own reason (surfaced via FR-020's `LastRejection`), because a conformance test that only asserts "failed" cannot show two implementations agree. + +**Rows are evaluated in the numerical order below and the FIRST match wins.** The conditions deliberately overlap — `{"foo":1}` satisfies both "a required key is missing" and "a key outside the four" — so without a stated precedence two conforming implementations could report different reasons for the same input, which is exactly the divergence the reason codes exist to rule out. + +| # | Condition | Reason code | +|---|---|---| +| 1 | file exceeds 4 KiB (checked **before** parsing) | `sidecar_too_large` | +| 2 | not a single JSON object / trailing bytes | `sidecar_malformed` | +| 3 | a required key is missing | `sidecar_missing_field` | +| 4 | any key outside the four | `sidecar_unknown_field` | +| 5 | a key appears more than once | `sidecar_duplicate_field` | +| 6 | any value has the wrong JSON type | `sidecar_type` | +| 7 | `sidecar_version != "1"` | `sidecar_version_unsupported` | +| 8 | `algorithm != "ed25519"` | `sidecar_algorithm_unsupported` | +| 9 | `key_id` not 64 lowercase hex | `sidecar_key_id_malformed` | +| 10 | `signature` is not exactly 88 canonical base64 chars, does not decode strictly, does not round-trip, or is not 64 bytes decoded | `sidecar_signature_malformed` | +| 11 | `key_id` not in the active authority's trust set | `key_not_trusted` | +| 12 | signature does not verify over the raw bundle bytes | `signature_invalid` | +| 13 | manifest `key_id` ≠ sidecar `key_id` ≠ verifying key's id | `key_id_mismatch` | + +Row 8 deserves emphasis: **an unknown algorithm identifier is a hard rejection, never a fallback to Ed25519 and never a skip.** The field exists so the scheme can evolve without breaking verification of *existing* artifacts — a v2 loader will accept both; a v1 loader accepts one. + +Rows 2, 4 and 5 each need their own mechanism, and one of them is easy to get wrong. Verified against Go 1.25: + +- **Row 4** (unknown key): `Decoder.DisallowUnknownFields()` — works as expected. +- **Row 5** (duplicate key): needs an **explicit scan**. `encoding/json` silently accepts duplicates and takes the LAST one, and `DisallowUnknownFields` does **not** catch them. Measured: `{"algorithm":"ed25519","algorithm":"rsa"}` decodes with `err == nil` and `algorithm == "rsa"`. On a field whose whole job is to pin the algorithm, that is the difference between a hard rejection and quietly reading a different value than a reviewer would. +- **Row 2** (trailing bytes): plain `json.Unmarshal` already rejects them — `invalid character '{' after top-level value` — so it is the simplest correct choice. If a streaming `Decoder` is used instead, decode once and then require the next `Decode` to return `io.EOF`; `Decoder.More()` also reports `true` for trailing bytes and garbage and `false` for trailing whitespace, but the `io.EOF` form states the intent directly and does not depend on how `More()` treats the top level. + +## 4. What the signature does and does not cover + +The signature is over **the exact raw bundle bytes**, not over the sidecar. Therefore: + +- **Covered**: every byte of the bundle. Any alteration → row 13. +- **Covered by agreement rules**: the sidecar's identity fields, because FR-003 requires manifest/sidecar/verifying-key id agreement → row 14. +- **NOT covered**: the sidecar's own encoding. Whitespace and key-order changes that alter neither the signature value nor an identity field are **not required to be rejected** — a detached signature cannot detect them. + +SC-001 is written against exactly this boundary, which is why it enumerates four tamper classes rather than claiming "every byte of the sidecar". + +## 5. Fixtures (shipped in `internal/security/bundlesig/testdata/`) + +**Accept** — one canonical valid pair (`valid.json` + `valid.json.sig`) plus: + +| Fixture | Property demonstrated | +|---|---| +| `accept-trailing-newline` | a trailing `\n` is ignored | +| `accept-all-zero-signature` | the 88-char canonical encoding of 64 zero bytes — pins the canonical form itself | + +**Reject** — one fixture per numbered row above, named `reject-`. Each asserts the **reason code**, not merely that loading failed. + +Two implementations agreeing on the accept set is not conformance. Agreeing on the reject set, reason by reason, is. diff --git a/specs/101-tpa-db/data-model.md b/specs/101-tpa-db/data-model.md new file mode 100644 index 00000000..65acd4be --- /dev/null +++ b/specs/101-tpa-db/data-model.md @@ -0,0 +1,123 @@ +# Phase 1 Data Model: tpa-db + +**Feature**: `101-tpa-db` · **Plan**: [plan.md](plan.md) · **Decisions**: [research.md](research.md) + +Three stores, deliberately separated by who can write them and what survives a data-directory wipe. That separation *is* the security model — every FR that says "config-resident" is choosing store 2 over store 3 for exactly that reason. + +| # | Store | Written by | Survives data-dir wipe | Trust role | +|---|---|---|---|---| +| 1 | **Binary** (`go:embed`) | a release build | yes | publisher keys, publisher epoch, embedded corpus + its sequence (the FR-009b publisher baseline) | +| 2 | **`mcp_config.json`** | the operator | yes | operator keys, operator baseline, operator epoch, active-authority declaration, `require_signed_bundle` | +| 3 | **`config.db`** (BBolt) | the running proxy | **no** | watermarks, ratchet, pin, deny-list, activation history — all best-effort (plan Complexity Tracking row 1) | + +--- + +## 1. Bundle manifest additions (inside the signed bytes) + +Additive to the Scanner Bundle Contract v0.1 line, so a v0.1 loader ignores them (FR-002). + +| Field | Type | Rules | +|---|---|---| +| `generated_at` | RFC3339 string | derived from release metadata (`SOURCE_DATE_EPOCH` or the release commit), **never** build wall-clock | +| `sequence` | **decimal string** | `^[1-9][0-9]{0,19}$`, value ≤ 2^64−1. Never a bare JSON number — double-precision round-trip corrupts >2^53−1 and admits exponent notation (FR-002) | +| `key_id` | 64 lowercase hex | canonical fingerprint of the signing public key; must equal the sidecar's `key_id` and the verifying trust-set key's id (FR-003) | +| `trust_epoch` | integer ≥ 0 | the authority's epoch this artifact was published under; must **equal** the binary-trusted epoch of the verifying authority (FR-005a) | +| `signatures[]` | array | see [contracts/bundle-signatures-section.md](contracts/bundle-signatures-section.md) (FR-001a/D4) | + +**Why these live inside the signed bytes**: the epoch especially. If it lived only in the binary, a bump-and-revoke would let an old bundle signed by a surviving key be silently reinterpreted as belonging to the new epoch, clear the re-baselined watermark and activate at full coverage — reinstating the blind spots the recovery release existed to close (FR-005a). + +## 2. Sidecar + +Separate file, never signed by itself. Five keys, total rejection grammar — [contracts/sidecar-format.md](contracts/sidecar-format.md). + +## 3. Signing authority + +```go +type Authority string // "publisher" | "operator" +``` + +Not a key set — a **namespace for anti-downgrade state** (FR-004a). All publisher keys, including ones mid-rotation, belong to one publisher authority precisely so a rotated-in key inherits the existing watermark instead of starting a fresh one. + +| Property | Publisher | Operator | +|---|---|---| +| Keys | embedded in the binary (store 1) | config (store 2) | +| Epoch advanced by | a binary release | a config edit | +| Baseline (FR-009b) | the embedded corpus's sequence, **re-applied every startup** | declared in config; **may be absent** → authority runs explicitly degraded | +| Active by default | yes (FR-004b) | no | + +Exactly one authority is **active** at a time, declared in config. A candidate whose verifying key belongs to a non-active authority is refused `authority_not_active`. Without that declaration, per-authority watermarks open the hole they were introduced to close: whoever writes the bundle path would choose the authority, and any operator-signed artifact clearing the operator baseline would swap the install onto a narrower corpus while still reporting `signature_verified=true` at full coverage. + +**The declaration governs the serving bundle, not only future candidates.** On a change, an active external bundle of the now-inactive authority is deactivated and the install falls back to the embedded default under FR-009a's rules. + +## 4. Per-authority persisted state (BBolt, store 3) + +One record per authority in the new `TPABundleStateBucket`, plus one history list. **All members of one `Update` transaction** (D6/FR-011a). + +```go +type authorityState struct { + Authority Authority + Epoch uint64 // the epoch these values are scoped to + Watermark uint64 // highest sequence ever activated for (authority, epoch) + WatermarkSetBy string // full SHA-256 of the bundle that set it + SignatureRatchet bool // has a signature-verified bundle ever activated + Pin *pinEntry + DenyList []denyEntry // keyed (authority, epoch, sequence, fingerprint) +} +``` + +**Scoping is `(authority, epoch, sequence)`, ordered by epoch first** (FR-005a). Every comparison is same-authority and same-epoch: a cross-authority numeric comparison is meaningless — an operator corpus at sequence 900 says nothing about a publisher corpus at 100 — and making one would permanently suspend auto-approval on an install whose publisher fallback is perfectly current (FR-009a). + +**`WatermarkSetBy` and `DenyList` are load-bearing transaction members, not bookkeeping.** A crash that preserved a rollback while losing the deny-list would let the next refresh immediately re-activate the exact release the operator just rejected — silently undoing the rollback (FR-011a). + +### Watermark lifecycle + +```text +seed from baseline (every startup, publisher) ──► raised by each higher-sequence activation + ▲ │ + │ rollback does NOT lower it + epoch bump re-baselines (expressed as pin + deny-list instead) + and retires prior-epoch pins/deny-list +``` + +Never empty for any authority that has a baseline — an absent watermark accepts anything, which is the whole point of FR-009b. The single exception is a baseline-less operator authority, which is run **explicitly degraded** rather than as a silent zero. + +## 5. Activation history (append-only, store 3) + +One record per activation, retained across restarts (FR-010): + +`(authority, trust_epoch, sequence, full SHA-256 digest, generated_at, activated_at, source)` + +**This is new.** `BundleInfo` reports only the currently active artifact, so without a history an operator has no trustworthy way to learn the digest of the known-good artifact they intend to revert to — and rollback authorization is by full digest. Being data-dir resident, it carries the same best-effort caveat as the ratchet: it makes a legitimate rollback *possible*, it does not *authorize* one. + +## 6. `BundleInfo` additions (FR-020) + +Existing fields keep their meaning. `Fingerprint` stays the 12-hex short form — it is load-bearing in current operator output — and a full digest is **added** beside it rather than widening it (R3). + +| New field | Purpose | +|---|---| +| `FullDigest` | complete SHA-256; the only value rollback accepts | +| `Sequence` | decimal string, as published | +| `Authority`, `TrustEpoch` | which authority is serving, under which epoch | +| `SignatureVerified` | bool | +| `StalenessDays` | derived from `generated_at` | +| `CoverageDegraded`, `DegradedReason` | FR-007/FR-009a — a silently suspended `scan`-mode auto-approval is otherwise indistinguishable from a quiet install | +| `LastRejection` | `{reason, at}` — one of `tamper`, `downgrade`, `stale_epoch_replay`, `unsigned_refused_by_ratchet`, `authority_not_active`, `rollback_target_mismatch`, `gate_failure`, `ceiling_exceeded`, `not_regular_file`, `read_timeout` | + +## 7. Signature (TPA record) + +The catalog unit SC-004 counts: `TPA-YYYY-NNNN` id, category, gating intent, detector(s), level, confidence, provenance reference, licence, and the `rule_ids` implementing it. Many-to-one to `rules[]` — one campaign class routinely needs several patterns (a hidden-instruction signature already implies distinct rules for `` blocks, HTML comments and zero-width runs). + +## 8. Coverage state (the derived value that matters most) + +`coverageOK` today is `bundlePresent && ChecksFailed == 0` (R5). It becomes: + +```text +degraded ⟸ an unsigned EXTERNAL bundle is active (FR-007) + ∨ the embedded default is serving AND its sequence is below + the PUBLISHER watermark (FR-009a a) + ∨ the operator authority is active AND its previously-activated + external bundle is gone or invalid (FR-009a b) + ∨ a baseline-less operator authority's bundle is active (FR-009b) +``` + +**And explicitly NOT degraded** in the two cases a careless reading would catch: an ordinary fresh install on the embedded default, and an embedded default at or above the publisher watermark after an external file was removed. A false degrade silently suspends `scan`-mode auto-approval fleet-wide, which is why both negatives are named here and tested in the plan's Test Strategy. diff --git a/specs/101-tpa-db/plan.md b/specs/101-tpa-db/plan.md new file mode 100644 index 00000000..ebbf752f --- /dev/null +++ b/specs/101-tpa-db/plan.md @@ -0,0 +1,168 @@ +# Implementation Plan: Versioned, Signed, Offline-First TPA Signature Database + +**Branch**: `101-tpa-db` | **Date**: 2026-08-26 | **Spec**: [spec.md](spec.md) +**Input**: Feature specification from `/specs/101-tpa-db/spec.md` (roadmap P1 epic `tpa-db`) + +## Summary + +Make the TPA signature database a first-class security artifact: a **signed, sequence-versioned publishable pair** (the existing `scanner-bundle.json` + a detached Ed25519 sidecar), a **verify-before-parse loader** with real ceilings and per-authority anti-downgrade state, an **explicit fingerprint-authorized rollback**, a **seed corpus of ≥25 cataloged campaign signatures** with provenance and licences, and the **observability** to tell "current and verified" from "stuck on last-known-good, and why". + +Technical approach, grounded in the code: + +- **Verification is a new leaf package, everything else extends what shipped.** `internal/security/bundlesig/` holds sidecar parsing and Ed25519 verification as pure functions over bytes — no I/O, no globals, no `scanner` import — because that is the one piece with an independent wire contract two implementations must agree on byte-for-byte (D3/D10). The loader, ceilings, activation and status work extend `internal/security/scanner/tpa_bundle_source.go` and `tpa_bundle.go`, where Spec 086 actually shipped them. **Spec 087's plan names a package `internal/security/bundle` that does not exist** (research.md R1); reading it as a description of the tree is the fastest way to write tasks against wrong paths. +- **The load pipeline gains a first stage and grows ceilings throughout.** FR-008's order is not a preamble to the existing pipeline but interleaved with it, because the checks become knowable at different points: sidecar cap → parse sidecar → bundle byte ceiling → **signature over the raw bytes** → parse manifest → rule-count ceiling → epoch match → sequence/fingerprint → version compat → compile-all-or-reject → runnable>0 → self-check → activate. Today's `loadBundleFromFile` does `os.ReadFile` with no cap, no file-type check and no deadline (R2), so FR-011's numbers — bundle ≤ 8 MiB, rules ≤ 2000, sidecar ≤ 4 KiB, 5 s read budget — are new code at three different points, not one guard. +- **Candidate opening is descriptor-based, non-blocking, and platform-split.** `os.OpenFile(..., O_NOFOLLOW|O_NONBLOCK)` → `fstat` the descriptor → refuse non-regular → clear `O_NONBLOCK` → read the ceiling from that same `FileInfo`, never re-resolving the path (D5). `O_NONBLOCK` is not defensive: opening a writer-less FIFO **blocks inside `open(2)`**, before `fstat` and before any timeout exists to observe it, so without it SC-011(c) cannot pass by construction. The time bound is a context-cancelled reader goroutine, **not** `SetReadDeadline` (`ErrNoDeadline` on regular files), and abandoned readers are capped at one per path so releasing the slot cannot leak an fd per refresh. Windows gets its own build-tagged implementation and an explicitly **weaker** guarantee — `os.Root` blocks path escape but still follows symlinks resolving inside the root. +- **Anti-downgrade state is per signing authority, one BBolt transaction, and content-addressed on disk.** A new `TPABundleStateBucket` (`internal/storage/models.go`) holds, per authority: watermark, its epoch, the fingerprint that set it, the signature ratchet, the FR-010 pin and deny-list — plus an append-only activation history, all in a single `Update`. Last-known-good bundles are stored as `tpa-lkg/.json` and referenced by digest from that record, so writing a new one never destroys the old and no crash can pair new bytes with old state — the failure a fixed `last-known-good.json` filename would reintroduce. Write-temp → fsync file → rename → **fsync the directory** (a file fsync does not make its directory entry durable) → commit → publish; startup reconciles unreferenced files (D6). +- **`BundleInfo` gains a full digest rather than a widened one.** Today's `Fingerprint` is `hex(sha256)[:12]` — 48 bits (R3), fine as a human label and load-bearing in existing output, but not as authorization for the one path that bypasses the watermark. Rollback accepts only the full digest, and status surfaces both. +- **`coverageOK` is where the trust story meets the approval gates.** `inprocess.go` currently computes it from mere bundle presence (R5). FR-007's unsigned-external degradation and FR-009a's fallback demotion both land on that one expression — the narrowest and most safety-critical edit in the feature, and the one whose regression is silent (a quietly suspended `scan`-mode auto-approval looks exactly like a quiet install, which is why FR-020 requires surfacing it). +- **The gate must be taught to see the database before the corpus grows.** `cmd/scan-eval` never loads a bundle and `gateChecks()` omits `BundleCheck` entirely (R6), so FR-016/SC-005 would pass with the corpus absent. `--bundle` + registration through the **production** loader (D8), `categoryCheck` entries for every new class, and a deterministic negative control whose gated-malicious set is one control sample alone (D9). +- **Telemetry needs typed backstop entries, not a relaxation.** `scanV8TPAScanner` whitelists four integer keys and rejects anything else before transmit (R7). Four typed entries plus one payload-level cross-field rule (`sequence` present while `bundle_version == "other"` is a violation), with the provenance gate in the reporting layer where authority is known (D7). +- **Scope stops at the network boundary, behind a source-neutral seam.** Spec 087 is specified but not implemented (R8) and FR-018 forbids inventing a second refresh lifecycle, so this feature delivers the file-drop path end to end and leaves fetch to 087 (D11). SC-003's air-gapped install reaches full P1 capability on that path alone. The cut is only safe because the pipeline takes **bounded bytes + source metadata**, not a path: file-drop and 087's fetch are both thin adapters over one verifier, so the one component that must never exist twice cannot be forked. SC-007's fetch clause moves to 087 **in spec.md** rather than shipping permanently half-met. + +## Technical Context + +**Language/Version**: Go 1.25.5 module toolchain (`go.mod`). Backend-only; no frontend, tray, or Swift changes beyond status fields already rendered generically. +**Primary Dependencies**: **stdlib only for the new work** — `crypto/ed25519`, `crypto/sha256`, `encoding/json`, `syscall`. No `ed25519` usage exists in the tree today (R10). Existing: `go.etcd.io/bbolt` (state), `go.uber.org/zap`, Cobra (CLI). **No new dependencies.** +**Storage**: one new BBolt bucket (`TPABundleStateBucket`) in the existing database; no schema migration, no Bleve change. +**Testing**: `go test -race ./internal/...` (locally `internal/server` needs the CI skip regex); fixture-driven conformance tests for the sidecar format; `cmd/scan-eval --gate` in CI (`.github/workflows/eval.yml`). +**Target Platform**: Linux/macOS/Windows core binary, personal + server editions — pure `internal/`, edition-agnostic. `O_NOFOLLOW` is POSIX-only; Windows gets the same descriptor-level guarantee via `os.Root`-scoped opens (D5). +**Project Type**: Single Go project (core server) + one new leaf package. +**Performance Goals**: SC-008 — a database at the ceilings (≥500 RE2 rules) moves p95 scan latency <10% versus the 6-signature baseline, and load-plus-verify completes under one second. Verification is one Ed25519 check over ≤8 MiB; the cost is compilation, which is bounded by the rule ceiling. +**Constraints**: verify-before-parse (FR-006) is an ordering constraint on every candidate path; all patterns stay RE2-only; the embedded default is exempt from watermark and ratchet (FR-009) so scanning never stops. +**Scale/Scope**: ~5 packages — `internal/security/bundlesig` (new), `internal/security/scanner`, `internal/storage`, `internal/config`, `internal/telemetry` — plus `cmd/scan-eval`, `cmd/mcpproxy` (CLI status/rollback), the corpus, docs, and the CI signing job. + +## Constitution Check + +*GATE: must pass before Phase 0 and be re-checked after Phase 1.* — Constitution v1.1.0. + +| Principle | Assessment | +|-----------|------------| +| **I. Performance at Scale** | PASS. BM25 search and indexing untouched. Verification is one signature check per candidate load — not per scan. The rule ceiling (2000) is what bounds compile cost, and SC-008 measures the hot path directly rather than asserting it. | +| **II. Actor-Based Concurrency** | PASS. The activation swap is Spec 087 FR-010's existing atomic in-memory swap; this feature adds durable state written under a single BBolt transaction on the same path. No new goroutine except D5's bounded reader, which is scoped to one candidate read and abandoned rather than joined — stated as a residual, not hidden. | +| **III. Configuration-Driven Architecture** | PASS. New config: operator keys, operator baseline, operator epoch, active-authority declaration, `require_signed_bundle`. All config-resident **by design, not convenience** — FR-004b, FR-009b and FR-005a each turn on the config-write attacker being out of the threat model while the bundle-path attacker is in it. | +| **IV. Security by Default** | PASS, and this is the feature's whole subject. Verify-before-parse; ceilings before verification's own inputs; per-authority watermarks so neither authority can lock the other out; a config-resident active-authority declaration so file-drop cannot choose the authority; full-digest rollback authorization; a signature ratchet; degraded coverage rather than silent full trust whenever the trust basis is weaker than it looks. The one place the design accepts less than the spec's literal text is recorded in Complexity Tracking. | +| **V. Test-Driven Development** | PASS. Every SC has a named fixture; the sidecar format ships byte-level accept/reject fixtures (D3) so conformance is testable by construction, and D9's control asserts the gate itself is not vacuous. | +| **VI. Documentation Hygiene** | PASS. FR-022 is explicit and in scope: operator lifecycle (verify, drop, roll back, read status) and contributor rules (author a signature, provenance/licence, eval samples, how the gate blocks a bad corpus). | + +**Result**: PASS. One new leaf package, one new BBolt bucket, no new dependency. Re-checked after Phase 1 design: unchanged. + +## Key Decisions (normative for tasks) + +Resolved in [research.md](research.md); recorded here as the plan of record. + +| # | Decision | Where resolved | +|---|----------|----------------| +| D1 | Publication channel = **GitHub Releases on the corpus repo**, immutable per-sequence asset names, `latest` resolved via the Releases API (not a mutable file) | R-D1 | +| D2 | **LOCKED: `require_signed_bundle` stays default-OFF.** Recommendation was to flip it on; maintainer chose off, keeping existing 086 unsigned file-drops working across the upgrade. Consequence: FR-007's degraded coverage + FR-020's surfacing are the only mitigation for a fresh install, so they are mandatory rather than polish | R-D2 | +| D3 | Sidecar = one-line JSON, exactly five keys (`sidecar_version`, `algorithm`, `key_id`, `signature`, `sequence`), total deterministic rejection, Ed25519-only in v1 | R-D3, contracts/sidecar-format.md | +| D4 | `signatures[]` additive top-level section keyed by TPA id, many-to-one to `rules[]`, bidirectional referential integrity enforced at build, licence from an SPDX allowlist | R-D4, contracts/bundle-signatures-section.md | +| D5 | Read budget = `O_NOFOLLOW\|O_NONBLOCK` open + descriptor `fstat` + ceiling from that `FileInfo`; context-cancelled reader goroutine, never `SetReadDeadline`; abandoned readers capped at one per path; Windows build-tagged and explicitly weaker | R-D5 | +| D6 | Activation state = one new `TPABundleStateBucket`, one BBolt `Update`; LKG bundles **content-addressed** (`tpa-lkg/.json`) and referenced by digest; fsync file **and directory**, commit, publish; startup reconciles orphans | R-D6 | +| D7 | Telemetry = four **typed** backstop entries + **whole-tuple** validation (not one direction); `schema_version` + sequence instead of a per-release enum the binary cannot know; provenance gate still in the reporting layer, but the backstop no longer depends on it | R-D7 | +| D8 | `scan-eval --bundle` loads through the **production** loader and registers `BundleCheck`; same path as the activation self-check | R-D8 | +| D9 | Negative control = one signature whose sample no built-in check flags, evaluated over a corpus whose gated-malicious set is that sample alone; no-overlap asserted in CI | R-D9 | +| D10 | Structure = new leaf `internal/security/bundlesig/` for verification only; everything else extends `internal/security/scanner/` | R-D10 | +| D11 | Scope = **file-drop path end to end** behind a source-neutral `Candidate{bytes, source}` pipeline so 087 cannot fork the verifier; SC-007's fetch clause moves to 087 in spec.md | R-D11 | + +## Project Structure + +### Documentation (this feature) + +```text +specs/101-tpa-db/ +├── spec.md # Feature spec (normative) +├── plan.md # This file +├── research.md # Phase 0: R1–R11 findings + D1–D11 decisions +├── data-model.md # Phase 1: state records, manifest additions, entity relations +├── quickstart.md # Phase 1: sign, drop, verify, roll back, read status +├── contracts/ +│ ├── sidecar-format.md # D3 wire format + accept/reject fixtures +│ ├── bundle-signatures-section.md # D4 signatures[] schema + integrity rules +│ └── bundle-status-surface.md # FR-020 fields across CLI / REST / Web UI +└── tasks.md # Phase 2 (/speckit.tasks) +``` + +### Source Code (repository root) — REAL paths + +```text +internal/ +├── security/ +│ ├── bundlesig/ # NEW leaf package (D10) — pure, no I/O, no globals +│ │ ├── sidecar.go # D3 parse + total rejection grammar +│ │ ├── verify.go # Ed25519 verify over raw bundle bytes; key-id agreement (FR-003) +│ │ └── testdata/ # byte-level valid + invalid sidecar fixtures +│ └── scanner/ +│ ├── candidate_open_posix.go # NEW (//go:build !windows): O_NOFOLLOW|O_NONBLOCK + fstat (D5) +│ ├── candidate_open_windows.go # NEW: handle-based open; weaker guarantee, documented (D5) +│ ├── tpa_candidate.go # NEW: source-neutral Candidate{bytes, source} pipeline (D11) +│ ├── tpa_bundle_source.go # file-drop ADAPTER over that pipeline; sidecar cap; +│ │ # verify-before-parse (FR-006, read once, never re-read); +│ │ # BundleInfo gains full digest, sequence, authority, epoch, +│ │ # signature_verified, staleness, last rejection reason (FR-020) +│ ├── tpa_bundle.go # rule-count ceiling; signatures[] parse + integrity (D4); +│ │ # sequence grammar (FR-002 decimal string, ≤ 2^64-1) +│ ├── tpa_bundle_state.go # NEW: per-authority watermark/ratchet/pin/deny-list/history +│ │ # read+write through storage (D6); epoch scoping (FR-005a) +│ ├── tpa_bundle_rollback.go # NEW: FR-010 full-digest rollback, pin + deny-list +│ ├── inprocess.go # coverageOK: degrade on unsigned-external and on the two +│ │ # FR-009a fallback conditions ONLY (R5) — the narrowest edit +│ └── bundled/scanner-bundle.json # seed corpus grows to ≥25 signatures / ≥8 classes (SC-004) +├── storage/ +│ ├── models.go # TPABundleStateBucket const +│ └── tpa_bundle_state.go # NEW: single-Update transaction over that bucket (D6) +├── config/ +│ └── config.go # SecurityConfig: operator keys, operator baseline, operator +│ # epoch, active-authority declaration, require_signed_bundle +└── telemetry/ + └── anonymity.go # scanV8TPAScanner: four typed entries + cross-field rule (D7) +cmd/ +├── scan-eval/ +│ ├── main.go # --bundle flag (D8) +│ └── gate.go # register BundleCheck; categoryCheck entries per new class; +│ # D9 control + its no-overlap assertion +└── mcpproxy/ # security status output (FR-020) + rollback command (FR-010) +.github/workflows/ +├── eval.yml # D2 job now scores the bundle (D8) +└── (corpus repo) # gate-then-sign-then-publish (D1) — not this repo +docs/ # FR-022 operator + contributor lifecycle +``` + +**Structure Decision**: one new leaf package for verification, everything else in place. `bundlesig` earns its own package on the Spec-085 `toolsig` argument — a pure, fixture-testable contract with an independent consumer (any publisher tool) — which does **not** apply to the loader, the state, or the status surface. + +## Test Strategy + +**Fixture-first, because almost every requirement here is a rejection boundary.** A rejection you cannot drive one unit past is not tested. + +- **Sidecar conformance** (D3): byte-level accept and reject fixtures in `bundlesig/testdata/` — missing key, unknown key, duplicate key, wrong type, trailing bytes, non-hex `key_id`, wrong-length signature, unknown algorithm, `sidecar_version != "1"`, malformed `sequence`. Each asserts a *specific* rejection reason, not merely failure, so two implementations can be compared on refusal as well as acceptance. +- **Verify-before-parse** (FR-006/SC-001): a candidate whose bytes are altered after the digest is computed must be refused **before** any pattern compiles — asserted by a compile counter, not by output alone. Plus the four SC-001 tamper classes (bundle byte, signature value, sidecar identity field, mismatched pair) and the explicit non-requirement (sidecar whitespace/key-order changes are NOT required to be rejected). +- **Ceilings** (SC-010a): both sides of every boundary — 8 MiB and 8 MiB+1, 2000 rules and 2001, 4 KiB and 4 KiB+1, and a read exceeding 5 s — with last-known-good serving throughout. +- **Race-free opening** (SC-011): FIFO swapped in between path resolution and open; symlink-to-FIFO; a never-written FIFO whose slot is released within the budget so a later legitimate update still activates. +- **Anti-downgrade** (SC-002/SC-002a): same-authority same-epoch comparisons; the max-sequence lockout and its epoch-bump recovery; the post-bump stale-epoch replay refusal (a2); the two-authority isolation (b); the authority-not-active refusal (b2); and the baseline-less operator authority activating **degraded** rather than fully trusted (c). +- **Rollback** (SC-010): the 12-hex-prefix collision fixture — an attacker-substituted, publisher-signed, current-epoch bundle sharing the abbreviated fingerprint but differing in the full digest — refused as a target mismatch; plus N-activation history replay across a process restart. +- **Crash consistency** (FR-011a): a fault injected between LKG persist, state commit and pointer publish, asserting every recovery lands on a consistent (bundle, watermark) pair, and specifically that a preserved rollback never loses its deny-list. +- **State substitution** (SC-012): a prepared `config.db` with a lowered watermark, a cleared ratchet and a forged pin — in each case the outside-the-data-dir anchors still hold and the install reports degraded. **No fixture may assert the ratchet/watermark/pin survive**; they are best-effort against this attacker by construction (R11). +- **Coverage degradation** (FR-007/FR-009a): the four states that must degrade, and — equally important — the two that must **not**: an ordinary fresh install on the embedded default, and an embedded default at or above the publisher watermark after an external file was removed. A false degrade silently disables `scan`-mode auto-approval fleet-wide. +- **Gate non-vacuity** (SC-005/D9): the control signature's removal turns the gate red with exit `exitGateBreach`, plus the standalone no-overlap assertion that keeps the control honest as built-in checks grow. +- **Telemetry backstop** (D7): each typed entry accepted; a free-string version rejected; the cross-field violation rejected; and an operator-built corpus reporting `other` with `sequence` omitted **even when it claims a genuine publisher version string**. +- **Air-gapped parity** (SC-003): the file-drop path with the network stack unavailable produces byte-identical behavior to a connected install's file-drop path. + +## Documentation & wiring checklist (Constitution VI) + +- `docs/features/security-quarantine.md` + the tool-scanner docs: database lifecycle, degraded-coverage states and what each means. +- New operator doc: verify, drop, roll back, read status; the air-gapped walkthrough. +- New contributor doc: author a signature, provenance/licence rules, eval-sample requirements, how the gate blocks a bad corpus. +- `docs/configuration.md`: operator keys, baseline, epoch, active authority, `require_signed_bundle`. +- `make swagger` after the status-surface change. +- `CLAUDE.md`: security-model line + Recent Changes entry. + +## Complexity Tracking + +| Violation | Why Needed | Simpler Alternative Rejected Because | +|-----------|------------|-------------------------------------| +| **Persisted anti-downgrade state is best-effort against an attacker who can substitute `config.db`** (spec §Threat-model boundary; SC-012) | The bundle path may be configured inside the data directory, so the in-scope bundle-path attacker may also reach the state store. BBolt is transactional and checksummed, **not** authenticated (R11). | Authenticating the state with a key outside the writable data directory means a second key-management story — key location, rotation, and its own compromise path — to protect state whose loss already degrades safely to the FR-009b seeded baseline and the config-resident anchors. The spec accepts this explicitly and SC-012 tests the boundary rather than a guarantee that does not exist. **The plan's obligation is to never present the ratchet, watermark, pin or deny-list as surviving that attacker** — and to keep the anchors that do survive genuinely outside the data dir. | +| **A read blocked on a stalled filesystem leaves a goroutine parked** (D5) | Go cannot interrupt a blocked read on a regular file; `SetReadDeadline` returns `ErrNoDeadline`, which is why FR-011 forbids specifying the bound that way. | Every alternative that truly kills the read is a subprocess or an OS-specific `aio` path — a large, platform-divergent surface to reclaim one goroutine. The property that matters is *the single-flight refresh slot is released and a rejection reason recorded*, so a legitimate update still activates; that is delivered, tested (SC-011c), and stated rather than implied. | +| **SC-007's fetch clause moves to Spec 087** (D11) | SC-007 measures publish → active → fixture fires "within one daily refresh cycle on a fetch-enabled install". There is no refresh cycle: Spec 087 is unimplemented (R8), and FR-018 forbids building a second one here. | Implementing the fetch loop here would either duplicate 087 or absorb it, and absorbing it makes this feature the whole epic. But leaving the criterion in place and half-met is not the alternative: a success criterion no fixture can exercise is a **spec defect**, and maintainer assent does not make it testable. So this is an amendment task against spec.md, not an accepted residual — the fetch clause moves to 087, the manual-drop clause ("and immediately on manual drop") stays here where it is demonstrated end to end. Recorded in this table only because it changes what the spec promises. | + +| **The Windows candidate open gives a weaker guarantee than POSIX** (D5) | `O_NOFOLLOW` does not exist on Windows, and `os.Root` — the obvious substitute — blocks path *escape* but still follows a symlink resolving inside the root, so a final symlink to a regular file passes `fstat` cleanly. | Reaching parity means a handle-based open with reparse-point controls (`FILE_FLAG_OPEN_REPARSE_POINT`), which is a real amount of platform-specific code for a path the operator already controls. Design may take that route; what is NOT acceptable is implying parity. If the weaker guarantee is accepted, it is written down: on Windows the candidate path trusts its containing directory and carries symlink-substitution risk POSIX does not. | + +No other deviations: no new dependency, one new leaf package the verification contract justifies, one new BBolt bucket the crash-consistency requirement names. diff --git a/specs/101-tpa-db/quickstart.md b/specs/101-tpa-db/quickstart.md new file mode 100644 index 00000000..ac10283f --- /dev/null +++ b/specs/101-tpa-db/quickstart.md @@ -0,0 +1,148 @@ +# Quickstart: tpa-db + +**Feature**: `101-tpa-db` · Sign a corpus, drop it, verify it activated, roll it back, read the status. + +Everything here works with **no network** — that is SC-003, not a footnote. The fetch path is Spec 087's (plan D11). + +## 0. Build + +```bash +go build -o mcpproxy ./cmd/mcpproxy +``` + +Use an isolated instance for all of this — a high port and a scratch `--data-dir` **and** `--config`. The watermark, ratchet and pin are per-data-directory, so testing against your real one leaves state behind that is deliberately hard to undo. + +```bash +SCRATCH=$(mktemp -d) +cat > "$SCRATCH/config.json" <<'EOF' +{ "listen": "127.0.0.1:18701", "api_key": "dev", "mcpServers": [] } +EOF +``` + +## 1. Generate a signing key + +```bash +go run ./cmd/tpa-db keygen --out "$SCRATCH/keys" +# writes keys/operator.key (private, 0600) and keys/operator.pub +# prints the key id: the lowercase-hex SHA-256 of the 32-byte raw public key +``` + +The **key id is a fingerprint of the key**, never a label you choose (FR-003). That is what makes "the manifest names one publisher, the bytes are signed by another" detectable rather than cosmetic. + +## 2. Sign a bundle + +```bash +go run ./cmd/tpa-db sign \ + --bundle internal/security/scanner/bundled/scanner-bundle.json \ + --key "$SCRATCH/keys/operator.key" \ + --sequence 1 \ + --epoch 0 \ + --out "$SCRATCH/scanner-bundle.json" +# writes scanner-bundle.json and scanner-bundle.json.sig +``` + +`sign` refuses a bundle that fails the eval gate — FR-016 means a bundle that cannot pass is never signed, so the gate runs here rather than only in CI. + +Inspect the sidecar; it is one line, five keys, nothing else ([contract](contracts/sidecar-format.md)): + +```bash +cat "$SCRATCH/scanner-bundle.json.sig" | jq . +``` + +## 3. Configure the operator authority and drop the bundle + +An operator-signed bundle needs three config decisions, all of them **config-resident on purpose**: the config-write attacker is out of the threat model, the bundle-path attacker is in it. + +```json +{ + "security": { + "tpa_bundle_path": "/scanner-bundle.json", + "tpa_active_authority": "operator", + "tpa_operator_keys": [""], + "tpa_operator_baseline": 1, + "tpa_operator_epoch": 0, + "require_signed_bundle": true + } +} +``` + +Declaring `tpa_operator_baseline` matters more than it looks. Without it the authority runs **explicitly degraded** rather than silently trusted — with no baseline there is no anti-downgrade at all after a data-directory wipe, and an attacker who can write the bundle path could replay any older genuinely operator-signed bundle at full coverage (FR-009b). + +Start the proxy; the file watcher picks the bundle up without a restart. + +```bash +./mcpproxy serve --config "$SCRATCH/config.json" --data-dir "$SCRATCH/data" --log-level=debug +``` + +## 4. Verify it activated + +```bash +./mcpproxy security bundle -o json --config "$SCRATCH/config.json" --data-dir "$SCRATCH/data" | jq . +``` + +Expect `signature_verified: true`, `authority: "operator"`, `trust_epoch: 0`, `sequence: "1"`, a `full_digest`, `coverage_degraded: false`. + +**If `coverage_degraded` is true, read `degraded_reason` before anything else** — it distinguishes "you forgot the baseline" (`operator_authority_unbaselined`) from "the bundle is unsigned" (`unsigned_external_active`) from "we fell back to the embedded default" (`embedded_below_watermark`). A degraded install has `scan`-mode auto-approval suspended, which is otherwise invisible. + +## 5. Watch a downgrade be refused + +Sign the same corpus at a lower sequence and drop it: + +```bash +go run ./cmd/tpa-db sign --bundle … --sequence 1 --epoch 0 --out "$SCRATCH/old.json" +cp "$SCRATCH/old.json"* "$SCRATCH/" # overwrite the configured path +``` + +The sequence-1 artifact is now at or below the watermark, so it is refused and the current bundle keeps serving: + +```bash +./mcpproxy security bundle -o json … | jq '.last_rejection' +# { "reason": "downgrade", "at": "…" } +``` + +Nothing about this is fatal — every rejection keeps last-known-good. That is the point: a refused candidate must never leave the scanner with an empty or partial rule set. + +## 6. Roll back — by digest, not by path + +Read the history to learn the digest you want: + +```bash +./mcpproxy security bundle history -o json … | jq '.[] | {sequence, full_digest, activated_at}' +``` + +Then roll back naming **the full digest**: + +```bash +./mcpproxy security bundle rollback \ + --path "$SCRATCH/old.json" \ + --digest … +``` + +The path is a *hint about where to find those bytes*; the digest is the authorization. Rollback is the one path that deliberately bypasses the watermark, so identifying its target by an attacker-writable path would hand the attacker the choice of what the bypass lands on. Supply the wrong digest and it is refused `rollback_target_mismatch` — try it, it is one command: + +```bash +./mcpproxy security bundle rollback --path "$SCRATCH/old.json" --digest 0000… # refused +``` + +**The 12-hex short fingerprint is not accepted here.** It is 48 bits — fine as a human label, not as authorization for a watermark bypass where the attacker chooses the substituted bytes. + +After a rollback the artifact is **pinned** and the release you rolled away from is **deny-listed**, so the next refresh cycle cannot re-activate the very thing you just rejected. Clear the pin to return to normal watermark-governed activation: + +```bash +./mcpproxy security bundle unpin … +``` + +## 7. Air-gapped check (SC-003) + +Steps 1–6 with the network unavailable behave identically. That is the criterion — the file-drop path is not a degraded mode, it is the primary one, and the fetch path adds convenience rather than capability. + +## 8. Contributor loop + +```bash +go run ./cmd/tpa-db validate --bundle # schema + signatures[] integrity + licences +go run ./cmd/scan-eval --gate --bundle # recall ≥ 0.90 gated, hard-negative FP ≤ 0.05 +``` + +`validate` enforces the [signatures[] contract](contracts/bundle-signatures-section.md) in both directions — every signature names a real rule, **and** every gating rule is named by exactly one signature. The second direction is the one that is easy to skip and the one that matters: without it a gating rule can ship with no provenance, no licence and no eval evidence. + +`--gate` is the same code path activation runs (plan D8), so a bundle that passes here is one that will activate — the CI layer and the activation layer cannot drift apart. diff --git a/specs/101-tpa-db/research.md b/specs/101-tpa-db/research.md new file mode 100644 index 00000000..163586c1 --- /dev/null +++ b/specs/101-tpa-db/research.md @@ -0,0 +1,234 @@ +# Phase 0 Research: Versioned, Signed, Offline-First TPA Signature Database + +**Feature**: `101-tpa-db` · **Spec**: [spec.md](spec.md) · **Date**: 2026-08-26 + +Two kinds of entry below. **D**ecisions resolve a `[NEEDS CLARIFICATION]` marker or a "MUST be specified in design" instruction from the spec. **R**esults are mechanical findings about the tree as it stands — each one is a fact a task will trip over if the plan does not state it. + +--- + +## Mechanical findings (the tree as it is today) + +### R1 — The loader is in `internal/security/scanner/`, not the package Spec 087's plan named + +Spec 087's plan proposes a new package `internal/security/bundle` owning the file lifecycle. **That package does not exist.** What shipped (Spec 086) lives in `internal/security/scanner/`: + +| File | Role | +|---|---| +| `tpa_bundle_source.go` | `ConfigureBundle(path, logger)`, `BundleStatus() BundleInfo`, `loadEmbeddedBundle`, `loadBundleFromFile`, `loadBundleWithInfo`, and the `storeBundle`/`snapshotBundle` global-state pair | +| `tpa_bundle.go` | `rawBundle`/`rawRule`/`rawSkip` wire types, `bundleMetadata`, `loadBundleCheck`, `checkBundleVersion`, `BundleCheck` (the `detect.Check`) | +| `registry_bundled.go` | embedded default via `go:embed` | +| `bundled/scanner-bundle.json` | the shipped corpus | + +**Consequence for the plan**: this feature extends those files rather than creating 087's package. Reading 087's plan as a description of the current tree is the single most likely source of wrong file paths in tasks — treat 087's *spec* as normative for the refresh lifecycle and its *plan* as superseded on structure. + +### R2 — Candidate reads are completely unbounded today + +`tpa_bundle_source.go` reads a candidate with a bare `os.ReadFile(path)` carrying a `//nolint:gosec` comment that reasons "operator-configured path, same trust level as `mcp_config.json`". There is **no size check, no file-type check, no deadline, and no re-read protection**. `loadBundleCheck` (`tpa_bundle.go`) applies **no rule-count limit**. FR-011's ceilings are therefore entirely new code, not a tightening — and the existing nolint rationale is exactly the assumption FR-011 overturns, since the bundle path is in the threat model while the config path is not. That comment must be updated, not just the code beside it. + +### R3 — `BundleInfo.Fingerprint` is 48 bits + +`tpa_bundle_source.go` computes `Fingerprint: hex.EncodeToString(sum[:])[:12]` — the first 12 hex characters of the SHA-256. FR-010 requires the **full** digest for rollback authorization, so `BundleInfo` needs a second field rather than a widened one: the short form is load-bearing in existing operator-facing output and in `logBundle`. + +### R4 — The bundle has nowhere to put per-signature metadata + +Top-level keys of the shipped `scanner-bundle.json`: `bundle_version`, `generated_from`, `rules`, `schema_version`, `signature_count`, `skipped`. A `rules[]` entry is keyed `category, confidence, detector, engine, flags, id, indicators, level, pattern, severity, target, type`. **No provenance field, no license field, anywhere.** This is what FR-001a exists to fix, and it means FR-012's "each signature MUST carry provenance and a redistributable license" cannot be satisfied by any rearrangement of today's shape. + +### R5 — `coverageOK` is derived from mere bundle presence + +`internal/security/scanner/inprocess.go`: `coverageOK = bundlePresent && result.Coverage.ChecksFailed == 0`. Nothing consults signature state, authority, or watermark. FR-007's degradation rule and FR-009a's fallback demotion both land on this one expression, which makes it the narrowest and most safety-critical edit in the feature. + +### R6 — The eval gate cannot see the bundle at all + +`cmd/scan-eval/gate.go`'s `gateChecks()` returns only the four built-in `detect.Check`s; `scanner.BundleCheck` — which production **does** append in `inprocess.go` — is absent, and `scan-eval` never loads a bundle. `categoryCheck` maps a category to a check id, and `gatedCategory()` enforces a category **only** when its mapped check id is registered. So today FR-016/SC-005 would pass with the corpus entirely absent. This is FR-016a, and it is a prerequisite for the seed-corpus work rather than a parallel task: growing the corpus while the gate cannot score it is how a vacuous pass ships. + +### R7 — The telemetry backstop rejects strings and booleans by construction + +`internal/telemetry/anonymity.go` `scanV8TPAScanner` whitelists the `tpa_scanner` sub-object to `{scans_completed, scans_failed, scans_with_findings, findings}` and rejects anything else with `"carries a key outside the whitelist"`. A string `bundle_version`/`source` or a boolean `signature_verified` is refused **before transmit**. FR-021's typed extension is therefore a change to the backstop itself, and the cross-field invariant FR-021 names (`sequence` present while `bundle_version` is `other` → reject) is enforceable right there on the payload. + +### R8 — Spec 087 is specified but not implemented + +No `internal/security/bundle`, no `security.bundle` config section, no refresh ticker. `SecurityConfig` carries exactly one bundle-related field, `TPABundlePath` (`config.go:2751`), plus the `MCPPROXY_TPA_BUNDLE_PATH` env override resolved in `EffectiveTPABundlePath`. **Sequencing consequence**: FR-018 forbids this feature from inventing a second refresh lifecycle, but there is no first one to insert stages into. The spec's own Assumptions anticipate this — US1's signature/sequence verification is reachable through the existing Spec-086 `ConfigureBundle` hot-reload/file-drop path — so the plan phases the fetch-dependent parts behind 087 and delivers the file-drop path first. + +### R9 — Persistence has a home but no bucket + +BBolt buckets are declared in `internal/storage/models.go` (`UpstreamsBucket`, `ToolApprovalBucket`, `MetaBucket`, …). FR-011a's one-transaction requirement maps cleanly onto a single BBolt `Update` over a new bucket; nothing in the current schema needs to change. + +### R10 — Ed25519 is stdlib and unused here + +No `crypto/ed25519` import exists in `internal/` or `cmd/` (the only `ed25519` matches are SSH-key *path strings* in the sensitive-data detectors). Verification is stdlib — **no new dependency**, consistent with the repo's stated stance. + +### R11 — The `config.db` trust boundary is a boundary, not a guarantee + +BBolt is transactionally consistent and checksummed against corruption; it is not authenticated. Spec §Threat-model boundary already says so. Every persisted item this feature adds — watermark, ratchet, pin, deny-list, activation history — inherits that. The plan must never present them as surviving a prepared-database substitution (SC-012 asserts exactly this). + +--- + +## Decisions + +### D1 — Publication channel: **GitHub Releases on the corpus repo** (resolves FR-017's marker) + +**Decision**: publish artifact pairs as GitHub Release assets on the tpa-db corpus repository, with **content-addressed** asset names (`scanner-bundle-.json` + `.sig`) plus a **signed release index** mapping sequence → digest, and a `latest` pointer inside that index. + +**"Immutable per-release asset names" would have been wrong.** GitHub Release assets are mutable: a maintainer (or a compromised token) can delete an asset and upload a different one under the same name, and releases and tags can be edited after publication. A sequence-bearing filename asserts immutability the platform does not provide. Content-addressing makes the name a checksum instead of a promise, so a substituted asset simply fails to match the digest the index — itself signed — names. Publication automation MUST additionally refuse to replace an existing asset, and historical artifacts needed for FR-010 rollback MUST be archived independently of the Release that first carried them. + +**Rationale**: the alternative (a static path under mcpproxy.app) puts corpus publication on the Cloudflare Pages deploy pipeline, which is already the site's critical path and whose failure mode is silent — [the appcast feed has gone stale this way before](../../docs/). Releases give immutable asset URLs, a publication timestamp SC-006's server-side age join needs, and a signing job that lives in the same repo as the corpus it signs, so FR-016's "never signed or published without a green gate" is one workflow rather than a cross-repo handoff. + +**Alternatives considered**: mcpproxy.app static path — rejected for the silent-failure coupling above and because it separates the artifact from the gate that must precede it. A dedicated CDN — rejected as unjustified operational surface for a file fetched at most daily per install. + +**Consequence for the 087 handoff**: `latest` resolves through the signed index, not a fixed href — and the index must be reachable as a **cacheable object at a stable URL**, not only through the Releases API. Relying on the API alone adds an availability dependency and a rate limit that a fleet shares: unauthenticated GitHub API calls are limited per source IP, so installs behind one NAT doing a synchronized daily refresh can exhaust it and all fail together. 087 MUST therefore specify conditional requests (`ETag`/`If-None-Match`), refresh jitter so a fleet does not tick in lockstep, and explicit behavior when the index is unreachable or rate-limited (keep last-known-good, record the reason, do not treat it as a verification failure). + +### D2 — `require_signed_bundle` stays **opt-in indefinitely** (resolves the Assumptions marker) + +**Decision (LOCKED 2026-08-26, maintainer): `require_signed_bundle` stays DEFAULT-OFF.** The trade below was put to the maintainer with the recommendation to flip it on; the call is to keep it off for now. + +**Why the draft rationale fails.** It argued that a flip "would change behavior only for installs that have never seen a signed bundle". That is true — and it is the wrong conclusion, because **those installs are exactly the unprotected population**. The ratchet only arms after a first signed activation; before that, an attacker who can write the bundle path (in scope) can drop an unsigned corpus and have it activate. Saying the ratchet "does the work" describes installs that are already safe. + +**The real trade is back-compat, and it is not free either way.** + +| | Default OFF (draft) | Default ON with opt-out (review's proposal) | +|---|---|---| +| Fresh install, attacker writes bundle path | unsigned corpus **activates** (degraded coverage, but serving) | refused | +| Existing 086 user with an unsigned file drop, upgrading | keeps working | **breaks** until they set the flag | +| Air-gapped / self-built corpus | works out of the box | works after one explicit opt-out | + +Unsigned file drops are a shipped 086 capability (`security.tpa_bundle_path`), so default-on is a **user-visible breaking change on upgrade** — loud rather than silent (the rejection reason is surfaced per FR-020), but breaking. + +**Recommendation as put** was to flip to default-on for EXTERNAL candidates with an explicit opt-out. **The maintainer chose default-off**, which keeps every existing 086 unsigned file-drop working across the upgrade — the breaking change was the whole cost of the alternative, and the population it would protect (installs that have never activated a signed bundle) is the same population that has not yet adopted signing at all. + +**What that obliges this spec to carry, so the residual is not silent**: while an unsigned EXTERNAL bundle is active, FR-007 already degrades coverage and suspends `scan`-mode auto-approval, and FR-020 requires that state and its reason to be surfaced. Those are now the ONLY thing standing between a fresh install and an attacker-supplied unsigned corpus, so they are not optional polish — a task that ships the loader without the degraded-state surfacing has shipped the risk without the mitigation. + +**Revisit trigger**: SC-006 telemetry showing a material population running unsigned external bundles long-term. That is the evidence a future flip would need, and it is now measurable by design. + +### D3 — Sidecar wire format (resolves FR-003's "MUST be specified in design") + +**Decision**: a JSON object, one line, with exactly five required keys and no others. Full grammar and fixtures in [contracts/sidecar-format.md](contracts/sidecar-format.md). + +```json +{"sidecar_version":"1","algorithm":"ed25519","key_id":"<64 lowercase hex>","signature":"","sequence":""} +``` + +- `sidecar_version` — exactly `"1"` in v1; any other value is a hard rejection (not a fallback). +- `algorithm` — exactly `"ed25519"`. **An unknown algorithm is a hard rejection, never a skip.** +- `key_id` — the canonical full-length fingerprint of the public key: lowercase hex SHA-256 of the 32-byte raw Ed25519 public key. Not an operator-chosen label (FR-003). +- `signature` — standard base64 with padding, decoding to exactly 64 bytes. +- `sequence` — the decimal-string form of FR-002's counter, duplicated here so a mismatch against the signed manifest is detectable **before** the bundle is parsed. + +Rejection is deterministic and total: a missing key, an unknown key, a duplicate key (JSON with repeated names), a wrong type, trailing bytes after the object, a `key_id` that is not 64 lowercase hex, a `signature` that does not decode to 64 bytes, or a `sequence` failing FR-002's grammar. **No leniency anywhere** — this is the artifact an independent publisher must reproduce exactly. + +**Rationale**: JSON because the bundle is JSON and the repo already parses it strictly; five fields because each one is consumed before verification and nothing else is; `sequence` duplicated because it is the one manifest field whose disagreement with the sidecar is cheap to detect early and is otherwise only visible after parsing the thing the signature protects. + +**Alternatives considered**: a binary/`minisign`-style format — rejected as a second parser and a second toolchain for publishers, for no property JSON lacks here. Embedding the signature in the bundle — rejected outright: it makes the signed bytes self-referential. + +### D4 — `signatures[]` section shape (resolves FR-001a) + +**Decision**: an additive top-level `signatures[]` array keyed by TPA id, with rules referencing signatures many-to-one. Full schema in [contracts/bundle-signatures-section.md](contracts/bundle-signatures-section.md). + +```json +{"signatures":[{"id":"TPA-2026-0001","category":"instruction_injection","gating":true, + "provenance":{"source":"https://…","published":"2026-01-14"}, + "license":"CC-BY-4.0","rule_ids":["r-hidden-important-block"]}]} +``` + +Build-time referential integrity, enforced in **both** directions and rejected whole on failure: every `signatures[]` entry names ≥1 existing rule id; every rule with `gating: true` is named by exactly one signature entry; TPA ids are unique; licenses come from a fixed allowlist of redistributable SPDX ids. + +**Rationale**: additive keeps contract §4 forward-compat, so a v0.1 loader ignores it (FR-001a). Many-to-one because a campaign class routinely needs several patterns — one hidden-instruction signature already implies distinct rules for `` blocks, HTML comments and zero-width runs. Keying by TPA id rather than by rule is what makes SC-004's "at least 25 signatures spanning 8 classes" a countable property. + +**Alternatives considered**: per-rule `provenance`/`license` fields — rejected: it duplicates the record across every rule implementing one campaign and gives no place to hang gating intent or the class identity SC-004 counts. + +### D5 — Read budget mechanism (resolves FR-011's "design MUST name the concrete mechanism") + +**Decision**: three distinct mechanisms, because one does not fit all three cases. + +| Case | Mechanism | +|---|---| +| Local file open | `os.OpenFile(path, os.O_RDONLY\|syscall.O_NOFOLLOW\|syscall.O_NONBLOCK, 0)`, then `f.Stat()` on the **descriptor**, refuse unless `Mode().IsRegular()`, clear `O_NONBLOCK` on the surviving regular file, then `io.ReadFull` bounded by the ceiling read from that same `FileInfo` | +| Local file read time | a goroutine performing the read, selected against a `context.WithTimeout(5s)`; on expiry the **slot is released and a rejection reason recorded** while the goroutine is abandoned | +| Fetch body | `http.Client` timeout + request context + `io.LimitReader(body, ceiling+1)` | + +**`O_NONBLOCK` is load-bearing, not defensive.** Opening a FIFO for reading with no writer attached **blocks in `open(2)` itself** — before `fstat`, before the reader goroutine, before any timeout can observe anything. Without it the whole descriptor-based design is defeated by the very fixture it exists to pass: SC-011(c)'s never-written FIFO would park the loader inside `open`, and the 5 s budget could not release the single-flight slot because nothing has started measuring yet. With `O_NONBLOCK` the open returns immediately, `fstat` sees a FIFO, and the candidate is refused `not_regular_file` without a single byte read. The flag is then cleared for the regular-file case, so an ordinary read keeps blocking semantics and `io.ReadFull` behaves normally rather than spinning on `EAGAIN`. + +The abandoned-goroutine case is stated as an accepted residual, not hidden: a read blocked on a stalled network filesystem cannot be interrupted in Go, so the guarantee is *the refresh path stays available*, not *the goroutine dies*. **Windows is a weaker guarantee, stated rather than claimed away.** `O_NOFOLLOW` does not exist there, and `os.Root` is **not** a substitute: it prevents a path from escaping the root, but a symlink resolving *within* the root is still followed, and a final symlink to a regular file passes `fstat` cleanly. So the no-follow property is genuinely not delivered on Windows by that mechanism. Design MUST either open by handle with reparse-point controls (`FILE_FLAG_OPEN_REPARSE_POINT`) and inspect the opened handle, or record explicitly that the Windows candidate path trusts its containing directory and carries symlink-substitution risk that POSIX does not. Silently implying parity is the one option that is not acceptable. + +**This needs build-tagged files, and the plan's file list must say so.** `syscall.O_NOFOLLOW` and `syscall.O_NONBLOCK` do not compile on Windows, so the candidate open is a platform-split helper — `candidate_open_posix.go` (`//go:build !windows`) and `candidate_open_windows.go` — each with its own test. A single `syscall`-using file would simply fail the Windows build. + +**The residual must be bounded, or releasing the slot converts a stall into a leak.** Releasing the single-flight slot lets the next cycle start, which opens the same path again — so a path that stays stalled accumulates one parked goroutine and one file descriptor per refresh, indefinitely (a daily tick makes that slow, not harmless). Design MUST therefore cap concurrently-abandoned readers for a given candidate path at a small number (1 is sufficient: a second stall on the same path means the first has not returned) and, while that cap is reached, refuse the candidate immediately with the existing `read_timeout` reason instead of opening it again. That keeps the availability property — the refresh path is never *blocked* — while making the fd/goroutine cost O(1) rather than O(refreshes). Fixture: SC-011(c) extended to run a second and third refresh cycle against the same never-written FIFO and assert the process holds no more than one parked reader for it. + +**Rationale**: FR-011 already rules out `SetReadDeadline` (unsupported on regular files — returns `ErrNoDeadline`). The descriptor-level check is the only form that closes the stat/open substitution window. + +### D6 — Crash-consistent activation state = **one BBolt bucket, one `Update`** (resolves FR-011a) + +**Decision**: a new `TPABundleStateBucket` in `internal/storage/models.go` holding one record per authority plus one activation-history list, written in a single BBolt `Update` transaction. The recoverable last-known-good bundle bytes are made durable **before** the transaction commits and before the in-memory pointer is published. + +**LKG files are CONTENT-ADDRESSED, and that is what makes the ordering safe.** An earlier draft said "persist LKG bytes → commit → publish" and claimed a pre-commit crash leaves the old (bundle, watermark) pair. That is false if the new bytes land at a fixed filename: overwriting or renaming onto `last-known-good.json` before the commit means a pre-commit crash leaves **new bytes under old state** — the same split the requirement forbids, reached from the other side. + +The protocol is therefore: + +```text +write /tpa-lkg/.json.tmp +fsync that file +rename to /tpa-lkg/.json # atomic; readers never see a partial file +fsync /tpa-lkg/ # NOT optional — see below +commit the BBolt state record, which REFERENCES the digest +publish the in-memory pointer +``` + +Because the filename **is** the digest, writing a new LKG never destroys the old one, so a crash at any point leaves every previously-committed (state → bundle) pair intact and resolvable. The state record is the only thing that says which digest is current, and it becomes durable in one transaction. + +**The directory `fsync` is a separate step and is not optional.** `fsync` on a newly created file flushes its *contents*; it does not make the *directory entry* naming it durable. Skip it and a crash can leave the state record committed — BBolt fsyncs its own file on commit — while the bundle it references has no name on disk. + +**Recovery must reconcile orphans**, since content-addressing trades a lost-file failure for an unreferenced-file one: on startup, delete any `tpa-lkg/*.json` no state record references (a crash between rename and commit), and if the referenced digest is *missing*, fall back to the embedded default under FR-009a rather than starting with no corpus. + +Note BBolt's own durability needs no help here: it fsyncs on commit by default, and `NoSync` must stay off on this path. + +**Rationale**: BBolt gives the single-transaction property for free and is already the durable store for approvals and quarantine — the state this record sits beside. R9 confirms no schema change is needed. The deny-list and the watermark's setting-fingerprint are members of the same record precisely because FR-011a names the crash that splits them. + +### D7 — Telemetry: typed whitelist entries + a payload-level cross-field invariant (resolves FR-021) + +**Decision**: extend `scanV8TPAScanner` with four **typed** entries rather than relaxing it: integer `sequence`, enum-bounded `source ∈ {embedded, file, fetch}`, boolean `signature_verified`, and a bounded `bundle_version`. Enforce the complete allowed **tuple** at the backstop, not a single one-directional rule. + +**`bundle_version` cannot be both verbatim and a compiled enum.** A binary shipped today cannot enumerate release lines published after it, so a static enum would classify every *new* corpus as `other` until the binary itself updates — defeating the point of an independently updatable database, and making SC-006's freshness metric blind to exactly the installs that are current. Accepting an arbitrary signed string is not bounded at all. The resolution is a **normalization rule independent of any individual release**: transmit the corpus's `schema_version` (a small, slow-moving set the binary does know) plus the `sequence`, and let the server side join sequence → publication date from what the publisher already knows. The near-unique free string never leaves the install. + +**The cross-field rule must be bidirectional.** "`sequence` present while `bundle_version == "other"` is a violation" catches one direction and leaves three holes: a publisher-looking version with no sequence, a publisher-looking version from `source: file` that never verified, and `signature_verified: false` alongside publisher provenance. The backstop MUST validate the whole tuple: `(publisher schema_version, sequence)` appear **together** and **only** when `source == "embedded"`, or `source ∈ {file, fetch}` **and** `signature_verified == true`; every other combination MUST be `other` with `sequence` omitted. The reporting layer still applies the provenance gate — it is the only layer that knows which authority verified — but the backstop no longer depends on the reporting layer being correct, which is the entire reason a backstop exists. + +The provenance gate lives in the reporting layer, where authority is known: verbatim version + sequence are reported only when the active bundle's provenance is the publisher authority (the embedded default, or an external bundle signature-verified against a publisher key). Every other bundle reports `other` with `sequence` omitted **regardless of the version string it claims** — a string allowlist alone is defeated by an operator corpus copying a genuine publisher version verbatim. + +**Rationale**: R7 shows the backstop rejects the naive addition outright, and FR-021 explicitly forbids relaxing it to free strings. Typed entries keep the backstop's purpose (no near-unique value reaches the payload) while making the new fields expressible. + +### D8 — `scan-eval` loads the bundle through the **production** loader (resolves FR-016a) + +**Decision**: `cmd/scan-eval` gains a `--bundle ` flag defaulting to the embedded default, loads it via `scanner`'s existing loader (never a parallel parser), and registers the resulting `BundleCheck` in `gateChecks()`. The same loading path is what the Spec-087 activation self-check runs. + +**Rationale**: FR-016a names the shared path explicitly, and the reason is drift: two parsers would let CI and activation disagree while SC-005 claims they are independent checks of the same artifact. Sharing the loader makes them independent *runs* of one implementation, which is the property actually wanted. + +**Consequence**: `gateChecks()` stops being a pure function of the built-in set, so its "MUST mirror" doc comment needs rewriting rather than deleting. + +### D9 — Control signature for the negative control (resolves FR-016b) + +**Decision**: designate one control signature whose canonical malicious sample is verified to be hard-flagged by **no** built-in check and by no other bundle signature, and evaluate the control over a corpus whose gated-malicious set is **that sample alone**, so removal drives recall to 0 and `decide()` fails unambiguously. + +The no-overlap property is asserted in CI as its own test, so a later built-in check gaining coverage of the control sample turns the control **red** rather than silently re-vacuating it. + +**Rationale**: FR-016b spells out both vacuity paths. The "restrict to the control's category" shortcut is specifically rejected there — ten gated samples in a category land on exactly 0.90 after one miss, and `decide()` fails only on `< minRecall`. + +### D10 — Structure: extend `internal/security/scanner/`, add `internal/security/bundlesig/` for verification only + +**Decision**: the loader, ceilings, activation and status work extend the existing `scanner` files (R1). Signature verification and sidecar parsing go in a new leaf package `internal/security/bundlesig/` — pure functions over bytes, no I/O, no global state, no scanner imports. + +**Rationale**: verification is the one piece with a genuinely independent contract (D3's format), it is where an independent implementation would be compared byte-for-byte, and a leaf package is what makes the fixture-driven conformance tests cheap. Everything else is already `scanner`'s job and moving it would be churn. Deliberately **not** 087's `internal/security/bundle` — that name describes a lifecycle that shipped elsewhere. + +### D11 — Phasing against unimplemented Spec 087 (resolves the R8 sequencing problem) + +**Decision**: this feature delivers the **file-drop path end to end** (format, signing, verification, ceilings, watermark, rollback, status, seed corpus, gate) and stops at the network boundary. The fetch path stays 087's, and this spec's FR-008 stages are written so 087 inserts them without a second lifecycle. + +**Rationale**: FR-018 forbids a second refresh lifecycle, and R8 shows there is no first one. Building the fetch loop here would either duplicate 087 or absorb it — both worse than shipping the artifact and trust story that 087 then consumes. SC-003 (air-gapped install reaches full P1 capability) is satisfied entirely by the file-drop path, which is the strongest signal that this is the right cut. + +**The loader MUST be source-neutral from day one, or 087 will fork it.** Cutting scope at the network boundary is only safe if the seam is in the right place. A file-oriented loader (path in, bundle out) forces 087 to either duplicate verification or invent temp-file semantics so a fetched body can be fed to a path-shaped API — and a duplicated verifier is the one piece of this feature that must never exist twice. Design MUST therefore define ONE candidate pipeline taking **bounded raw bytes plus source metadata**: + +```text +Candidate{ BundleBytes, SidecarBytes []byte; Source (embedded|file|fetch); Origin string } + → verify → parse → ceilings → epoch → sequence → compile → self-check → activate +``` + +The file-drop path becomes a thin adapter (open, fstat, bounded read → `Candidate`), and 087's fetch path becomes a second adapter (`http` + `io.LimitReader` → `Candidate`) that shares every stage after it. The ceilings then apply identically to both, which FR-011 requires anyway. + +**Consequence for SC-007**: its fetch half is not demonstrable by this feature. Rather than leaving a success criterion permanently half-met — which is a spec defect, not a plan caveat, and no amount of assent makes it testable — **spec.md MUST be amended to move the fetch-cycle clause of SC-007 to Spec 087**, leaving the manual-drop clause here where it is fully demonstrable. That edit is a task, not a footnote. diff --git a/specs/101-tpa-db/spec.md b/specs/101-tpa-db/spec.md new file mode 100644 index 00000000..a29eb53e --- /dev/null +++ b/specs/101-tpa-db/spec.md @@ -0,0 +1,246 @@ +# Feature Specification: Versioned, Signed, Offline-First TPA Signature Database (tpa-db) + +**Feature Branch**: `101-tpa-db` +**Created**: 2026-08-23 +**Status**: Draft +**Input**: Roadmap P1 epic `tpa-db` — "Build a versioned, offline-first signature/pattern database (known TPA campaigns, malicious phrase corpora, IoC hashes) that the engine consumes — bundled with the binary, refreshable out-of-band, community-contributable, and guarded by the existing scan-eval recall/FP CI gate." Tasks: `tpa-db-format` (signature DB format + loader: versioned, signed, bundled default), `tpa-db-corpus` (seed corpus of known public TPA campaigns/patterns), `tpa-db-refresh` (out-of-band refresh, offline-friendly, eval-gated). + +## Positioning *(context)* + + + +## User Scenarios & Testing *(mandatory)* + +### User Story 1 - Signed database format + trust-anchored, downgrade-proof loader (Priority: P1) + +An operator receives a TPA signature database out of band — a `scanner-bundle.json` plus its detached signature file — and drops both into the configured bundle location (or lets the opt-in Spec 087 fetch retrieve them). Before anything else touches the bytes, mcpproxy verifies the signature against its built-in publisher trust anchors (or an operator-configured key), checks that the candidate's publish sequence is not older than what is already active, and only then runs the existing validate-before-activate pipeline (version compatibility, pattern compilation, activation self-check). A tampered bundle, a bundle signed by an unknown key, or a replayed older bundle never activates: the last-known-good database keeps serving and the rejection reason is recorded and surfaced. When the operator genuinely needs to roll back (a bad signature pushed by the publisher), an explicit, loudly-logged rollback action reverts to the embedded default or a named prior bundle. + +**Why this priority**: The database is a code-adjacent security input: whoever controls its bytes controls what the scanner ignores. Without signing and anti-rollback, the "refreshable signature DB" pillar creates a new supply-chain surface (swap the file → blind the scanner; replay an old bundle → resurrect a fixed blind spot). Signing + sequence + fail-closed loading is what makes every other story safe to ship, so it goes first. + +**Independent Test**: Fully offline, no live upstream. Sign a fixture bundle with a test key configured as a trust anchor; drop bundle+signature and assert activation with `signature_verified=true` and the new sequence recorded. Then assert each of the following is rejected with the active bundle unchanged and a machine-readable reason: (a) one flipped byte in the bundle, (b) a signature from a non-trusted key, (c) a missing sidecar with `require_signed_bundle=true`, (d) a validly-signed bundle whose sequence is lower than the active one, (e) a validly-signed bundle failing the activation self-check, (f) an unsigned bundle dropped after a signed one was active (ratchet), (g) an unsigned bundle claiming a huge sequence — assert the watermark is unchanged so a later legitimate signed release still activates, and (h) on a FRESH install with no persisted state, a genuinely publisher-signed current-epoch bundle whose sequence is older than the binary's embedded corpus — assert it is refused by the FR-009b seeded baseline rather than activating at full coverage, and (i) with an operator key configured but the publisher authority declared active, a valid operator-signed bundle clearing the operator baseline — assert it is refused as "authority not active" (FR-004b) and the publisher bundle keeps serving. Finally, invoke the explicit rollback action and assert the embedded default is restored, the override is logged, the high-watermark is unchanged, and a subsequent refresh does not re-activate the deny-listed release; and assert that a rollback whose named file has been substituted (valid signature, current epoch, but a fingerprint other than the one the operator supplied) is REFUSED as a target mismatch rather than activated through the watermark bypass (FR-010). + +**Acceptance Scenarios**: + +1. **Given** a candidate bundle whose detached signature verifies against a configured trust anchor and whose sequence is greater than the active bundle's, **When** refresh runs, **Then** the candidate proceeds through the existing validation pipeline and, on success, becomes active with its sequence persisted and `signature_verified=true` surfaced in bundle status. +2. **Given** a candidate whose bytes were altered after signing (signature mismatch), **When** refresh runs, **Then** the candidate is rejected BEFORE its JSON is parsed or any pattern compiled, the active bundle is unchanged, and a "signature verification failed" reason is recorded. +3. **Given** a validly-signed candidate whose sequence is lower than its OWN signing authority's watermark — the highest sequence that authority has ever activated on this install within the current trust epoch (FR-004a/FR-005a), never a cross-authority comparison — **When** refresh runs, **Then** it is refused as a downgrade, the active bundle is unchanged, and a "downgrade refused" reason names both sequences. +4. **Given** `require_signed_bundle` is enabled and a file-drop candidate has no signature sidecar, **When** refresh runs, **Then** the candidate is refused (fail-closed) with a "signature required" reason; with the setting disabled (default) and no signature-verified bundle ever having been active on this install, an unsigned drop is accepted through the existing Spec 086 validation path but surfaced as `signature_verified=false`, does NOT advance the sequence watermark, and marks coverage degraded so `scan`-mode auto-approval is suspended while it is active. +4a. **Given** an install that has already activated a signature-verified bundle, **When** an unsigned candidate is dropped (with `require_signed_bundle` still off), **Then** it is refused by the signature ratchet — signing cannot be stripped back off an install by whoever can write the bundle path — and an unsigned candidate claiming an arbitrarily high sequence never becomes the watermark, so it cannot lock out subsequent legitimate signed releases. +5. **Given** a network-fetched candidate (Spec 087 opt-in path), **When** it arrives, **Then** signature verification is ALWAYS mandatory regardless of `require_signed_bundle` — an unsigned or wrongly-signed fetched bundle is never activated (parity with Spec 087 FR-013/FR-014). +6. **Given** an operator invokes the explicit rollback action naming the embedded default (or a prior bundle file), **When** it runs, **Then** the target is re-validated and activated even though its sequence is lower, the historical high-watermark is NOT lowered, the artifact being rolled away from is deny-listed so the next refresh cycle cannot silently re-activate the very release the operator just rejected, and the override is loudly logged and visible in bundle status. +7. **Given** an air-gapped host with no network at any point, **When** any of the above scenarios run via file drop, **Then** behavior is identical — signature verification, sequence checks, and rollback require no network. + +--- + +### User Story 2 - Seed corpus: catalog known public TPA campaigns, eval-gated (Priority: P2) + +A security-conscious developer installs mcpproxy and gets, out of the box, a signature database that actually catalogs the publicly known Tool Poisoning Attack landscape — not 6 demo signatures. Hidden-instruction blocks (the `TPA-2026-0001` `…read ~/.ssh/id_rsa…` class), hidden HTML/comment directives, tool shadowing and cross-server override instructions, exfiltration redirects ("also send the result to…"), sensitive-file coaxing, and rug-pull phrasings each have cataloged signatures with a stable `TPA-YYYY-NNNN` identity, a provenance record pointing at the public disclosure that motivated them, and a redistributable license. Every signature ships with labeled eval samples, and no corpus change — addition, tightening, or removal — can land unless the existing scan-eval CI gate stays green. + +**Why this priority**: The format (US1) is worthless empty, and the refresh channel (US3) has nothing to carry without a corpus. This is the "database" in tpa-db. It is P2 only because the P1 trust machinery must exist before a larger corpus becomes an attractive distribution target. + +**Independent Test**: Run the corpus build and assert every signature carries id, category, detector(s), provenance source, and license; assert every gated signature has ≥1 labeled malicious eval sample and ≥1 category-matched hard-negative in the eval dataset; run `go run ./cmd/scan-eval --corpus specs/065-evaluation-foundation/datasets/detect_corpus_v1.json --gate --min-recall 0.90 --max-fp 0.05` with the new embedded default and assert PASS — and, because that command today scores built-in checks only, assert the accompanying NEGATIVE CONTROL that proves the bundle is being scored at all: with the designated FR-016b control signature removed from the bundle, the gate must exit with the gate-breach code when run over the FR-016b control scope (the corpus restricted to that signature's category, or the equivalent per-category recall assertion). The control MUST NOT be expressed as "remove the signature and re-run the full-corpus command" — one miss out of a 25-signature corpus leaves aggregate recall at roughly 0.96, comfortably above the 0.90 threshold, so the full-corpus run would stay green and the control would prove nothing. Feed each campaign class's canonical fixture through the offline scanner and assert a hard-tier finding naming the TPA id. + +**Acceptance Scenarios**: + +1. **Given** the seed corpus is built into the embedded default bundle, **When** the offline scanner inspects a tool description carrying any cataloged campaign class's canonical payload (e.g. the TPA-2026-0001 hidden-instruction string), **Then** a hard-tier finding fires naming the matched `TPA-YYYY-NNNN` id, and a `scan`-mode gate holds the tool/server. +2. **Given** any signature in the corpus, **When** its metadata is inspected, **Then** it carries provenance (public source reference) and a license permitting redistribution; a contribution without a redistributable license is refused at corpus build time (Spec 087 FR-021 parity). +3. **Given** a proposed corpus change that would drop gated-category recall below 0.90 or push hard-negative false positives above 0.05 on the frozen eval dataset, **When** the CI gate runs, **Then** the change fails CI and cannot merge; the same regressing bundle, if force-published, is additionally rejected at activation by the Spec 087 self-check (two independent layers). +4. **Given** a new signature intended to gate approvals, **When** it is authored, **Then** it emits hard-tier signals (the eval gate scores hard tier only) and its eval samples follow the dataset validator's conventions, including `hn__*` naming for hard-negatives. +5. **Given** benign-but-spicy tool descriptions (security tooling that legitimately mentions credentials, docs that quote attack examples), **When** scanned with the full seed corpus, **Then** they do not fire hard-tier findings — each campaign class is paired with hard-negatives that pin this down. + +--- + +### User Story 3 - Publication channel + adoption loop (freshness you can see, refresh that re-protects) (Priority: P3) + +The publisher cuts a new database release (new signature for a fresh campaign), signs it, and publishes it to a stable public location. A connected install with the Spec 087 opt-in fetch enabled picks it up within one daily cycle; an air-gapped operator downloads the same two files from any machine and drops them in. In both cases the operator can see — in `mcpproxy security` CLI output, the REST status, and the Web UI — which database version/sequence is active, whether its signature verified, and how stale it is. After a new database activates, mcpproxy re-evaluates the already-approved toolsets it has cached against the new signatures off the hot path and surfaces any new hits as review findings — without auto-revoking approvals — so a signature published today protects servers approved last month. Anonymous telemetry gains the active bundle version/sequence and source so the funnel "published → fetched → active → detected" is finally measurable. + +**Why this priority**: This closes the loop that makes the DB worth updating. Verified adoption analysis (2026-08-22) showed TPA adoption is structurally gated: refresh as spec'd in 087 only swaps the bundle and never re-scans, so a refreshed DB would protect only future admissions. P3 because it strictly builds on US1's artifacts and 087's lifecycle, and the product is already safer with US1+US2 alone. + +**Independent Test**: Serve a signed release from a local test endpoint; enable the 087 fetch against it; assert activation within one refresh cycle and that bundle status (CLI, REST, Web UI) reports the new version, sequence, source=fetch, signature_verified=true, and generated_at. Separately: with a server already approved whose cached tool description matches ONLY a signature added in the new bundle, activate the new bundle and assert a review finding naming the TPA id appears for that tool without the approval being revoked or the server quarantined. Assert heartbeat telemetry (when enabled) carries the active bundle version/sequence/source, and carries nothing when telemetry is opted out. + +**Acceptance Scenarios**: + +1. **Given** a signed database release published at the channel's stable location, **When** an install with the opt-in fetch enabled completes its next daily cycle, **Then** the new database is verified, gated, and active, and the same artifact pair downloaded manually and file-dropped on an air-gapped install activates identically. +2. **Given** an active database, **When** the operator inspects bundle status on any surface (CLI, REST, Web UI), **Then** they see bundle version, sequence, fingerprint, source (embedded/file/fetch), signature-verified state, generated_at, and last refresh outcome incl. the last rejection reason. +3. **Given** a newly-activated database containing a signature absent from the previous one, **When** the post-activation re-scan runs over cached, already-approved tool metadata, **Then** tools matching the new signature surface as review findings naming the TPA id — approvals are NOT auto-revoked and servers are NOT auto-quarantined by refresh alone. +4. **Given** telemetry is enabled, **When** the heartbeat fires, **Then** it includes active bundle version, sequence, and source alongside the existing anonymous TPA-scanner stats; **Given** telemetry is disabled, **Then** nothing bundle-related is sent (existing opt-out respected). + +--- + +### Edge Cases + +- **Air-gapped host, forever**: every P1 behavior (verification, sequence check, rollback, activation self-check) is network-free; the trust anchors ship inside the binary; staleness is surfaced but never blocks scanning — an old database is degraded coverage, not an outage. +- **Threat-model boundary — attacker with config write**: FR-004 lets operator-configured keys join the trust set, so anyone who can write `mcp_config.json` can trust their own key and sign anything. This is explicitly OUT of the threat model and is not a regression: an attacker with config write can already disable quarantine outright or set a server's trust mode to `auto`. The attacker this design defends against is the one who can write the BUNDLE PATH or intercept the FETCH but not the config — which is precisely why the signature ratchet (FR-009) is keyed to install state rather than to the bundle path, why the active-authority declaration (FR-004b) and the operator baseline (FR-009b) are config-resident, and why `require_signed_bundle` is config-resident (none of them must be defeatable from the data dir alone). The boundary MUST be read conservatively: because the bundle path may be configured to live inside the data directory, the design MUST NOT assume the bundle-path attacker cannot also delete or reset the data-directory persisted security state (watermark record, ratchet flag, pin/deny-list) — the worst such an attacker can achieve is exactly the "anti-downgrade state lost" wipe below, against which the guarantees that survive are the ones anchored outside the data dir: the FR-009b publisher baseline (re-seeded from the binary at every startup), the config-resident operator baseline, `require_signed_bundle`, and the FR-004b active-authority declaration. The ratchet and the pin/deny-list are data-dir resident and are therefore explicitly best-effort hardening against this attacker, never the sole defense. What the design DOES assume is that the attacker cannot substitute or corrupt the persisted state with CHOSEN values (e.g. rewrite the watermark to a lower number without triggering the seeded-baseline floor): the state store is the same local BBolt database (`config.db`) that already holds approvals and quarantine state. That database is transactionally consistent and checksummed against corruption, NOT cryptographically authenticated: nothing in it resists an attacker who can substitute the file with a prepared one. The assumption is therefore a TRUST BOUNDARY, not a cryptographic property — an attacker who can write chosen records into `config.db` is already past every guarantee mcpproxy makes (approvals, tool-quarantine baselines, OAuth state), not just this feature's. The design MUST NOT be read as claiming the ratchet, watermark, pin or deny-list survive that attacker; the guarantees that do survive are only the ones anchored outside the data directory, enumerated above. Conformance fixtures MUST therefore cover state SUBSTITUTION (a prepared database carrying a chosen watermark, cleared ratchet, or forged pin/deny-list entry) and assert the outside-the-data-dir floors still hold — deletion and reset fixtures alone do not exercise this boundary. +- **Signature key rotation**: the trust anchor is a SET of publisher keys embedded per binary release. Rotation is a binary-release event: a release ships old+new keys, bundles are signed with the new key, the old key is removed in a later release. Keys are NEVER delivered over the network, and rotation is not done casually — every shipped binary pins its key set, so a hasty rotation strands older installs on unsigned-refresh only (same philosophy as the app-update signing keys). Key compromise → remove the key in the next release, **bump the trust epoch (FR-005a)**, and publish a bundle in the new epoch signed by the surviving key. The epoch bump is what makes recovery possible at all: a compromised key can sign at the maximum representable sequence, and against a monotonic watermark no surviving key could ever outbid that within the same epoch, so revocation alone would leave installs permanently unable to accept any future release. +- **Bundle signed by a formerly-trusted, now-removed key**: verification fails on binaries that dropped the key (fail-closed to last-known-good); older binaries still trusting it keep working — the downgrade watermark still blocks sequence replay. +- **Tampered bundle / tampered sidecar / swapped pair** (valid signature belonging to a different bundle's bytes): all reduce to signature-verification failure before parse; never a crash, never partial load. +- **Downgrade attack via the embedded default**: a fresh binary whose embedded bundle sequence is LOWER than a previously-activated external bundle's persisted watermark does not silently downgrade — the external candidate re-validates and wins by precedence; if only the embedded bundle is available, it serves (scanning must never stop) with the watermark retained and the situation surfaced, but per FR-009a as DEGRADED coverage, so an attacker who merely deletes the external bundle cannot use the weaker embedded corpus to obtain `scan`-mode auto-approvals. This is one of SC-002's exactly two exemptions (the other is FR-009b's baseline-less operator authority). +- **Anti-downgrade state lost** (data directory wiped/reset): the persisted watermark resets to the FR-009b seeded baseline — NOT to zero, so a wiped install still refuses any publisher-signed bundle older than the corpus its own binary ships. For the operator authority the same protection holds only if the operator declared a config-resident baseline; without one, operator-authority anti-downgrade does not survive the wipe and the install runs degraded while an operator-signed bundle is active, by FR-009b, rather than pretending to a guarantee it does not have. What a wipe does lose is the FR-009 signature-ratchet state and any FR-010 pin/deny-list (both data-dir resident), so a wiped install accepts an unsigned drop again. This is accepted — the ratchet is best-effort hardening on top of signing, not the sole defense, and a wiped data dir already resets approvals. Operators who need this closed absolutely set `require_signed_bundle`, which is config-resident rather than data-dir-resident and therefore survives the wipe. +- **Equal sequence, different bytes** (two distinct bundles claiming the same sequence): refused as suspicious unless it is the currently-active fingerprint (idempotent re-drop stays a no-op per Spec 087 FR-011). +- **Clock skew**: no validity windows or expiry in v1 — sequence ordering, not wall-clock time, is the trust signal; `generated_at` is advisory freshness only. +- **Huge or adversarial database** (thousands of rules, pathological regex) **or an adversarial sidecar**: the sidecar byte cap (read before it is parsed, since verification's own inputs come from it), the regular-file requirement and read deadline that stop a FIFO or stalled stream from pinning the refresh path, plus the bundle size and rule-count ceilings, are enforced at load; every pattern remains RE2 (linear-time, no catastrophic backtracking by construction); a candidate exceeding ceilings is rejected whole, keeping last-known-good. +- **Post-activation re-scan storms**: the re-scan is off the hot path, rate-limited, runs once per activation over cached metadata only (no upstream reconnects), and produces at most one review finding per tool per bundle activation — no notification storm. +- **Signature removed from a newer bundle** (false-positive retired): tools previously held by it are NOT auto-approved by refresh; the held state persists for human review (state changes only flow toward review, never silently toward approval). + +## Requirements *(mandatory)* + +### Functional Requirements + +**Database format & signing** + +- **FR-001**: The publishable database artifact MUST be the existing compiled bundle (`scanner-bundle.json`, Scanner Bundle Contract) plus a detached signature sidecar over the exact bundle bytes. Bundle bytes MUST be deterministic for a given **(corpus, manifest metadata)** pair — rebuilding the same corpus with the same manifest inputs MUST be byte-identical, so the signature and the fingerprint are stable. Determinism is deliberately NOT claimed over the corpus alone: FR-002 binds release-identifying metadata (`generated_at`, `sequence`, key id, trust epoch) into the signed bytes, so two releases of an unchanged corpus legitimately differ in bytes, fingerprint and signature. The reproducibility check MUST therefore pin the manifest inputs (a fixed `SOURCE_DATE_EPOCH` and the release's sequence/key id/epoch) and assert byte-equality under those pinned inputs; asserting byte-equality across two different releases of the same corpus would fail by construction and MUST NOT be a gate. +- **FR-001a**: The bundle contract MUST gain a defined home for the per-signature metadata FR-012 requires. Today's compiled bundle carries `rules[]` entries keyed `category, confidence, detector, engine, flags, id, indicators, level, pattern, severity, target, type` and a `skipped[]` list — there is no provenance field and no license field anywhere in the artifact, so FR-001 ("the existing compiled bundle") and FR-012 ("each signature MUST carry provenance … and a redistributable license") cannot both hold as written. Design MUST resolve this by adding a `signatures[]` section keyed by the stable `TPA-YYYY-NNNN` id, carrying provenance, license, gating intent, and the ids of the `rules[]` entries that implement it — additive, so a v0.1 loader ignores it (contract §4 forward-compat). The relationship MUST be explicitly many-to-one (several rules may implement one TPA id) and the build MUST enforce referential integrity in both directions: every `signatures[]` entry references at least one existing rule id, every gating rule is referenced by exactly one signature entry, and TPA ids are unique. A bundle failing either check MUST be rejected whole, like any other contract violation. +- **FR-002**: The bundle manifest MUST gain additive metadata within the supported schema line: `generated_at` (RFC3339 build stamp — the loader already surfaces it), `sequence` (a strictly-increasing publish counter, canonically a `uint64` serialized as a JSON **decimal string** — never a bare JSON number, whose double-precision round-trip in generic consumers silently corrupts values above 2^53-1 and admits exponent notation; the loader MUST reject a value that is not a bare decimal string of 1–20 digits with no sign, leading zeros, whitespace or exponent, and MUST reject anything above 2^64-1, which also fixes the "MAXIMUM representable sequence" SC-002a exercises), a publisher key identifier, and the **trust epoch** the artifact was published under (FR-005a). Because these keys live inside the signed bytes, the epoch is cryptographically bound to the bundle and cannot be reinterpreted after the fact. These keys MUST be additive so existing v0.1 loaders ignore them (contract §4 forward-compat preserved). `generated_at` MUST be derived from deterministic release metadata (the release commit's timestamp or `SOURCE_DATE_EPOCH`), NEVER from build wall-clock time — a wall-clock stamp would change the bundle bytes, fingerprint, and signature on every rebuild of the SAME release, which is exactly the byte-equality FR-001 pins its reproducibility check to. (Two different releases of an unchanged corpus differing in bytes is expected and permitted; see FR-001.) +- **FR-003**: The signature scheme MUST be Ed25519 detached signatures (or an equivalent modern EdDSA scheme selected in design); the sidecar MUST carry the algorithm identifier and signing key id so the scheme can evolve without breaking verification of existing artifacts. **The sidecar wire format MUST be specified in design, not left to the implementation**, because it is the one artifact an independent publisher must be able to produce and every loader must reject identically: design MUST fix its own schema version key, the exact encodings of the algorithm identifier, key id and signature value, the complete set of required fields, and the deterministic rejection behaviour for a missing field, an unknown or duplicate field, a duplicate key, an unknown algorithm, and trailing bytes. v1 MUST verify Ed25519 and nothing else — an unknown algorithm identifier is a hard rejection, never a fallback or a skip — and the spec MUST ship byte-level valid and invalid sidecar fixtures so two implementations can be shown to agree on both acceptance and refusal. A key id MUST be a canonical full-length fingerprint of the public key itself (not an operator-chosen label), the trust set MUST reject duplicate ids at build/config time, and verification MUST succeed only when the manifest key id, the sidecar key id, and the id of the trust-set key whose signature actually verified are all identical — a manifest naming one publisher while the bytes are signed by another MUST be a verification failure, not a cosmetic mismatch. +- **FR-004**: The set of trusted verification keys MUST be the union of (a) publisher public keys embedded in the binary at build time and (b) operator-configured keys (the Spec 087 FR-013 key). Keys MUST NOT be retrieved over the network, and verification MUST work fully offline. These are two distinct **signing authorities**, and the design MUST keep them so: all publisher keys (including keys being rotated in or out, FR-005) belong to ONE publisher authority, while operator-configured keys form a separate operator authority. +- **FR-004a**: Anti-downgrade state MUST be tracked per signing authority, not in one global namespace. An operator maintaining a self-built corpus is an explicitly supported persona (air-gapped installs, private signature sets), and with a single shared counter their locally-signed bundle at sequence N would raise the one global watermark above the publisher's release line and permanently block official updates — or, symmetrically, a publisher release would block the operator's own corpus. Each authority MUST therefore carry its own watermark, and a candidate MUST be compared only against the watermark of the authority whose key verified it. Choosing an operator-signed corpus over a higher-sequence publisher one is a deliberate operator decision (declared per FR-004b) and MUST NOT by itself be treated as a downgrade or as degraded coverage — the operator configured that key precisely to assert their own corpus — but the active authority MUST be visible in bundle status (FR-020) so the choice is never invisible. Grouping all publisher keys into one authority is what keeps per-authority watermarks compatible with key rotation: a rotated-in key inherits the publisher authority's watermark rather than starting a fresh one. +- **FR-004b**: The **active signing authority MUST be a config-resident declaration**, defaulting to the publisher authority, and MUST NOT be inferred solely from whichever key happens to verify a candidate. A candidate whose verifying key belongs to a non-active authority MUST be refused with an "authority not active" reason, exactly as any other failed candidate (last-known-good keeps serving). Without this, per-authority watermarks (FR-004a) open the very hole they were introduced to avoid: because each authority is compared only against its own counter, an attacker who can write the bundle path — squarely inside the threat model, unlike the config-write attacker — can replace a current publisher bundle at sequence 100 with ANY operator-signed artifact that merely clears the operator authority's FR-009b baseline, silently swapping the install onto an older or deliberately narrower corpus while it still reports `signature_verified=true` at FULL coverage. The publisher watermark cannot detect this, since the comparison never crosses namespaces, and no operator decision is involved — whoever writes the path picks the authority. Config-residency is what puts that choice out of the attacker's reach, matching `require_signed_bundle` (FR-007) and the operator baseline (FR-009b), which are config-resident for the same reason. Both key sets MAY remain configured simultaneously; only artifacts of the declared active authority activate, and switching authorities is a config edit, not a file drop. **The declaration governs the SERVING bundle, not only future candidates**: when the declaration changes, an active external bundle belonging to the now-inactive authority MUST NOT keep serving as full coverage — it is deactivated (loudly logged, surfaced per FR-020) and the install falls back to the embedded default under FR-009a's rules until a bundle of the newly-active authority activates. Without this the switch is advisory only: fail-safe keep-last-known-good would leave the old authority's corpus serving indefinitely, and an attacker who keeps the new authority's candidates invalid (bundle-path writes are in scope) could perpetuate that state at full coverage — the operator's explicit change of trust would simply never take effect. Merely declaring the operator authority active does not by itself degrade coverage: until an operator bundle activates, the embedded default serves exactly as on a fresh install (FR-009a condition (b) concerns a previously-activated operator bundle that was lost, not one that never existed). +- **FR-005**: Key rotation MUST be supported by trusting multiple keys simultaneously: a binary release MAY add a new key while retaining the old for at least one release cycle, and key removal is also a binary-release event. The design MUST document the compromise procedure and the "do not rotate casually" operational stance. +- **FR-005a**: Each signing authority's trust set MUST carry a **trust epoch** — a monotonic per-authority integer that advances only through that authority's own trust root: the PUBLISHER epoch advances ONLY via a binary release, while the OPERATOR epoch is config-resident, declared alongside the operator keys and baseline, and advances only by a config edit. The split is forced by the same recovery argument that motivates the epoch itself: a compromised OPERATOR key can sign the maximum representable sequence exactly as a publisher key can, and no binary release can coordinate an epoch bump for a private operator authority the publisher has never heard of — a binary-only rule would make operator-key compromise an unrecoverable permanent lockout of the operator authority (replacing the key in config does not help, since the new key inherits the same-epoch watermark). A config-resident operator epoch is sound for the same reason the operator baseline is (FR-009b): the operator authority's entire trust root — its keys, baseline, and the active-authority declaration — is already config-resident, so the epoch adds no new attack surface for the config-write attacker, who is out of the threat model. Bumping the operator epoch in config behaves exactly as a publisher binary bump: it re-baselines the operator watermark from the config-declared baseline (or marks the authority baseline-less-degraded per FR-009b) and retires prior-epoch operator pins and deny-list entries (FR-010). Every anti-downgrade comparison (FR-009) MUST be scoped `(authority, epoch, sequence)`, ordered by epoch first. The epoch MUST be carried inside the SIGNED manifest (FR-002) and a candidate MUST be accepted only when its manifest epoch EQUALS the current binary-trusted epoch of the authority whose key verified it — never merely less-than-or-equal. Binding the epoch to the signed bytes and requiring exact equality is what stops the epoch bump from becoming a replay window of its own: if the epoch lived only in the binary, then after a bump-and-revoke the re-baselined watermark would start from zero, and an OLD, weaker bundle legitimately signed by a still-trusted surviving key would be silently reinterpreted as belonging to the new epoch, clear the fresh watermark, and activate with `signature_verified=true` and full coverage — reinstating exactly the blind spots the recovery release was published to close. Without this, key-compromise recovery is impossible rather than merely painful: an attacker holding a compromised publisher key signs a bundle at the maximum representable sequence, and because the watermark is monotonic and no surviving key can ever exceed that sequence, the install is permanently unable to accept any future legitimate release — a one-shot permanent denial of updates, downgraded only to the FR-009a fallback. A binary release that revokes the compromised key MUST therefore also bump the epoch, and a validly-signed bundle whose manifest epoch matches a newly-bumped binary epoch MUST supersede the retained watermark and re-baseline it; the bump likewise retires the authority's prior-epoch FR-010 pins and deny-list entries, so stale rollback state cannot block the recovery bundle (FR-010). A bundle whose manifest epoch is BELOW the current binary-trusted epoch MUST be refused as stale-epoch replay even when its signature verifies against a still-trusted key. Epoch advancement is safe to treat as authoritative because it arrives only through the authority's own trust root — a release binary for the publisher epoch (the binary's integrity is already the root of trust for the embedded key set, FR-004) and `mcp_config.json` for the operator epoch (already the root of trust for the operator keys themselves) — never over the bundle or fetch path, which is the attacker-reachable surface. The accepted consequence of exact-equality matching is that a binary still on the OLD epoch will refuse the new epoch's bundles and stop receiving database updates until it is itself updated. That is deliberate, not incidental: an epoch bump means a key was compromised, and an un-updated binary still trusts that compromised key, so freezing its database is safer than feeding it artifacts whose trust basis it cannot evaluate. It also matches the existing "a hasty rotation strands older installs" stance, and it MUST be surfaced as a stale-epoch degraded state (FR-020) rather than failing silently. + +**Loader & activation** + +- **FR-006**: For any candidate carrying a signature, verification MUST run over the exact raw candidate bytes BEFORE those bytes are parsed as JSON or any pattern is compiled; a candidate failing verification MUST be rejected without further processing, keeping last-known-good (extends the Spec 086 fail-closed pipeline with a new first stage). The bytes that are verified MUST be the same bytes that are subsequently parsed (read once into memory, verify, then parse that buffer) — the file MUST NOT be re-read from disk after verification, or a writer racing the loader could swap content between verify and parse. +- **FR-007**: Signing policy MUST be: network-fetched candidates ALWAYS require a valid signature; file-drop candidates are verified whenever a sidecar is present; a new `require_signed_bundle` setting (default off for back-compat with existing unsigned drops) makes the sidecar mandatory for file drops too; the embedded default needs no sidecar (its integrity rides on binary distribution integrity). Bundle status MUST always report whether the active bundle was signature-verified. While an unsigned EXTERNAL (file-dropped or fetched) bundle is active, the scan coverage the approval gates consume MUST be marked degraded so `scan`-mode auto-approval is suspended and changes route to human review — today `internal/security/scanner/inprocess.go` sets `coverageOK` from mere bundle *presence*, so without this an attacker-supplied unsigned corpus would keep silently auto-approving under weakened rules. This degradation rule applies to EXTERNAL candidates ONLY: the embedded default is binary-trusted (its integrity rides on the signed binary distribution, not on a sidecar) and MUST be full-coverage on an ordinary install, or every fresh install would ship with `scan` mode silently disabled. The embedded default is degraded only in the FR-009a cases — when it is serving as a fallback BELOW the retained publisher watermark, or when the active operator authority's previously-activated bundle has gone missing, i.e. when the install has evidence that a corpus it should be running is newer than, or not covered by, the embedded one. This requirement supersedes Spec 087 FR-014's precondition: because FR-004 makes publisher keys embedded in the binary part of the trust set, a fetch is no longer refused merely because the operator configured no key of their own; the "never activate an unverified fetched bundle" guarantee of Spec 087 FR-013/FR-014 is preserved unchanged. +- **FR-008**: The full validate-before-activate order MUST be: sidecar byte cap → parse sidecar → bundle byte-size ceiling → signature over the raw bytes (per policy, FR-006) → parse manifest → rule-count ceiling → epoch match (FR-005a) → sequence/fingerprint anti-downgrade (FR-009) → version compatibility → compile-all-or-reject → runnable-rules > 0 → activation self-check (Spec 087 FR-008) → atomic activation. FR-011's ceilings MUST therefore be split across the pipeline rather than applied as one stage: the sidecar cap comes first because the sidecar is read before any signature exists to check, bundle byte size is measurable on raw bytes and so guards the verification step itself, while rule count is only knowable after parsing and so cannot precede it. The same reasoning applies to sequence, a manifest field necessarily read after parsing: what MUST precede parsing is signature verification, not the sequence or rule-count check — an unverified candidate's manifest is never trusted for any decision. Any failure at any stage MUST keep last-known-good and record a machine-readable reason and timestamp (surfaced via the existing bundle status), never an empty or partial rule set. +- **FR-009**: The system MUST persist the highest sequence ever activated — the **watermark**, scoped per signing authority and trust epoch per FR-004a/FR-005a — and refuse any candidate with a lower sequence, or an equal sequence with a different content fingerprint, as a downgrade/replay, unless an explicit operator rollback (FR-010) is in effect. **The watermark is a property of SIGNED artifacts only**: `sequence` lives in the manifest, so an unsigned candidate's sequence is attacker-chosen. An unsigned/unverified candidate MUST NOT advance the watermark (otherwise a single unsigned drop claiming a huge sequence permanently locks out every legitimate signed release), and once an install has ever activated a signature-verified bundle, unsigned candidates MUST be refused thereafter regardless of `require_signed_bundle` — a signature ratchet, so signing cannot be stripped back off an install by whoever can write the bundle path. Both "ever" clauses — the highest-ever watermark and the once-signed-always-signed ratchet — are claims about RETAINED data-directory state, not absolutes that survive its destruction: after a data-directory reset only the FR-009b seeded and config-resident baselines still hold, the ratchet re-arms on the next signed activation, and the accepted post-reset guarantees are exactly those of the "Anti-downgrade state lost" edge case — fixtures for FR-009 and SC-002 MUST test against those, not against a stronger guarantee the threat model disclaims. The embedded default is exempt from BOTH the watermark and the ratchet — it is the availability fallback of last resort and must never be locked out (scanning must never stop), which is exactly why FR-009a demotes it to degraded coverage instead of refusing it. +- **FR-009b**: Each authority's watermark MUST be SEEDED from a trusted baseline rather than starting empty, because an absent watermark accepts anything — with exactly one exception, carved out at the end of this requirement: an operator authority configured without a baseline, which has no seed available and is therefore run as explicitly degraded rather than as a silent zero. Otherwise the anti-downgrade guarantee simply does not exist at first use or after the data directory is reset: an attacker who can write the bundle path drops an OLD but genuinely publisher-signed, current-epoch bundle — say sequence 5 while the shipping release is at 100 — and with no watermark to compare against it activates as a fully signature-verified bundle at FULL coverage, silently reinstating every blind spot fixed between 5 and 100. Neither the FR-009 ratchet nor the FR-010 pin/deny-list helps here; both only bind once persisted state exists. Therefore: the **publisher** authority's watermark MUST be seeded from the sequence of the bundle embedded in the running binary — a candidate older than what the binary already ships is strictly worse than the fallback and MUST be refused — and this baseline MUST be re-applied on every startup, so wiping the data directory cannot lower it below the binary's own corpus. The **operator** authority has no binary-trusted baseline available, so its baseline MUST be declared explicitly in configuration alongside the operator key, and it MUST live in `mcp_config.json` rather than the data directory — that is what makes it durable across the wipe, since a baseline stored in resettable state provides no protection precisely when it is needed. A silent zero default MUST NOT be used: with a zero baseline the operator authority has NO anti-downgrade after state loss at all, and a bundle-path attacker — who is squarely inside the threat model, unlike the config-write attacker — could replay any older genuinely operator-signed bundle at full coverage on a fresh or wiped install. Therefore: if an operator key is configured WITHOUT a baseline, the system MUST treat operator-authority anti-downgrade as unsupported and mark coverage degraded (per FR-007) while an operator-signed bundle is active, rather than silently presenting it as fully trusted. The data-directory watermark then tracks activations above the configured baseline and may only raise the effective minimum, never lower it. The seeded baseline also makes the epoch re-baseline of FR-005a safe: after an epoch bump the new epoch's watermark starts at the authority's baseline, not at zero. +- **FR-009a**: When no valid external candidate is available, the embedded default MUST still serve so scanning never stops, even if its sequence is below the watermark. Where the install has NO retained signed watermark and no missing previously-active external bundle (the ordinary fresh install), this is simply the normal full-coverage state. Two — and only two — conditions demote that fallback to degraded coverage (auto-approval suspended, as in FR-007's unsigned-external case): (a) the embedded default's sequence is below the **publisher** authority's retained watermark, or (b) the **operator** authority is the active one (FR-004b) and its previously-activated external bundle is no longer available or no longer valid. The comparison in (a) MUST be same-namespace: the embedded default is a publisher artifact, and sequences of different authorities are independent counters, so it MUST be compared ONLY against the publisher watermark and never against an operator watermark — a cross-authority numeric comparison is meaningless and would, for an operator corpus at sequence 900 against an embedded publisher corpus at 100, permanently suspend auto-approval on an install whose publisher fallback is perfectly current. Condition (b) is what a cross-authority comparison was reaching for, restricted to the case where the fallback cannot be shown to be at least as fresh as what was lost: an operator corpus is not comparable to the publisher's, so a missing one leaves the install with no evidence the embedded default covers it. It MUST NOT extend to a missing PUBLISHER external bundle, because there condition (a) already decides the question on the same axis — an embedded default at or above the publisher watermark is demonstrably no older than the bundle that went missing (the ordinary "external file removed after a binary upgrade shipped a newer corpus" case) and MUST stay full coverage rather than being degraded merely because an older external file once existed. In either case the retained watermark, the active authority, and the degraded reason MUST be surfaced, and it MUST NOT count as an anti-downgrade violation. This fallback is one of SC-002's two explicit exemptions; the other is FR-009b's baseline-less operator authority. +- **FR-010**: An explicit rollback action MUST let an operator revert to the embedded default or a named bundle file: the target is re-validated (signature policy and self-check still apply), activated despite a lower sequence, and the override is loudly logged and visible in bundle status. **The rollback target MUST be identified by content fingerprint, not by pathname alone.** Rollback is the one path that deliberately bypasses the watermark, so identifying its target by an attacker-writable path hands the adversary the choice of what the bypass lands on: with a legitimate bundle at sequence 100 active and the operator intending to revert to a known-good sequence 90, an attacker who can write the bundle path — in scope, unlike the config-write attacker — swaps the named file for a genuinely publisher-signed, current-epoch bundle at sequence 5 before the rollback reads it. It passes signature, epoch, compilation, and plausibly the self-check; the watermark that would normally refuse it is switched off by the rollback itself; and the result is then PINNED, so the downgrade persists across refresh cycles. FR-006's read-once rule closes the verify/parse race but not substitution before the read. Therefore the operator MUST supply the expected fingerprint, the path is only a hint for where to find those bytes, and a loaded artifact whose fingerprint differs MUST be refused as a rollback-target mismatch rather than activated. **The rollback fingerprint MUST be the complete SHA-256 digest of the bundle bytes, compared byte-for-byte.** The abbreviated fingerprint the loader surfaces today is the first 12 hex characters of that digest (`internal/security/scanner/tpa_bundle_source.go` — `hex.EncodeToString(sum[:])[:12]`), i.e. 48 bits: adequate as a human-comparable identity label, but NOT as the authorization for the one path that deliberately bypasses the downgrade watermark, where the attacker chooses the substituted bytes and need only match a 48-bit prefix. Bundle status MUST therefore surface the full digest alongside the abbreviation, and the rollback interface MUST accept only the full digest. **Bundle status history is a new requirement, not an existing surface**: no activation history exists today (`BundleInfo` reports only the currently active artifact), so this spec MUST add a durable, append-only activation record — (authority, trust epoch, sequence, full SHA-256 digest, generated_at, activated_at, source) per activation, retained across restarts — because without it an operator has no trustworthy way to learn the digest of the known-good artifact they intend to revert to. That record is data-dir resident and therefore carries the same best-effort caveat as the ratchet: it makes a legitimate rollback possible, it does not by itself authorize one. A rollback target MUST additionally still be at or above **its own authority's** FR-009b baseline — the publisher baseline for a publisher-signed target, the config-declared operator baseline for an operator-signed one. The comparison MUST NOT be made against the publisher baseline in both cases: sequences of different authorities are independent counters (FR-004a), so measuring an operator target at sequence 9 against a publisher embedded corpus at 100 would refuse every legitimate operator rollback. Where an authority has NO declared baseline — possible only for the operator authority, which FR-009b explicitly permits to run without one — there is no floor to enforce and none is invented: the rollback proceeds and the install is marked degraded for exactly as long as that operator-signed bundle is active, which is already FR-009b's treatment of a baseline-less operator authority. Neither refusing the rollback (which would break FR-010's guarantee for a configuration FR-009b allows) nor assuming a zero floor (which would claim an anti-downgrade guarantee that does not exist) is acceptable. The embedded default is the one target always permitted, since it IS the publisher baseline and is the availability fallback of last resort. Rollback MUST NOT lower the historical high-watermark: the watermark is monotonic and normal activation MUST still require exceeding it. Rollback is instead expressed as a **pinned exception** — the operator pins the rolled-back artifact by fingerprint, and the (sequence, fingerprint) being rolled away from is deny-listed so the very release the operator just rejected cannot be re-activated by the next refresh cycle (and neither can any intermediate release between the pin and the watermark). Clearing the pin returns the install to normal watermark-governed activation. **Pin and deny-list state MUST be scoped like the watermark it excepts**: each entry is keyed by (authority, trust epoch, sequence, fingerprint) and binds only while its authority is the active one (FR-004b) and its epoch equals the binary's current epoch for that authority. Switching the active authority SUSPENDS (does not delete) the non-active authority's pin and deny-list, so switching back restores them unchanged. A binary release that bumps an authority's trust epoch (FR-005a) RETIRES that authority's prior-epoch pins and deny-list entries, loudly logged: the epoch bump is the key-compromise recovery path and MUST NOT be blockable by stale pin state — a retained old-epoch pin could not re-activate its artifact anyway (the pinned manifest's epoch no longer matches, FR-005a), so honoring it could only strand the install on the embedded fallback while blocking the very recovery bundle the bump exists to deliver. +- **FR-011**: The loader MUST enforce load ceilings and reject a candidate exceeding them as a whole; all patterns remain RE2-only. **The ceilings are numbers in this spec, not a design deferral**, because every one of them is a rejection boundary a fixture has to drive one unit past — a ceiling described only as "fixed in design" cannot be tested, and today none exists at all: `internal/security/scanner/tpa_bundle_source.go` reads the candidate with a bare `os.ReadFile(path)` (no size check, no type check), and `loadBundleCheck` (`tpa_bundle.go`) applies no rule-count limit. The values are: **bundle ≤ 8 MiB**, **rules ≤ 2000** (SC-008 exercises ≥ 500, so this leaves headroom for four more corpus generations without a spec change), **sidecar ≤ 4 KiB**, and a **read budget of 5 s per candidate** (local file or fetch body). A candidate at exactly the ceiling MUST load; one byte or one rule over MUST be rejected whole, with the reason recorded in bundle status. Design MAY tighten these but MUST NOT loosen them without amending this FR. **The sidecar needs its own, much smaller ceiling and a bounded read**, because it is the one artifact that must be read and parsed BEFORE any signature has been verified (its algorithm identifier, key id, and signature value are the inputs to verification), which puts it outside the protection FR-006 gives the bundle. Without it the in-scope bundle-path attacker simply writes a multi-gigabyte or never-terminating sidecar next to an otherwise acceptable bundle and exhausts memory or stalls the refresh before anything can be rejected — a denial of the scanner's own update path. The sidecar MUST therefore be read through a hard byte cap sized to its fixed small structure and rejected on overrun, before it is parsed, on both the file-drop and fetch paths. A byte cap alone is not sufficient, because it bounds bytes consumed but not time spent: the same attacker replaces the sidecar (or bundle) path with a FIFO and never writes to it, so the read blocks forever without ever approaching the cap and Spec 087's single-flight refresh stays occupied, blocking every legitimate database update while the loader waits. Both local candidate artifacts MUST therefore be REGULAR files — a FIFO, device, socket, or a symlink resolving to one is refused outright, not read. **This check MUST be made on the OPEN DESCRIPTOR, never on the pathname.** A `stat`-then-open sequence is itself a substitution window: the in-scope bundle-path attacker replaces the path between the two calls, the stat sees a regular file and the open lands on their FIFO. The loader MUST open the candidate with the platform's no-follow protection (`O_NOFOLLOW`; on Go, `os.OpenFile` with `syscall.O_NOFOLLOW`, or `os.Root`-scoped opens where the containing directory is itself trusted), `fstat` the resulting descriptor, refuse anything whose mode is not a regular file, and then read the size ceiling from that same descriptor — never re-resolving the path. **The time bound MUST NOT be specified as an I/O deadline on the file itself**: Go's `SetReadDeadline` is not supported for regular files and returns `ErrNoDeadline`, so a requirement phrased that way is unimplementable as written and would be silently dropped. The requirement is instead that every candidate read complete within a bounded wall-clock budget enforced OUTSIDE the read — a context-cancelled reader goroutine for local files (whose expiry releases the single-flight slot and records a rejection reason even if the blocked read itself cannot be interrupted), and `http.Client`/`context` deadlines plus a capped `io.LimitReader` for fetch response bodies. Design MUST name the concrete mechanism and its per-platform limits, including the case where a stalled network filesystem leaves the abandoned read goroutine parked: the slot is released and the refresh path stays available, which is the property being guaranteed. A database at the ceilings MUST NOT measurably regress the scan hot path (see SC-008). +- **FR-011a**: Activation MUST be crash-consistent, not merely atomic in memory. Spec 087 FR-010 makes the rule-set swap atomic with respect to concurrent scans; that says nothing about persisted state, and a process death between "activated" and "watermark written" reopens exactly the replay window FR-009 exists to close (while writing the watermark first can leave the install with only an older recoverable bundle). The activation record — active fingerprint, sequence, per-authority watermarks and their epochs, the fingerprint each watermark was set by, signature-verified state, any FR-010 pin, and the FR-010 deny-list — MUST be persisted as ONE transaction, and the recoverable last-known-good bundle MUST be durable before the in-memory pointer is published. The deny-list and watermark fingerprint are load-bearing members of that transaction, not bookkeeping: a crash that preserved a rollback while losing the deny-list would let the next refresh immediately re-activate the exact release the operator had just rejected, silently undoing the rollback. Startup, refresh, hot-reload, and rollback MUST all serialize through that same transaction, so any crash leaves the install on a consistent (bundle, watermark) pair. + +**Seed corpus** + +- **FR-012**: The seed corpus MUST catalog known public TPA campaign/technique classes as signatures with stable `TPA-YYYY-NNNN` identities, covering at minimum: hidden-instruction blocks (the TPA-2026-0001 class), hidden HTML/comment directives, tool shadowing and cross-server override, exfiltration redirects, sensitive-file coaxing (SSH keys, cloud credentials), and rug-pull phrasing patterns. Each signature MUST carry category, detector(s), level, confidence, provenance (public source reference), and a redistributable license; corpus build MUST refuse contributions lacking either (Spec 087 FR-021 parity). +- **FR-013**: Every signature intended to gate approvals MUST ship with eval evidence in the frozen eval dataset: at least one labeled gated-malicious sample and at least one category-matched hard-negative following the dataset validator's conventions (including `hn__*` naming), so the gate can measure both its recall contribution and its false-positive risk. +- **FR-014**: Gating signatures MUST emit hard-tier signals (the eval gate and the `scan`-mode approval gates score hard tier only); any soft-tier-only signature MUST be explicitly marked non-gating in its metadata and MUST NOT be counted toward gate coverage. +- **FR-015**: Signatures MUST be authored in the corpus source-of-truth pipeline (signature sources compiled to the bundle); mcpproxy MUST continue to consume only the compiled bundle and never parse signature sources (Spec 087 FR-002 parity). The contribution pipeline MUST run the same validation the loader runs (compile, schema, license, eval gate) before a bundle can be signed. +- **FR-016**: The CI eval gate (`cmd/scan-eval --gate`, recall ≥ 0.90 over gated categories, hard-negative FP rate ≤ 0.05, existing vacuity guard) MUST pass for: every change to the embedded default corpus (merge-blocking, existing `eval.yml` D2 job) and every published bundle BEFORE it is signed — a bundle that fails the gate is never signed or published. Thresholds are reused verbatim, not redefined. +- **FR-016a**: The gate harness MUST first be taught to see the database at all — today it does not, so FR-016 and SC-005 would otherwise pass vacuously. `cmd/scan-eval/gate.go`'s `gateChecks()` returns only the four built-in `detect.Check`s (`unicode.hidden`, `shadowing.cross_server`, `payload.decoded`, `phrase.injection`); `scanner.BundleCheck` — which production DOES append in `internal/security/scanner/inprocess.go` — is absent, and `scan-eval` never loads a bundle. Therefore: (a) `scan-eval` MUST accept a candidate bundle path, load it through the production loader (never a parallel parser), and register the resulting bundle check, defaulting to the embedded default so the existing `eval.yml` D2 invocation starts scoring the shipped corpus; (b) `gate.go`'s `categoryCheck` map MUST gain an entry for every new TPA campaign category, since `gatedCategory()` enforces a category ONLY when its mapped check id is registered — an unmapped category's malicious samples are tallied and reported but excluded from `gatedMalicious`/`OverallRecall`, so its MISSES can never fail the gate (its false positives still count, because `FPRate` is computed over the whole hard-negative set regardless of category mapping). A category whose recall is unenforceable is precisely the vacuous-pass failure mode; (c) the same candidate-loading path MUST be what the Spec 087 FR-008 activation self-check runs, so the CI layer and the activation layer cannot drift apart while SC-005 claims they are independent. +- **FR-016b**: The FR-016a/SC-005 negative control MUST be constructed to fail DETERMINISTICALLY, not left as "remove any signature and hope the gate turns red". As written it can pass vacuously twice over: `OverallRecall` is an aggregate `gatedDetected/gatedMalicious` over ALL gated categories, so with a 25-signature corpus dropping one signature typically moves recall by well under the 0.10 headroom the 0.90 threshold allows; and the dropped signature's malicious sample may still be flagged at hard tier by a built-in check (`phrase.injection` in particular overlaps most instruction-injection payloads), in which case recall does not move at all. Therefore the corpus MUST designate at least one **control signature** whose canonical malicious sample is verified to be hard-flagged by NO built-in `detect.Check` and by no other bundle signature, and the control MUST assert the gate's exit code is `exitGateBreach` when that signature alone is removed. The control MUST be evaluated over a scope in which the removal PROVABLY crosses the threshold, and merely "restricting to the control signature's category" does not establish that: `decide()` fails only on `OverallRecall < minRecall`, so a control category holding ten gated malicious samples lands on exactly 0.90 after one miss and the gate stays green on the boundary. The canonical construction is therefore a control corpus whose gated malicious set is the control signature's sample ALONE, so removal drives recall to 0 — any other scope MUST be accompanied by an explicit assertion that the control category's recall is 1.0 with the signature present and strictly below the threshold without it, computed rather than assumed. The no-overlap property of the control sample MUST itself be asserted in CI so that a later built-in check gaining coverage of it turns the control red rather than silently re-vacuating it. + +**Publication & refresh channel** + +- **FR-017**: Published databases MUST be versioned, signed artifact pairs (bundle + sidecar) at a stable public location with a discoverable "latest" reference; the Spec 087 opt-in fetch consumes this channel unchanged, and manual download + file drop MUST remain a first-class, documented, equally-capable path. [NEEDS CLARIFICATION: hosting location for the published artifacts — GitHub Releases on the tpa-db corpus repo vs. a mcpproxy.app static path (affects URL stability, availability SLO, and bandwidth, not behavior)] +- **FR-018**: This feature MUST NOT introduce a second refresh lifecycle: cadence, single-flight, on-demand trigger, activation self-check, and fail-safe behavior are Spec 087's; this spec only inserts the signature and sequence stages (FR-008) and defines the artifacts flowing through it. +- **FR-019**: After a new database activates, the system MUST run a post-activation informational re-scan over cached tool metadata of already-approved toolsets, off the hot path, without connecting to upstreams: new hard-tier hits surface as review findings naming the TPA id. Refresh alone MUST NOT auto-revoke approvals, auto-quarantine servers, or auto-approve previously-held tools; all state changes from re-scan flow toward human review only. + +**Observability, telemetry & docs** + +- **FR-020**: Bundle status on all existing surfaces (CLI security commands, REST security overview, Web UI) MUST additionally report sequence, signing authority and trust epoch, signature-verified state, and staleness (age since `generated_at`), alongside the existing version/fingerprint/source/load-error fields, and MUST show the last rejection reason (tamper, downgrade, stale-epoch replay, unsigned-refused-by-ratchet, authority-not-active, rollback-target mismatch, gate failure) so an operator can tell "current and verified" from "stuck on last-known-good and why". Where the active database has demoted the install to DEGRADED coverage (FR-007 unsigned-active, FR-009a embedded fallback), that state and its reason MUST be surfaced too — a silently suspended `scan`-mode auto-approval is otherwise indistinguishable from a quiet install. +- **FR-021**: The existing anonymous TPA-scanner telemetry MUST gain the active bundle version, sequence, and source (and signature-verified state) so database adoption and freshness are measurable fleet-wide; the existing telemetry opt-out MUST fully cover these fields, and no new identifying data is introduced. This is NOT a drop-in addition: `internal/telemetry/anonymity.go`'s schema-v8 backstop hard-whitelists the `tpa_scanner` sub-object to `{scans_completed, scans_failed, scans_with_findings, findings}` and requires every scalar to be a non-negative integer, so a string `bundle_version`/`source` or a boolean `signature_verified` would be rejected before transmit as "carries a key outside the whitelist". The design MUST therefore extend that backstop with explicitly TYPED entries (integer `sequence`, enum-bounded `source` ∈ {embedded, file, fetch}, boolean signature-verified) rather than relaxing it to accept arbitrary keys or free strings. `bundle_version` MUST be constrained to publisher-issued values (unrecognized/operator-built corpus versions reported as a single `other` bucket): the backstop exists precisely to stop a bespoke, near-unique string from acting as a fleet identifier, and a self-built corpus version would be exactly that. **The same reasoning MUST be applied to `sequence`, not only to `bundle_version`**: an operator picks their own corpus's sequence values freely — a unix timestamp or a private build number is the obvious choice — so a raw integer sequence from an operator-built corpus is every bit the near-unique fleet identifier the `other` bucket was introduced to suppress, and bucketing the version while transmitting the sequence verbatim would defeat it. `sequence` MUST therefore be reported only when `bundle_version` resolves to a publisher-issued value (a known, low-cardinality release line); when the version falls into the `other` bucket the sequence MUST be omitted. **"Resolves to a publisher-issued value" MUST be a provenance test, never a match on the claimed version string**: an operator-signed or unsigned corpus controls its own manifest and can copy a genuine publisher version string verbatim while carrying a unique sequence, which would sail through a string allowlist and transmit the fingerprinting sequence anyway. The verbatim version and the sequence MUST therefore be reported only when the active bundle's provenance is the publisher authority — the embedded default, or an external bundle signature-verified against a publisher key (FR-004) — and every other active bundle reports `other` with sequence omitted REGARDLESS of the version string it claims. The reporting layer is where authority is known, so the gate lives there; the anonymity backstop additionally enforces the cross-field invariant it can see on the payload alone — a heartbeat carrying a `sequence` while `bundle_version` is `other` is rejected before transmit. Nothing analytic is lost: SC-006 already scopes fleet-freshness to publisher-issued corpora and reports operator-built ones as an unaged bucket. +- **FR-022**: Documentation MUST cover the database lifecycle end to end for operators (verify, drop, fetch, roll back, read status) and for contributors (author a signature, provenance/license rules, eval-sample requirements, how the gate blocks a bad corpus), updating the existing security-quarantine/tool-scanner docs. + +### Key Entities *(include if feature involves data)* + +- **Signature Database (bundle + sidecar)**: the publishable unit — the deterministic compiled `scanner-bundle.json` plus a detached signature over its exact bytes. Identified by (bundle_version, sequence, fingerprint). +- **Bundle Manifest metadata**: `bundle_version`/`schema_version` (existing), plus additive `generated_at`, `sequence`, publisher key id. +- **Publisher Trust Anchor**: the set of publisher public keys embedded in a binary release, unioned with operator-configured keys; the only roots of trust for verification, never network-delivered. Each authority carries its own **trust epoch** (FR-005a), a monotonic integer advanced only through that authority's trust root (a binary release for the publisher, a config edit for the operator) and that a candidate's signed manifest must match exactly. +- **Signing Authority**: the grouping anti-downgrade state is keyed by (FR-004a) — one publisher authority spanning all publisher keys including those mid-rotation, and a separate operator authority for operator-configured keys. Each holds its own watermark so neither can lock the other out. Exactly one authority is **active** at a time, declared in config (FR-004b, default publisher); artifacts of a non-active authority are refused, so per-authority isolation cannot be turned into an authority-swap by whoever can write the bundle path. +- **Signature (TPA record)**: one cataloged campaign/technique — `TPA-YYYY-NNNN` id, detectors, category, level, confidence, tier intent (gating vs non-gating), provenance reference, license. +- **Eval Sample Pair**: the labeled gated-malicious sample(s) plus category-matched hard-negative(s) a gating signature must contribute to the frozen eval dataset. +- **Sequence Watermark**: the persisted highest-activated sequence, tracked per signing authority and scoped by trust epoch (FR-004a, FR-005a); the anti-downgrade/anti-replay state. Never empty for any authority that has a baseline — it is seeded from a trusted one (the embedded bundle's sequence, for the publisher authority) per FR-009b, so "no state yet" is not an accept-anything state. The single exception is an operator authority configured without a baseline, which FR-009b permits precisely because no binary-trusted seed exists for it; there the install carries no anti-downgrade guarantee and is marked degraded while such a bundle is active rather than being presented as fully trusted. It is MONOTONIC — rollback does not lower it (FR-010 expresses rollback as a pinned exception plus a deny-list instead), and the only thing that re-baselines it is a higher trust epoch arriving through the authority's trust root (a binary release for the publisher authority, a config edit for the operator authority — FR-005a). +- **Publication Channel**: the stable public location of versioned signed artifact pairs with a "latest" reference; consumed by the Spec 087 fetch or by manual download. +- **Post-Activation Re-scan**: the one-shot, off-hot-path evaluation of cached approved tool metadata against a newly-activated database, emitting review findings only. +- **Active Database / Last-Known-Good**: unchanged from Spec 086/087 — the serving bundle and the fail-closed fallback target. + +## Success Criteria *(mandatory)* + +### Measurable Outcomes + +- **SC-001**: 100% of tampered candidates are rejected before parsing, with the last-known-good database still serving and the reason surfaced — demonstrated by fixture in CI. "Tampered" is defined precisely, because the signature covers the bundle bytes and NOT the sidecar's own encoding: any altered bundle byte, any altered signature value, any altered sidecar identity field (algorithm or key id, whose agreement with the manifest FR-003 already requires), and any mismatched bundle/sidecar pair. Byte changes to the sidecar that alter neither the signature value nor an identity field (whitespace, key order) are explicitly NOT required to be rejected — a detached signature cannot detect them — so the criterion is stated in terms of the fields that carry meaning rather than an unachievable "every byte of the sidecar". +- **SC-002**: A validly-signed bundle whose sequence is below its OWN signing authority's watermark for the current trust epoch is never activated by any non-rollback path (drop, fetch, restart) — "lower-sequence" is always a same-authority, same-epoch comparison per FR-004a/FR-005a, so a deliberate FR-004b authority switch followed by the newly-active authority's bundle clearing its own baseline is by definition not a downgrade and needs no exemption — including on a fresh install or after a data-directory wipe, where the FR-009b seeded baseline stands in for the absent persisted watermark — and an unsigned candidate never advances the watermark nor survives the signature ratchet (the ratchet claim holds while data-directory state is retained; after a wipe it re-arms on the next signed activation, per FR-009) — demonstrated by fixture in CI. There are exactly two exemptions, and both are degraded-coverage states rather than silent ones: the FR-009a embedded-default availability fallback, and the baseline-less operator authority of FR-009b (SC-002a fixture (c)), which has no seed to compare against and so carries no anti-downgrade guarantee to claim. Neither counts as a downgrade; both MUST surface as degraded. An explicit rollback succeeds, leaves an audit trail, does not lower the historical watermark, activates only the fingerprint the operator named (a substituted file at the same path is refused), and the deny-listed release it rolled away from is not re-activated by the following refresh cycle. +- **SC-002a**: The watermark cannot be weaponized into a permanent update lockout, demonstrated by the following fixtures: (a) a bundle signed by a to-be-revoked key at the MAXIMUM representable sequence, after which a binary carrying a bumped trust epoch and the surviving key successfully activates a new release — proving key-compromise recovery is possible rather than merely documented; (a2) immediately after that bump, an OLD pre-bump bundle carrying the previous epoch and signed by the still-trusted surviving key is REFUSED as stale-epoch replay rather than clearing the re-baselined watermark — the epoch bump must not itself become a downgrade window; and (b) with the operator authority declared active (FR-004b), an operator-authority-signed bundle at a high sequence that leaves the publisher authority's watermark untouched, so after the declaration is switched back the next official publisher release still activates — proving the two signing authorities cannot lock each other out. A fixture (b2) covers the declaration itself: the same operator-signed bundle dropped while the PUBLISHER authority is declared active is refused as "authority not active" with the publisher bundle still serving, so the choice of authority cannot be made by whoever writes the bundle path. A third fixture (c) covers the operator authority after a data-directory wipe: with a config-resident baseline declared, an older operator-signed bundle is refused; with no baseline declared, it may activate but the install reports degraded coverage rather than full trust. +- **SC-003**: An air-gapped install (zero network, ever) reaches full P1 capability: verified activation of a file-dropped signed database, downgrade refusal, and rollback, with behavior byte-identical to a connected install's file-drop path. +- **SC-004**: The shipped seed corpus contains at least 25 signatures spanning at least 8 distinct public campaign/technique classes, every one carrying provenance + redistributable license, and every gating signature backed by its eval sample pair; the canonical fixture of each class produces a hard-tier finding naming its TPA id. +- **SC-005**: The scan-eval CI gate (recall ≥ 0.90 gated categories, hard-negative FP ≤ 0.05) passes with the full seed corpus embedded AND actually scored — i.e. the gate run registers the bundle check and maps every new campaign category (FR-016a), proven by a **deterministic** negative control (FR-016b): removing a designated control signature from the bundle must turn the gate red. Without that control the criterion is vacuous, since today's gate would pass with the bundle entirely absent. A deliberately-regressing corpus change is then blocked twice independently: at CI (cannot merge) and at activation (self-check rejects it) — both demonstrated by fixture. +- **SC-006**: Fleet freshness is answerable from telemetry: for opted-in installs, the distribution of active database version/sequence/source is reportable, enabling the metric "% of active installs on a **publisher-issued** database ≤ 30 days old" — a metric that is impossible to compute today. Age MUST be derived server-side by joining the reported publisher-issued version/sequence to the publication date the publisher already knows for its own releases, NOT by adding a `generated_at` or age field to the heartbeat: a build timestamp is precisely the kind of near-unique value FR-021's backstop exists to keep out of the payload, and it would undo that protection to buy nothing the join does not already give. Installs on an operator-built corpus (reported in the `other` bucket per FR-021) are consequently NOT ageable and MUST be reported as their own unaged bucket rather than silently folded into either side of the ratio — the metric's denominator is publisher-corpus installs. +- **SC-007**: A newly published signature reaches detection within one daily refresh cycle on a fetch-enabled install (and immediately on manual drop), measured end to end: publish → active → the new signature's fixture fires. +- **SC-008**: With a database at the design load ceilings (≥ 500 regex rules), p95 scan latency over a 100-tool server changes by less than 10% versus the 6-signature baseline, and bundle load-plus-verify completes under one second on commodity hardware. +- **SC-009**: After activating a database containing a new signature, an already-approved tool whose cached description matches it is surfaced for review within the same refresh cycle, with zero approvals auto-revoked and zero servers auto-quarantined by the refresh itself. +- **SC-010**: Rollback authorization is bound to the whole digest and to a real history: a fixture in which an attacker-substituted, publisher-signed, current-epoch bundle shares the operator-supplied abbreviated fingerprint (first 12 hex chars) but differs in the full SHA-256 is REFUSED as a rollback-target mismatch; and after N activations the activation record replays all N entries — (authority, epoch, sequence, full digest, generated_at, activated_at, source) — across a process restart, so the digest of a known-good artifact is recoverable without trusting the candidate file. +- **SC-010a**: Every FR-011 ceiling is exercised at its boundary: a bundle of exactly 8 MiB and one of 8 MiB + 1 byte, a corpus of exactly 2000 rules and one of 2001, a sidecar of exactly 4 KiB and one of 4 KiB + 1 byte, and a candidate whose read exceeds 5 s — the first of each pair activates, the second is rejected whole with its reason in bundle status, and the last-known-good corpus keeps serving throughout. +- **SC-011**: Candidate opening is race-free and time-bounded, demonstrated by fixtures that (a) swap the candidate path for a FIFO between path resolution and open and are refused by the descriptor-level `fstat` check rather than blocking, (b) point the path at a symlink to a FIFO and are refused without following it, and (c) leave a never-written FIFO in place and see the single-flight refresh slot released within the bounded budget with a recorded rejection reason, so a subsequent legitimate database update still activates. +- **SC-012**: The persisted-state trust boundary is exercised by SUBSTITUTION, not only deletion: fixtures replace `config.db` with a prepared database carrying (a) a lowered watermark, (b) a cleared signature ratchet, and (c) a forged pin/deny-list entry, and in every case the guarantees anchored outside the data directory still hold — the FR-009b publisher baseline re-seeded from the binary, the config-resident operator baseline, `require_signed_bundle`, and the FR-004b active-authority declaration — while the install reports itself degraded. No fixture may assert that the ratchet, watermark, pin or deny-list themselves survive; they are best-effort against this attacker by construction. + +## Assumptions + +- Spec 087's refresh lifecycle (daily tick, single-flight, activation self-check, opt-in fetch, fail-safe) is implemented before or together with this feature's refresh story; this spec inserts stages into that pipeline rather than duplicating it. US1's signature/sequence verification is also exercised by the existing hot-reload/file-drop path (Spec 086 `ConfigureBundle`), so US1 does not hard-depend on 087 landing first. +- The Scanner Bundle Contract v0.1 line remains the wire format; signing metadata is additive and the sidecar is a separate file, so existing loaders and the byte-determinism guarantee are unaffected. +- The corpus source-of-truth pipeline (signature sources → compiled bundle) lives outside this repo's runtime (the tpa-db authoring pipeline); this repo consumes compiled bundles only and embeds one at build time (a release build with no embeddable bundle fails, per Spec 087 FR-001). +- The eval gate's thresholds, dataset conventions (including hard-negative naming), vacuity guard, and hard-tier-only scoring are reused verbatim; growing the corpus means growing the dataset alongside it. +- The offline tier's runnable surface remains `engine: regex` × `target: tool_description` for v1; `structural_diff`, `resource_content`, and `server_manifest` rules stay declared-not-runnable (skipped, never clean coverage) exactly as today. +- Cached tool metadata already held by mcpproxy (approval baselines, index) is sufficient for the post-activation re-scan; no upstream connection is initiated by refresh. +- [NEEDS CLARIFICATION: back-compat window for unsigned file drops — should `require_signed_bundle` flip to default-on after one or two release cycles once signed publishing is live, or remain opt-in indefinitely for air-gapped/self-built-corpus operators? Note this is now a smaller question than it looks: FR-009's signature ratchet already means any install that has ever run a signed bundle refuses unsigned drops, and FR-007 suspends auto-approval while an unsigned bundle is active, so the default-on flip would only change behavior for installs that have never seen a signed bundle at all.] + +## Out of Scope + +- Any change to the detect engine's check semantics, tiers, thresholds, or the shared position classifier (a corpus change must never require touching `ClassifyPosition`; recall fixes belong in signatures + eval samples, not classifier cues). +- Running `structural_diff`/stateful rules, or adding `resource_content`/`server_manifest` scan surfaces. +- LLM-assisted or networked detection tiers; this database feeds the deterministic offline tier only. +- Remote/networked trust-anchor distribution, certificate hierarchies, or transparency logs (key set ships with the binary; revisit only if the publisher set ever grows beyond the project). +- Auto-revoking approvals, auto-quarantining, or auto-approving anything as a side effect of a database refresh (re-scan is informational; state moves toward review only). +- IoC hash feeds and package/registry reputation data (the roadmap note's "IoC hashes" are deferred: v1 targets are description-borne patterns; the format's additive versioning leaves room for later rule engines). +- mcpproxy binary self-update or release-awareness changes (Spec 087 US3 owns that surface). +- Changing the scan-eval gate's thresholds or scoring. + +## Constitution Check *(note)* + +Principle IV (Security by Default) governs this spec: the database is treated as a hostile input until proven otherwise (verify-before-parse, fail-closed to last-known-good, anti-downgrade watermark, mandatory signatures on the network path), and refresh can never silently widen approvals. Principle III (Configuration-Driven Architecture): `require_signed_bundle`, operator keys, the active-authority declaration and its baseline, and the existing bundle path/fetch settings live in `mcp_config.json` with env override and hot-reload; no hardcoded URLs or paths. Principle V (TDD): every rejection class (tamper, downgrade, unsigned, gate regression, ceiling breach) is built against failing fixtures first, and the corpus itself is test-gated by scan-eval. Principle I (Performance at Scale): verification, activation, and the post-activation re-scan stay off the scan hot path with atomic swaps (SC-008). + +## Commit Message Conventions *(mandatory)* + +When committing changes for this feature, follow these guidelines: + +### Issue References +- ✅ **Use**: `Related #[issue-number]` - Links the commit to the issue without auto-closing +- ❌ **Do NOT use**: `Fixes #[issue-number]`, `Closes #[issue-number]`, `Resolves #[issue-number]` - These auto-close issues on merge + +**Rationale**: Issues should only be closed manually after verification and testing in production, not automatically on merge. + +### Co-Authorship +- ❌ **Do NOT include**: `Co-Authored-By: Claude ` +- ❌ **Do NOT include**: "🤖 Generated with [Claude Code](https://claude.com/claude-code)" + +**Rationale**: Commit authorship should reflect the human contributors, not the AI tools used. + +### Example Commit Message +``` +feat(security): [brief description of change] + +Related #[issue-number] + +[Detailed description of what was changed and why] + +## Changes +- [Bulleted list of key changes] +- [Each change on a new line] + +## Testing +- [Test results summary] +- [Key test scenarios covered] +```