feat(tbtc): native signer state-anchor service, trust startup, and bootstrap ceremony (ABI 4.3) - #4199
Open
mswilkison wants to merge 91 commits into
Open
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…iness Adds Go consumers for the Rust signer's ABI 3.5 inventory and state-witness symbols, plus interactive-signing readiness checks wired into the node. INCOMPLETE - committed to preserve the work, not because it is ready: - Native inventory parsing is unfixed. The Rust keyGroup is a lowercase 66-character compressed SEC1 key; the parser must accept both valid 64-character x-only test keys and 66-character compressed keys via TaprootOutputKeyFromTBTCSignerKey, and require the resulting x-only key to equal walletID. Frozen vector to pin: walletID = 11*32, keyGroup = 02 + 11*32, threshold 2, participants 3, epoch 0, public commitment 33*32, seat 1/3 commitments 44*32 and 55*32, expected inventory commitment bd6ec36fa27a57dd9926883bb2ff4dee7ececd28de940df7294f0e0f0dedd150. - The read-only state anchor is still an interface. It needs a concrete authenticated Read + CompareAndSwap service with a durable PostgreSQL schema, fresh nonce, store/protocol/trust-domain/operator/authority/client identity binding, signed responses with bounded ancestry verification, unknown-outcome recovery by readback, and rejection of stale, divergent, ahead, over-window or wrong-identity states. It must anchor every security-relevant native mutation before Round1 commitments or Round2 shares leave the process. An interface-only implementation is not acceptable. - Casing asymmetry to verify against the Rust side: DurableStoreIdentityResult is snake_case, while RetainedKeyPackageInventoryResult, StateWitnessProofRequest (deny_unknown_fields, maximumEntries is u16 bounded 1..=256) and StateWitnessProofResult are camelCase. Verified only that `go build ./pkg/tbtc/... ./pkg/frost/...` succeeds. No test run, no cgo integration run, no cross-language vector check. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012KZb3W6TwYtKTw1mb2sfpS
Lockstep bump for the signer's ABI 3.5 -> 4.0 change (keep-core #4198). Per this file's own rule, the Go constants and ci/frost-signer-pin.env move together with the lib commit that provides them. Under ABI 3, `durable_store_fingerprint` mixed the stable `.store-id` with the parent directory's path string and inode, the filesystem device, and the advisory lock file's inode — and that composite was the first field of every state commitment. So deleting the zero-byte lock file, restoring the data directory from backup at the same path, renaming it, or remounting where st_dev is unstable made every committed record unverifiable and the signer unstartable, with no in-band recovery: the failure is raised inside StateFileLock::acquire, before the corruption policy can apply. ABI 4.0 binds commitments to the stable `.store-id` alone. The path, filesystem and lock fingerprints are still computed and still enforced at open — they simply no longer enter any committed transcript. The symbol set is unchanged, but the VALUES of existing wire fields differ, so an ABI-3 library must not be linked by this bridge. requiredTBTCSignerABIMajor 3 -> 4 requiredTBTCSignerABIMinMinor 5 -> 0 FROST_SIGNER_MIRROR_REF 6e0fa97 -> a5363e9 Minor 0 is the initial ABI-4 surface and carries forward every symbol added through 3.5 under the v2 identity and transcript schemas. Verified: go build ./pkg/frost/... ./pkg/tbtc/... clean; go test ./pkg/frost/signing/ passes, including the contract-pin guard. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012KZb3W6TwYtKTw1mb2sfpS
Authenticate a complete sequence-one recovery artifact, select the durable-intent suffix against it, execute the transition with at most one typed-recovery retry, and validate the result under an explicit recoveryReplay marker: a recovered transition must be an exact idempotent replay whose witness base equals the recovered certified floor, and can never double as an exact-head replay. Decode and classify the typed state-anchor trust-recovery error across the FFI boundary. Anchored interactive calls advance at most three Rust generations (prepared-witness reconciliation, one sweep/repair snapshot that also covers protected retirement, and the operation's own write), matching the engine's production barrier bound. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DpftF6aYHh56KgiHU6D2NQ
Interactive aggregate memoization exists only on the FROST/ROAST path. A legacy ECDSA wallet folds an empty key group into its stable ROAST session ID, so in frost-native builds two wallets signing the same message at the same start block would contend for one memo session and the second would fail signing outright. Gate memo ownership on the executor's Schnorr material; Release stays nil-safe on the legacy path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DpftF6aYHh56KgiHU6D2NQ
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DpftF6aYHh56KgiHU6D2NQ
Introduce the offline-authorized four-phase initial anchor ceremony for
the FROST native tBTC signer:
- pkg/frost/signing: strict bootstrap-facts codec pinned to the exact
generation-one store genesis, a strict-JSON preflight (duplicate,
case-folded-alias, depth, and trailing-data rejection) now fronting
every native-signer decoder, the dedicated four-field bootstrap
provisioning config installer, and the StateAnchorBootstrapFacts
bridge operation with a fail-closed default build.
- pkg/tbtc: plan/core/detached-signature/final/output-bundle artifacts
with strict codecs and validation. Prepare re-derives every stream,
binding, SPKI, store, and checkpoint relationship; Initialize verifies
the detached core authorization and requires an exactly reconciled
create-if-absent record; Finalize validates the second detached
signature and emits one atomic bundle whose decoder re-validates the
embedded certificate chain and canonical signer config, with Finalize
round-tripping its own output through that decoder. The identity wire
decoder names its single zero-allowed pin (loopback endpoint leaf)
instead of relying on positional indexing.
- pkg/tbtc: openat-based no-replace artifact reader/writer enforcing
owner-only 0700/0600 posture, O_NOFOLLOW opens, fsync-before-publish,
and an explicit short-write guard.
- cmd: "tbtc-signer anchor bootstrap {facts,core,initialize,finalize}"
accepting detached signatures only; initialize fails closed until a
separately reviewed transport supplies its narrow client factory.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DpftF6aYHh56KgiHU6D2NQ
- pkg/frost/signing: bootstrap-facts round trip plus a strict-decode table (schema, non-canonical/zero/missing fingerprints, cross-store checkpoint, non-genesis generation and predecessor, commitment mismatch, duplicate and case-folded members, trailing data, depth bomb, non-canonical numbers) and direct preflight tests exercising every rejection class with an exact depth-bound corpus. - pkg/tbtc: round trips and tamper tables for the plan, core artifact, detached signature, final artifact, and output bundle; Initialize divergence coverage against a simulated history service (binding hash, operation ID, transition digest, checkpoint, epoch, revision, previous event root, tampered/missing acknowledgement, missing read recovery) plus stage/digest/authority signature rejections; Finalize rejection of wrong-stage, tampered signatures, tampered certificates, and a non-canonical base-config table; output-bundle rejection of rehashed tampered chains, duplicated chains, and non-canonical configs. - pkg/tbtc: provisioning artifact writer/reader coverage - exact 0600 bytes, refused overwrite, no temporary residue, symlink and non-0700 directory rejection, byte-bound and mode enforcement. - cmd: end-to-end facts->core->initialize->finalize ceremony through the injected client factory with real Ed25519 authority keys and a local reimplementation of the acknowledgement transcripts, plus transport-unavailable, non-canonical-path, required-flag, and fail-closed facts assertions. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DpftF6aYHh56KgiHU6D2NQ
Implement the online transport behind the bootstrap initialize ceremony
phase as a hardened HTTP client beside the runtime anchor client:
- New initialize endpoint (path suffix "initialize" beside
read/advance/history) with request schema
tbtc-frost-native-signer-state-anchor-initialize-request/v1 and its own
signing domain. Both bootstrap request kinds ("initialize" and the
reconciliation "read") are client-signed over a domain-separated
fixed-width transcript binding schema, kind, binding hash, nonce,
operation ID, transition digest, the genesis checkpoint fields, and the
client SPKI; no JSON bytes are ever signed.
- The client re-derives the offline core digests and verifies the offline
authority signature before any network activity, then requires the
existing checkpoint-acknowledgement schema in response (applied on first
create, already-applied on idempotent replay) verified through the
existing acknowledgement validator, and always reconciles through a
fresh signed exact read whose retained JSON and expiry feed the client
result consumed by InitializeFrostNativeSignerAnchorBootstrap.
- An authenticated, request-bound read showing a different genesis record
(checkpoint, binding hash, operation ID, or transition digest) poisons
the client permanently; transport-shaped failures stay retryable, and
every post-send verification failure is ambiguous rather than terminal.
- Canonical strict-JSON client config
tbtc-frost-native-signer-anchor-bootstrap-client-config/v1 (endpoint,
raw response key plus SPKI pin, endpoint leaf SPKI hash zero only for
numeric loopback HTTP, PKCS#8 PEM client key path, request timeout)
validated with the provisioning canonicality machinery, plus the same
transport hardening and byte caps as the runtime anchor client.
- cmd/tbtc-signer anchor bootstrap initialize now constructs the real
transport from --client-config; the transport-unavailable error remains
only for nil-factory injection.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DpftF6aYHh56KgiHU6D2NQ
Exercise the bootstrap initialize HTTP client against an httptest fake history service with real Ed25519 keys: - happy path: applied acknowledgement plus reconciling read passes InitializeFrostNativeSignerAnchorBootstrap and finalize end-to-end. - idempotent retry: a commit followed by a malformed response reconciles through the fresh signed read; the second full call succeeds through the already-applied sentinel, and both results are identical except the nonce-fresh exact-read wrapper. - divergence: an authenticated read showing a foreign genesis record poisons the client permanently; repeated calls return the identical error with zero additional network requests, as does a signed absent read that contradicts a verified applied sentinel. - decode strictness table: wrong acknowledgement schema, wrong status, tampered signature, oversized response, wrong content type, non-200, and trailing data are all ambiguous and retryable, never accepted and never poisoning. - endpoint/config negative tables: non-canonical endpoint forms, HTTPS-without-pin, plaintext non-loopback, duplicate/unknown/case- folded members, non-canonical numbers, zero or aliased client keys. - transcript stability: frozen digests, length, and signature for the exact initialize and read request transcripts. - cmd: the default initialize factory reaches the client-config loader instead of the nil-factory transport-unavailable sentinel. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DpftF6aYHh56KgiHU6D2NQ
mswilkison
force-pushed
the
codex/frost-preauth-outbox-restored
branch
from
July 27, 2026 00:08
67e591f to
e037450
Compare
mswilkison
changed the title
frostWalletIDFromSigner already treats FrostTBTCSignerV1 material whose key group is folded from the legacy wallet public key as a legacy wallet; the retained key-group binding resolver did not, so a stored scaffold-era wallet failed startup by validating a taproot-derived key group against its legacy wallet ID. Mirror the KeyGroupSource discrimination so both resolvers agree on what is a FROST wallet. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DpftF6aYHh56KgiHU6D2NQ
The rebuilt lineage's first tagged CI run surfaced harness gaps rather than product bugs: - Multiproc e2e children (shape-B bootstrap/workers, distributed-DKG workers) never installed the process-local test state anchor, so every request-taking native operation was blocked; they now run the same anchor setup as the in-process suites, and the setup itself gained the ABI 4.3 trust head and window bounds the barrier now requires. - Harnesses that drive the ROAST runner directly now begin the outer interactive aggregate memo session the production executor owns. - Shape-B asserts the finalized signing subset aggregates one signature while a committed-but-omitted seat fails closed with the engine's aggregate-authorization rejection: separate processes share no aggregate memo, so an omitted observer cannot obtain the signature locally. - The native-anchor store-mismatch manifest case recomputes the stream ID over the mutated identity so the store check is actually reachable. - gofmt on the two state-anchor trust test files. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DpftF6aYHh56KgiHU6D2NQ
mswilkison
marked this pull request as ready for review
July 27, 2026 01:25
Three interacting defects on the native FROST signing path. An active canonical quarantine reached the activation handshake but never the signing path. reconcile() copied QuarantineCount into its snapshot and no consumer read it, so a quarantined wallet stayed in state.Wallets, kept full native signer expectations, and kept signing Bitcoin - while lifting that same quarantine required a manifest-pinned authority quorum certificate. Quarantine records, including recovery-required ones, are raised against exactly one WalletID and lifted against that same WalletID, so the snapshot now carries every unlifted record joined to its canonical Bridge wallet, and the pre-sign authorization gate refuses to authorize or keep authorizing a batch for that wallet. Node-wide emptiness stays a bootstrap requirement of the handshake and is documented as such. The cached readiness path compared the whole native inventory snapshot by value, including the state generation and anchor revision. The authorized signing window is exactly what advances those: admission reserves their durable cost up front and the engine persists a consumption marker before releasing each share. Every guard and every monitor tick therefore saw G+1 != G and failed, and the monitor latches the first error permanently, so native signing aborted deterministically. Identity, trust head and key material stay strictly pinned; the state checkpoint, anchor revision and derived headrooms may only advance monotonically and only while the anchor stays out of the rotation-warning band. A rollback, an equal-generation fork or a rotated trust certificate still fail closed. The journal stamp check used TryLock and reported contention as a fatal authorization error. The journal mutex is held for a whole reconcile, including paginated independent-history reads, so any other wallet's authorization or the handshake exporter's background reconciliation could cancel a valid signing session. Contention is now waited out under the caller's context, which is the pre-caching behaviour; only a genuine stamp change fails. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Batch publication and its state checkpoint are two separate writes. A failure between them leaves the batch durable and the checkpoint one sequence behind, and because batch files are immutable no-replace publications every later reconciliation recomputed the same sequence and failed with "immutable journal file already exists". The journal then stayed wedged in-process until an operator restarted the node, even though initialize() already adopts exactly that shape on startup. Reconciliation now performs the same adoption for both the canonical and the quarantine journal before it reads any history. Adoption is not a trust shortcut: each batch is revalidated against its exact durable predecessor, persisted lift certificates are rechecked, and the reconciliation that follows still rejects the whole durable prefix if the canonical history rewrote, omitted or reordered an adopted event. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The read path refuses any journal file above 8 MiB, but the write path had no matching bound, so an oversized batch was published and fsynced successfully and only the next initialize() discovered it - by which point the journal was permanently unopenable and the file could no longer be attributed to the write that produced it. Persist now fails closed with the offending file name before the temporary file is created. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The native signer reserves six terminal witness-journal records so an in-flight interactive request can finish before a checkpoint has to be acknowledged, and enforces rotationThreshold + 6 <= maximumRecords at both of its geometry intakes: configured_state_anchor behind frost_tbtc_init_signer_config, and parse_endpoint behind frost_tbtc_transition_state_witness_anchor. Go still enforced the retired two-record reserve in seven independent copies of the same arithmetic. Go is the layer that mints and pre-verifies the offline-authority-signed trust certificates and validates the installed init configuration, so a geometry such as maximum 64 with threshold 60 passed the operator plan, the certificate endpoint, the startup manifest, the anchor identity, the activation manifest, and the installed init config, was signed by the offline authority, and was first rejected by the signer at node startup - which can only be undone by re-running the whole ceremony. Every geometry with a maximum in [4,7] was Go-legal and signer-invalid outright. Replace all seven copies with one exported reservation constant and one validation helper in pkg/frost/signing, mirroring the signer's error wording so operators can correlate the two layers. The trust-head decoder is tightened with the rest: its input is emitted from an endpoint the signer parsed through this same rule, so decoding at exactly that rule cannot reject a head the signer can produce, while decoding any looser lets a readback the signer would refuse to install look installable to the Go startup path. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…flow Pre-sign admission multiplied the hard per-operation generation bound by every anchored call in the workflow. With production parameters that made a maximum-size deposit sweep (21 inputs, 5 signing attempts) cost 4158 generations for an operator holding three local seats, above the 4096 certified proof window, so authorize() failed outright before any headroom accounting - a freshly rotated anchor could not admit it either. Three or four local seats is an ordinary sortition result in a hundred-seat wallet, and every excluded seat is lost to the wallet's signing threshold on every batch of that size, so enough of them stop the group from assembling a threshold at all while each node reports only a local admission error. The per-operation bound of three is correct and stays: the signer's terminal reserve of six journal records is three snapshots, and node startup installs this same constant as the output barrier's MaximumStateGenerationAdvancePerOperation, where a call advancing further poisons the process. What was wrong is multiplying it by the call count. The third advance of any single call is always a repair - both the expiry sweep's second snapshot and the Round2/Aggregate marker re-persist require a pending persistence operation that only a failed snapshot creates, and the marker re-persist excludes that call's own mutation because flushing the marker makes the replay gate reject the retry. A snapshot that failed advanced no generation, so its call leaves exactly the advance the later repair lands. Reserve the amortized two advances per call instead, plus one workflow-wide terminal allowance for the two interleavings that outrun it and cannot recur: a witness committed before a failed post-commit revalidation (replace_state revalidates first, so every later call then fails closed), and one witness prepared before admission (prepare_witness refuses a second). Both restart windows are finite, so a seat ceiling remains - four local seats at the maximum batch size, up from two. Report it in the rejection itself, with the input count, the ceiling, and the threshold consequence, so an operator that crosses it is not left inferring the cause from a wallet that quietly stops signing full-size batches. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The identity-frozen HTTPS round tripper had no end-to-end coverage: the existing tests exercise the pieces around it - peer keys, the history cap, the live-channel bookkeeping, and the TLS dialer's address fail-over - but nothing drove a real request through RoundTrip, so the verification chain that actually enforces the frozen TLS identity was untested. Stand up a live JSON-RPC TLS server, dial it through the production constructor, and assert both directions. The frozen peer must round-trip a real eth_chainId call; a second TLS server serving the same SPIFFE identity from a rotated key at the same pinned loopback address must be refused, even though it satisfies PKIX, the TLS 1.3 http/1.1 profile, the hostname, and the frozen address set, because it never passed the guarded dialer and was therefore never recorded as an authenticated peer. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The periodic replay pass discarded the Bitcoin-side result of every rebroadcast it performed: a rejection produced no log line, no error, and no counter, so a reservation the node had been failing to broadcast for hours looked exactly like a healthy one from the outside. Report the rejection as a transient replay failure, the same shape this file already uses for recoverable candidate failures. Retry and backoff are unchanged - the record keeps its authorized variant and the next tick retries it, start-up recovery still proceeds because the failure is not fatal - but the periodic ticker now logs it, and the attempt counter travels with the message so a stuck entry is distinguishable from a single mempool hiccup. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The retained peer history was keyed by the full peer identity, so every leaf certificate the retained endpoint ever presented took its own slot. That rotation is routine and expected - registerRetainedPeer already pins the leaf SPKI to the frozen identity, so a rotated leaf keeps its key and changes only the certificate around it - and once the bounded history filled, the cap latched a permanent separation failure on a healthy endpoint, which verifyIndependence can never clear. Key the history by the state the policy actually compares across endpoints: address, leaf SPKI, and SPIFFE authorities. Certificate aliasing is already reported by the SPKI comparison, because identities sharing a certificate necessarily share its public key, so collapsing observations that differ only in the certificate leaves every independence verdict unchanged. The cap still latches for 4096 genuinely distinct separation identities. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The journal stamp compared before the cached snapshots are read covers journal state only, which reads as if the native signer inventory and interactive signing readiness were exported unchecked. They are not: verifyFrostProductionSignerReadinessUnchanged re-reads live native signer state at the signing boundary and refuses to sign unless it still equals the cached value the payload carries. Record that where the snapshots are taken, together with the residual window a consumer of a signed handshake actually has to tolerate. No behaviour changes; TestFrostActivationHandshakeExporter_RevalidatesNativeSignerStateBeforeSigning already pins the revalidation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… bound The signer keeps a PREPARE/COMMIT pair above the terminal band so a corrupt state image can still be quarantined once an interrupted retry has parked the journal at the terminal limit, and enforces rotationThreshold + 6 + 2 <= maximumRecords at both geometry intakes. The Go bound mirrored only the six terminal records, so Go would again mint and pre-verify offline-authority certificates for geometries the signer rejects at node startup - the same drift the shared helper was introduced to end, one reservation later. Fold the quarantine pair into the same helper and pin both constants, so raising either one in Rust fails the Go test instead of surfacing after an offline ceremony. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The pin pointed at codex/dkg-retirement-signer, a branch in no open pull request, which forked from this stack four hardening commits before the signer PR's head. That commit supplies the ABI 4.4 retirement symbol this branch requires, but not the witness rollback, state-loading, rotation boundary and interrupted-retry fixes, so the cgo gate never built the signer source actually under review and the two pull requests could not merge as a pair - this branch requires minor 4 and the signer PR published 4.3, so the bridge would reject the library at load. Move the pin to the signer head that carries both: the reviewed hardening commits and the retirement engine restacked on top of them. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The Go geometry test can only pin Go's own constants: the signer crate lives on the mirror branch, so nothing in this branch notices the signer raising a reservation. That is precisely the drift that shipped once already, when Rust moved the terminal reserve from two records to six and seven Go validators kept the old bound, and again when the quarantine pair was added on top. The cgo gate is the one place that holds both sources at once, because it checks out the pinned crate to build the library. Compare the reservation constants there, so a Rust-side change fails in the same gate that builds the library rather than after an operator has run the offline signing ceremony against a geometry the node rejects at startup. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two follow-ups to the cached-readiness fix. The stability check inside a single verification was left comparing the two native signer reads by value. Both reads verify the state tip against the authenticated external anchor, so a full round trip separates them, and the authorized signing window's own consumption marker routinely persists in that gap: the reconciliation and every revalidation therefore still failed with "native signer state changed during readiness verification", and the pre-sign authorization monitor latches the first such error permanently. The check now applies the same advance-versus-change discipline the cached comparison uses. A rollback, an equal-generation fork, a rotated trust head or any strict-field change between the two reads still fails closed. The activation handshake exported the native signer state from the reconciliation cache, which is keyed on the finalized Ethereum point and refreshed only when finality advances. The state generation, the commitment chain, the anchor revision and both restartable headrooms are exactly the fields the revalidation now permits to move, so a concurrent authorized batch could leave the node signing a nonce-bound attestation claiming a headroom stale by orders of magnitude. The revalidation now returns the live inventory it read and the payload is built from that, so the attested volatile values are bounds taken at the signing boundary rather than the age of the cache. The strict fields, including AnchorRotationWarning, stay pinned to the reconciled values, so a healthier-than-reality anchor still cannot be attested. The comment at the cache read said the revalidation refused to sign unless live state equalled the exported value; it now states what is pinned, what may drift, and in which direction. The handshake test double still compared the whole inventory by value, so its "native anchor advances" case asserted a 503 production no longer returns. It now mirrors the production comparison, and a new test pins that an authorized advance is answered from the same cache with the live values in the signed payload. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The generation accounting around the pre-sign reservation documented two guarantees the signer does not provide. The per-operation enumeration counted three durable writes per anchored call and called that three generations. It is not: one generation is one committed state witness, and replace_state commits up to two per write - reconcile_pending_witness first commits a witness an earlier call prepared and left uncommitted after its rename won, then the write prepares, renames, and commits its own. A call whose sweep repairs such a carry-in reaches four (reconcile plus own commit in the sweep's first snapshot, the second snapshot the repair unblocked, the endpoint's own mutation), and the output barrier poisons the process at four. That interleaving predates this accounting and the constant stays at three: raising it widens the only check on a call mutating more than admission reserved, and would have to raise the frozen barrier ceiling with it. Say so instead of implying four cannot happen, in both places that enumerate the bound - the barrier's copy still carried the older, differently wrong version, claiming the sweep and the protected retirement share a snapshot. The amortization argument was also incomplete. "A failed persist advanced no generation" holds only before the rename; a persist that commits a carried-in witness and then fails, or commits its own and fails the post-commit revalidation, advances one while returning an error, and the repair call that follows still pays its own sweep, its repair snapshot, and its own mutation. Each post-rename persist failure therefore costs about one generation more than the amortized two reserve. Keep the reservation - nothing bounds how often a failing store can fail after rename, so a larger allowance would buy a false sense of coverage on a node that should fail closed - and document the residual with what an operator sees when it bites: request-taking calls blocked for want of certified generation window, new pre-sign work rejected, recovery by the offline anchor rotation window exhaustion always requires. No share is released and no replay gate weakens. Pin both gaps in tests, and pin the seat exclusion the amortized reservation still leaves: four local seats at the maximum batch means a five-seat holder in a hundred-seat wallet - an ordinary sortition result one seat over - is refused every standard 20-deposit sweep, topping out at 19 inputs, and a six-seat node at 16. Moving that needs the certified windows or the attempt limit to change, so the rejection now also reports the batch size this node's own seats could have signed; both levers are the operator's and neither is inferable from the other. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ssion
Round 1 covered the destructive half of frostOrphanedDKGReconciler. The
mirror-image half was still open, and it is the half that matters now: with
the eager retirement removed from the DKG error exits, this reconciler is the
only code that ever destroys a FROST wallet's durable material, so preserving
a wallet that is registered or whose DKG is unresolved is what protects the
key shares.
No preserve-class test built a candidate with a LOCAL SESSION, so nothing
pinned the archive side of preservation. A reconciler that skipped the
preserve check whenever a candidate had a local session - archiving the Go
session and retiring its native key group while the wallet is registered or
its DKG is still unresolved - passed the entire `-tags frost_native
./pkg/tbtc/` suite. Cover all five combinations (registered, AwaitingResult,
Challenge; with and without native inventory) and assert the ABSENCE of the
destructive calls: no engine retirement, no archive, the session still cached,
no retained key-group binding written, no anchor capacity charged. Each case
also asserts the finalized snapshot WAS read, so a reconciler that returned
before classifying anything cannot pass by doing nothing.
The crash half-state test claimed "no native inventory means nothing to charge
the anchor for" but asserted anchorAdmission.reserved == {}, which reconcile
restores through its deferred Release() whether or not a reservation was made.
Charging one revision per retirement regardless of native inventory left that
assertion green. Count headroom reads instead, which distinguishes "never
charged" from "charged and released", and say so in the comment.
Finally, the boundary-before-execution ordering was pinned only at
admitFrostDKGAttempt; the caller that has to honour its failure was untested
because executeFrostDKGIfPossible first requires InteractiveSigningReady(),
whose engine-provider registration takes an unexported pkg/frost/signing
interface no other package can satisfy. Route that one gate through a package
variable production never reassigns, and test the caller itself: a boundary
that cannot be persisted must leave the event unhandled, must not reach anchor
admission, and must not start the attempt goroutine. A positive control on the
same fixture proves the goroutine probe fires when the attempt IS admitted, so
the negative case is a real absence rather than a dead detector.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The HTTPS round-trip test left a misaligned struct field that the client-format CI job rejects. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…le dependencies Two follow-ups to the first round of FROST review fixes. The 8 MiB journal file bound was reachable by ordinary canonical history, so mirroring it onto the write path turned a latent post-restart failure into an immediate one. A batch carries the suffix of a single complete-history read, which reconcile() caps at 4096 mutations, and one mutation with every field at its widest accepted value marshals to 10248 bytes - encoding/json renders each [32]byte as an array of decimal numbers, and a maximum-length reason of control characters escapes six bytes to one. The full retained-wallet set admitted with 100-seat groups and then transitioned is 4096 ordinary mutations that serialize to about 13 MB in one batch, so a healthy mainnet node could hit the cap on write and, before the write bound existed, publish a file no later initialize() would reopen. Neither has a recovery path on an append-only journal. The bound is now derived from the mutation count and per-mutation width the producer already enforces, read and write still share the one constant, and a replayed batch is held to the same mutation count so the derivation stays exact. The authorization monitor still latched permanently on transient failures. The cached readiness path now waits for the journal lock, but the cache-miss route runs a whole readiness reconciliation - paginated canonical-history reads and an anchor read - every time the finalized point advances, and the monitor cancels the signing session on the first error revalidation returns. A single RPC timeout during a multi-minute signing window therefore killed a session nothing was wrong with. Revalidation now retries a dependency that is merely unreachable, under the caller's context, for a bounded budget. Only transport-level causes qualify: a raised quarantine, a rollback, a fork, a rewritten proposal or a moved readiness stamp is a comparison against data that was read successfully, so it returns on the first attempt and still latches. A pass never substitutes a stale answer for a fresh one, and an outage that outlives the budget is reported so the session still fails closed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…r anchor The state-anchor barrier authenticates the remote anchor before every request-taking FFI call, and until now any failure of that check poisoned the package-global barrier. Poisoning is process-lifetime - only a restart clears it - so an anchor-service redeploy, a TLS or DNS hiccup, a reset connection or a momentary load spike permanently disabled FROST signing on the node, over a fault that healed on its own. The single remote read behind that check had no retry anywhere either, so one lost packet was enough. The pre-operation check now classifies its failure. Reaching it means the local readback and trust head already matched the installed identity, no FFI symbol has been entered and no Rust generation has advanced, so a failure to REACH the service cannot have diverged anything: it takes the barrier's existing non-poisoning refusal, releases the mutex, and the next call re-reads and re-authenticates from scratch. Only a positively recognized transport cause qualifies - the same set pkg/tbtc's pre-sign monitor uses, duplicated rather than imported because pkg/tbtc imports this package. Everything else still poisons, because a stale, rolled-back, forked or unauthenticated anchor is a comparison against data that was read successfully and genuinely means it is unsafe to proceed. Both branches fail closed for the operation in hand; the only difference is whether the next one may try again. The post-operation commit is deliberately NOT declassified, and now says so. By then the native call has advanced durable Rust state, the CAS is not idempotent, and the anchor client has already exhausted the only safe disambiguation of an ambiguous CAS, so a transport failure there is indistinguishable from the rollback the anchor exists to catch. Releasing without poisoning would also gain nothing: the committed process tip would mismatch on the next call and poison there instead, one call later and without the original cause. Recovery for a local-ahead state is a restart under reconcileStartup, not a retry. The authenticated read itself now rides out a single blip. Up to three attempts with a doubling 50ms backoff share ONE existing request-timeout budget, so total wall time is exactly what a single attempt could already take. Only unreachability is retried - transport causes plus the two statuses a restarting service or an intermediary answers with while it cannot answer at all. Anything the service actually said is deterministic and still surfaces on the first attempt. The CAS is left alone. Finally, the poisoning is now observable at its source. Nothing was logged where it fires; the cause only reached a log at WARN from one caller per attempt. recordNativeTBTCSignerStateAnchorPoisoning is now the single place the barrier latches, logs one ERROR naming the cause and the remedy - the barrier re-reports the same cause on every later call, so a line per refusal would flood - and keeps the first cause rather than a later consequence. NativeTBTCSignerStateAnchorPoisoned exposes that state to health and admission callers through a lock-free mirror, because the barrier mutex is held across a whole signing operation and a health probe must not stall behind one. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Five Go source strings tell an operator that "offline anchor rotation is required". Nothing in docs/ said what that is, how often it is needed, how to see it coming, or how to perform it - and there is no `tbtc-signer anchor rotate` subcommand to discover by trying. Derive the cadence from the code rather than assuming rotation is rare. frostPreSignAnchoredWorkflowCost reserves inputs * (1 + attempts * (4*seats + 1)) revisions and twice that plus three generations against two 4096-entry windows, so at 21 inputs and five attempts a 4-seat node reserves 3615 generations while a completed sweep consumes only a few hundred. That is one to two full-size sweeps per anchor epoch at 4 seats, and the refusal arrives with 3508 of 4096 generations still unspent. State the gaps instead of papering over them: both headroom values are readable only from the loopback activation-handshake JSON and appear in no metric or log; the certificate chain must start at sequence one and holds at most 64 entries, capping a signer store at 63 rotations for its lifetime; and no code here produces a rotation certificate, only a bootstrap one. Placed under docs/development/ with the other FROST operator documents, since docs/operations/ does not exist and the readiness manifest already records the decision not to create it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…chor The state-anchor barrier guards every request-taking call into the native tBTC signer, and once it latches its terminal failure the node keeps running while refusing every one of those calls until it is restarted. None of the terms of frostActivationHandshakeHealthy moved when that happened: interactive readiness is opt-in configuration plus a registered engine, and the whole native signer snapshot is read through no-arg entry points - state witness tip, retained key package inventory, anchor trust head - that never enter the barrier's request-taking path. A poisoned member therefore signed an attestation asserting health while it could not contribute a single signature share, so a permissioned FROST set could stop reaching threshold with every member attesting healthy and no node reporting a cause. Add the barrier verdict to the signed payload as nativeSignerState.stateAnchorPoisoned and make it a health term, so a consumer reads why the verdict flipped rather than only that it did, and can recompute the verdict from the transcript it verified. The verdict is read live at the signing boundary rather than taken from the finality-keyed reconciliation cache, which is refreshed only when finality advances and would otherwise attest away a poisoning that happened since. It is re-read under the signing lock, because the window between building the payload and signing it spans the checkpoint self-verification and one activation point check over the network; a latch inside that window refuses the attestation as retryable instead of signing a health claim the node has already contradicted. The latch is one-way and sticky until restart, so the retry answers with the poisoned verdict rather than flapping. The node also logs the barrier cause and the restart remedy on every attestation it answers while poisoned, since the handshake endpoint is the only place this fact is reachable today. Adding a field to the signed payload bumps the handshake schema to v5, as v2 -> v3 and v3 -> v4 did before it: an auditor pinned to v4 is refused rather than handed a payload whose health verdict it cannot recompute. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The state-anchor admission controller is the single chokepoint for pre-sign, native DKG and DKG retirement reservations, and it could not see the one failure that matters most. currentRestartableHeadroom reads the witness tip through the inventory bridge and authenticates it against the anchor service; neither path takes the request-taking barrier, so a barrier that has already latched a terminal failure still reports perfectly healthy headroom. A poisoned node therefore kept accepting workflows whose every signer call was already doomed, and the wallet lost that member's seats with no node reporting a cause. Refuse in reserve() while frostsigning.NativeTBTCSignerStateAnchorPoisoned reports a latched failure, before the headroom read and outside the admission mutex: one atomic load is cheaper than an anchor round trip, and nothing clears poisoned in-process, so no later headroom value can make the workflow admissible. All three reservation paths share reserve(), so DKG and DKG retirement are covered too. The refusal names the process restart, because that is the only remedy and it is not guessable from the message. Name the remedy in the unreserved-headroom refusals as well. They are the first refusal a healthy, correctly configured node produces - a four-seat node is refused its next maximum-size sweep with 3508 of 4096 generations still unspent - so the numbers read healthy and the refusal looks transient. Neither certified window refills on its own, so both messages now say that offline anchor rotation is required, matching the sibling rotation-floor refusal and docs/development/frost-anchor-rotation.adoc. The doc path itself is not cited: no production error string in this tree references docs/. Add frostNativeSignerAnchorWorkloadRotationWarning, which measures headroom against the largest workflow this node can actually be admitted for rather than against the flat 256 floor. The flat floor is kept as a hard lower bound and the workload term is added to it, not substituted: 256 guards the rotation-blocked refusal, where all new work has already stopped, so it fires roughly 3359 generations after a four-seat node stops being able to sign a maximum-size batch. Each window is compared against its own dimension's cost, and the measuring stick is the largest ADMISSIBLE batch so that a node above the seat ceiling is not warned - and marked unhealthy - permanently for an exclusion no rotation fixes. The local seat count is a parameter because this layer has no view of the node's sortition result. Add rejection counters keyed by cause - seat-ceiling, unreserved headroom, rotation floor, poisoned - following the roast_interactive_signing_metrics.go pattern, so a monitoring system can alert on any non-zero seat-ceiling or poisoned value. Nothing anchor-related was registered with clientinfo before, so a node refusing every signing request was indistinguishable from one that had not been asked. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ivation Two conditions stop this node from signing without stopping the process, and neither was observable from outside the box. The state-anchor barrier latches terminally poisoned and refuses every later request-taking native call until a restart; the certified restart windows drain until admission refuses work. In both cases the node keeps running, keeps its wallet seats, and keeps attesting healthy on the signed activation handshake. In a permissioned FROST set several members can be in that state at once while every one of them attests healthy, and the only symptom is a wallet that stops reaching its signing threshold with no node reporting a cause. Three gauges and a counter are now registered with clientinfo, next to the ROAST and interactive-signing counters and on exactly the same terms: not gated on FROST being active, because a source that only appears once the gated path runs reports nothing during the window an operator is trying to observe. frost_native_signer_anchor_poisoned reads the barrier's lock-free mirror, so it is authoritative in every build the moment it is registered. The two headroom gauges publish the restartable revision and generation counts that until now existed only as JSON on the activation-handshake endpoint, whose validator requires a loopback host and so cannot be scraped from a monitoring host at all. The headroom pair is mirrored rather than recomputed on the scrape. Every accessor that can answer the question from live state goes through currentRestartableHeadroom, which reads the local tip and then authenticates it against the remote anchor while holding the anchor binding mutex - the same mutex CommitNativeTBTCSignerStateTransition holds across the remote CAS for the whole of a signing commit. Putting that on the clientinfo tick would stall the scrape behind an in-flight signing operation and put a network call on a timer, so the node publishes what the readiness reconciliation already computed instead. That is a decorator on the readiness verifier, so it covers the startup verification and every later pre-sign authorization without a second reader. Only successful reconciliations publish: zero is the value that means the windows are exhausted, and an unreachable anchor service must not be reported as that. headroom_observations_total exists so "nothing has ever reported headroom" is distinguishable from "the windows are empty" - both read as zero on the gauges, and alerts must require the counter to be non-zero. The seat ceiling is now stated when the wallet executor is built rather than weeks later. A node whose local seat count exceeds what anchor admission can serve for a maximum-size batch is refused every full-size deposit sweep for that wallet's whole life, and the refusal is whole-node - reservePreSign is charged for the complete local seat set at once - so all of its seats are lost to the wallet's signing threshold, not just the surplus ones. Under production parameters the ceiling is four seats: five seats tops out at 19 inputs and six at 16, which are ordinary sortition results. Until now the only place that said so was the text of an authorization the node had already been refused, which an operator first sees when a 20-deposit backlog forms. The warning names the seat count, the ceiling, the batch size that produced it, and the largest batch this node can actually sign, and it is emitted only for wallets that sign with Schnorr, since a FROST-enabled node builds a gate for its draining ECDSA wallets too. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
RegisterFrostNativeSignerAnchorAdmissionMetrics was defined and unit tested but called from nothing, so the admission controller's refusal counters incremented internally and never reached a scrape - the exact failure the comment above this block warns about for the counters that came before them. The two causes worth alerting on are the ones an operator cannot infer any other way: a seat-ceiling refusal, which excludes a node from every full-size sweep for the life of the wallet, and a poisoned-anchor refusal, which means the node has stopped accepting work until it is restarted. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The runbook was written against the v4 handshake schema, which the poisoned-anchor health term has since replaced with v5, so its worked example sent a request the node now refuses. It also described only one way for a node to stop signing. There are two, they present identically from outside - a running process answering everything except signing requests - and the ceremony in this document fixes only one of them. An operator who reaches for rotation on a poisoned anchor performs an offline signing ceremony and is no better off. Name the second failure, the string that identifies it, and the fact that only a restart clears it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The runbook implied a restore from backup was the threat. It is not: nonces are never written to disk, and interactive_session_open zeroizes any prior nonce on every new attempt, so a file-level restore plus a restart destroys the spent nonce rather than resurrecting it. The threat is a memory-inclusive rollback - a VM snapshot, a CRIU checkpoint, a live-migration rollback - which is the only thing that brings a consumed nonce back. Round 2 pins the message and the member's own commitment but not the signing subset, so a resurrected nonce signed against enough distinct subsets recovers the key share. An operator who does not know that will snapshot a running signer. Also correct the per-call cost table's preamble. The barrier skips the remote CAS when the Rust tip is unchanged, so Round1 - which persists nothing - spends neither a revision nor a generation. The table is an upper bound that the admission reservation is charged against, not real consumption. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Pre-sign anchor admission charged a whole batch's worst case up front:
revisions = inputCount * (1 + attempts*(4*seats + 1))
generations = 2*revisions + 3
At production parameters (21 inputs, 5 signing attempts) that admitted
at most FOUR local seats against the 4096-entry certified proof window.
Mainnet runs a hundred-seat wallet across roughly twenty operators with
replacement sortition and no per-operator cap, so the average operator
holds five seats: most of the stake-weighted seats were refused every
full-size sweep, wallets formed normally and then could not move the
deposits they had already received.
The arithmetic was wrong rather than merely tight. A batch signs its
inputs strictly sequentially, so it never needs more than one input's
worth of unconsumed window at any instant. Charge one input instead:
cost = 2*(1 + attempts*(4*seats + 1)) + 3 == 40*seats + 15
That is 4015 generations for a wallet's entire hundred-seat set, so the
unchanged 4096-entry window covers every protocol-legal seat count and
the ceiling disappears rather than moving. The unit is an input rather
than an attempt so an admitted input keeps its whole retry budget.
signBatchWithTaprootPolicy's per-input loop now takes and releases the
reservation for each input through a new admitInput closure threaded
alongside the authorization guard. The per-input body moved into
signBatchInputUnderAnchorAdmission so the release is a defer and covers
the admission failing, the policy binding failing, the signing failing,
context cancellation and panic unwind alike. admitInput is a method on
frostPreSignAuthorizationGate, so no signing path or test double can
reach native signing without one, and the Schnorr precondition check
refuses a batch that arrives without it.
authorize() keeps its reservation, sized at one input, so the on-chain
relay stays gated by admission; walletTransactionExecutor releases it
as the per-input admissions take over, because holding both would
charge two inputs for one input's work and reinstate a ceiling at fifty
seats. Release is sync.Once-guarded and the existing deferred release
still covers every earlier exit.
What is given up is the whole-batch promise: a batch can be admitted
for input 0 and refused at input 7. That refusal is fail-closed - the
input's reservation is released, nothing is partially applied, no share
is produced, the wallet action fails naming the rotation remedy - and
it is now separately countable as
frost_native_signer_anchor_admission_pre_sign_input_rejected_total,
which is the difference between a batch refused before its relay and
one abandoned after it. The batch-wide charge never prevented the
window running out mid batch; it prevented most operators from
starting one.
DKG and DKG-retirement reservations are untouched. Batch size is no
longer a lever, so the seat-ceiling advice, the startup warning and the
rotation runbook drop it and say so; the ceiling guard itself stays,
unreachable under the shipped constants, to catch a future change to
the windows, the attempt limit or the seat cap.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The per-input arithmetic section already says Round1 spends nothing; the terms table still claimed every request-taking call advances a revision. Make them agree. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Based on the frost-schnorr migration scaffold. Draft — the keep-core side is complete and tested; activation remains gated on the external anchor-service deployment.
Go consumers for the Rust native signer (keep-core #4198, pinned via
ci/frost-signer-pin.envat ABI 4.3): retained-inventory/state-witness consumers, signing readiness, an authenticated state-anchor Read/CAS client with admission control, offline-authority trust-transition startup with crash recovery, and the anchor bootstrap ceremony (CLI + HTTP initialize client).Since the previous head (67e591f)
The three "NOT done" items from the old description are resolved in this lineage:
New on top of the restacked lineage:
state-anchor trust-recoveryerror from the engine selects an exact durable-intent suffix from the authenticated sequence-one artifact; the recovered result must be an exact idempotent replay at the recovered certified floor (recoveryReplayvalidator branch, fully tested).tbtc-signer anchor bootstrap {facts,core,initialize,finalize}— strict bootstrap-facts decode, offline core certificate, detached signatures, an idempotent signed initialize POST (applied/already-appliedsentinel, mandatory fresh signed exact-read reconciliation, divergence poisoning), and a round-trip-validated output bundle.Verification
go build ./...,go vet, and fullgo test ./pkg/tbtc/ ./pkg/frost/... ./cmd/... -count=1green at the pushed head;-tags frost_nativevet clean. The cgo integration gate pins the #4198 signer headb35637ed01693014b6b7ac94586541538eaaa446.