Skip to content

Align fetched snapshot verification with local joins - #8403

Closed
cjen1-msft wants to merge 5 commits into
microsoft:mainfrom
cjen1-msft:defer-recovery-snapshot-verification
Closed

cjen1-msft wants to merge 5 commits into
microsoft:mainfrom
cjen1-msft:defer-recovery-snapshot-verification

Conversation

@cjen1-msft

@cjen1-msft cjen1-msft commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Joiners accept local snapshots signed by a previous service identity, but reject fetched copies because the fetch path checks the signer against the current service certificate.

This PR removes that extra check. Fetched snapshots use the same receipt checks as local join snapshots, both during recovery and after the service opens.

The production change is one line:

-verify_snapshot(segments, join_config.service_cert);
+verify_snapshot(segments);

Digest and Merkle proof checks remain. Snapshot contents are authenticated by GCM with ledger keys during full deserialisation, immediately for an open service or during private recovery. The configured service certificate still authenticates the connection to the service.

Before

flowchart TD
    F["Fetch snapshot"] --> D["Check digest and Merkle proof"]
    D --> S{"Signed by current service identity?"}
    S -->|Yes| J["Receive authenticated join response"]
    S -->|No| R["Reject fetched snapshot"]
    J --> I["Continue through existing join path"]
    I --> G["Authenticate contents with GCM during full deserialisation"]
Loading

After

flowchart TD
    F["Fetch snapshot"] --> D["Check digest and Merkle proof"]
    D --> J["Receive authenticated join response"]
    J --> I["Continue through existing join path"]
    I --> G["Authenticate contents with GCM during full deserialisation"]
Loading

Existing recovery trust assumption

This PR adopts the existing local-snapshot trust model. During recovery, joiners use public snapshot state before GCM authentication. The safety of those early effects, including network.secrets hooks, is a separate concern shared with the local path. This PR does not establish that safety or change those hooks.

Validation

  • logging builds.
  • recovery_snapshot_endorsements_test passes. The join scenario checks that local and fetched previous-identity snapshots are used during recovery and after opening, rather than falling back to ledger replay.
  • scripts/ci-checks.sh passes.
  • The earlier Sphinx build was blocked by Doxygen 1.17.0 rejecting the existing ccf::pal::ccf::ds::Mutex alias. It has not been rerun for this revision.

Authenticate snapshot state with trusted ledger keys before using its service identity to verify the receipt. Suppress initial snapshot secret hooks so unauthenticated state cannot supply recovery keys. Cover local and fetched snapshots, malformed receipts, and hook isolation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Fetched snapshots were rejected when not signed by the current service
identity, even during recovery when the primary only has snapshots signed
by its predecessor. Local snapshots were accepted in the same situation.

Keep the eager check, but on mismatch fall back to the digest and proof
check used for local snapshots and record that the signer is unverified.
When the join response shows the service is open, discard such a snapshot
and retry the join; when it is recovering, accept it as for a local one.

Also suppress network.secrets hooks while a public-only join snapshot is
loaded, so unauthenticated snapshot writes cannot seed private recovery.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@cjen1-msft cjen1-msft changed the title Defer recovery join snapshot signatures until private recovery completes Accept previous-identity fetched snapshots while recovering Sep 18, 2026
Use the existing local-join receipt checks for fetched snapshots, without requiring the current service identity. Remove the verification flag, retry handling, and separate hook-hardening changes. Cover previous-identity snapshots for local and fetched joins during recovery and after opening.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@cjen1-msft cjen1-msft changed the title Accept previous-identity fetched snapshots while recovering Align fetched snapshot verification with local joins Sep 18, 2026
@cjen1-msft cjen1-msft closed this Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants