Skip to content

Add a chunked encryption recipe implementing c2sp.org/chunked-encryption#15144

Open
alex wants to merge 5 commits into
mainfrom
claude/chunked-encryption-rust-dw7f80
Open

Add a chunked encryption recipe implementing c2sp.org/chunked-encryption#15144
alex wants to merge 5 commits into
mainfrom
claude/chunked-encryption-rust-dw7f80

Conversation

@alex

@alex alex commented Jul 3, 2026

Copy link
Copy Markdown
Member

Resolves #2358

This adds cryptography.chunked_encryption, a top-level recipe (à la Fernet) for streaming authenticated encryption of large messages, implementing the C2SP chunked-encryption specification instantiated with SHA-256 and AES-128-GCM:

from cryptography.chunked_encryption import Decrypter, Encrypter

key = Encrypter.generate_key()
enc = Encrypter(key, context=b"myapp backup encryption")
ciphertext = enc.update(b"a secret message") + enc.finalize()
dec = Decrypter(key, context=b"myapp backup encryption")
assert dec.update(ciphertext) + dec.finalize() == b"a secret message"

Design notes:

  • The core is implemented in Rust, on top of the existing AesGcm AEAD and HkdfExpand implementations. For each message a fresh AEAD key, base nonce, and 32-byte key commitment are derived with HKDF-Expand-SHA-256 from the input key, a random 24-byte salt, and a caller-provided context; the message is encrypted in 16 KiB chunks with AES-128-GCM, with the chunk counter XOR'd into the base nonce. The ciphertext is salt || commitment || chunks.
  • Buffering is minimal: full 16 KiB chunks are encrypted/decrypted directly from the caller's input into the output, so only sub-chunk remainders are copied into the internal buffer. update_into() variants let callers supply their own output buffers.
  • The internals are parameterized over the AEAD (IANA name, key/tag lengths) so other AEADs could be added later, but the public API is AES-128-GCM only, per the spec's recommended instantiation.
  • The decrypter checks the commitment before decrypting anything, releases plaintext only for authenticated chunks, and enforces the short-final-chunk truncation rule at finalize().
  • Auth failures raise cryptography.exceptions.InvalidTag; any further use of a finalized or failed context raises AlreadyFinalized.

Tests cover the test vectors from the chunked reference implementation (vendored into cryptography_vectors), plus round-trip, streaming-granularity, tampering, reordering, truncation, extension, and API misuse cases.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Uyk58oD6F8BJwKHE4qCMA8

reaperhulk pushed a commit that referenced this pull request Jul 3, 2026
These are the test vectors from the chunked reference implementation
(https://github.com/FiloSottile/chunked), for use by the chunked
encryption recipe being added in #15144.


Claude-Session: https://claude.ai/code/session_01Uyk58oD6F8BJwKHE4qCMA8

Co-authored-by: Claude <noreply@anthropic.com>
claude added 4 commits July 3, 2026 18:09
This adds cryptography.chunked_encryption, a top-level recipe (like
Fernet) for streaming authenticated encryption of large messages,
implementing the C2SP chunked-encryption specification
(https://c2sp.org/chunked-encryption) instantiated with SHA-256 and
AES-128-GCM.

The core is implemented in Rust: for each message a fresh key, base
nonce, and key commitment are derived with HKDF-Expand-SHA-256 from the
input key, a random 24-byte salt, and a caller-provided context; the
message is encrypted in 16 KiB chunks with AES-128-GCM, with the chunk
counter XOR'd into the base nonce. Full chunks are encrypted/decrypted
directly from the caller's input, so only sub-chunk remainders are
buffered internally, and update_into variants allow callers to supply
output buffers.

The internals are parameterized over the AEAD so that additional AEADs
could be supported later, but the public API is AES-128-GCM only.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uyk58oD6F8BJwKHE4qCMA8
- Use the existing AesGcm AEAD implementation for chunk
  encryption/decryption instead of using OpenSSL's EVP interface
  directly (AESGCM.decrypt_into is now pub(crate) for this).
- Use the existing HkdfExpand implementation for key derivation.
- Fold the error state into the finalized state: a context that hit an
  error raises AlreadyFinalized on further use.
- Replace the let-else in Decrypter::update_impl with a match that
  yields the cipher and buffer fields.
- Replace the in-test reference implementation with the test vectors
  from the chunked reference implementation
  (https://github.com/FiloSottile/chunked), vendored into
  cryptography_vectors.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uyk58oD6F8BJwKHE4qCMA8
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uyk58oD6F8BJwKHE4qCMA8
- Extract the chunk counter/nonce logic into a ChunkNonces struct with
  Rust unit tests covering the 2**38-chunk limit, which can't be
  reached from Python tests.
- Track the Decrypter state as Option<DecrypterState> and process both
  active states in a single exhaustive match, removing the
  unreachable!() arms; finalize() now consumes the state on all paths.
- Drop the Encrypter's error poisoning: the only errors it guarded
  against are the capacity check (which fails before any state is
  modified) and internal OpenSSL errors.
- Test that a Decrypter is unusable after update_into() raises
  InvalidTag.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Uyk58oD6F8BJwKHE4qCMA8
@alex alex force-pushed the claude/chunked-encryption-rust-dw7f80 branch from d94f3cf to 896f9cc Compare July 3, 2026 18:11
Comment thread docs/chunked-encryption.rst Outdated
Comment thread src/rust/src/chunked_encryption.rs
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Implement (or document) streaming API

2 participants