T6: exactly-once acceptance tests + smoke for incremental sink reads#158
Merged
philcunliffe merged 1 commit intoJun 26, 2026
Merged
Conversation
Add the LLP 0040 exactly-once proof the T4/T5 unit suites can't give (they stub storage): a deterministic acceptance test driving the REAL kernel cache, retention enforcer, maintainCache compaction, BOTH sinks (central forward + core local-fs blob), and the driver outbox respool. Covers: ~0 bytes on a no-new-rows tick, ~N on an N-new tick, exactly-once across a retention front-prune and a compaction generation swap for both sinks, and watermark vs. driver-outbox respool composition (suffix-only replay + idempotent batch-id / re-PUT). Adds a hermetic smoke proving the blob sink reads straight through a compaction generation swap via the real driver. Anchors LLP 0040 section 5 so the @ref resolves. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Task-Id: T6
35d77bd
into
integration/incremental-sink-reads
6 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements task T6 of the incremental-sink-reads change set (LLP 0042).
The exactly-once proof. T4/T5 ship comprehensive unit suites, but those stub
storage and so cannot prove the design's load-bearing claim: that a row-resident
_hyp_ingest_seqwatermark survives the two cache rewrites that motivate thewhole design (LLP 0039) — a retention front-prune and a compaction generation
swap.
What this adds
test/core/sink-incremental-acceptance.test.js— a deterministic acceptancesuite driving the real kernel cache (
createQueryStorageService), thereal retention enforcer, real
maintainCachecompaction, bothsinks (central
forward+ corelocal-fsblob via their ownactivate/create), and the real sink driver's outbox respool. Eight tests:
(asserts the generation directory actually swapped);
reads only the un-acked suffix with a stable
X-Hyp-Batch-Id;(idempotent overwrite, no duplicate blob).
hypaware-core/smoke/flows/incremental_sink_compaction.js— a hermeticacceptance smoke proving the blob sink (parquet + local-fs) reads straight
through a compaction generation swap via the real driver across three ticks.
{#exactly-once-argument}anchor to LLP 0040 §5 so the new@refs resolve.Checks
npm test(1446 pass / 1 pre-existing skip),npm run typecheck,npm run lintall green;
npm run smoke -- incremental_sink_compactionpasses.Task-Id: T6
🤖 Generated with Claude Code