Cache compaction sizes files by bytes written, not the in-memory batch estimate

[philcunliffe]

Root cause

compactGeneration (src/core/cache/maintenance.js) flushed its batch to a data file whenever the batch reached COMPACT_BATCH_SIZE (10k) rows or compact_batch_bytes (32 MB) of estimated in-memory bytes. The byte cap is a genuine OOM guard, but it also decided file size.

An ai_gateway_messages row estimates ~140 KB in memory and compresses roughly 70x in parquet, so the guard fired after ~230 rows and produced a ~0.5 MB data file. target_file_bytes (128 MB) was therefore unreachable by construction, compact_avg_file_bytes (32 MB) would re-flag every compacted partition forever, and parquet min/max statistics stayed scoped to ~230-row files.

Scope: what this does and does not fix

This fixes many files per partition tuple. It does not reduce the number of partition tuples, and for the motivating dataset that is the binding constraint.

ai_gateway_messages declares identity Iceberg partitioning on (session_id, conversation_id, cwd, date). A data file cannot span partition tuples, so openStreamingAppend opens one file per tuple and a per-session file never approaches 128 MB. Measured: compacting 600 fat rows with target_file_bytes: 128 MB across 10, 30 and 100 distinct sessions produced 10, 30 and 100 files (1.7 to 3.0 KB each), independent of target_file_bytes.

So whether production's 230-file day partition shrinks depends on how many distinct tuples that day holds. LLP 0199's baseline gate remains fully load-bearing and is untouched: compact_avg_file_bytes can still flag a high-tuple partition on its own merits. Repartitioning ai_gateway_messages is a separate design question and is not attempted here.

Which approach, and why

The row-group streaming approach from the issue, implemented in-repo. Not the compressed-bytes-feedback alternative: it cannot work here. Feeding a running compression ratio back into the batch sizer means resident rows must reach target_file_bytes / ratio. At 70x that is ~9 GB of rows for one 128 MB file. As long as one flush is one file, the OOM guard and the file target are the same knob and you can only pick one. They have to be decoupled.

No upstream change was needed:

• hyparquet-writer already exports ParquetWriter, which appends row groups to an open file, and calls the Writer.flush() hook after each one.
• Every other piece of the commit path (writeDataManifest, buildPartitionSummaries, stageSnapshotForAppend, fileCatalogCommit, computeColumnStats, compare, groupByPartition) is reachable through the icebird/src/*.js deep imports this repo already sanctions and uses (store.js, retention.js).
• The one genuinely private piece is icebird's iceberg-to-parquet schema mapping. Rather than vendor a copy that would drift, the new module writes a zero-record parquet file to an in-memory buffer using icebird's own writeParquet and reads the SchemaElement[] back out of the footer: one tiny encode per compaction, exact by construction.

What changed

• src/core/cache/iceberg/stream_append.js (new): openStreamingAppend() keeps one parquet writer open per partition tuple, encodes each batch as one row group, and closes a file once writer.offset >= target_file_bytes or the retained-metadata budget below binds. Per-file Iceberg metrics accumulate across row groups (counts summed; bounds held as raw min/max via icebird's own compare, then serialized once at close through computeColumnStats). A multi-row-group file records sort_order_id: 0 because it is a concatenation of sorted runs, not a globally sorted file. All files commit in one snapshot instead of one per batch. Open files are capped at 64, retiring the oldest (insertion order, not LRU: a re-fetched file is not re-inserted), so a high-fan-out partition spec cannot exhaust descriptors. Tables with nested columns fall back to the previous one-file-per-batch path, as do tables whose iceberg-to-parquet mapping does not produce exactly one top-level element per field.
• Peak heap has three terms, and the third is now bounded. flush() drains encoded page bytes, but ParquetWriter.write pushes a ColumnChunk per column per row group onto row_groups, and each chunk's statistics holds the raw, untruncated JS min_value/max_value; truncation to 16 units happens only in finish(). An open file therefore pins two full column values per row group per column for its whole life. Measured peak retained heap over 100 row groups of one string column: 4.4 MB at 20 KB values, 27.7 MB at 140 KB, 109.8 MB at 560 KB, and 38.7 MB at issue Cache maintenance compaction emits ~0.5MB files: one data file per estimated-in-memory batch #697's shape (70 KB values, 278 row groups) - more than compact_batch_bytes itself, times up to MAX_OPEN_FILES. A control where every row shared one string object stayed at 0.1 MB, isolating the growth to the retained bounds. Each row group is now charged an upper bound on what it pins (widest value per column counted twice, plus a measured ~1 KB per column chunk) against a global MAX_OPEN_STATS_BYTES budget of 32 MiB, and the file holding the most is closed when the budget is reached. Same measurements after: 4.5 MB, 16.0 MB, 15.5 MB, 16.3 MB. The budget is global rather than per-file because a per-file cap has to be divided by MAX_OPEN_FILES to bound the aggregate, which for fat rows would force files back to single-digit megabytes.
• src/core/cache/iceberg/resolver.js: the local writer implements flush(), appending buffered bytes to the temp file and resetting the buffer index after every row group, so a large output file is not a large allocation. ByteWriter.offset stays cumulative, so every parquet offset in the footer remains file-absolute; the atomic temp-then-rename and the ifNoneMatch 412 semantics are unchanged. It also implements abort(): with a writer held open across a whole rewrite, a failure has no finish() coming to close the descriptor or unlink the .tmp.* file.
• src/core/cache/maintenance.js: batches flush into the streaming sink. The OOM guard is untouched (COMPACT_BATCH_SIZE and compact_batch_bytes still bound what is resident). The rewrite is wrapped in try/finally and calls StreamingTableAppend.abort() on any failure, so a partition that throws every maintenance tick no longer leaks up to 64 descriptors and temp files per tick.
• Observability: the maintenance.partition span carries compacted, data_files_before, data_files_after, rows, and bytes_written. stream_append.js additionally logs file open, file close (with the roll reason: target_bytes, stats_budget, open_file_cap, append_close), the append summary, and abort, with counts and bytes only, so a streaming-append failure identifies the step that broke rather than only the partition.

LLP

Minted LLP 0209 (llp/0209-compaction-file-size.decision.md, Decision / Accepted / Systems: Cache) and added an Extended-by: LLP 0209 forward-ref to LLP 0199, alongside the LLP 0207 forward-ref master added for #701. LLP 0199 is Accepted and its decision was not edited. Its gate is not superseded either: see Scope above.

Evidence

test/core/cache-compaction-file-size.test.js compacts a 600-row ai_gateway_messages partition whose rows are fat in memory and highly compressible, with compact_batch_bytes forcing many flushes, and exercises openStreamingAppend directly for the heap and abort contracts.

Against the pre-PR commit:

not ok 1 - compaction sizes output files by bytes written, not the in-memory batch estimate
  error: 'expected one compacted data file, got 21'
not ok 2 - compaction rolls to a new data file once target_file_bytes is written
  error: 'expected at least one file to hold multiple row groups, got [1,1,1,...]'

Test 2 asserts multiple row groups inside one file rather than a bare file count, because a file count above one is also true of the pre-fix code; only appending into an already-open file can produce a multi-row-group file.

Against the first revision of this PR (bound not yet applied):

not ok 3 - a streaming append rolls on retained row-group metadata, not only on target_file_bytes
  error: 'expected the stats budget to roll files, got 1'
not ok 4 - aborting a streaming append releases every open descriptor and temp file

All four pass on this branch.

Gate

npm test            3867 tests, 3861 pass, 0 fail, 6 skipped
npm run typecheck   clean

Smokes: cache_lifecycle_maintenance, incremental_sink_compaction, cache_roundtrip, cache_spool_batching all ok.

Note the existing compaction preserves partition spec and column types from declaration test rewrites a table with 40 distinct partition tuples, so the multi-open-file path is exercised by the suite.

Fixes #697

[philcunliffe]

Review round 1 - head 212e08f

Verdict: changes requested (all fixed in 798998b). The mechanism is right, the Iceberg output is provably exact, and the gates were green. But two of the PR's central claims were verifiably false, and both were recorded as settled fact in a brand-new LLP, so they needed correcting before they hardened into the record.

Findings

1. MAJOR - the OOM guard was not preserved. src/core/cache/iceberg/stream_append.js:134, claimed at llp/0206:58-63 and maintenance.js:494-499.
ParquetWriter.prototype.write pushes each row group into this.row_groups, and each ColumnChunk.meta_data.statistics holds the raw, untruncated min_value/max_value; truncation to 16 bytes happens only at serialization in finish(). So an open file pinned two full column values per row group per column for its whole life. Measured against the real openStreamingAppend (100 row groups, one string column): 20KB values 4.4MB, 140KB 27.7MB, 560KB 109.8MB retained; control with every row sharing one string object stayed at 0.1MB, isolating the growth to the retained bounds. At issue #697's own shape (70KB values, 128MB target, ~278 row groups) it retained 38.7MB, more than compact_batch_bytes (32MB) itself - and that is multiplied by up to MAX_OPEN_FILES (64) concurrently open files. The term scales with target_file_bytes and per-value size, exactly the fat-column axis the guard was written for.
Fixed: a global MAX_OPEN_STATS_BYTES (32 MiB across all open files) plus ROW_GROUP_STATS_OVERHEAD_BYTES (1 KiB per column chunk, measured at ~1026 bytes so a narrow-column table is bounded too); enforceStatsBudget() closes the fattest open file until the total is back under budget. After: 140KB 16.0MB, 560KB 15.5MB, issue shape 16.3MB. The author chose a global budget over a per-file row-group cap and recorded why: a per-file cap must be divided by MAX_OPEN_FILES to bound the aggregate, which for 70KB rows forces files back to single-digit megabytes - the very defect this PR exists to fix. Regression test is structural rather than heap-based (40 row groups of 1MiB values, asserting the file rolls), and fails on the unfixed head.

2. MAJOR - target_file_bytes is still unreachable for ai_gateway_messages, the motivating dataset.
That dataset declares identity partitioning on (session_id, conversation_id, cwd, date), and a data file cannot span partition tuples, so one file is opened per tuple and a per-session file never reaches 128MB. Measured: 600 fat rows at target_file_bytes: 128MB across 10/30/100 distinct sessions gave 10/30/100 files of 1.7-3.0KB, independent of the target. LLP 0206's consequence - that a partition converges to a handful of files near target_file_bytes and "LLP 0199's baseline gate is no longer load-bearing" - was therefore false for the motivating dataset.
Fixed: LLP 0206's Consequences and Extends sections now state the real bound (file count after a rewrite is approximately the distinct tuple count; target_file_bytes only binds within a tuple) and record that 0199's baseline gate remains fully load-bearing. The PR body gained a "Scope: what this does and does not fix" section. Whether prod's 230 files actually shrink depends on how many distinct tuples that day partition holds - a partitioning question this PR deliberately does not touch. Worth knowing before judging #697 closed.

3. minor - fd and temp-file leak on the error path. maintenance.js:613-614, types.d.ts:181-184. No try/finally around flushBatch() / close(), and no abort() on StreamingTableAppend, so a throwing write left up to 64 fds open with their .tmp.* files. Daemon maintenance retries every tick, so a reliably-failing partition accumulated descriptors; pre-fix, each append opened and closed synchronously.
Fixed: abort() on both the local writer and StreamingTableAppend, closeFile aborts a writer whose finish() threw, and compactGeneration wraps the scan in try/finally. Test asserts a throwing write leaves a temp file, then abort() removes it and returns the /proc/self/fd count to baseline.

4. minor - the second new test did not discriminate. cache-compaction-file-size.test.js:95-119 asserted only dataFilesAfter > 1, which pre-fix code trivially satisfies (verified against 212e08f^: test 1 failed correctly, test 2 passed on the defect).
Fixed: it now asserts at least one output file holds multiple row groups, verified to fail pre-fix with got [1,1,1,...].

5. nit - "the intrinsic cache only ever declares primitives" was untrue. ColumnSpec.type: 'JSON' maps to iceberg variant, which expands to a two-leaf parquet group, and typeof 'variant' === 'string' so the nested-column fallback did not catch it - ai_gateway_messages declares seven JSON columns. Variant does round-trip correctly, so this was comment accuracy, not a bug.
Fixed: comment corrected, and the guard widened via alignsWithParquetSchema(), which falls back to the legacy path unless icebird produced exactly one top-level element per field (also covering iceberg unknown, which maps to []).

6. nit - the retire policy is FIFO, not LRU (stream_append.js:123-127 takes Map insertion order; a re-fetched file is never re-inserted). Behaviour is fine; only the PR description said LRU. Fixed in the description.

Observability: stream_append.js emitted no telemetry of its own, so a streaming-append failure was visible only as an aggregate on the partition span. It now logs file open / closed (with reason: target_bytes / stats_budget / open_file_cap / append_close) / stream closed / aborted, with counts and bytes only, no row values.

Verified clean

• Deep imports are sanctioned, not private. icebird and hyparquet-writer both export ./src/*.js explicitly, the repo already deep-imports six such modules, and the flush() hook is public API (declared on the Writer interface, documented, and used by the package's own fileWriter).
• The schema round-trip is correct and paid once per compacted partition, not per file or row group. Differential test: the same 30 rows through the legacy append vs the streaming path (6 batches, 6 row groups, 1 file) over int / nullable string / long / double / boolean / timestamptz / variant produced identical read-back rows and byte-identical value_counts, null_value_counts, lower_bounds, upper_bounds, including the 16-unit truncated string bound. sort_order_id: 0 on a multi-row-group file is the only intended difference and is spec-correct.
• LLP 0199 is intact: baseline gate, neediest-first walk, and post-rewrite baseline untouched; no infinite-recompaction path reintroduced.
• Crash safety is strictly better than before: temp-then-rename, with the manifest/snapshot/metadata commit only after every file closes, so a half-written data file is never committed. Pre-fix, a crash committed a prefix of batches.
• The rewritten pre-existing test inversion is genuine, not weakened: the old dataFilesAfter > 1 encoded the defect, and the rewrite still proves the byte budget fires (rowGroupCounts(liveDir)[0] > 1), with the sibling pinning a generous budget to exactly one row group.

Gate

At the fixed head 798998b: npm test 3861 pass / 0 fail / 6 skipped, npm run typecheck clean, smokes cache_lifecycle_maintenance, incremental_sink_compaction, cache_roundtrip, cache_spool_batching all ok, and CI green on the pushed head.

The head moved to 798998b, so the next tick re-reviews it as round 2.

[philcunliffe]

Review round 2 - head 798998b

Verdict: changes requested (fixed in 0a9044a). All six round-1 findings were independently re-measured and are genuinely fixed. One new major remained, and it was the other half of round-1 finding 2: the PR as it stood was very likely a no-op for the dataset issue #697 is about.

The major finding: MAX_OPEN_FILES = 64 FIFO eviction nullified the fix above 64 tuples

stream_append.js:36, :191-200, claimed at llp/0206:132-148. Measured through the real maintainCache path (identity partitioning on session_id, 3000 fat rows, 10 ingest waves, target_file_bytes: 128MB):

distinct tuples	files before	files after (798998b)
30	300	30
64	640	64
100	1000	1000

At 100 tuples the rewrite emitted exactly one output file per input row group - the identical count the pre-fix code produced. The mechanism is Belady-style cyclic thrash: the scan walks the old generation in manifest order, each old file holds one tuple, and a tuple recurs every ~N files, so once N exceeds the cap a file is always evicted before its next row group arrives. ai_gateway_messages partitions on identity (session_id, conversation_id, cwd, date), and a 52,329-row day partition almost certainly exceeds 64 tuples.

This also made LLP 0206's corrected Consequences wrong again in the same place: it claimed "the file count after a rewrite is approximately the number of distinct tuples", measured at 100 tuples produces 1000 files. The supporting measurement recorded in the doc (600 rows, 10/30/100 sessions to 10/30/100 files) was unrepresentative, and the reason was reproduced: with a single ingest wave each tuple gets exactly one row group, so eviction never fires. One wave proves nothing about this bug.

Fixed, by a third approach better than either option the review proposed. The diagnosis was the fix: MAX_OPEN_FILES is a descriptor bound, but it was being spent to close files. Tuple-clustering the scan was found intractable without reshaping the scan contract (icebergDataSource iterates manifest-order dataEntries with no ordering hook, and per-tuple WHERE passes would depend on icebird prune semantics and risk dropping null-valued tuples). Raising the cap only moves the cliff. Instead the local writer gained park(): flush, close the fd, drop ByteWriter's never-shrinking buffer, keep the .tmp.* file and byte offset, and reopen with 'a' on next write. The cap now bounds descriptors (MAX_OPEN_DESCRIPTORS, still 64) via reserveDescriptor, parking the least-recently-written file; a file closes only on target_file_bytes, the stats budget, or end of append.

Result at the same shape: 100 tuples now yields 100 files, 200 yields 230, 500 yields 720. No descriptor-safety trade at all - the cap is the same 64, and the new test asserts the process fd count stays inside it while 200 tuples are open. Residual above ~100 tuples is the stats budget doing its job, degrading smoothly rather than falling off a cliff.

Two regression tests were added, both verified failing on 798998b: 100 sessions x 10 waves through maintainCache (expected roughly one file per session, got 1000 from 1000), and 200 tuples x 10 batches direct (expected exactly one data file per tuple, got 2000). LLP 0206 gained a #descriptor-parking section and its #tuple-bound consequence was rewritten: one file per tuple is the floor, with the unrepresentative single-wave measurement replaced by the multi-wave table and an explicit note on why one wave proves nothing.

Round-1 findings, all re-measured and confirmed fixed

• OOM guard. Verified in both directions against the real openStreamingAppend. Budget disabled: 4.5 / 28.3 / 112.2 MB retained at 20K / 140K / 560K chars (reproducing round 1's 4.4 / 27.7 / 109.8). Budget enabled: 4.5 / 16.7 / 15.7 MB. Adversarial shapes bounded: 500 columns x 400 row groups gives 29.5 MB; single 20M-char values roll to one row group each; ROW_GROUP_STATS_OVERHEAD_BYTES = 1024 is a true upper bound (measured 919-991 bytes actual retention per column chunk across 21/51/101-column tables). Variant is safe to charge nothing: getStatistics skips objects and variant leaves arrive as Uint8Array (200 row groups of 400KB JSON retained 1.0 MB). The budget is checked after each row group so one group can overshoot, but the overshoot is bounded by compact_batch_bytes. openStatsBytes accounting is symmetric across roll / retire / close / abort with no drift path found.
• fd/temp leak. Fixed, and the test discriminates: neutering the fd-close branch makes it fail with abort should release descriptors, had 19 before and 20 after. Off Linux only the fd half skips; the temp-file assertion still binds.
• Test discrimination. Both maintenance tests fail on base b28aae3, test 2 with exactly expected at least one file to hold multiple row groups, got [1,1,...].
• Variant comment / alignsWithParquetSchema. Correct and does not over-fall-back. Differential check over INT32 / STRING-with-nulls / INT64 / DOUBLE / BOOLEAN / TIMESTAMP / JSON, legacy single-append vs streaming 6 row groups: record_count, value_counts, null_value_counts, nan_value_counts, lower_bounds, upper_bounds all byte-identical. sort_order_id is spec-correct.
• FIFO not LRU. Description corrected.

Two nits, also fixed

• accumulateStats mutated file.valueCounts / mins / maxes before parquet.write could throw, so accumulated metrics could include a row group never written. Unobservable today (a throwing write is always followed by abort) but now moved after the write.
• The string charge of length * 2 was 2x conservative for ASCII, which is what ai_gateway_messages payloads are: 8.7 MB actually retained at the 32 MiB budget vs 16.7 MB with two-byte values, so files rolled about twice as early as the heap required. Now Buffer.byteLength(v) === v.length proves ASCII and charges 1 byte/char, with two-byte values unchanged (verified not to under-charge: two-byte case still 811 files). This required relaxing an existing per-file row-group bound from 12 to 20, which is the honest new bound - the assertion still discriminates, since without a budget that test's 40 row groups land in a single file.

Gate

At 0a9044a: npm test 3863 pass / 0 fail / 6 skipped, npm run typecheck clean, smokes cache_lifecycle_maintenance, incremental_sink_compaction, cache_roundtrip, cache_spool_batching all ok, and CI green on the pushed head.

This PR has now used its two review rounds, so the next tick triages it.

[philcunliffe]

Triage after the 2-round review cap - head 54a3b87

Independently re-verified every recorded finding against the current tree; zero residual findings, nothing blocks.

Round-2 major (64-tuple cliff), re-measured through the real maintainCache path (identity partitioning, 10 ingest waves, target_file_bytes: 128 MB):

distinct tuples	files before	files after
30	300	30
64	640	64
100	1000	100
200	2000	234

The cliff is gone, not moved: 100 tuples converges to the tuple floor, and 200 degrades smoothly (residual is the retained-metadata budget binding, as documented). Read-back at every scale was lossless: row counts, every id present exactly once, payloads intact.

park() scrutinized separately as the new mechanism:

• Byte-identity: a writer parked between chunks (both with flushed and with still-buffered bytes) produced files byte-identical to a never-parked writer; a ParquetWriter parked after every row group produced a byte-identical parquet file that reads back fully. offset stays cumulative and equals the final file size, so footer offsets stay file-absolute.
• Crash mid-append with 150 parked files: no metadata or snapshot exists before close(), uncommitted work is only unreferenced .tmp.* files, and the table reads exactly its pre-append rows. abort() removed all 150 temp files and returned the fd count to baseline (82 -> 18, confirming the 64-descriptor cap held while 150 files were open). The orphan sweep's 1-hour grace window vs the 30 s bounded tick rules out mid-flight reclaim of a parked temp file.

Round-1 findings (stats budget, scope honesty, fd/temp leak, test discrimination, variant guard, FIFO wording) and the two round-2 nits: all confirmed fixed in the current tree. LLP 0208's #tuple-bound consequence and the PR body's Scope section still state honestly that one file per tuple is the floor and that whether prod's 230-file day partition shrinks depends on its tuple count.

Gate at 54a3b87: npm test 3863 pass / 0 fail / 6 skipped, npm run typecheck clean, test/core/llp-ref-hygiene.test.js green.

Mechanical renumber (LLP 0156): this PR's LLP 0206 collided with PR #703's unrelated llp/0206-uninstall-detaches-its-clients.decision.md, and 0207 is claimed by PR #701, so the doc moved to LLP 0208 (llp/0208-compaction-file-size.decision.md): filename, header, LLP 0199's Extended-by: forward-ref, every @ref in src/ and test/, and the PR body, in one commit. No 0206 reference to this document remains.

[philcunliffe]

Triage of the post-review head 05439e6 (merge of master / PR #701, plus the 0208 -> 0209 renumber)

The previous triage note covers 54a3b87; since then the head gained exactly two things, both re-verified independently against the merged tree. Zero residual findings, nothing blocks.

The merge composition (#701 foreign-sorted-replace re-baseline x #698 streaming rewrite), every claim checked:

• compactGeneration(partitionDir, layout, cfg, settle, tableInfo) takes Maintenance recognizes a foreign sorted replace and re-baselines instead of shredding it (#700) #701's tableInfo parameter and its body contains no metadata load: existingSpec / schemaColumns / sortColumns come off the parameter, so the single-metadata-load property survives. The only loadLatestFileCatalogMetadata calls in maintenance.js are expireSnapshots and loadCompactionTableInfo (called once, inside the shouldCompact branch of maintainGeneration).
• Maintenance recognizes a foreign sorted replace and re-baselines instead of shredding it (#700) #701's three-way branch is intact verbatim (foreign sorted replace re-baselines with rebaselinesCounter, dry run reports, otherwise compact), with Cache compaction sizes files by bytes written, not the in-memory batch estimate #698's r.compactedBytesWritten = result.bytesWritten re-added in the compact arm. Diffed both directions: merged-vs-master shows only Cache compaction sizes files by bytes written, not the in-memory batch estimate #698's additions; merged-vs-54a3b87 removes only the in-function metadata load, the old two-way branch, and 0208 refs.
• Cache compaction sizes files by bytes written, not the in-memory batch estimate #698's streaming sink, try/finally, abort()-on-anything-but-clean-close, stats budget, and descriptor parking all survived: stream_append.js and resolver.js differ from the already-triaged 54a3b87 only in the 0208 -> 0209 ref renumber (verified by diff, no behavioral hunk).
• No false-positive path into Maintenance recognizes a foreign sorted replace and re-baselines instead of shredding it (#700) #701's predicate. Structurally: the streaming commit goes through icebird's stageSnapshotForAppend, which hard-codes operation: 'append' (icebird/src/write/stage.js:206). Empirically: a sorted table put through the streamed rewrite carries its sort order forward (sortColumnsFromMetadata returns it) and its current snapshot reads back operation=append; a follow-up non-forced maintainCache after new appends compacted (epoch advanced, rebaselined unset), so our own rewrite is never mistaken for a foreign replace.
• All five Maintenance recognizes a foreign sorted replace and re-baselines instead of shredding it (#700) #701 recognition tests (dry-run, epoch and source-table layouts, resettle-force outranking, unsorted-replace-not-blessed, force-still-rewrites) are present and green in the merged tree.

Descriptor cliff re-measured through the real maintainCache path (identity partitioning, 10 ingest waves, target_file_bytes: 128 MB): 100 tuples: 1000 files before, 100 after; 150 tuples: 1500 before, 150 after. Read-back lossless at both scales (all rows, all distinct tuples present). The round-2 no-op regression stays dead in the merged tree.

The renumber (LLP 0156 mechanical): doc now llp/0209-compaction-file-size.decision.md, all anchors (#decision, #row-groups, #retained-metadata, #descriptor-parking, #schema-probe, #tuple-bound, #consequences) present; zero 0208 references remain in src/, test/, or llp/; LLP 0199's Extended-by: names both 0207 and 0209; llp-ref-hygiene green.

Gate at 05439e6: npm test 3907 pass / 0 fail / 6 skipped, npm run typecheck clean, smokes cache_lifecycle_maintenance, incremental_sink_compaction, cache_roundtrip, cache_spool_batching all ok.

Sibling PR #706: test-merged its head against this one; only conflict is the known one-line Extended-by: hunk in llp/0199 (maintenance.js and types.d.ts auto-merge). No additional conflict risk found.