diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index cb7337c0..3be92d73 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -374,6 +374,8 @@ Pure utility functions with no domain or infrastructure coupling: - **`kdfPolicy.js`** — KDF parameter validation and sensible defaults. - **`aesGcmMeta.js`** — AES-GCM metadata validation (IV length, tag length). - **`canonicalBase64.js`** — base64 encoding round-trip integrity check. +- **`boundedPromiseCache.js`** — fixed-residency LRU coalescing for immutable + async reads; rejected work is never retained. ## Storage Model diff --git a/CHANGELOG.md b/CHANGELOG.md index 57880656..7df39765 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,23 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Added + +- **Direct bundle references** - `bundles.getMemberReference()` and + `bundles.iterateMemberReferences()` validate bundle structure, canonical + descriptors, direct Git edges, and direct target object types without + recursively resolving each member's support graph. Existing member and root + methods retain complete-validation semantics. + +### Performance + +- **Bounded immutable Git metadata coalescing** - repeated exact tree-entry and + object-info reads now share successful in-flight and completed work through a + fixed-residency LRU. BundleService separately retains only structural + descriptor bytes under both entry and byte bounds. Rejected work is evicted, + mutable records are cloned for callers, and no application payloads or ref + state enter either cache. + ## [6.4.0] — 2026-07-17 ### Added diff --git a/README.md b/README.md index da38eb60..dd5292bf 100644 --- a/README.md +++ b/README.md @@ -138,8 +138,8 @@ Core capabilities: read-only, and `sweep()` can release only markers whose expiry has passed. - **Application storage**: `assets`, `pages`, `bundles`, `retention`, and `publications` compose streaming CAS writes, bounded structured - materializations, targeted member reads, reachability roots, - compare-and-swap refs, and immutable lifecycle evidence. + materializations, direct-reference or complete-validation member reads, + reachability roots, compare-and-swap refs, and immutable lifecycle evidence. - **Scoped staging workspaces**: `workspaces.open()` mirrors application writes behind one renewable temporary RootSet, returns only after each handle is anchored, promotes destination-first, and exposes bounded age, expiry, diff --git a/docs/API.md b/docs/API.md index f6924a95..9d732ec9 100644 --- a/docs/API.md +++ b/docs/API.md @@ -1077,6 +1077,47 @@ The canonical bundle token is: git-cas:1:bundle:fanout-tree::: ``` +### Direct bundle references + +```javascript +const reference = await cas.bundles.getMemberReference({ + handle: materializationHandle, + path: 'nodes/root', +}); + +for await (const reference of cas.bundles.iterateMemberReferences({ + handle: materializationHandle, +})) { + indexHandle(reference.path, reference.handle); +} +``` + +`getMemberReference()` and `iterateMemberReferences()` are the explicit lazy +integrity surfaces. They validate the bundle root, persisted limits, fanout +summaries, canonical member descriptor, direct Git tree edge, and direct target +object type. They do not recursively resolve the referenced member's complete +support graph. Use them for indexes and manifests that dereference only selected +members. + +Each immutable `BundleMemberReference` contains `version`, `path`, `handle`, +`type`, and declared `size`. Missing paths return `null`; malformed bundle +structure, mismatched edges, and missing or mistyped direct targets fail closed. +Completing reference iteration still validates root and fanout summaries. + +The Git persistence adapter coalesces successful exact tree-entry and +object-info reads in one process-local LRU with a fixed 2,048-entry default. +Bundle traversal separately retains immutable structural descriptor bytes under +a 1,024-entry and 16 MiB total bound. Rejected work is removed, returned +tree-entry records are cloned, and application payloads, refs, and collection +state are never cached. Advanced direct adapter construction may set +`metadataCacheEntries` to another positive safe integer. + +Object identity is immutable, but external pruning can change whether an +unretained object remains available. Keep roots retained for the adapter's +operation lifetime and do not race active reads with destructive repository +maintenance. Payload access performs authoritative Git I/O if availability +changes despite that precondition. + ### `bundles.getMember()`, `bundles.iterateMembers()`, and `bundles.openMember()` ```javascript diff --git a/docs/design/0050-lazy-bundle-reference-reads/lazy-bundle-reference-reads.md b/docs/design/0050-lazy-bundle-reference-reads/lazy-bundle-reference-reads.md new file mode 100644 index 00000000..99eadac0 --- /dev/null +++ b/docs/design/0050-lazy-bundle-reference-reads/lazy-bundle-reference-reads.md @@ -0,0 +1,351 @@ +--- +schema: 'method.design/v1' +cycle: '0050' +slug: 'lazy-bundle-reference-reads' +status: 'active' +sponsor_human: 'James Ross' +sponsor_agent: 'Codex' +created: '2026-07-18' +updated: '2026-07-18' +issue: 'https://github.com/git-stunts/git-cas/issues/81' +milestone: 'v6.5.0' +--- + +# PERF-0050 - Lazy Bundle Reference Reads + +## Linked Issue + +- [#81 - perf: add lazy bundle references and bounded immutable metadata reads](https://github.com/git-stunts/git-cas/issues/81) + +## Design Type + +- Public additive API +- Git read-path performance correction +- Bounded-residency infrastructure +- Integrity-boundary clarification + +## Decision Summary + +Add public `getMemberReference()` and `iterateMemberReferences()` bundle +operations. These operations validate the bundle descriptors and the selected +direct Git tree edges without recursively resolving every member target. +Existing `getMember()`, `iterateMembers()`, `openMember()`, and `resolveRoot()` +retain their complete target-validation behavior. + +The Git persistence adapter will coalesce successful immutable object-info and +exact tree-entry reads through a fixed-size least-recently-used map. +`BundleService` will separately coalesce raw structural descriptor bytes under +both entry and byte bounds. Neither cache owns application payloads, full +arbitrary trees, refs, or mutable collection state. + +## Sponsored Human + +Application operators need exact reads to remain responsive when a retained +materialization references a graph much larger than memory. + +## Sponsored Agent + +Agents need a precise API that distinguishes locating an immutable member edge +from proving the complete transitive support graph. They also need stable +command-count evidence rather than undocumented process-local shortcuts. + +## Hill + +A consumer can open a retained structured manifest, inspect only its direct +member references, and dereference only the requested data while Git command +count stays proportional to the selected path rather than the transitive graph. + +## Current Truth + +- `iterateMembers()` is the explicit full-index validation surface and validates + each member support graph before yielding it. +- `BundleService` already contains internal direct-reference methods used by + CacheSet and ExpiringSet indexes. +- Those methods are absent from the frozen public facade and declarations. +- Every `readTreeEntry()` and object-info call currently starts another Git + command, even when an immutable OID/path was read moments earlier. +- A four-node git-warp retained-property read performs 192 Git commands and + takes approximately 3-5 seconds on the development host. +- A controlled direct-reference A/B performs 92 commands and takes 1.44 + seconds with the same result, zero replay, and no whole-state cache. +- Public references plus immutable Git metadata coalescing reduce the same + one-read fixture to 71 commands, 1.215 seconds wall, and 80.947 ms Node CPU. + A 16-read run proves structural descriptor rereads remain the dominant warm + data-plane cost until they are coalesced separately. + +## Problem + +Consumers that need only an immutable manifest edge must currently choose +between an internal API and a public API that recursively validates unrelated +targets. The latter defeats bounded causal reads. Repeated descriptor and +object-info reads then multiply process startup cost even though Git objects are +immutable by OID. + +## Scope + +- Promote direct-reference lookup and iteration as public bundle capabilities. +- Define and declare `BundleMemberReference` separately from fully resolved + `BundleMember`. +- Preserve direct-edge, descriptor, fanout, path, and summary checks. +- Add fixed-residency immutable metadata and structural descriptor coalescing. +- Add unit, real-Git, facade, declaration, and command-count proof. +- Document the integrity distinction and migration guidance. + +## Non-Goals + +- Changing complete validation semantics. +- Caching application payload pages or blobs, arbitrary full trees, refs, or + CacheSet state. +- Moving CAS or cache ownership into git-warp. +- Replacing `@git-stunts/plumbing` or introducing a long-lived Git subprocess. +- Claiming a performance result before same-fixture measurements pass. + +## Runtime / API Contract + +```ts +interface BundleMemberReference { + readonly version: 1; + readonly path: string; + readonly handle: ApplicationHandle; + readonly type: 'blob' | 'tree'; + readonly size: number | null; +} + +interface BundleCapability { + getMemberReference(options: { + handle: BundleHandleInput; + path: string; + }): Promise; + + iterateMemberReferences(options: { + handle: BundleHandleInput; + }): AsyncIterable; +} +``` + +A reference result proves: + +1. the bundle root descriptor and persisted limits decoded successfully +2. the fanout path and summaries are structurally consistent +3. the named leaf descriptor is canonical +4. the direct Git tree edge exists and agrees with the encoded handle/type + +It does not prove that the referenced target's complete support graph is +healthy. `getMember()`, `iterateMembers()`, `openMember()`, and `resolveRoot()` +remain the complete-validation and dereference surfaces. + +## Data / State Model + +No persisted schema changes. Reference reads consume existing bundle root, +fanout-node, and leaf descriptors. Metadata caches are process-local, +non-authoritative accelerators keyed only by immutable Git OID, structural +descriptor OID plus read bound, or exact `(treeOid, path)` identity. + +## Architecture / Anti-SLUDGE Posture + +- Bundle semantics remain in `BundleService`. +- Git command coalescing remains in `GitPersistenceAdapter`. +- Structural bundle descriptor coalescing remains in `BundleService`; the + persistence adapter does not learn descriptor semantics. +- The facade exposes named capabilities; consumers do not import domain + services or Git plumbing. +- One small bounded cache helper may be introduced if it removes duplicated LRU + and rejected-promise handling. +- git-warp remains unable to manage raw CAS objects or invent a second cache. + +## Cost / Residency Posture + +- Every metadata cache has a fixed entry maximum. +- Structural descriptor bytes also have a fixed 16 MiB aggregate maximum; + descriptors larger than the budget are returned but not retained. +- Values are immutable metadata, exact tree-entry records, or structural bundle + descriptors, never application payloads. +- Successful concurrent reads for one key share one promise. +- Rejected reads are removed so transient failures are not cached. +- Access refreshes recency; insertion evicts the oldest entry. +- Returning data clones mutable record shapes so callers cannot mutate cached + truth. +- Direct-reference iteration remains streaming and bounded by fanout depth plus + descriptor limits. + +## Git Substrate Impact + +No new refs or object formats. The change reduces repeated `ls-tree`, +`cat-file --batch-check`, and structural descriptor `cat-file blob` +invocations. Cache acquisition refs continue to pin a generation and retain +their explicit release contract. + +## Compatibility / Migration Posture + +The API is additive. Existing methods and result types are unchanged. Consumers +that need transitive integrity continue using full-validation methods. +Consumers that need a manifest/index edge and will validate payloads on access +may adopt the reference methods. + +## Error Contract + +Reference methods retain existing bundle corruption, path, descriptor, fanout, +handle-edge, and direct target type/existence errors. They intentionally defer +errors below the direct target root until that target is dereferenced or fully +validated. Cached failures are never replayed as durable conclusions. + +## Security / Trust / Redaction Posture + +Direct references are not witnesses of transitive target health. Documentation +and types must not call them resolved or validated members. A malicious or +corrupt direct edge still fails closed. Full validation remains available and +is still mandatory for retention, publication, and doctor paths. + +## Lower Modes + +Memory persistence and non-Node runtimes keep the same domain semantics. +Metadata coalescing is an optimization of the default Git adapter only. + +## Accessibility Posture + +No visual interface changes. API and diagnostic terminology must distinguish +"reference" from "resolved member" without relying on color or symbols. + +## User-Facing Text / Directionality + +New prose uses plain English and code identifiers. No localization or text +direction assumptions are introduced. + +## Agent Inspectability / Explainability Posture + +Command-count tests expose whether a future change restores per-edge Git +chatter. Public method names encode the integrity level instead of hiding it in +an option boolean. + +## Design Alternatives Considered + +### Change `iterateMembers()` to shallow validation + +Rejected. The published contract explicitly promises complete support-graph +validation. Silent weakening would be a security and compatibility regression. + +### Add `validation: false` + +Rejected. A boolean hides which guarantees remain. Separate reference methods +make the weaker but useful integrity posture explicit in code review. + +### Cache whole trees and blobs + +Rejected. Arbitrary Git trees and payloads can exceed memory. This would undo +the bounded-residency work the optimization exists to protect. + +### Cache materializations in git-warp + +Rejected. git-cas owns CAS lifecycle and caching. Duplicating that policy in a +consumer recreates split authority and stale-state risk. + +### Use only targeted reads with no metadata coalescing + +Rejected. It preserves memory bounds but repeats identical immutable Git +processes across hot reads. Fixed-size metadata coalescing has a stronger +latency/complexity tradeoff without becoming authoritative state. + +## Decision + +Ship explicit direct-reference bundle APIs and fixed-residency immutable +metadata coalescing together. Measure each contribution independently. + +## Proof Surface + +- `BundleService` unit tests distinguish direct-edge and transitive validation. +- Facade tests lock public method names and frozen capability shape. +- Declaration tests lock `BundleMemberReference` and method signatures. +- Git persistence tests prove coalescing, cloning, rejection eviction, and LRU + bounds. +- Real-Git tests record command histograms for first and repeated reads. +- git-warp reruns the exact retained-property benchmark after publication. + +## Implementation Slices + +1. RED public reference API tests and design checkpoint. +2. Public facade, declarations, docs, and reference implementation. +3. RED metadata coalescing and residency tests. +4. Fixed-size adapter implementation and real-Git command-count proof. +5. Witness, self-review, code-lawyer review, PR, and release. + +## Tests To Write First + +- A direct reference survives a missing nested support object while + `getMember()` fails complete validation. +- Reference iteration never calls the target resolver. +- Full iteration still detects a missing nested target. +- Duplicate object-info reads execute plumbing once. +- Duplicate exact tree-entry reads execute plumbing once. +- Concurrent duplicate reads share in-flight work. +- Failed reads are retried, not cached. +- Capacity overflow evicts the least recently used metadata entry. + +## Acceptance Criteria + +- Public declarations and frozen facade expose both reference methods. +- Existing full-validation tests remain green unchanged. +- No application payload or mutable ref state enters the metadata cache. +- Cache residence is mechanically bounded and tested. +- Direct-reference iteration is independent of nested target cardinality. +- Same-fixture git-warp command count and wall/CPU time materially improve. +- README/API/CHANGELOG and witness material state the exact guarantees. + +## Validation Plan + +- Focused BundleService, facade, declaration, and persistence tests. +- Real-Git integration and command-count tests. +- `pnpm run lint`. +- `pnpm test`. +- `pnpm test:integration` where the host permits it. +- Type declaration compile check. +- Package dry run/release verification before publication. + +## Playback / Witness + +Human playback questions: + +1. Can an operator see before/after command count, CPU, wall, and RSS? +2. Does the design still fail closed for complete validation? +3. Is peak metadata residency explicitly bounded? + +Agent playback questions: + +1. Can an agent select the integrity posture from the method name alone? +2. Can it distinguish a direct-edge reference from a resolved member? +3. Can it consume a manifest without materializing unrelated targets? + +Witness output belongs under `witness/` and includes exact commands, test +counts, and before/after JSON. + +## Risks + +- Consumers may mistake a reference for transitive integrity evidence. +- A cache may accidentally retain rejected work or mutable records. +- Too-small bounds may reduce hit rate; too-large bounds may waste memory. +- Command-count wins on one fixture may not generalize to deep fanout. +- Per-read acquisition refs may remain the next fixed-cost bottleneck. +- Successful metadata can outlive externally pruned, unretained objects within + one adapter lifetime. Consumers must retain roots for the operation lifetime; + destructive external pruning must not race active reads. Payload access + performs authoritative Git I/O if that precondition is violated. + +## Follow-On Debt + +Persistent `cat-file` batching and longer-lived cache acquisition scopes remain +separate decisions unless evidence proves this cycle cannot meet its hill. + +## Tracker Disposition + +Issue #81 remains open until implementation, witness, PR review, and release +evidence satisfy every acceptance criterion. + +## Done Does Not Mean + +- Every git-warp operation is fast. +- Full support-graph validation is cheap. +- Payloads fit in memory. +- Cache acquisitions may be leaked or held forever. + +## Retrospective + +Pending merge and production benchmark evidence. diff --git a/docs/design/0050-lazy-bundle-reference-reads/witness/verification.md b/docs/design/0050-lazy-bundle-reference-reads/witness/verification.md new file mode 100644 index 00000000..047e1567 --- /dev/null +++ b/docs/design/0050-lazy-bundle-reference-reads/witness/verification.md @@ -0,0 +1,225 @@ +# Lazy Bundle Reference Reads Verification Witness + +Generated: 2026-07-18 02:27:45 PDT + +Issue: [#81](https://github.com/git-stunts/git-cas/issues/81) + +Feature commit: `7ddbda5d369b4c0694b1bbf337834a7fc6e776cb` + +Concurrency repair commit: `d7841acbaffbc4c5b14c78d31d3dc65ac4618cce` + +Cache settlement repair commit: `f1d219973d40bfa8a728093d39f793c3486037ad` + +Review repair commit: `b7695abc5d87038d9c79f42fc3c77c3580008a24` + +## Public Contract + +The frozen `bundles` facade exposes direct lookup and streaming reference +iteration without exposing `BundleService` itself. + +[cite: `index.js#215-224@7ddbda5d369b4c0694b1bbf337834a7fc6e776cb`] + +The declarations distinguish `BundleMemberReference` from a fully resolved +`BundleMember` and preserve the existing complete-validation methods. + +[cite: `index.d.ts#1463-1499@7ddbda5d369b4c0694b1bbf337834a7fc6e776cb`] + +## Integrity Boundary + +Targeted lookup validates the root and selected child summaries, charges every +descriptor on the selected path against the persisted descriptor-byte budget, +and follows only the selected fanout edge. + +[cite: `src/domain/services/BundleService.js#114-155@7ddbda5d369b4c0694b1bbf337834a7fc6e776cb`] + +Reference iteration remains streaming while preserving descriptor, member, +root-summary, and fanout-summary checks. Existing `iterateMembers()` selects +the same traversal with recursive target validation enabled. + +[cite: `src/domain/services/BundleService.js#229-275@7ddbda5d369b4c0694b1bbf337834a7fc6e776cb`] + +Tests prove that a corrupt root summary fails closed, a direct reference can be +read without recursively resolving missing nested support, and reference +iteration does not invoke target resolvers. + +[cite: `test/unit/domain/services/BundleService.test.js#282-335@7ddbda5d369b4c0694b1bbf337834a7fc6e776cb`] + +## Bounded Residency + +Structural bundle descriptor blobs use a 1,024-entry, 16 MiB cache. Every hit +returns a fresh byte array, so a decoder or caller cannot mutate cached bytes. + +[cite: `src/domain/services/BundleService.js#15-29@7ddbda5d369b4c0694b1bbf337834a7fc6e776cb`] +[cite: `src/domain/services/BundleService.js#613-622@7ddbda5d369b4c0694b1bbf337834a7fc6e776cb`] + +The shared helper enforces positive entry bounds, optional aggregate weight, +least-recently-used eviction, in-flight coalescing, and rejection eviction. +Bookkeeping now settles before the shared promise becomes observable, so an +immediate retry cannot recover a stale rejected promise. Resolved weights must +also be non-negative safe integers or the entry is rejected and evicted. + +[cite: `src/helpers/boundedPromiseCache.js#35-101@f1d219973d40bfa8a728093d39f793c3486037ad`] + +The Git adapter uses one bounded cache for exact immutable tree-entry and +object-info reads. Tree-entry records are frozen internally and cloned before +return; failed reads are not retained. The adapter contract explicitly +requires retained roots and prohibits destructive pruning races during active +operations. + +[cite: `src/infrastructure/adapters/GitPersistenceAdapter.js#27-50@7ddbda5d369b4c0694b1bbf337834a7fc6e776cb`] +[cite: `src/infrastructure/adapters/GitPersistenceAdapter.js#150-166@7ddbda5d369b4c0694b1bbf337834a7fc6e776cb`] +[cite: `src/infrastructure/adapters/GitPersistenceAdapter.js#215-269@7ddbda5d369b4c0694b1bbf337834a7fc6e776cb`] + +Unit coverage proves concurrent and sequential coalescing, immediate +transient-failure retry, resolved-weight validation, configurable residency +validation, and least-recently-used eviction. + +[cite: `test/unit/helpers/boundedPromiseCache.test.js#42-60@f1d219973d40bfa8a728093d39f793c3486037ad`] +[cite: `test/unit/helpers/boundedPromiseCache.test.js#63-123@f1d219973d40bfa8a728093d39f793c3486037ad`] + +[cite: `test/unit/infrastructure/adapters/GitPersistenceAdapter.readTree.test.js#92-114@7ddbda5d369b4c0694b1bbf337834a7fc6e776cb`] +[cite: `test/unit/infrastructure/adapters/GitPersistenceAdapter.readTree.test.js#147-213@7ddbda5d369b4c0694b1bbf337834a7fc6e776cb`] + +A code-lawyer probe retried immediately after directly awaiting a rejected +cache promise. Before the settlement repair, all 10,000 iterations recovered +the stale rejection. The same probe completed with zero stale rejections after +the repair. + +## Concurrent Root-Set Safety + +The release gate exposed a second-order failure under repeated concurrent +cache replacement. Git occasionally returned exit code 128 with empty output +for the exact guarded `update-ref` command. An independent ref read showed +that the competing writer had advanced the head, but the missing text prevented +the plumbing classifier from identifying the compare-and-swap race. + +Root-set persistence now accepts that narrow evidence bundle as a conflict: +fatal exit 128, exact managed command operands, a valid observed head, and an +observed head different from the caller's expectation. It does not normalize +same-head failures, other-ref commands, or nonfatal exits. + +[cite: `src/domain/services/RootSetPersistence.js#17-70@d7841acbaffbc4c5b14c78d31d3dc65ac4618cce`] +[cite: `src/domain/services/RootSetPersistence.js#267-337@d7841acbaffbc4c5b14c78d31d3dc65ac4618cce`] + +The regression suite preserves both sides of that boundary. It proves the +empty-diagnostic advanced-head case is retryable and proves three nearby error +shapes remain terminal. + +[cite: `test/unit/domain/services/RootSetPersistence.test.js#71-125@d7841acbaffbc4c5b14c78d31d3dc65ac4618cce`] +[cite: `test/unit/domain/services/RootSetPersistence.test.js#217-245@d7841acbaffbc4c5b14c78d31d3dc65ac4618cce`] + +A pre-fix Docker stress run reproduced the terminal failure at iteration 149. +The repaired implementation completed 500 consecutive two-writer guarded +replacements with exactly one accepted winner per iteration. The underlying +Node stderr-drain defect is tracked as +[git-stunts/plumbing#9](https://github.com/git-stunts/plumbing/issues/9). + +### Root-Set Stress Provenance + +The recovered inline stress program is preserved as a checked-in diagnostic. +It creates a temporary bare repository, opens two cache handles per namespace, +and races two guarded replacements of the same original handle. Every iteration +must return exactly one accepted result. + +[cite: `scripts/diagnostics/stress-root-set-replacement.js#1-54@684e6731aacbebf8777f87f2a9b93552c14d2db7`] + +The pre-fix run used a worktree based on +`5e3fe13f816b22b5d74790178eb873b2508e8667`, an iteration cap of 200, and the +same fixture algorithm; it failed at iteration 149. The repaired run used the +implementation now recorded by `b7695abc5d87038d9c79f42fc3c77c3580008a24` +with an iteration cap of 500 and passed all 500 races. The exact current +invocation is: + +```sh +docker compose run --build --rm -T test-node \ + node scripts/diagnostics/stress-root-set-replacement.js 500 +``` + +The container contract is the repository `test-node` target: Ubuntu 24.04 base +`sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90` +and Node 22 base +`sha256:5647be709086c696ff32edaaf1c70cd26d1da6ab2b39c32f3c7b4c4a31957e37`. +The observed tool versions were Node `v22.23.1`, pnpm `10.34.5`, and Git +`2.43.0`. Docker `29.5.2` with Compose `v5.1.4` ran on an Apple M1 Pro host with +16 GB RAM and macOS 26.3. + +[cite: `Dockerfile#1-32@7ddbda5d369b4c0694b1bbf337834a7fc6e776cb`] +[cite: `docker-compose.yml#1-11@7ddbda5d369b4c0694b1bbf337834a7fc6e776cb`] + +## Verification Results + +| Command | Result | +| ------------------------------------------------------ | ----------------------------------------- | +| `pnpm exec vitest run` over the six changed unit files | 6 files, 87 tests passed | +| `pnpm test` | 220 files passed; 2,019 passed, 2 skipped | +| `pnpm run lint` | passed | +| `pnpm run test:integration:node` | 12 files, 192 tests passed in Docker | +| `pnpm run release:verify` | 14/14 steps; 6,622 observed tests passed | +| guarded-replacement Docker stress | 500 consecutive races passed | +| `git diff --check` | passed | + +The real-Git proof requires a cold read to issue Git metadata commands, then +requires an identical warm read to issue zero additional commands. A separate +comparison requires direct reference iteration to issue fewer total and fewer +object-inspection commands than complete recursive validation. + +[cite: `test/integration/bundle-reference-performance.test.js#111-151@7ddbda5d369b4c0694b1bbf337834a7fc6e776cb`] + +## Same-Fixture Performance Evidence + +These exploratory measurements use the same four-node git-warp retained +property fixture. They are diagnostic evidence, not portable latency promises; +the command-count integration test above is the stable regression contract. + +The host was the Apple M1 Pro machine described above, using Node `v26.0.0`, +npm `11.12.1`, and Apple Git `2.50.1`. The fixture was a four-node directed +chain with three edges and one deterministic 128-byte `payload` property per +node. Reads targeted `node:00000003`; one-read and 16-read samples used the +same repository-construction path. + +The git-warp checkout was at +`5460be0518c769e20c5eb2919cbddf25019c5449` with the pending direct-reference +adapter and performance-harness worktree changes. The candidate git-cas package +was packed from `7ddbda5d369b4c0694b1bbf337834a7fc6e776cb` and overlaid into that +checkout. The exact final-sample command, run once with `GIT_WARP_PERF_READS=1` +and once with `GIT_WARP_PERF_READS=16`, was: + +```sh +GIT_WARP_PERF_READS=16 node --input-type=module -e \ + "import { preparePerformanceFixture } from './dist/scripts/performance/PerformanceFixture.js'; import { runPerformanceWorker } from './dist/scripts/performance/PerformanceWorker.js'; const fixture = await preparePerformanceFixture('warm-property-read', { nodeCount: 4, propertyBytesPerNode: 128 }); try { const sample = await runPerformanceWorker({ repositoryPath: fixture.repositoryPath, scenario: 'warm-property-read' }); process.stdout.write(JSON.stringify(sample, null, 2) + '\\n'); } finally { await fixture.cleanup(); }" +``` + +Because the git-warp adapter migration and harness were not yet committed, the +latency rows below are an exact environment record but not a commit-replayable +benchmark. They are intentionally excluded from merge gates and will be +superseded by git-warp's committed v19 benchmark suite after git-cas 6.5.0 is +published. The committed real-Git command-count test remains the reproducible +proof for this change. + +| Posture | Reads | Git commands | Wall time | Node CPU | +| --------------------------------------------------- | ----: | -----------: | ------------------: | ---------: | +| git-cas 6.4.0, complete recursive validation | 1 | 192 | approximately 3-5 s | 210-234 ms | +| Internal direct-reference A/B | 1 | 92 | 1.443 s | 89 ms | +| Public references plus immutable Git metadata cache | 1 | 71 | 1.215 s | 80.947 ms | +| Public references plus descriptor cache | 1 | 69 | 1.291 s | 92.065 ms | +| Before descriptor cache | 16 | 521 | 8.943 s | 626.554 ms | +| After descriptor cache | 16 | 309 | 5.173 s | 397.1 ms | + +The one-read command count fell from 192 to 69, a 64.1% reduction. The +16-read descriptor-cache comparison cut 212 commands and 42.2% wall time. +Peak RSS for the final 16-read diagnostic was 117,129,216 bytes. The run +performed zero causal replay and did not materialize or cache whole graph +state. + +## Residual Cost + +This cycle does not establish that git-warp is fast. The final 16-read command +histogram still contains 48 `rev-parse`, 16 `for-each-ref`, 16 `rev-list`, 48 +`symbolic-ref`, and 32 `update-ref` invocations. That is ten acquisition/ref +commands per read before selected payload access. + +The next optimization belongs in git-warp: hold one explicit materialization +acquisition for a runtime or coordinate lifetime, release it through runtime +resource closure, and reacquire only when the selected generation changes. +That design must preserve git-cas as the sole owner of cache objects and +retention refs. diff --git a/docs/design/README.md b/docs/design/README.md index d893f115..ceee34a5 100644 --- a/docs/design/README.md +++ b/docs/design/README.md @@ -11,6 +11,7 @@ process in [docs/method/process.md](../method/process.md). ## Active METHOD Cycles +- [0050-lazy-bundle-reference-reads - lazy-bundle-reference-reads](./0050-lazy-bundle-reference-reads/lazy-bundle-reference-reads.md) - [0049-scoped-staging-workspaces — scoped-staging-workspaces](./0049-scoped-staging-workspaces/scoped-staging-workspaces.md) - [0048-scoped-cache-acquisitions — scoped-cache-acquisitions](./0048-scoped-cache-acquisitions/scoped-cache-acquisitions.md) - [0020-method-adoption — adopt-method](./0020-method-adoption/adopt-method.md) diff --git a/index.d.ts b/index.d.ts index a9250f37..f396a571 100644 --- a/index.d.ts +++ b/index.d.ts @@ -564,7 +564,11 @@ export declare class GitRefPortBase { /** Git-backed implementation of the persistence port. */ export declare class GitPersistenceAdapter extends GitPersistencePortBase { - constructor(options: { plumbing: unknown; policy?: unknown }); + constructor(options: { + plumbing: unknown; + policy?: unknown; + metadataCacheEntries?: number; + }); setMaxBlobSize(maxBlobSize: number): void; } @@ -1456,12 +1460,15 @@ export type BundleMemberInput = | PageSource | { source: PageSource; maxBytes?: number }; -export interface BundleMember { +export interface BundleMemberReference { readonly version: 1; readonly path: string; readonly handle: ApplicationHandle; readonly type: 'blob' | 'tree'; readonly size: number | null; +} + +export interface BundleMember extends BundleMemberReference { readonly logicalBytes: number; } @@ -1483,6 +1490,11 @@ export interface BundleCapability { handle: BundleHandleInput; path: string; }): Promise; + getMemberReference(options: { + handle: BundleHandleInput; + path: string; + }): Promise; + iterateMemberReferences(options: { handle: BundleHandleInput }): AsyncIterable; iterateMembers(options: { handle: BundleHandleInput }): AsyncIterable; openMember(options: { handle: BundleHandleInput; path: string }): AsyncIterable; } diff --git a/index.js b/index.js index f7d3b93e..387a7e76 100644 --- a/index.js +++ b/index.js @@ -216,7 +216,11 @@ export default class ContentAddressableStore { put: async (options) => (await this.#getBundleService()).put(options), putOrdered: async (options) => (await this.#getBundleService()).putOrdered(options), getMember: async (options) => (await this.#getBundleService()).getMember(options), + getMemberReference: async (options) => ( + await this.#getBundleService() + ).getMemberReference(options), iterateMembers: (options) => this.#iterateBundleMembers(options), + iterateMemberReferences: (options) => this.#iterateBundleMemberReferences(options), openMember: (options) => this.#openBundleMember(options), }); this.retention = Object.freeze({ @@ -486,6 +490,12 @@ export default class ContentAddressableStore { yield* bundles.openMember(options); } + /** @returns {AsyncIterable} */ + async *#iterateBundleMemberReferences(options) { + const bundles = await this.#getBundleService(); + yield* bundles.iterateMemberReferences(options); + } + /** @returns {AsyncIterable} */ async *#iterateBundleMembers(options) { const bundles = await this.#getBundleService(); diff --git a/scripts/diagnostics/stress-root-set-replacement.js b/scripts/diagnostics/stress-root-set-replacement.js new file mode 100644 index 00000000..413c0edf --- /dev/null +++ b/scripts/diagnostics/stress-root-set-replacement.js @@ -0,0 +1,53 @@ +import { execFileSync } from 'node:child_process'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { tmpdir } from 'node:os'; +import path from 'node:path'; +import { inspect } from 'node:util'; +import ContentAddressableStore from '../../index.js'; +import { createGitPlumbing } from '../../src/infrastructure/createGitPlumbing.js'; + +const iterations = positiveSafeInteger(process.argv[2] ?? '500'); +const repoDir = mkdtempSync(path.join(tmpdir(), 'cas-root-race-')); +const git = (args) => + execFileSync('git', args, { + cwd: repoDir, + encoding: 'utf8', + }).trim(); + +try { + git(['init', '--bare']); + const plumbing = await createGitPlumbing({ cwd: repoDir }); + const cas = new ContentAddressableStore({ plumbing }); + + for (let index = 0; index < iterations; index += 1) { + const namespace = `stress/${index}`; + const left = await cas.caches.open({ namespace }); + const right = await cas.caches.open({ namespace }); + const original = await cas.pages.put({ source: Buffer.from(`original-${index}`) }); + const leftPage = await cas.pages.put({ source: Buffer.from(`left-${index}`) }); + const rightPage = await cas.pages.put({ source: Buffer.from(`right-${index}`) }); + await left.put('shared', original.handle); + + const results = await Promise.all([ + left.replace('shared', leftPage.handle, { expectedHandle: original.handle }), + right.replace('shared', rightPage.handle, { expectedHandle: original.handle }), + ]); + if (results.filter((result) => result.accepted).length !== 1) { + throw new Error( + `Expected exactly one replacement winner at iteration ${index}: ${inspect(results, { depth: 8 })}` + ); + } + } + + process.stdout.write(`${JSON.stringify({ iterations, passed: iterations })}\n`); +} finally { + rmSync(repoDir, { recursive: true, force: true }); +} + +function positiveSafeInteger(raw) { + const parsed = Number(raw); + if (!Number.isSafeInteger(parsed) || parsed < 1) { + throw new TypeError('iterations must be a positive safe integer'); + } + return parsed; +} diff --git a/src/domain/services/BundleService.js b/src/domain/services/BundleService.js index c2bcbf1a..3454f114 100644 --- a/src/domain/services/BundleService.js +++ b/src/domain/services/BundleService.js @@ -7,12 +7,15 @@ import BundleHandle from '../value-objects/BundleHandle.js'; import BundleLimits from '../value-objects/BundleLimits.js'; import normalizeBundlePath from '../value-objects/BundlePath.js'; import StagedBundle from '../value-objects/StagedBundle.js'; +import BoundedPromiseCache from '../../helpers/boundedPromiseCache.js'; import BundleDescriptorCodec, { BUNDLE_INDEX_ENTRY } from './BundleDescriptorCodec.js'; import BundleFanoutBuilder from './BundleFanoutBuilder.js'; import StagingEvidence from './StagingEvidence.js'; const DEFAULT_CLOCK = Object.freeze({ now: () => new Date() }); const DEFAULT_MAX_NESTING_DEPTH = 32; +const DESCRIPTOR_CACHE_ENTRIES = 1_024; +const DESCRIPTOR_CACHE_BYTES = 16 * 1024 * 1024; /** * Builds and traverses deterministic, targeted structured bundle trees. @@ -20,6 +23,10 @@ const DEFAULT_MAX_NESTING_DEPTH = 32; export default class BundleService { #clock; #codec; + #descriptorBlobs = new BoundedPromiseCache(DESCRIPTOR_CACHE_ENTRIES, { + maxWeight: DESCRIPTOR_CACHE_BYTES, + weightOf: (value) => value.byteLength, + }); #limits; #maxNestingDepth; #openHandle; @@ -104,14 +111,30 @@ export default class BundleService { return member === null ? null : await this.#resolveMemberReference(member, BundleHandle.from(value)); } - /** @internal Returns one descriptor after validating only its direct Git edge. */ + /** Returns one descriptor after validating the bundle structure and direct Git edge. */ async getMemberReference({ handle: value, path: rawPath }) { const handle = BundleHandle.from(value); const path = normalizeBundlePath(rawPath, this.#limits.maxMemberPathBytes); const root = await this.#readRoot(handle); let nodeOid = root.index.oid; + let expectedSummary = null; + let parentTreeOid = handle.oid; + let descriptorBytes = root.descriptorBytes; while (true) { const node = await this.#readNode(nodeOid, root.descriptor.limits); + descriptorBytes += node.descriptorBytes; + if (descriptorBytes > root.descriptor.limits.maxDescriptorBytes) { + throw corrupt('Bundle descriptors exceed their persisted byte limit', { + descriptorBytes, + maxDescriptorBytes: root.descriptor.limits.maxDescriptorBytes, + }); + } + const actualSummary = summaryOf(node.descriptor, nodeOid); + if (expectedSummary === null) { + assertSummary(root.descriptor, actualSummary); + } else { + assertChildSummary(expectedSummary, actualSummary, parentTreeOid); + } if (node.descriptor.kind === 'leaf') { return await this.#referenceFromLeaf({ bundleHandle: handle, @@ -125,6 +148,8 @@ export default class BundleService { return null; } const edge = await this.#requiredEdge(nodeOid, child.slot, 'tree'); + expectedSummary = child; + parentTreeOid = nodeOid; nodeOid = edge.oid; } } @@ -206,7 +231,7 @@ export default class BundleService { yield* this.#iterateMemberReferences(value, { validateTargets: true }); } - /** @internal Streams descriptors after validating only their direct Git edges. */ + /** Streams descriptors after validating bundle structure and direct Git edges. */ async *iterateMemberReferences({ handle: value }) { yield* this.#iterateMemberReferences(value, { validateTargets: false }); } @@ -587,7 +612,11 @@ export default class BundleService { async #readDescriptorBlob(oid, meta, maxBytes) { try { - return await this.#persistence.readBlob(oid, maxBytes); + const bytes = await this.#descriptorBlobs.getOrCreate( + `${oid}\0${maxBytes}`, + () => this.#persistence.readBlob(oid, maxBytes), + ); + return Uint8Array.from(bytes); } catch (error) { throw corrupt('Bundle descriptor blob is missing or unreadable', { ...meta, oid, originalError: error }); } diff --git a/src/domain/services/RootSetPersistence.js b/src/domain/services/RootSetPersistence.js index 128349f0..bb117225 100644 --- a/src/domain/services/RootSetPersistence.js +++ b/src/domain/services/RootSetPersistence.js @@ -1,6 +1,7 @@ import CasError from '../errors/CasError.js'; import { ErrorCodes } from '../errors/index.js'; import { errorDetailsText, isGitMissingRefError } from '../helpers/gitRefErrors.js'; +import Oid from '../value-objects/Oid.js'; import RootSetRef from '../value-objects/RootSetRef.js'; import RootSetMetadataCodec, { ROOT_SET_METADATA_ENTRY } from './RootSetMetadataCodec.js'; import RootSetTreeCodec from './RootSetTreeCodec.js'; @@ -11,16 +12,14 @@ const UPDATE_REF_CONFLICT_MARKERS = Object.freeze({ referenceAlreadyExists: 'reference already exists', }); const GIT_REPOSITORY_LOCKED = 'GIT_REPOSITORY_LOCKED'; +const GIT_FATAL_EXIT_CODE = 128; -function classifyStructuredUpdateRefConflict(err, { +function hasExactUpdateRefArgs(err, { rootSetRef, newCommit, expectedHeadOid, }) { const details = err?.details && typeof err.details === 'object' ? err.details : {}; - if (details.code !== GIT_REPOSITORY_LOCKED) { - return null; - } if (!Array.isArray(details.args)) { return false; } @@ -35,6 +34,48 @@ function classifyStructuredUpdateRefConflict(err, { expectedArgs.every((arg, index) => details.args[index] === arg); } +function classifyStructuredUpdateRefConflict(err, { + rootSetRef, + newCommit, + expectedHeadOid, +}) { + const details = err?.details && typeof err.details === 'object' ? err.details : {}; + if (details.code !== GIT_REPOSITORY_LOCKED) { + return null; + } + return hasExactUpdateRefArgs(err, { rootSetRef, newCommit, expectedHeadOid }); +} + +function isObservedUpdateRefConflict(err, { + rootSetRef, + newCommit, + expectedHeadOid, + actualHeadOid, +}) { + const details = err?.details && typeof err.details === 'object' ? err.details : {}; + if (!Oid.isValid(actualHeadOid)) { + return false; + } + const canonicalActualHeadOid = Oid.from(actualHeadOid).toString(); + const canonicalExpectedHeadOid = Oid.isValid(expectedHeadOid) + ? Oid.from(expectedHeadOid).toString() + : expectedHeadOid; + return details.code === GIT_FATAL_EXIT_CODE && + canonicalActualHeadOid !== canonicalExpectedHeadOid && + hasExactUpdateRefArgs(err, { rootSetRef, newCommit, expectedHeadOid }); +} + +function hasTextualUpdateRefConflict(err, rootSetRef) { + const normalized = errorDetailsText(err).toLowerCase(); + if (!normalized.includes(rootSetRef.toLowerCase())) { + return false; + } + return normalized.includes(UPDATE_REF_CONFLICT_MARKERS.cannotLockRef) && ( + normalized.includes(UPDATE_REF_CONFLICT_MARKERS.butExpected) || + normalized.includes(UPDATE_REF_CONFLICT_MARKERS.referenceAlreadyExists) + ); +} + /** * Stateless persistence boundary for one root-set ref and snapshot format. */ @@ -238,7 +279,11 @@ export default class RootSetPersistence { }); } catch (err) { const meta = await this.#updateFailureMeta(err, commitOid, expectedHeadOid); - if (this.#isConflict(err, commitOid, expectedHeadOid)) { + if (this.#isConflict(err, { + newCommit: commitOid, + expectedHeadOid, + actualHeadOid: meta.actualHeadOid, + })) { throw new CasError( 'Concurrent root-set update detected', ErrorCodes.ROOT_SET_CONFLICT, @@ -272,7 +317,7 @@ export default class RootSetPersistence { }; } - #isConflict(err, newCommit, expectedHeadOid) { + #isConflict(err, { newCommit, expectedHeadOid, actualHeadOid }) { const meta = err?.meta && typeof err.meta === 'object' ? err.meta : {}; if (Object.hasOwn(meta, 'expectedOldOid') && Object.hasOwn(meta, 'actualOldOid')) { return true; @@ -285,14 +330,17 @@ export default class RootSetPersistence { if (structuredConflict !== null) { return structuredConflict; } - const normalized = errorDetailsText(err).toLowerCase(); - if (!normalized.includes(this.rootSetRef.toLowerCase())) { - return false; + // A failed exact CAS command plus an independently observed head advance + // remains sufficient conflict evidence when Git emits no diagnostic text. + if (isObservedUpdateRefConflict(err, { + rootSetRef: this.rootSetRef, + newCommit, + expectedHeadOid, + actualHeadOid, + })) { + return true; } - return normalized.includes(UPDATE_REF_CONFLICT_MARKERS.cannotLockRef) && ( - normalized.includes(UPDATE_REF_CONFLICT_MARKERS.butExpected) || - normalized.includes(UPDATE_REF_CONFLICT_MARKERS.referenceAlreadyExists) - ); + return hasTextualUpdateRefConflict(err, this.rootSetRef); } #isMissingRefError(err) { diff --git a/src/helpers/boundedPromiseCache.js b/src/helpers/boundedPromiseCache.js new file mode 100644 index 00000000..2bb1bb96 --- /dev/null +++ b/src/helpers/boundedPromiseCache.js @@ -0,0 +1,108 @@ +/** + * Fixed-residency LRU for in-flight and completed immutable reads. + * Rejected promises remove themselves so transient failures remain retryable. + */ +export default class BoundedPromiseCache { + /** @type {Map, weight: number }>} */ + #entries = new Map(); + #maxEntries; + #maxWeight; + #totalWeight = 0; + #weightOf; + + /** + * @param {number} maxEntries + * @param {object} [options] + * @param {number} [options.maxWeight=Infinity] + * @param {(value: unknown) => number} [options.weightOf] + */ + constructor(maxEntries, { maxWeight = Number.POSITIVE_INFINITY, weightOf = () => 0 } = {}) { + if (!Number.isSafeInteger(maxEntries) || maxEntries < 1) { + throw new TypeError('maxEntries must be a positive safe integer'); + } + if ( + maxWeight !== Number.POSITIVE_INFINITY && + (!Number.isSafeInteger(maxWeight) || maxWeight < 0) + ) { + throw new TypeError('maxWeight must be a non-negative safe integer or Infinity'); + } + if (typeof weightOf !== 'function') { + throw new TypeError('weightOf must be a function'); + } + this.#maxEntries = maxEntries; + this.#maxWeight = maxWeight; + this.#weightOf = weightOf; + } + + /** + * @template T + * @param {string} key + * @param {() => Promise|T} factory + * @returns {Promise} + */ + getOrCreate(key, factory) { + const cached = this.#entries.get(key); + if (cached !== undefined) { + this.#entries.delete(key); + this.#entries.set(key, cached); + return cached.promise; + } + + const source = Promise.resolve().then(factory); + const entry = { promise: source, weight: 0 }; + const tracked = source.then( + (value) => { + if (this.#entries.get(key) === entry) { + let weight; + try { + weight = this.#resolvedWeight(value); + } catch (error) { + this.#remove(key, entry); + throw error; + } + if (weight > this.#maxWeight) { + this.#remove(key, entry); + return value; + } + entry.weight = weight; + this.#totalWeight += entry.weight; + this.#evict(); + } + return value; + }, + (error) => { + if (this.#entries.get(key) === entry) { + this.#remove(key, entry); + } + throw error; + }, + ); + entry.promise = tracked; + this.#entries.set(key, entry); + this.#evict(); + return tracked; + } + + #resolvedWeight(value) { + const weight = this.#weightOf(value); + if (!Number.isSafeInteger(weight) || weight < 0) { + throw new TypeError('weightOf must return a non-negative safe integer'); + } + return weight; + } + + #evict() { + while (this.#entries.size > this.#maxEntries || this.#totalWeight > this.#maxWeight) { + const oldestKey = this.#entries.keys().next().value; + const oldest = this.#entries.get(oldestKey); + this.#remove(oldestKey, oldest); + } + } + + #remove(key, entry) { + if (entry === undefined || !this.#entries.delete(key)) { + return; + } + this.#totalWeight -= entry.weight; + } +} diff --git a/src/infrastructure/adapters/GitPersistenceAdapter.js b/src/infrastructure/adapters/GitPersistenceAdapter.js index 495f07ae..b1d7883d 100644 --- a/src/infrastructure/adapters/GitPersistenceAdapter.js +++ b/src/infrastructure/adapters/GitPersistenceAdapter.js @@ -4,6 +4,7 @@ import os from 'node:os'; import path from 'node:path'; import GitPersistencePort from '../../ports/GitPersistencePort.js'; import { CasError, createCasError, ErrorCodes } from '../../domain/errors/index.js'; +import BoundedPromiseCache from '../../helpers/boundedPromiseCache.js'; /** * Default resilience policy: 30 s timeout (no retry). @@ -15,8 +16,10 @@ import { CasError, createCasError, ErrorCodes } from '../../domain/errors/index. */ const DEFAULT_POLICY = Policy.timeout(30_000); export const DEFAULT_MAX_BLOB_SIZE = 10 * 1024 * 1024; +const DEFAULT_METADATA_CACHE_ENTRIES = 2_048; const MIN_READ_BLOB_LIMIT = 1; const MIN_MAX_BLOB_SIZE = 1024; +const MIN_METADATA_CACHE_ENTRIES = 1; const MAX_BLOB_SIZE_LIMIT = Number.MAX_SAFE_INTEGER; const OBJECT_INFO_ARGUMENT = '--batch-check=%(objectname) %(objecttype) %(objectsize)'; const GIT_OBJECT_TYPES = new Set(['blob', 'tree', 'commit', 'tag']); @@ -25,19 +28,25 @@ const GIT_OBJECT_TYPES = new Set(['blob', 'tree', 'commit', 'tag']); * {@link GitPersistencePort} implementation backed by `@git-stunts/plumbing`. * * All Git I/O is wrapped with a configurable resilience {@link Policy} - * (30 s timeout by default). + * (30 s timeout by default). Successful metadata reads assume referenced roots + * remain retained while this adapter is in use; destructive external pruning + * must not race active operations. */ export default class GitPersistenceAdapter extends GitPersistencePort { #maxBlobSize = DEFAULT_MAX_BLOB_SIZE; + #metadataCache; /** * @param {Object} options * @param {import('@git-stunts/plumbing').default} options.plumbing - GitPlumbing instance. * @param {import('@git-stunts/alfred').Policy} [options.policy] - Resilience policy (defaults to 30 s timeout, no retry). + * @param {number} [options.metadataCacheEntries=2048] - Maximum immutable metadata entries retained by this adapter. */ - constructor({ plumbing, policy }) { + constructor({ plumbing, policy, metadataCacheEntries = DEFAULT_METADATA_CACHE_ENTRIES }) { super(); + GitPersistenceAdapter.#assertMetadataCacheEntries(metadataCacheEntries); this.plumbing = plumbing; this.policy = policy ?? DEFAULT_POLICY; + this.#metadataCache = new BoundedPromiseCache(metadataCacheEntries); } /** @@ -145,12 +154,15 @@ export default class GitPersistenceAdapter extends GitPersistencePort { * @returns {Promise<{ mode: string, type: string, oid: string, name: string }|null>} */ async readTreeEntry(treeOid, treePath) { - return this.policy.execute(async () => { + const key = `tree\0${treeOid}\0${treePath}`; + const entry = await this.#metadataCache.getOrCreate(key, () => this.policy.execute(async () => { const output = await this.plumbing.execute({ args: ['ls-tree', '-z', treeOid, '--', treePath], }); - return GitPersistenceAdapter.#parseTreeOutput(output)[0] || null; - }); + const found = GitPersistenceAdapter.#parseTreeOutput(output)[0] || null; + return found === null ? null : Object.freeze(found); + })); + return entry === null ? null : { ...entry }; } /** @@ -209,32 +221,51 @@ export default class GitPersistenceAdapter extends GitPersistencePort { * @returns {Promise<{ oid: string, type: string, size: number }>} Object metadata. */ async #readObjectInfo(oid) { - const rawOutput = await this.policy.execute(() => this.plumbing.execute({ - args: ['cat-file', OBJECT_INFO_ARGUMENT], - input: `${oid}\n`, - })); - const output = typeof rawOutput === 'string' ? rawOutput.trim() : ''; - if (output === `${oid} missing`) { - throw new CasError(`Git object not found: ${oid}`, ErrorCodes.GIT_OBJECT_NOT_FOUND, { - oid, - }); - } + return this.#metadataCache.getOrCreate(`object\0${oid}`, async () => { + const rawOutput = await this.policy.execute(() => this.plumbing.execute({ + args: ['cat-file', OBJECT_INFO_ARGUMENT], + input: `${oid}\n`, + })); + const output = typeof rawOutput === 'string' ? rawOutput.trim() : ''; + if (output === `${oid} missing`) { + throw new CasError(`Git object not found: ${oid}`, ErrorCodes.GIT_OBJECT_NOT_FOUND, { + oid, + }); + } - const fields = output.split(' '); - const size = Number(fields[2]); + const fields = output.split(' '); + const size = Number(fields[2]); + if ( + fields.length !== 3 || + fields[0] !== oid || + !GIT_OBJECT_TYPES.has(fields[1]) || + !Number.isSafeInteger(size) || + size < 0 + ) { + throw new CasError(`Git object has invalid metadata: ${oid}`, ErrorCodes.GIT_ERROR, { + oid, + output: rawOutput, + }); + } + return Object.freeze({ oid: fields[0], type: fields[1], size }); + }); + } + + /** + * @param {number} metadataCacheEntries + */ + static #assertMetadataCacheEntries(metadataCacheEntries) { if ( - fields.length !== 3 || - fields[0] !== oid || - !GIT_OBJECT_TYPES.has(fields[1]) || - !Number.isSafeInteger(size) || - size < 0 + Number.isSafeInteger(metadataCacheEntries) && + metadataCacheEntries >= MIN_METADATA_CACHE_ENTRIES ) { - throw new CasError(`Git object has invalid metadata: ${oid}`, ErrorCodes.GIT_ERROR, { - oid, - output: rawOutput, - }); + return; } - return Object.freeze({ oid: fields[0], type: fields[1], size }); + throw createCasError( + 'Git metadata cache entries must be a positive safe integer', + ErrorCodes.INVALID_OPTIONS, + { option: 'metadataCacheEntries', metadataCacheEntries }, + ); } /** diff --git a/test/integration/bundle-reference-performance.test.js b/test/integration/bundle-reference-performance.test.js new file mode 100644 index 00000000..a42da69c --- /dev/null +++ b/test/integration/bundle-reference-performance.test.js @@ -0,0 +1,152 @@ +/** + * Real-Git command-count proof for direct bundle references and immutable + * metadata coalescing. + * + * MUST run inside Docker (GIT_STUNTS_DOCKER=1). Refuses to run on the host. + */ + +import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest'; +import { mkdtempSync, rmSync } from 'node:fs'; +import { spawnSync } from 'node:child_process'; +import os from 'node:os'; +import path from 'node:path'; +import ContentAddressableStore from '../../index.js'; +import { createGitPlumbing } from '../../src/infrastructure/createGitPlumbing.js'; + +if (process.env.GIT_STUNTS_DOCKER !== '1') { + throw new Error( + 'Integration tests MUST run inside Docker (GIT_STUNTS_DOCKER=1). ' + + 'Use: npm run test:integration:node' + ); +} + +vi.setConfig({ testTimeout: 20_000, hookTimeout: 30_000 }); + +let repoDir; +let writer; +let outer; + +function git(args) { + const result = spawnSync('git', args, { cwd: repoDir, encoding: 'utf8' }); + if (result.error) { + throw result.error; + } + if (result.status !== 0) { + throw new Error(`${result.stderr || result.stdout || 'git failed'}`.trim()); + } + return result.stdout.trim(); +} + +function operationOf(args) { + if (args[0] === 'cat-file' && args.some((arg) => arg.startsWith('--batch-check='))) { + return 'cat-file:batch-check'; + } + return args[0]; +} + +async function countingReader() { + const plumbing = await createGitPlumbing({ cwd: repoDir }); + const counts = new Map(); + const record = (options) => { + const operation = operationOf(options.args); + counts.set(operation, (counts.get(operation) ?? 0) + 1); + }; + const counted = { + execute(options) { + record(options); + return plumbing.execute(options); + }, + executeStream(options) { + record(options); + return plumbing.executeStream(options); + }, + }; + return { + cas: new ContentAddressableStore({ plumbing: counted }), + snapshot() { + return new Map(counts); + }, + }; +} + +function count(snapshot, operation) { + return snapshot.get(operation) ?? 0; +} + +function total(snapshot) { + return [...snapshot.values()].reduce((sum, value) => sum + value, 0); +} + +function delta(after, before) { + const result = new Map(); + for (const [operation, value] of after) { + result.set(operation, value - count(before, operation)); + } + return result; +} + +async function collect(iterable) { + const values = []; + for await (const value of iterable) { + values.push(value); + } + return values; +} + +beforeAll(async () => { + repoDir = mkdtempSync(path.join(os.tmpdir(), 'cas-bundle-reference-perf-')); + git(['init', '--bare']); + writer = new ContentAddressableStore({ + plumbing: await createGitPlumbing({ cwd: repoDir }), + }); + const child = await writer.pages.put({ source: Buffer.from('child') }); + const nested = await writer.bundles.put({ members: { child: child.handle } }); + outer = await writer.bundles.put({ members: { nested: nested.handle } }); +}); + +afterAll(() => { + rmSync(repoDir, { recursive: true, force: true }); +}); + +describe('real-Git direct bundle reference reads', () => { + it('coalesces immutable Git metadata across repeated targeted reads', async () => { + const reader = await countingReader(); + await expect( + reader.cas.bundles.getMemberReference({ + handle: outer.handle, + path: 'nested', + }) + ).resolves.toMatchObject({ path: 'nested', type: 'tree' }); + const cold = reader.snapshot(); + + await reader.cas.bundles.getMemberReference({ handle: outer.handle, path: 'nested' }); + const warm = delta(reader.snapshot(), cold); + + expect(count(cold, 'ls-tree')).toBeGreaterThan(0); + expect(count(cold, 'cat-file:batch-check')).toBeGreaterThan(0); + expect(count(warm, 'ls-tree')).toBe(0); + expect(count(warm, 'cat-file:batch-check')).toBe(0); + expect(total(warm)).toBe(0); + }); + + it('does less Git work than complete recursive member validation', async () => { + const direct = await countingReader(); + const references = await collect( + direct.cas.bundles.iterateMemberReferences({ + handle: outer.handle, + }) + ); + const directCounts = direct.snapshot(); + + const complete = await countingReader(); + const members = await collect(complete.cas.bundles.iterateMembers({ handle: outer.handle })); + const completeCounts = complete.snapshot(); + + expect(references).toHaveLength(1); + expect(members).toHaveLength(1); + expect(total(directCounts)).toBeLessThan(total(completeCounts)); + expect(count(directCounts, 'cat-file:batch-check')).toBeLessThan( + count(completeCounts, 'cat-file:batch-check') + ); + }); +}); diff --git a/test/unit/domain/services/BundleService.test.js b/test/unit/domain/services/BundleService.test.js index 1652aba3..a1f7f557 100644 --- a/test/unit/domain/services/BundleService.test.js +++ b/test/unit/domain/services/BundleService.test.js @@ -263,6 +263,78 @@ describe('BundleService prevalidated references', () => { }); }); +describe('BundleService descriptor cache', () => { + it('coalesces bounded descriptor blobs across repeated reference reads', async () => { + const { bundles, pages, persistence } = makeServices(); + const page = await pages.put({ source: Buffer.from('page') }); + const staged = await bundles.put({ members: { page: page.handle } }); + const readBlob = vi.spyOn(persistence, 'readBlob'); + + await bundles.getMemberReference({ handle: staged.handle, path: 'page' }); + const coldReads = readBlob.mock.calls.length; + await bundles.getMemberReference({ handle: staged.handle, path: 'page' }); + + expect(coldReads).toBeGreaterThan(0); + expect(readBlob).toHaveBeenCalledTimes(coldReads); + }); +}); + +describe('BundleService public reference integrity', () => { + it('rejects a root summary that disagrees with the selected fanout root', async () => { + const { bundles, persistence } = makeServices(); + const staged = await bundles.put({ members: { selected: Buffer.from('value') } }); + const handle = await rewriteBundleRoot(persistence, staged.handle, (descriptor) => { + descriptor.memberCount += 1; + }); + + await expect(bundles.getMemberReference({ handle, path: 'selected' })) + .rejects.toMatchObject({ + code: 'BUNDLE_CORRUPT', + meta: { field: 'count', expected: 2, actual: 1 }, + }); + }); +}); + +describe('BundleService public references', () => { + it('reads public references without resolving nested support graphs', async () => { + const { bundles, pages, persistence } = makeServices(); + const child = await pages.put({ source: Buffer.from('child') }); + const nested = await bundles.put({ members: { child: child.handle } }); + const outer = await bundles.put({ members: { nested: nested.handle } }); + persistence.deleteObject(child.handle.oid); + + await expect(bundles.getMemberReference({ + handle: outer.handle, + path: 'nested', + })).resolves.toMatchObject({ + path: 'nested', + handle: nested.handle, + type: 'tree', + }); + await expect(bundles.getMember({ + handle: outer.handle, + path: 'nested', + })).rejects.toMatchObject({ code: 'HANDLE_TARGET_MISSING' }); + }); + + it('iterates public references without invoking target resolvers', async () => { + const { bundles, pages } = makeServices(); + const page = await pages.put({ source: Buffer.from('page') }); + const staged = await bundles.put({ members: { page: page.handle } }); + const resolveRoot = vi.spyOn(pages, 'resolveRoot'); + + const references = []; + for await (const reference of bundles.iterateMemberReferences({ handle: staged.handle })) { + references.push(reference); + } + + expect(references).toEqual([ + expect.objectContaining({ path: 'page', handle: page.handle, type: 'blob' }), + ]); + expect(resolveRoot).not.toHaveBeenCalled(); + }); +}); + describe('BundleService targeted reads and corruption evidence', () => { it('opens only the selected member payload', async () => { const { bundles, pages, persistence } = makeServices({ diff --git a/test/unit/domain/services/RootSetPersistence.test.js b/test/unit/domain/services/RootSetPersistence.test.js index 6d464549..314e7132 100644 --- a/test/unit/domain/services/RootSetPersistence.test.js +++ b/test/unit/domain/services/RootSetPersistence.test.js @@ -68,6 +68,20 @@ function structuredUpdateRefLock({ }); } +function emptyUpdateRefFailure({ + args = exactUpdateRefArgs(), + code = 128, +} = {}) { + return Object.assign(new Error(`Git command failed with code ${code}`), { + details: { + code, + args, + stderr: '', + stdout: '', + }, + }); +} + async function expectStructuredLockCode(error, expectedHeadOid, code) { const persistence = mockPersistence({ writeBlob: vi.fn().mockResolvedValue('a'.repeat(40)), @@ -83,6 +97,33 @@ async function expectStructuredLockCode(error, expectedHeadOid, code) { .rejects.toMatchObject({ code }); } +async function expectEmptyUpdateRefFailureCode({ + actualHeadOid, + args = exactUpdateRefArgs(), + gitExitCode = 128, + expectedCode, +}) { + const persistence = mockPersistence({ + writeBlob: vi.fn().mockResolvedValue('a'.repeat(40)), + writeTree: vi.fn().mockResolvedValue('b'.repeat(40)), + }); + const ref = mockRef({ + resolveRef: vi.fn().mockResolvedValue(actualHeadOid), + createCommit: vi.fn().mockResolvedValue(NEW_COMMIT_OID), + updateRef: vi.fn().mockRejectedValue(emptyUpdateRefFailure({ + args, + code: gitExitCode, + })), + }); + const rootSet = new RootSetPersistence({ rootSetRef: REF, persistence, ref }); + + await expect(rootSet.write({ entries: [], expectedHeadOid: EXPECTED_HEAD_OID })) + .rejects.toMatchObject({ + code: expectedCode, + meta: { expectedHeadOid: EXPECTED_HEAD_OID, actualHeadOid }, + }); +} + describe('RootSetPersistence snapshot writes', () => { it('writes real tree edges and a parentless current-generation commit', async () => { const persistence = mockPersistence({ @@ -173,6 +214,40 @@ describe('RootSetPersistence structured lock conflicts', () => { }); }); +describe('RootSetPersistence observed compare-and-swap conflicts', () => { + it('normalizes an empty fatal response when the exact managed ref advanced', async () => { + await expectEmptyUpdateRefFailureCode({ + actualHeadOid: 'e'.repeat(40), + expectedCode: 'ROOT_SET_CONFLICT', + }); + }); + + it.each([ + ['the managed ref did not advance', { + args: exactUpdateRefArgs(), + actualHeadOid: EXPECTED_HEAD_OID, + }], + ['the managed ref differed only by OID case', { + args: exactUpdateRefArgs(), + actualHeadOid: EXPECTED_HEAD_OID.toUpperCase(), + }], + ['the failed command targeted another ref', { + args: exactUpdateRefArgs('refs/heads/other'), + actualHeadOid: 'e'.repeat(40), + }], + ['the failure was not a fatal Git exit', { + args: exactUpdateRefArgs(), + actualHeadOid: 'e'.repeat(40), + gitExitCode: 1, + }], + ])('keeps the failure terminal when %s', async (_label, options) => { + await expectEmptyUpdateRefFailureCode({ + ...options, + expectedCode: 'ROOT_SET_REF_UPDATE_FAILED', + }); + }); +}); + describe('RootSetPersistence malformed structured locks', () => { const exactArgs = exactUpdateRefArgs(); const conflictText = diff --git a/test/unit/facade/ContentAddressableStore.application-storage.test.js b/test/unit/facade/ContentAddressableStore.application-storage.test.js index 50580bd8..555b70c3 100644 --- a/test/unit/facade/ContentAddressableStore.application-storage.test.js +++ b/test/unit/facade/ContentAddressableStore.application-storage.test.js @@ -35,7 +35,9 @@ describe('ContentAddressableStore application storage capabilities', () => { 'put', 'putOrdered', 'getMember', + 'getMemberReference', 'iterateMembers', + 'iterateMemberReferences', 'openMember', ]); expect(Object.keys(cas.caches)).toEqual(['open']); diff --git a/test/unit/helpers/boundedPromiseCache.test.js b/test/unit/helpers/boundedPromiseCache.test.js new file mode 100644 index 00000000..8fc207d9 --- /dev/null +++ b/test/unit/helpers/boundedPromiseCache.test.js @@ -0,0 +1,128 @@ +import { describe, expect, it, vi } from 'vitest'; +import BoundedPromiseCache from '../../../src/helpers/boundedPromiseCache.js'; + +describe('BoundedPromiseCache construction', () => { + it('rejects invalid residency options', () => { + expect(() => new BoundedPromiseCache(0)).toThrow('positive safe integer'); + expect(() => new BoundedPromiseCache(1, { maxWeight: -1 })).toThrow( + 'non-negative safe integer' + ); + expect(() => new BoundedPromiseCache(1, { weightOf: null })).toThrow( + 'weightOf must be a function' + ); + }); +}); + +describe('BoundedPromiseCache coalescing', () => { + it('coalesces in-flight work and retries rejected work', async () => { + let resolveValue; + const pending = new Promise((resolve) => { + resolveValue = resolve; + }); + const cache = new BoundedPromiseCache(2); + const sharedFactory = vi.fn().mockReturnValue(pending); + + const first = cache.getOrCreate('shared', sharedFactory); + const second = cache.getOrCreate('shared', sharedFactory); + resolveValue('value'); + + await expect(Promise.all([first, second])).resolves.toEqual(['value', 'value']); + expect(sharedFactory).toHaveBeenCalledTimes(1); + + const retryFactory = vi + .fn() + .mockRejectedValueOnce(new Error('transient')) + .mockResolvedValueOnce('recovered'); + await expect(cache.getOrCreate('retry', retryFactory)).rejects.toThrow('transient'); + await expect(cache.getOrCreate('retry', retryFactory)).resolves.toBe('recovered'); + expect(retryFactory).toHaveBeenCalledTimes(2); + }); +}); + +describe('BoundedPromiseCache rejected work', () => { + it('evicts a rejection before the returned promise settles', async () => { + const cache = new BoundedPromiseCache(1); + const factory = vi + .fn() + .mockRejectedValueOnce(new Error('transient')) + .mockResolvedValueOnce('recovered'); + + let firstError; + try { + await cache.getOrCreate('key', factory); + } catch (error) { + firstError = error; + } + + expect(firstError).toMatchObject({ message: 'transient' }); + await expect(cache.getOrCreate('key', factory)).resolves.toBe('recovered'); + expect(factory).toHaveBeenCalledTimes(2); + }); +}); + +describe('BoundedPromiseCache residency', () => { + it('evicts the least-recently-used entry at its count bound', async () => { + const cache = new BoundedPromiseCache(2); + const factory = vi.fn((value) => Promise.resolve(value)); + + await cache.getOrCreate('first', () => factory('first')); + await cache.getOrCreate('second', () => factory('second')); + await cache.getOrCreate('first', () => factory('first')); + await cache.getOrCreate('third', () => factory('third')); + await cache.getOrCreate('second', () => factory('second')); + + expect(factory).toHaveBeenCalledTimes(4); + }); + + it('evicts completed values by total weight', async () => { + const cache = new BoundedPromiseCache(3, { + maxWeight: 5, + weightOf: (value) => value.length, + }); + const first = vi.fn().mockResolvedValue('aaa'); + const second = vi.fn().mockResolvedValue('bbb'); + + await cache.getOrCreate('first', first); + await cache.getOrCreate('second', second); + await cache.getOrCreate('second', second); + await cache.getOrCreate('first', first); + + expect(first).toHaveBeenCalledTimes(2); + expect(second).toHaveBeenCalledTimes(1); + }); + + it('does not retain a value larger than the weight bound', async () => { + const cache = new BoundedPromiseCache(3, { + maxWeight: 2, + weightOf: (value) => value.length, + }); + const oversized = vi.fn().mockResolvedValue('oversized'); + const resident = vi.fn().mockResolvedValue('ok'); + + await cache.getOrCreate('resident', resident); + await cache.getOrCreate('value', oversized); + await cache.getOrCreate('value', oversized); + await cache.getOrCreate('resident', resident); + + expect(oversized).toHaveBeenCalledTimes(2); + expect(resident).toHaveBeenCalledTimes(1); + }); +}); + +describe('BoundedPromiseCache resolved weights', () => { + it.each([Number.NaN, -1, 1.5, Number.MAX_SAFE_INTEGER + 1])( + 'rejects and evicts invalid resolved weight %s', + async (weight) => { + const cache = new BoundedPromiseCache(1, { weightOf: () => weight }); + const factory = vi.fn().mockResolvedValue('value'); + + await expect(cache.getOrCreate('key', factory)).rejects.toThrow( + 'weightOf must return a non-negative safe integer' + ); + await expect(cache.getOrCreate('key', factory)).rejects.toThrow( + 'weightOf must return a non-negative safe integer' + ); + expect(factory).toHaveBeenCalledTimes(2); + } + ); +}); diff --git a/test/unit/infrastructure/adapters/GitPersistenceAdapter.readTree.test.js b/test/unit/infrastructure/adapters/GitPersistenceAdapter.readTree.test.js index d9283dc8..14da376f 100644 --- a/test/unit/infrastructure/adapters/GitPersistenceAdapter.readTree.test.js +++ b/test/unit/infrastructure/adapters/GitPersistenceAdapter.readTree.test.js @@ -88,6 +88,30 @@ describe('GitPersistenceAdapter.readTreeEntry() – path lookup', () => { it('returns null when git finds no matching path', async () => { await expect(adapterFor('').readTreeEntry('tree-oid', 'missing')).resolves.toBeNull(); }); + + it('coalesces repeated immutable path reads and isolates returned records', async () => { + const plumbing = mockPlumbing('040000 tree abc123\tdemo%2Fhello\0'); + const adapter = new GitPersistenceAdapter({ plumbing, policy: noPolicy }); + + const first = await adapter.readTreeEntry('tree-oid', 'demo%2Fhello'); + first.name = 'mutated'; + const second = await adapter.readTreeEntry('tree-oid', 'demo%2Fhello'); + + expect(second.name).toBe('demo%2Fhello'); + expect(plumbing.execute).toHaveBeenCalledTimes(1); + }); + + it('does not retain failed path reads', async () => { + const plumbing = mockPlumbing('040000 tree abc123\tdemo%2Fhello\0'); + plumbing.execute.mockRejectedValueOnce(new Error('transient read failure')); + const adapter = new GitPersistenceAdapter({ plumbing, policy: noPolicy }); + + await expect(adapter.readTreeEntry('tree-oid', 'demo%2Fhello')) + .rejects.toThrow('transient read failure'); + await expect(adapter.readTreeEntry('tree-oid', 'demo%2Fhello')) + .resolves.toMatchObject({ oid: 'abc123' }); + expect(plumbing.execute).toHaveBeenCalledTimes(2); + }); }); describe('GitPersistenceAdapter.readObjectType()', () => { @@ -120,6 +144,74 @@ describe('GitPersistenceAdapter.readObjectType()', () => { }); }); +describe('GitPersistenceAdapter immutable metadata cache', () => { + it('coalesces concurrent and sequential metadata reads by immutable OID', async () => { + const oid = 'a'.repeat(40); + let resolveInspection; + const inspection = new Promise((resolve) => { + resolveInspection = resolve; + }); + const plumbing = mockPlumbing(); + plumbing.execute.mockReturnValue(inspection); + const adapter = new GitPersistenceAdapter({ plumbing, policy: noPolicy }); + + const type = adapter.readObjectType(oid); + const size = adapter.readObjectSize(oid); + resolveInspection(`${oid} blob 42`); + + await expect(type).resolves.toBe('blob'); + await expect(size).resolves.toBe(42); + await expect(adapter.readObjectType(oid)).resolves.toBe('blob'); + expect(plumbing.execute).toHaveBeenCalledTimes(1); + }); + + it('retries failed metadata reads instead of caching rejection', async () => { + const oid = 'b'.repeat(40); + const plumbing = mockPlumbing(`${oid} tree 12`); + plumbing.execute.mockRejectedValueOnce(new Error('transient metadata failure')); + const adapter = new GitPersistenceAdapter({ plumbing, policy: noPolicy }); + + await expect(adapter.readObjectType(oid)).rejects.toThrow('transient metadata failure'); + await expect(adapter.readObjectType(oid)).resolves.toBe('tree'); + expect(plumbing.execute).toHaveBeenCalledTimes(2); + }); +}); + +describe('GitPersistenceAdapter metadata cache residency', () => { + it('rejects an invalid metadata cache bound', () => { + expect(() => new GitPersistenceAdapter({ + plumbing: mockPlumbing(), + policy: noPolicy, + metadataCacheEntries: 0, + })).toThrow(expect.objectContaining({ + code: 'INVALID_OPTIONS', + meta: { option: 'metadataCacheEntries', metadataCacheEntries: 0 }, + })); + }); + + it('evicts least-recently-used object metadata at its configured bound', async () => { + const oids = ['a', 'b', 'c'].map((value) => value.repeat(40)); + const plumbing = mockPlumbing(); + plumbing.execute.mockImplementation(({ input }) => { + const oid = input.trim(); + return Promise.resolve(`${oid} blob 1`); + }); + const adapter = new GitPersistenceAdapter({ + plumbing, + policy: noPolicy, + metadataCacheEntries: 2, + }); + + await adapter.readObjectType(oids[0]); + await adapter.readObjectType(oids[1]); + await adapter.readObjectType(oids[0]); + await adapter.readObjectType(oids[2]); + await adapter.readObjectType(oids[1]); + + expect(plumbing.execute).toHaveBeenCalledTimes(4); + }); +}); + describe('GitPersistenceAdapter.readObjectSize()', () => { it('reads a safe byte size without materializing the object', async () => { const oid = 'a'.repeat(40); diff --git a/test/unit/types/declaration-accuracy.test.js b/test/unit/types/declaration-accuracy.test.js index a6b47f57..abdaae2f 100644 --- a/test/unit/types/declaration-accuracy.test.js +++ b/test/unit/types/declaration-accuracy.test.js @@ -121,3 +121,16 @@ describe('Application-storage declaration accuracy', () => { expect(read('index.js')).not.toMatch(/export \{ default as CacheAcquisition \}/u); }); }); + +describe('Bundle reference declaration accuracy', () => { + it('declares direct bundle references and bounded Git metadata caching', () => { + const declarations = read('index.d.ts'); + + expect(declarations).toContain('export interface BundleMemberReference {'); + expect(declarations).toContain('Promise;'); + expect(declarations).toContain( + 'iterateMemberReferences(options: { handle: BundleHandleInput })' + ); + expect(declarations).toContain('metadataCacheEntries?: number;'); + }); +});