From 9208871802a596cbc508be725353532110d40198 Mon Sep 17 00:00:00 2001 From: James Ross Date: Sun, 19 Jul 2026 09:13:03 -0700 Subject: [PATCH 1/3] perf: preserve coherent Git object sessions --- CHANGELOG.md | 8 + .../git-object-session-coherence.md | 234 ++++++++++++++++++ docs/design/README.md | 1 + .../adapters/GitPersistenceAdapter.js | 21 +- .../git-object-session-coherence.test.js | 99 ++++++++ ...ersistenceAdapter.sessionCoherence.test.js | 76 ++++++ .../GitPersistenceAdapter.sessions.test.js | 115 ++++----- 7 files changed, 471 insertions(+), 83 deletions(-) create mode 100644 docs/design/0053-git-object-session-coherence/git-object-session-coherence.md create mode 100644 test/integration/git-object-session-coherence.test.js create mode 100644 test/unit/infrastructure/adapters/GitPersistenceAdapter.sessionCoherence.test.js diff --git a/CHANGELOG.md b/CHANGELOG.md index dfc14bd..5c4e5e5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +### Performance + +- **Coherent Git object session reuse** - successful loose object and tree + writes preserve typed `cat-file` and `mktree` processes. Scoped bulk writes + preserve `cat-file` but still retire `mktree` after checkpoint because Git's + quick tree-entry validation does not refresh packs created after the + session's object-database snapshot. + ## [6.5.2] — 2026-07-19 ### Added diff --git a/docs/design/0053-git-object-session-coherence/git-object-session-coherence.md b/docs/design/0053-git-object-session-coherence/git-object-session-coherence.md new file mode 100644 index 0000000..e103177 --- /dev/null +++ b/docs/design/0053-git-object-session-coherence/git-object-session-coherence.md @@ -0,0 +1,234 @@ +--- +title: 'PERF-0053 - Git Object Session Coherence' +cycle: '0053' +task_id: 'git-object-session-coherence' +legend: 'PERF' +release_home: 'v6.5.3' +issue: 'https://github.com/git-stunts/git-cas/issues/94' +goalpost_issue: 'https://github.com/git-stunts/git-cas/issues/94' +tracker_source: 'github' +status: 'active' +base_commit: 'fb3d3c2b620bae11d52e8ecc7a78a7ea07f27e24' +owners: + - '@git-stunts' +sponsors: + human: 'James' + agent: 'Codex' +blocking_issues: [] +supersedes: [] +superseded_by: null +created: '2026-07-19' +updated: '2026-07-19' +--- + +# PERF-0053 - Git Object Session Coherence + +## Linked Tracker + +- Issue: [#94 - Preserve Git object sessions across coherent writes](https://github.com/git-stunts/git-cas/issues/94) +- Milestone: [`v6.5.3`](https://github.com/git-stunts/git-cas/milestone/13) +- Parent design: [PERF-0052](../0052-persistent-git-object-sessions/persistent-git-object-sessions.md) + +## Design Type + +This design is primarily: + +- [x] Runtime/API +- [x] Storage/substrate +- [x] Migration/release +- [ ] CLI/operator +- [ ] Docs/public guidance +- [ ] TUI/visual surface +- [x] Test/tooling + +## Decision Summary + +`GitPersistenceAdapter` will preserve a live `cat-file --batch-command` +session across every successful immutable object write. It will preserve a live +`mktree --batch -z` session across one-shot loose blob writes and loose tree +writes, but retire `mktree` after a scoped bulk `fast-import` write because +that operation may create a pack that an already prepared `mktree` process +cannot discover. The scoped `fast-import` process still checkpoints and closes +before any OID escapes. No public API, object identity, retention policy, or +lifecycle contract changes. + +## Hill + +By the end of this cycle, repeated git-warp materialization writes through one +CAS adapter reuse coherent Git reader and tree-writer processes instead of +restarting them after every loose object. Real-Git tests prove both sides of the +boundary: safe reuse after loose writes and mandatory `mktree` retirement +after a pack-producing bulk write. + +## Current Truth + +- A one-shot blob write retires both persistent object protocols after + `hash-object -w` succeeds. + [cite: `src/infrastructure/adapters/GitPersistenceAdapter.js#93-103@fb3d3c2b620bae11d52e8ecc7a78a7ea07f27e24`] +- A bulk blob write retires both persistent object protocols after checkpoint, + then separately retires the scoped `fast-import` process. + [cite: `src/infrastructure/adapters/GitPersistenceAdapter.js#114-151@fb3d3c2b620bae11d52e8ecc7a78a7ea07f27e24`] +- Both typed and fallback tree writes retire the persistent reader. + [cite: `src/infrastructure/adapters/GitPersistenceAdapter.js#160-178@fb3d3c2b620bae11d52e8ecc7a78a7ea07f27e24`] +- The session pool already supports independent protocol retirement, + invalidation, generation barriers, idle bounds, and explicit close. + [cite: `src/infrastructure/adapters/GitObjectSessionPool.js#39-117@fb3d3c2b620bae11d52e8ecc7a78a7ea07f27e24`] +- PERF-0052 correctly keeps individual blobs on one-shot `hash-object` and + scoped batches on checkpointed `fast-import`. + [cite: `docs/design/0052-persistent-git-object-sessions/persistent-git-object-sessions.md#220-240@fb3d3c2b620bae11d52e8ecc7a78a7ea07f27e24`] +- The existing real-Git gate proves scoped bulk writes reduce process count and + that one-shot writes recreate objects after external pruning. + [cite: `test/integration/bundle-reference-performance.test.js#193-247@fb3d3c2b620bae11d52e8ecc7a78a7ea07f27e24`] + +## Problem + +The implementation treats every successful object-database mutation as if it +invalidated every typed Git process. That is stricter than the protocols' +actual lookup behavior and collapses persistent sessions back into +process-per-object execution on alternating blob, tree, and read workloads. + +A 128-node retained git-warp materialization on `@git-stunts/git-cas@6.5.2` +opened 558 Git children: 140 `hash-object`, 82 `mktree`, and 80 +`cat-file`. A temporary audited candidate opened 401 children: the same 140 +one-shot blob writers, one `mktree`, and four `cat-file` processes. The +candidate therefore removes 157 unnecessary process launches without changing +the one-shot blob correctness boundary. + +## Audited Git Coherence Matrix + +The raw protocol matrix was exercised with Git 2.50.1 and Git 2.39.5 in both +SHA-1 and SHA-256 repositories. + +| Existing session | Subsequent write | Preserve? | Reason | +| -------------------------- | ------------------------------- | --------- | ------------------------------------------------- | +| `cat-file --batch-command` | loose `hash-object -w` | Yes | Direct lookup observes the new loose object. | +| `cat-file --batch-command` | checkpointed `fast-import` pack | Yes | Missing lookup refreshes packed object state. | +| `cat-file --batch-command` | loose `mktree` output | Yes | Direct lookup observes the new loose tree. | +| `mktree --batch -z` | loose `hash-object -w` | Yes | Loose lookup follows the prepared pack lookup. | +| `mktree --batch -z` | checkpointed `fast-import` pack | No | Quick lookup does not refresh prepared pack data. | + +The behavior follows upstream Git's lookup contract. `cat-file` uses +`odb_read_object_info_extended()` without `OBJECT_INFO_QUICK`, allowing a +second read to refresh source state. `mktree` explicitly requests +`OBJECT_INFO_QUICK`, which suppresses that second read. + +- [Upstream object lookup retry](https://github.com/git/git/blob/41365c2a9ba347870b80881c0d67454edd22fd49/odb.c#L550-L610) +- [Upstream packed-source refresh](https://github.com/git/git/blob/41365c2a9ba347870b80881c0d67454edd22fd49/odb/source-packed.c#L35-L58) +- [Upstream mktree quick lookup](https://github.com/git/git/blob/41365c2a9ba347870b80881c0d67454edd22fd49/builtin/mktree.c#L118-L143) + +## Scope + +This cycle includes exactly one runtime correction: + +- remove `cat-file` retirement after successful blob, bulk blob, and tree writes +- remove `mktree` retirement after successful one-shot loose blob writes +- retain `mktree` retirement after successful scoped bulk writes +- retain scoped `fast-import` checkpoint and retirement +- replace obsolete blanket-retirement tests with the exact coherence matrix +- add real-Git regression coverage and measured witness evidence + +## Non-Goals + +This cycle does not include: + +- changing the public API or TypeScript declarations +- reusing `fast-import` across individual writes or separate batches +- changing individual `writeBlob()` from one-shot `hash-object` +- adding workspace, RootSet, retention, TTL, LRU, or cache policy +- coalescing staging-workspace generations +- changing Git ref, commit, or history operations +- changing session idle timeout, retry, invalidation, or explicit close behavior +- claiming that every remaining git-warp Git process is necessary + +## Runtime Contract + +The externally observable contract is unchanged. Internally, successful writes +apply this retirement policy: + +| Operation | `cat-file` | `mktree` | `fast-import` | +| ---------------------------- | ---------- | ---------- | ------------- | +| One-shot loose blob write | Preserve | Preserve | Not opened | +| Scoped bulk blob write | Preserve | Retire | Retire | +| Typed or fallback tree write | Preserve | Preserve | Not opened | +| Failure in that protocol | Invalidate | Invalidate | Invalidate | +| Idle timeout | Retire | Retire | Retire | +| Adapter close | Close | Close | Close | + +## Determinism, Retention, and Lifecycle + +Session preservation changes process topology only. Content bytes, tree entry +ordering, Git OIDs, CAS handles, refs, retention witnesses, and publication +state remain unchanged. Existing protocol failures still invalidate their own +generation. Idle retirement still bounds abandoned reusable processes. +`ContentAddressableStore.close()` remains the deterministic resource boundary. + +## Risks + +- Removing `mktree` retirement after bulk writes would be a correctness bug + when `fast-import` leaves a pack above `fastimport.unpackLimit`. +- A timing-only benchmark could hide a process-topology regression in scheduler + noise. Tests must assert protocol opening counts directly. +- Small bulk batches may unpack into loose objects, so the real-Git regression + must exceed `fastimport.unpackLimit` to prove the unsafe packed case. +- External repository mutation can still affect a long-lived process. This + cycle governs writes performed through this adapter; it does not promise a + transactional snapshot over arbitrary concurrent Git maintenance. + +## Alternatives Considered + +### Preserve every session after every write + +Rejected. A pack-primed `mktree --batch -z` process cannot validate an object +introduced in a later pack because its quick lookup does not refresh prepared +pack state. + +### Retire every session after every write + +Rejected. This is the current behavior and needlessly discards coherent +`cat-file` sessions and `mktree` sessions after loose writes. + +### Raise the idle timeout + +Rejected. A 30-second temporary timeout produced the same 558-process +git-warp trace because explicit retirement, not idle expiry, caused churn. + +### Add a workspace batch API + +Rejected for this cycle. Trie branches depend on child handles and workspace +retention has separate safety obligations. Combining that change would violate +the one-change release boundary. + +## Tests To Write First + +1. A one-shot blob write preserves already opened `cat-file` and `mktree`. +2. Typed and fallback tree writes preserve already opened `cat-file`. +3. A scoped bulk write preserves `cat-file`, retires `mktree`, and retires + `fast-import`. +4. A real-Git session reads a new loose blob and tree without reopening. +5. A real-Git reader discovers a later pack without reopening. +6. A real-Git pack-primed `mktree` is replaced after a pack-producing batch + and validates the new object through the replacement. +7. Existing prune/rewrite, failure, idle, close, SHA-1, and SHA-256 gates pass. + +## Acceptance Criteria + +- [ ] Every checkbox in issue #94 is proven. +- [ ] No public export or declaration changes. +- [ ] The unit retirement matrix is explicit and complete. +- [ ] Real-Git tests exceed `fastimport.unpackLimit`. +- [ ] Full unit, integration, platform, lint, and release gates pass. +- [ ] Witness evidence distinguishes deterministic process counts from local timing. +- [ ] The complete diff passes self-review against `origin/main`. + +## Implementation Slices + +1. Commit design, failing matrix tests, and raw coherence evidence. +2. Apply the narrow adapter retirement correction. +3. Run full gates and record the witness. +4. Review, publish one PR, and release `v6.5.3`. + +## Follow-On Debt + +Git-warp still performs one-shot blob writes and frequent workspace RootSet, +ref, and commit updates. Their necessity and any safe coalescing contract remain +separate work. This cycle neither solves nor obscures that remaining cost. diff --git a/docs/design/README.md b/docs/design/README.md index 1e0bc76..a9377e5 100644 --- a/docs/design/README.md +++ b/docs/design/README.md @@ -11,6 +11,7 @@ process in [docs/method/process.md](../method/process.md). ## Active METHOD Cycles +- [0053-git-object-session-coherence - git-object-session-coherence](./0053-git-object-session-coherence/git-object-session-coherence.md) - [0050-lazy-bundle-reference-reads - lazy-bundle-reference-reads](./0050-lazy-bundle-reference-reads/lazy-bundle-reference-reads.md) - [0049-scoped-staging-workspaces — scoped-staging-workspaces](./0049-scoped-staging-workspaces/scoped-staging-workspaces.md) - [0048-scoped-cache-acquisitions — scoped-cache-acquisitions](./0048-scoped-cache-acquisitions/scoped-cache-acquisitions.md) diff --git a/src/infrastructure/adapters/GitPersistenceAdapter.js b/src/infrastructure/adapters/GitPersistenceAdapter.js index 3e31066..7ed0da6 100644 --- a/src/infrastructure/adapters/GitPersistenceAdapter.js +++ b/src/infrastructure/adapters/GitPersistenceAdapter.js @@ -99,7 +99,6 @@ export default class GitPersistenceAdapter extends GitPersistencePort { input: content, }) ); - await this.#retireSessions(['catFile', 'mktree']); return oid; } @@ -128,7 +127,8 @@ export default class GitPersistenceAdapter extends GitPersistencePort { const oids = await this.#sessions.writeBlobs(replayableContents, (operation) => this.policy.execute(operation) ); - await this.#retireSessions(['catFile', 'mktree']); + // mktree's quick lookup cannot discover packs created after its ODB snapshot. + await this.#sessions.retire('mktree'); result = [...oids]; } catch (error) { operationFailed = true; @@ -164,7 +164,6 @@ export default class GitPersistenceAdapter extends GitPersistencePort { const oid = await this.#sessions.writeTree(structured, (operation) => this.policy.execute(operation) ); - await this.#sessions.retire('catFile'); return oid; } const oid = await this.policy.execute(() => @@ -173,7 +172,6 @@ export default class GitPersistenceAdapter extends GitPersistencePort { input: `${entries.join('\n')}\n`, }) ); - await this.#sessions.retire('catFile'); return oid; }); } @@ -553,21 +551,6 @@ export default class GitPersistenceAdapter extends GitPersistencePort { return promise; } - async #retireSessions(protocols) { - const results = await Promise.allSettled( - protocols.map((protocol) => this.#sessions.retire(protocol)) - ); - const failures = results - .filter((result) => result.status === 'rejected') - .map((result) => result.reason); - if (failures.length === 1) { - throw failures[0]; - } - if (failures.length > 1) { - throw new AggregateError(failures, 'Multiple Git object sessions failed to retire'); - } - } - static #isObjectBufferLimit(error) { return error?.details?.code === 'OBJECT_BUFFER_LIMIT_EXCEEDED'; } diff --git a/test/integration/git-object-session-coherence.test.js b/test/integration/git-object-session-coherence.test.js new file mode 100644 index 0000000..deccb8e --- /dev/null +++ b/test/integration/git-object-session-coherence.test.js @@ -0,0 +1,99 @@ +/** + * Real-Git coherence proof for typed object sessions across CAS writes. + * + * MUST run inside Docker (GIT_STUNTS_DOCKER=1). Refuses to run on the host. + */ + +import { spawnSync } from 'node:child_process'; +import { mkdtempSync, rmSync } from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import { describe, expect, it, vi } from 'vitest'; +import { createCountingGitPlumbing } from '../../scripts/diagnostics/createCountingGitPlumbing.js'; +import GitPersistenceAdapter from '../../src/infrastructure/adapters/GitPersistenceAdapter.js'; + +if (process.env.GIT_STUNTS_DOCKER !== '1') { + throw new Error( + 'Integration tests MUST run inside Docker (GIT_STUNTS_DOCKER=1). ' + + 'Use: npm run test:integration:node' + ); +} + +vi.setConfig({ testTimeout: 30_000 }); + +describe.each(['sha1', 'sha256'])('real-Git %s object session coherence', (objectFormat) => { + it('preserves coherent sessions and retires mktree after a packed bulk write', async () => { + const repo = mkdtempSync(path.join(os.tmpdir(), `cas-session-coherence-${objectFormat}-`)); + initializeRepository(repo, objectFormat); + const counted = await createCountingGitPlumbing({ cwd: repo, sessions: true }); + const adapter = new GitPersistenceAdapter({ plumbing: counted.plumbing }); + + try { + const initialPacked = await adapter.writeBlobs(batch('initial', 128)); + const initialOid = requireLast(initialPacked, 'Initial packed fixture'); + await expect(adapter.readObjectType(initialOid)).resolves.toBe('blob'); + const initialTree = await adapter.writeTree([treeEntry(initialOid, 'initial')]); + await expect(adapter.readObjectType(initialTree)).resolves.toBe('tree'); + + const looseOid = await adapter.writeBlob(Buffer.from('later loose object')); + await expect(adapter.readObjectType(looseOid)).resolves.toBe('blob'); + const looseTree = await adapter.writeTree([treeEntry(looseOid, 'loose')]); + await expect(adapter.readObjectType(looseTree)).resolves.toBe('tree'); + + const beforePackedWrite = counted.snapshot(); + expect(count(beforePackedWrite, 'session:cat-file')).toBe(1); + expect(count(beforePackedWrite, 'session:mktree')).toBe(1); + + const laterPacked = await adapter.writeBlobs(batch('later', 128)); + const laterOid = requireLast(laterPacked, 'Later packed fixture'); + await expect(adapter.readObjectType(laterOid)).resolves.toBe('blob'); + const laterTree = await adapter.writeTree([treeEntry(laterOid, 'later')]); + await expect(adapter.readObjectType(laterTree)).resolves.toBe('tree'); + + const afterPackedWrite = counted.snapshot(); + expect(count(afterPackedWrite, 'session:cat-file')).toBe(1); + expect(count(afterPackedWrite, 'session:mktree')).toBe(2); + expect(count(afterPackedWrite, 'session:fast-import')).toBe(2); + expect(git(repo, ['count-objects', '-v'])).toMatch(/packs: 2/u); + } finally { + await adapter.close(); + rmSync(repo, { recursive: true, force: true }); + } + }); +}); + +function initializeRepository(repo, objectFormat) { + git(repo, ['init', '--bare', `--object-format=${objectFormat}`]); + git(repo, ['config', 'fastimport.unpackLimit', '100']); +} + +function git(repo, args) { + const result = spawnSync('git', args, { cwd: repo, encoding: 'utf8' }); + if (result.error) { + throw result.error; + } + if (result.status !== 0) { + throw new Error((result.stderr || result.stdout || 'git failed').trim()); + } + return result.stdout.trim(); +} + +function batch(prefix, pageCount) { + return Array.from({ length: pageCount }, (_, index) => Buffer.from(`${prefix}-${index}`)); +} + +function requireLast(oids, name) { + const oid = oids.at(-1); + if (oid === undefined) { + throw new Error(`${name} did not produce an object`); + } + return oid; +} + +function treeEntry(oid, name) { + return `100644 blob ${oid}\t${name}`; +} + +function count(snapshot, operation) { + return snapshot.get(operation) ?? 0; +} diff --git a/test/unit/infrastructure/adapters/GitPersistenceAdapter.sessionCoherence.test.js b/test/unit/infrastructure/adapters/GitPersistenceAdapter.sessionCoherence.test.js new file mode 100644 index 0000000..f3ea830 --- /dev/null +++ b/test/unit/infrastructure/adapters/GitPersistenceAdapter.sessionCoherence.test.js @@ -0,0 +1,76 @@ +import { describe, expect, it, vi } from 'vitest'; +import GitPersistenceAdapter from '../../../../src/infrastructure/adapters/GitPersistenceAdapter.js'; + +const noPolicy = { execute: (operation) => operation() }; + +describe('GitPersistenceAdapter write coherence matrix', () => { + it('preserves cat-file and mktree sessions after a one-shot loose blob write', async () => { + const blobOid = 'a'.repeat(40); + const cat = catSession(blobOid); + const mktree = mktreeSession('b'.repeat(40)); + const plumbing = sessionPlumbing({ cat, mktree }); + plumbing.execute.mockResolvedValue(blobOid); + const adapter = new GitPersistenceAdapter({ plumbing, policy: noPolicy }); + await adapter.writeTree([]); + await adapter.readObjectType(blobOid); + + await expect(adapter.writeBlob(Buffer.from('value'))).resolves.toBe(blobOid); + + expect(cat.close).not.toHaveBeenCalled(); + expect(mktree.close).not.toHaveBeenCalled(); + await adapter.close(); + }); + + it('preserves cat-file and retires mktree after a scoped bulk write', async () => { + const blobOid = 'a'.repeat(40); + const cat = catSession(blobOid); + const mktree = mktreeSession('b'.repeat(40)); + const fastImport = { + writeBlob: vi.fn().mockResolvedValue('c'.repeat(40)), + checkpoint: vi.fn().mockResolvedValue(undefined), + close: vi.fn().mockResolvedValue(undefined), + abort: vi.fn().mockResolvedValue(undefined), + }; + const plumbing = sessionPlumbing({ cat, mktree, fastImport }); + const adapter = new GitPersistenceAdapter({ plumbing, policy: noPolicy }); + await adapter.writeTree([]); + await adapter.readObjectType(blobOid); + + await expect(adapter.writeBlobs([Buffer.from('value')])).resolves.toEqual(['c'.repeat(40)]); + + expect(cat.close).not.toHaveBeenCalled(); + expect(mktree.close).toHaveBeenCalledTimes(1); + expect(fastImport.close).toHaveBeenCalledTimes(1); + await adapter.close(); + }); +}); + +function catSession(oid) { + return { + info: vi.fn().mockResolvedValue({ oid, type: 'blob', size: 1 }), + read: vi.fn(), + close: vi.fn().mockResolvedValue(undefined), + terminate: vi.fn().mockResolvedValue(undefined), + }; +} + +function mktreeSession(oid) { + return { + write: vi.fn().mockResolvedValue(oid), + close: vi.fn().mockResolvedValue(undefined), + terminate: vi.fn().mockResolvedValue(undefined), + }; +} + +function sessionPlumbing({ cat, mktree, fastImport } = {}) { + const plumbing = { + execute: vi.fn(), + executeStream: vi.fn(), + openCatFileSession: vi.fn().mockResolvedValue(cat), + openMktreeSession: vi.fn().mockResolvedValue(mktree), + }; + if (fastImport !== undefined) { + plumbing.openFastImportSession = vi.fn().mockResolvedValue(fastImport); + } + return plumbing; +} diff --git a/test/unit/infrastructure/adapters/GitPersistenceAdapter.sessions.test.js b/test/unit/infrastructure/adapters/GitPersistenceAdapter.sessions.test.js index c766492..ee48333 100644 --- a/test/unit/infrastructure/adapters/GitPersistenceAdapter.sessions.test.js +++ b/test/unit/infrastructure/adapters/GitPersistenceAdapter.sessions.test.js @@ -356,14 +356,14 @@ describe('GitPersistenceAdapter persistent write sessions', () => { }); describe('GitPersistenceAdapter bulk write retirement failures', () => { - it('preserves cache and bulk session retirement failures', async () => { - const catCloseError = new Error('cat-file close failed'); + it('preserves mktree and bulk session retirement failures', async () => { + const mktreeCloseError = new Error('mktree close failed'); const fastImportCloseError = new Error('fast-import close failed'); - const cat = fakeCatSession({ - info: vi.fn().mockResolvedValue({ oid: 'a'.repeat(40), type: 'blob', size: 1 }), - close: vi.fn().mockRejectedValue(catCloseError), + const mktree = { + write: vi.fn().mockResolvedValue('a'.repeat(40)), + close: vi.fn().mockRejectedValue(mktreeCloseError), terminate: vi.fn().mockResolvedValue(undefined), - }); + }; const fastImport = { writeBlob: vi.fn().mockResolvedValue('b'.repeat(40)), checkpoint: vi.fn().mockResolvedValue(undefined), @@ -371,77 +371,57 @@ describe('GitPersistenceAdapter bulk write retirement failures', () => { abort: vi.fn().mockResolvedValue(undefined), }; const adapter = new GitPersistenceAdapter({ - plumbing: sessionPlumbing({ catSessions: [cat], fastImportSession: fastImport }), - policy: noPolicy, - }); - await adapter.readObjectType('a'.repeat(40)); - - const failure = await adapter.writeBlobs([Buffer.from('value')]).catch((error) => error); - - expect(failure).toBeInstanceOf(AggregateError); - expect(failure.errors).toEqual([catCloseError, fastImportCloseError]); - expect(cat.terminate).toHaveBeenCalledTimes(1); - expect(fastImport.abort).toHaveBeenCalledTimes(1); - }); -}); - -describe('GitPersistenceAdapter cache session retirement failures', () => { - it('preserves every cache session failure after an object write', async () => { - const catCloseError = new Error('cat-file close failed'); - const mktreeCloseError = new Error('mktree close failed'); - const cat = fakeCatSession({ - info: vi.fn().mockResolvedValue({ oid: 'a'.repeat(40), type: 'blob', size: 1 }), - close: vi.fn().mockRejectedValue(catCloseError), - terminate: vi.fn().mockResolvedValue(undefined), - }); - const mktree = { - write: vi.fn().mockResolvedValue('b'.repeat(40)), - close: vi.fn().mockRejectedValue(mktreeCloseError), - terminate: vi.fn().mockResolvedValue(undefined), - }; - const adapter = new GitPersistenceAdapter({ - plumbing: sessionPlumbing({ catSessions: [cat], mktreeSession: mktree }), + plumbing: sessionPlumbing({ mktreeSession: mktree, fastImportSession: fastImport }), policy: noPolicy, }); await adapter.writeTree([]); - await adapter.readObjectType('a'.repeat(40)); - const failure = await adapter.writeBlob(Buffer.from('value')).catch((error) => error); + const failure = await adapter.writeBlobs([Buffer.from('value')]).catch((error) => error); expect(failure).toBeInstanceOf(AggregateError); - expect(failure.errors).toEqual([catCloseError, mktreeCloseError]); - expect(cat.terminate).toHaveBeenCalledTimes(1); + expect(failure.errors).toEqual([mktreeCloseError, fastImportCloseError]); expect(mktree.terminate).toHaveBeenCalledTimes(1); + expect(fastImport.abort).toHaveBeenCalledTimes(1); }); }); describe('GitPersistenceAdapter explicit retirement barrier', () => { - it('does not open a replacement while the previous process drains', async () => { - const firstOid = 'c'.repeat(40); - const secondOid = 'd'.repeat(40); + it('does not open a replacement mktree while post-bulk retirement drains', async () => { const retired = deferred(); - const cat = fakeCatSession({ - info: vi.fn().mockResolvedValue({ oid: firstOid, type: 'blob', size: 1 }), + const first = { + write: vi.fn().mockResolvedValue('c'.repeat(40)), close: vi.fn().mockReturnValue(retired.promise), - }); - const replacement = fakeCatSession({ - info: vi.fn().mockResolvedValue({ oid: secondOid, type: 'tree', size: 1 }), - }); - const plumbing = sessionPlumbing({ catSessions: [cat, replacement] }); - plumbing.execute.mockResolvedValue('e'.repeat(40)); + terminate: vi.fn().mockResolvedValue(undefined), + }; + const replacement = { + write: vi.fn().mockResolvedValue('d'.repeat(40)), + close: vi.fn().mockResolvedValue(undefined), + terminate: vi.fn().mockResolvedValue(undefined), + }; + const fastImport = { + writeBlob: vi.fn().mockResolvedValue('e'.repeat(40)), + checkpoint: vi.fn().mockResolvedValue(undefined), + close: vi.fn().mockResolvedValue(undefined), + abort: vi.fn().mockResolvedValue(undefined), + }; + const plumbing = sessionPlumbing({ mktreeSession: first, fastImportSession: fastImport }); + plumbing.openMktreeSession + .mockReset() + .mockResolvedValueOnce(first) + .mockResolvedValueOnce(replacement); const adapter = new GitPersistenceAdapter({ plumbing, policy: noPolicy }); - await adapter.readObjectType(firstOid); - const write = adapter.writeBlob(Buffer.from('value')); - await vi.waitFor(() => expect(cat.close).toHaveBeenCalledTimes(1)); - const nextRead = adapter.readObjectType(secondOid); + await adapter.writeTree([]); + const write = adapter.writeBlobs([Buffer.from('value')]); + await vi.waitFor(() => expect(first.close).toHaveBeenCalledTimes(1)); + const nextTree = adapter.writeTree([]); await Promise.resolve(); - expect(plumbing.openCatFileSession).toHaveBeenCalledTimes(1); + expect(plumbing.openMktreeSession).toHaveBeenCalledTimes(1); retired.resolve(); - await expect(write).resolves.toBe('e'.repeat(40)); - await expect(nextRead).resolves.toBe('tree'); - expect(plumbing.openCatFileSession).toHaveBeenCalledTimes(2); + await expect(write).resolves.toEqual(['e'.repeat(40)]); + await expect(nextTree).resolves.toBe('d'.repeat(40)); + expect(plumbing.openMktreeSession).toHaveBeenCalledTimes(2); await adapter.close(); }); }); @@ -483,14 +463,18 @@ describe('GitPersistenceAdapter bulk write recovery', () => { }); describe('GitPersistenceAdapter persistent tree writes', () => { - it('converts existing mktree lines into structured session entries', async () => { + it('converts mktree lines and preserves an opened persistent reader', async () => { + const cat = fakeCatSession({ + info: vi.fn().mockResolvedValue({ oid: 'd'.repeat(40), type: 'blob', size: 1 }), + }); const mktree = { write: vi.fn().mockResolvedValue('c'.repeat(40)), close: vi.fn().mockResolvedValue(undefined), terminate: vi.fn().mockResolvedValue(undefined), }; - const plumbing = sessionPlumbing({ mktreeSession: mktree }); + const plumbing = sessionPlumbing({ catSessions: [cat], mktreeSession: mktree }); const adapter = new GitPersistenceAdapter({ plumbing, policy: noPolicy }); + await adapter.readObjectType('d'.repeat(40)); await expect(adapter.writeTree([`${'100644'} blob ${'d'.repeat(40)}\tpage`])).resolves.toBe( 'c'.repeat(40) @@ -499,12 +483,14 @@ describe('GitPersistenceAdapter persistent tree writes', () => { expect(mktree.write).toHaveBeenCalledWith([ { mode: '100644', type: 'blob', oid: 'd'.repeat(40), name: 'page' }, ]); + expect(cat.close).not.toHaveBeenCalled(); expect(plumbing.execute).not.toHaveBeenCalled(); + await adapter.close(); }); }); describe('GitPersistenceAdapter fallback tree writes', () => { - it('retires the persistent reader after a one-shot tree write', async () => { + it('preserves the persistent reader after a one-shot tree write', async () => { const blobOid = 'd'.repeat(40); const treeOid = 'e'.repeat(40); const cat = fakeCatSession({ @@ -517,7 +503,8 @@ describe('GitPersistenceAdapter fallback tree writes', () => { await expect(adapter.writeTree([])).resolves.toBe(treeOid); - expect(cat.close).toHaveBeenCalledTimes(1); + expect(cat.close).not.toHaveBeenCalled(); + await adapter.close(); }); }); @@ -551,7 +538,7 @@ describe('GitPersistenceAdapter lifecycle', () => { await adapter.writeTree([]); await adapter.writeBlobs([Buffer.from('value')]); - expect(cat.close).toHaveBeenCalledTimes(1); + expect(cat.close).not.toHaveBeenCalled(); expect(mktree.close).toHaveBeenCalledTimes(1); await Promise.all([adapter.close(), adapter.close(), adapter[Symbol.asyncDispose]()]); From df05c4e40915fc510797e96c793f0e5f32889972 Mon Sep 17 00:00:00 2001 From: James Ross Date: Sun, 19 Jul 2026 09:22:30 -0700 Subject: [PATCH 2/3] docs: record Git session coherence witness --- .../git-object-session-coherence.md | 14 +-- .../witness/measurements.json | 95 ++++++++++++++++ .../witness/verification.md | 102 ++++++++++++++++++ 3 files changed, 204 insertions(+), 7 deletions(-) create mode 100644 docs/design/0053-git-object-session-coherence/witness/measurements.json create mode 100644 docs/design/0053-git-object-session-coherence/witness/verification.md diff --git a/docs/design/0053-git-object-session-coherence/git-object-session-coherence.md b/docs/design/0053-git-object-session-coherence/git-object-session-coherence.md index e103177..b396109 100644 --- a/docs/design/0053-git-object-session-coherence/git-object-session-coherence.md +++ b/docs/design/0053-git-object-session-coherence/git-object-session-coherence.md @@ -212,13 +212,13 @@ the one-change release boundary. ## Acceptance Criteria -- [ ] Every checkbox in issue #94 is proven. -- [ ] No public export or declaration changes. -- [ ] The unit retirement matrix is explicit and complete. -- [ ] Real-Git tests exceed `fastimport.unpackLimit`. -- [ ] Full unit, integration, platform, lint, and release gates pass. -- [ ] Witness evidence distinguishes deterministic process counts from local timing. -- [ ] The complete diff passes self-review against `origin/main`. +- [x] Every checkbox in issue #94 is proven. +- [x] No public export or declaration changes. +- [x] The unit retirement matrix is explicit and complete. +- [x] Real-Git tests exceed `fastimport.unpackLimit`. +- [x] Full unit, integration, platform, lint, and release gates pass. +- [x] Witness evidence distinguishes deterministic process counts from local timing. +- [x] The complete diff passes self-review against `origin/main`. ## Implementation Slices diff --git a/docs/design/0053-git-object-session-coherence/witness/measurements.json b/docs/design/0053-git-object-session-coherence/witness/measurements.json new file mode 100644 index 0000000..81deabe --- /dev/null +++ b/docs/design/0053-git-object-session-coherence/witness/measurements.json @@ -0,0 +1,95 @@ +{ + "schema": "git-cas.git-object-session-coherence-measurement/v1", + "generatedAt": "2026-07-19T16:16:43Z", + "implementationCommit": "9208871802a596cbc508be725353532110d40198", + "environment": { + "node": "v26.0.0", + "git": "git version 2.50.1 (Apple Git-155)", + "platform": "darwin", + "architecture": "arm64" + }, + "protocolMatrix": { + "gitVersions": ["2.50.1", "2.39.5"], + "objectFormats": ["sha1", "sha256"], + "results": { + "catFileAfterLooseBlob": "preserved-and-readable", + "catFileAfterLooseTree": "preserved-and-readable", + "catFileAfterPackedBulk": "preserved-and-readable", + "mktreeAfterLooseBlob": "preserved-and-writable", + "mktreeAfterPackedBulk": "must-retire" + } + }, + "consumerExploratory": { + "repository": "git-stunts/git-warp", + "commit": "0d0cefa9f8ade44e58c0a83b65f85ce557d3e9dd", + "fixture": "temporary 128-node retained materialization diagnostic", + "samples": 3, + "baseline": { + "gitCas": "6.5.2", + "gitChildProcesses": 558, + "majorProcessCounts": { + "hash-object": 140, + "rev-parse": 87, + "mktree": 82, + "cat-file": 80, + "update-ref": 43, + "symbolic-ref": 43, + "commit-tree": 39, + "rev-list": 37 + }, + "medianMaterializationSeconds": 14.24, + "medianProcessCpuSeconds": 15.74, + "medianWallSeconds": 17.39 + }, + "candidate": { + "implementationCommit": "9208871802a596cbc508be725353532110d40198", + "gitChildProcesses": 401, + "majorProcessCounts": { + "hash-object": 140, + "rev-parse": 87, + "mktree": 1, + "cat-file": 4, + "update-ref": 43, + "symbolic-ref": 43, + "commit-tree": 39, + "rev-list": 37 + }, + "medianMaterializationSeconds": 12.72, + "medianProcessCpuSeconds": 13.83, + "medianWallSeconds": 16.93 + }, + "deltaPercent": { + "gitChildProcesses": -28.1, + "cat-file": -95.0, + "mktree": -98.8, + "materializationTime": -10.7, + "processCpu": -12.1, + "wall": -2.6 + }, + "interpretation": { + "processCounts": "deterministic for this fixture", + "timings": "local three-run medians; not a cross-platform guarantee", + "reproducibility": "exploratory consumer fixture was temporary; repository integration counts are the durable gate" + } + }, + "verification": { + "releaseVerify": { + "stepsPassed": 14, + "stepsTotal": 14, + "observedTests": 6826 + }, + "nodeIntegrationAfterFinalAssertionPass": { + "filesPassed": 13, + "testsPassed": 198 + }, + "platformBats": { + "parallelCommand": "pnpm test:platforms", + "parallelResult": "host-harness-failure-before-tests", + "parallelCause": "Homebrew moreutils parallel is not GNU parallel", + "serialCommand": "bats test/platform/runtimes.bats", + "serialTestsPassed": 3, + "serialTestsTotal": 3 + }, + "publicApiSemverImpact": "none" + } +} diff --git a/docs/design/0053-git-object-session-coherence/witness/verification.md b/docs/design/0053-git-object-session-coherence/witness/verification.md new file mode 100644 index 0000000..53a144d --- /dev/null +++ b/docs/design/0053-git-object-session-coherence/witness/verification.md @@ -0,0 +1,102 @@ +# Git Object Session Coherence Verification Witness + +Issue: [#94](https://github.com/git-stunts/git-cas/issues/94) +Milestone: [`v6.5.3`](https://github.com/git-stunts/git-cas/milestone/13) +Implementation: `9208871802a596cbc508be725353532110d40198` + +## Decision Boundary + +This release makes one internal correction: immutable writes no longer retire +Git object sessions that can lawfully observe the new object. `cat-file` is +preserved across loose blob, tree, and checkpointed bulk writes. `mktree` is +preserved across loose writes and retired after a bulk write that may create a +new pack. Scoped `fast-import` still checkpoints and closes before OIDs escape. + +The implementation keeps individual blob writes on one-shot `hash-object`, +retires only `mktree` after a scoped bulk write, and leaves typed and fallback +tree writes coherent with the existing reader. +[cite: `src/infrastructure/adapters/GitPersistenceAdapter.js#93-175@9208871802a596cbc508be725353532110d40198`] + +## Protocol Evidence + +Raw protocol probes covered Git 2.50.1 and Git 2.39.5 in SHA-1 and SHA-256 +repositories. The observed matrix was: + +| Existing session | Later write | Result | +| -------------------------- | ----------------- | -------------------------- | +| `cat-file --batch-command` | loose blob | preserved and readable | +| `cat-file --batch-command` | loose tree | preserved and readable | +| `cat-file --batch-command` | checkpointed pack | preserved and readable | +| `mktree --batch -z` | loose blob | preserved and writable | +| `mktree --batch -z` | checkpointed pack | stale; retirement required | + +The committed real-Git test forces two 128-object packs above +`fastimport.unpackLimit`, opens the reader and tree writer before subsequent +writes, reads later blobs and trees, and asserts one reader but two `mktree` +sessions in both object formats. +[cite: `test/integration/git-object-session-coherence.test.js#24-60@9208871802a596cbc508be725353532110d40198`] + +The unit matrix separately fixes the retirement policy as a direct contract: +loose writes preserve both persistent protocols; bulk writes preserve the +reader, retire `mktree`, and close `fast-import`. +[cite: `test/unit/infrastructure/adapters/GitPersistenceAdapter.sessionCoherence.test.js#6-45@9208871802a596cbc508be725353532110d40198`] + +## Verification Results + +| Gate | Result | +| ------------------------------ | ---------------------------------------- | +| `pnpm lint` | PASS | +| Node unit | PASS - 2,081 tests, 2 skipped | +| Bun unit | PASS - 2,080 tests, 3 skipped | +| Deno unit | PASS - 2,071 tests, 12 skipped | +| Node integration | PASS - 198 tests | +| Bun integration | PASS - 198 tests | +| Deno integration | PASS - 198 tests | +| Public Deno type compatibility | PASS | +| Three executable examples | PASS | +| npm pack dry-run | PASS | +| JSR publish dry-run | PASS | +| `pnpm release:verify` | PASS - 14/14 steps, 6,826 observed tests | +| Bats platform cases, serial | PASS - 3/3 runtimes | +| Prettier check | PASS | +| `git diff --check` | PASS | +| Graft exported API review | PASS - semver impact `none` | + +`pnpm test:platforms` did not execute tests on this macOS host because Bats +resolved Homebrew `moreutils`' incompatible `parallel` executable. Running the +same `test/platform/runtimes.bats` cases serially passed Node, Bun, and Deno. +This is recorded as a harness limitation, not represented as a passing parallel +command. + +Lifecycle and error-path coverage still proves independent retirement barriers, +combined `mktree`/`fast-import` close failures, idempotent adapter close, and +reader shutdown at the explicit lifecycle boundary. +[cite: `test/unit/infrastructure/adapters/GitPersistenceAdapter.sessions.test.js#358-550@9208871802a596cbc508be725353532110d40198`] + +## Consumer A/B + +A temporary 128-node retained-materialization diagnostic in git-warp compared +released git-cas 6.5.2 with the implementation commit: + +| Metric | 6.5.2 | Candidate | Delta | +| -------------------------------- | ------: | --------: | -----: | +| Git child processes | 558 | 401 | -28.1% | +| `cat-file` processes | 80 | 4 | -95.0% | +| `mktree` processes | 82 | 1 | -98.8% | +| Median materialization test time | 14.24 s | 12.72 s | -10.7% | +| Median process CPU | 15.74 s | 13.83 s | -12.1% | +| Median wall time | 17.39 s | 16.93 s | -2.6% | + +The process topology is the high-confidence result. Timing values are local +three-run medians and are not a cross-platform performance promise. The +consumer fixture was temporary, so these values remain exploratory; the +committed real-Git session-count test is the durable regression gate. Full +machine-readable values and caveats are in +[`measurements.json`](./measurements.json). + +## Residual Scope + +This release does not persist `fast-import`, replace one-shot `hash-object`, +coalesce git-warp RootSet generations, change refs or retention, or promise +coherence across arbitrary external Git maintenance. Those costs and contracts +remain separate work. From 7adeabe950a3c841caffd1a3a4f82a696df86e71 Mon Sep 17 00:00:00 2001 From: James Ross Date: Sun, 19 Jul 2026 09:24:01 -0700 Subject: [PATCH 3/3] docs: normalize witness metadata formatting --- .../witness/verification.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/design/0053-git-object-session-coherence/witness/verification.md b/docs/design/0053-git-object-session-coherence/witness/verification.md index 53a144d..37ae9e1 100644 --- a/docs/design/0053-git-object-session-coherence/witness/verification.md +++ b/docs/design/0053-git-object-session-coherence/witness/verification.md @@ -1,7 +1,9 @@ # Git Object Session Coherence Verification Witness -Issue: [#94](https://github.com/git-stunts/git-cas/issues/94) -Milestone: [`v6.5.3`](https://github.com/git-stunts/git-cas/milestone/13) +Issue: [#94](https://github.com/git-stunts/git-cas/issues/94) + +Milestone: [`v6.5.3`](https://github.com/git-stunts/git-cas/milestone/13) + Implementation: `9208871802a596cbc508be725353532110d40198` ## Decision Boundary