feat(improvement): a verify disposition that can pass a tree and keep spending shots

CHANGELOG.md

@@ -1,5 +1,30 @@
 # Changelog
 
+## 0.174.0
+
+### `keepGoing` and `score`: spend the whole shot budget, ship the best tree
+
+`agenticGenerator`'s `Verifier` answered one boolean, and `ok` carried two meanings at once — "this tree is shippable" and "stop now". The shot loop returned inside `if (result.ok)` and `AgenticGeneratorOptions` exposed no field to override it, so ordinary best-of-n could not be expressed: a caller who wanted to spend the shots it was given and keep the best tree had no way to say so.
+
+A consumer building best-of-n hit exactly that and encoded it by hand (agent-lab's playproof study, `projects/playproof/verify-budget.ts`): reject every shot but the last whatever it measured, score each tree as it is produced, and physically write the best program back into the worktree just before the last shot is accepted — because with only one boolean, "ship the best tree" has to mean "make the best tree BE the worktree". Their measured runs are why it matters. Under first-acceptance-wins, 1 shot of 3 fired and the program that shipped was never once run by its author. Under the workaround, 3 of 3 fired and every shot ran its own program.
+
+`VerifyResult` gains two optional fields, so three separate questions get three separate answers:
+
+| field | question | omitted |
+|---|---|---|
+| `ok` | is this tree shippable | unchanged |
+| `keepGoing` | should the budget stop here | the first passing tree ends the candidate, exactly as before |
+| `score` | how does this tree rank against the other passing trees | every passing tree ties, so the later one wins |
+
+- **The loop owns the restore, and that is the point.** A passing tree whose verifier asks for another shot is snapshotted as a Git tree object (staged into a private index, so the index the driver commits from is untouched). When the budget ends, the highest-scoring tree is put back into the worktree — content, added files, and the removal of files only a losing shot wrote — and the restore is proved by re-snapshotting and comparing tree ids before the candidate is returned. The caller ranks; the runtime moves the bytes. Without this half, every caller wanting best-of-n still hand-rolls the write-back, which is the thing being fixed.
+- **Compatibility is an explicit gate, not a claim.** `tests/agentic-generator.test.ts` drives the real `agenticGenerator` with a verifier returning today's shape and asserts today's behaviour: 1 of 3 shots fires, the disposition stream is exactly `['accepted']` with `restoredFromShot: null`, and the accepted shot's own tree is what lands. A second gate holds the failure path — a verifier that never passes still feeds `verification FAILED` into the next shot and still ships nothing.
+- **A last shot that breaks or reverts the change no longer costs the candidate.** A banked tree passed verification, so it ships even when the final shot ends on a broken or empty tree. The failing shot's own `rejected` disposition is still emitted first, so its evidence survives. The invariant is intact: a tree that failed verification is never what ships.
+- **A tie keeps the LATER tree.** It is already on disk, so no restore is needed, and it is the author's own refinement of the tree it tied with.
+- **A set of trees that cannot be ordered fails the run.** Scoring one passing tree and not another throws rather than guessing an order, and a non-finite score throws. A tree that FAILED verification is never ranked, whatever it scored.
+- **`onShotDisposition` gains `kept`**, the shot that passed and was sent back: it carries the `score`, whether the tree `best`s the candidate so far, and the verifier's `feedback`. `accepted` gains `restoredFromShot` — non-null is the record that best-of-n moved bytes rather than only ranking them.
+- **The next-shot note is new text for a new state.** A passing tree sent back is told `verification PASSED`, that shots remain, and that the best version it produces is the one that ships — not the `verification FAILED` note, which would be a lie.
+- **`cli-worktree`, `cli-in-place` and `commandVerifier` are unchanged.** This is the verify/shot-loop contract only; `commandVerifier` still answers `{ok:true}` / `{ok:false, feedback}` and still stops at the first passing tree.
+
 ## 0.173.0
 
 ### `cli-in-place`: a local coding CLI on the worktree you hand it

api-surface.json

@@ -78,7 +78,7 @@
       "AgentTaskStatus": "type 4087243dc453",
       "AgenticGeneratorExecutorForWorktree": "type a1493d23fed8",
       "AgenticGeneratorOptions": "type 1d5e47e57c3b",
-      "AgenticGeneratorShotDisposition": "type 657d4fd128fa",
+      "AgenticGeneratorShotDisposition": "type dc9795eb57ec",
       "AgenticGeneratorShotExecution": "type c0739e352ea6",
       "AgenticGeneratorShotReceipt": "type c73c53ad3197",
       "AnalystRegistry": "type 4be7b4a3eda8",
@@ -323,7 +323,7 @@
       "VerifiedAgentCandidate": "type a8159a2b2fc2",
       "VerifiedAgentCandidateTaskOutcome": "type e78360db85cb",
       "Verifier": "type c596e7edac69",
-      "VerifyResult": "type fe21ce933c15",
+      "VerifyResult": "type 0864be50c38c",
       "VetoedFact": "type b762517293cc",
       "WorkerTraceEvidence": "type 34812cbffcbf",
       "WorkerTraceUnavailableReason": "type 0dcb14d1071b",

docs/api/index.md

@@ -4792,8 +4792,15 @@ so trace stitching survives nested orchestration.
 
 ### VerifyResult
 
-Outcome of verifying a candidate worktree. `feedback` (compiler errors,
- failing test output) is fed into the next shot when `ok` is false.
+Outcome of verifying a candidate worktree.
+
+`ok` answers "is this tree shippable". `keepGoing` answers "should the budget
+stop here", and `score` ranks this tree against the other trees the same
+candidate produced — three separate questions, so a verifier can pass a tree
+and still spend the shots it was given.
+
+`feedback` (compiler errors, failing test output, or the reason a passing
+tree is being sent back) is fed into the next shot.
 
 #### Properties
 
@@ -4805,6 +4812,28 @@ Outcome of verifying a candidate worktree. `feedback` (compiler errors,
 
 > `optional` **feedback?**: `string`
 
+##### keepGoing?
+
+> `optional` **keepGoing?**: `boolean`
+
+Spend the remaining shots instead of returning this tree now.
+
+Read only when `ok` is true: a failed verification already spends the next
+shot. Omitted means the first passing tree ends the candidate.
+
+##### score?
+
+> `optional` **score?**: `number`
+
+How good this tree is, for ranking it against the other passing trees of
+this candidate. Higher wins; a tie keeps the LATER tree, which is the one
+already on disk and the one the author refined last.
+
+Only a passing tree is ranked — a tree that failed verification is never a
+candidate, whatever it scored. Score every passing tree or none of them: a
+scored tree cannot be ranked against an unscored one, and mixing the two
+fails the run rather than guessing an order.
+
 ***
 
 ### AgenticGeneratorShotReceipt
@@ -5020,8 +5049,10 @@ readonly `ProposalFinding`[]
 Verify the worktree after each dirtying shot. When set, a candidate that
  fails verification is NOT returned — the failure feeds the next shot
  (verify-in-session), up to `maxShots`; a candidate that never verifies is
- discarded (`applied:false`), never shipped. Omitted means the first dirty
- shot is the candidate. See `commandVerifier`.
+ discarded (`applied:false`), never shipped. A verifier that returns
+ `keepGoing` passes a tree AND spends the remaining shots, and the
+ best-scoring tree is the one that ships. Omitted means the first dirty
+ shot is the candidate. See `commandVerifier` and `VerifyResult`.
 
 ##### isDirty?
 
@@ -11297,12 +11328,90 @@ Runtime's exact terminal turn plus its complete normalized event stream.
 
 ### AgenticGeneratorShotDisposition
 
-> **AgenticGeneratorShotDisposition** = \{ `kind`: `"clean"`; `worktreePath`: `string`; \} \| \{ `kind`: `"rejected"`; `worktreePath`: `string`; `stage`: `"raw-trace-evidence"` \| `"verification"`; `feedback`: `string` \| `null`; \} \| \{ `kind`: `"accepted"`; `worktreePath`: `string`; `verified`: `boolean`; \} \| \{ `kind`: `"setup-error"`; `worktreePath`: `string`; `stage`: `"worktree-inspection"` \| `"raw-trace-evidence"` \| `"verification"`; `error`: \{ `name`: `string`; `message`: `string`; \}; \}
+> **AgenticGeneratorShotDisposition** = \{ `kind`: `"clean"`; `worktreePath`: `string`; \} \| \{ `kind`: `"rejected"`; `worktreePath`: `string`; `stage`: `"raw-trace-evidence"` \| `"verification"`; `feedback`: `string` \| `null`; \} \| \{ `kind`: `"kept"`; `worktreePath`: `string`; `score`: `number` \| `null`; `best`: `boolean`; `feedback`: `string` \| `null`; \} \| \{ `kind`: `"accepted"`; `worktreePath`: `string`; `verified`: `boolean`; `restoredFromShot`: `number` \| `null`; \} \| \{ `kind`: `"setup-error"`; `worktreePath`: `string`; `stage`: `"worktree-inspection"` \| `"raw-trace-evidence"` \| `"verification"`; `error`: \{ `name`: `string`; `message`: `string`; \}; \}
 
 Worktree decision emitted before a completed shot is retried, accepted, or
  discarded. The callback runs while `worktreePath` is still available, so
  callers can persist the exact diff.
 
+#### Union Members
+
+##### Type Literal
+
+\{ `kind`: `"clean"`; `worktreePath`: `string`; \}
+
+***
+
+##### Type Literal
+
+\{ `kind`: `"rejected"`; `worktreePath`: `string`; `stage`: `"raw-trace-evidence"` \| `"verification"`; `feedback`: `string` \| `null`; \}
+
+***
+
+##### Type Literal
+
+\{ `kind`: `"kept"`; `worktreePath`: `string`; `score`: `number` \| `null`; `best`: `boolean`; `feedback`: `string` \| `null`; \}
+
+###### kind
+
+> `readonly` **kind**: `"kept"`
+
+The tree passed verification and the verifier asked for another shot,
+ so it was snapshotted and the budget continues.
+
+###### worktreePath
+
+> `readonly` **worktreePath**: `string`
+
+###### score
+
+> `readonly` **score**: `number` \| `null`
+
+The rank the verifier gave this tree, or null when it scored nothing.
+
+###### best
+
+> `readonly` **best**: `boolean`
+
+Whether this tree is now the best one this candidate has produced.
+
+###### feedback
+
+> `readonly` **feedback**: `string` \| `null`
+
+***
+
+##### Type Literal
+
+\{ `kind`: `"accepted"`; `worktreePath`: `string`; `verified`: `boolean`; `restoredFromShot`: `number` \| `null`; \}
+
+###### kind
+
+> `readonly` **kind**: `"accepted"`
+
+###### worktreePath
+
+> `readonly` **worktreePath**: `string`
+
+###### verified
+
+> `readonly` **verified**: `boolean`
+
+###### restoredFromShot
+
+> `readonly` **restoredFromShot**: `number` \| `null`
+
+One-based shot whose tree was put back into the worktree because it
+ outranked the tree on disk; null when the tree on disk is the one that
+ ships. Non-null is the record that best-of-n moved bytes rather than
+ only ranking them.
+
+***
+
+##### Type Literal
+
+\{ `kind`: `"setup-error"`; `worktreePath`: `string`; `stage`: `"worktree-inspection"` \| `"raw-trace-evidence"` \| `"verification"`; `error`: \{ `name`: `string`; `message`: `string`; \}; \}
+
 ***
 
 ### AgenticGeneratorExecutorForWorktree

docs/api/primitive-catalog.md

@@ -7,7 +7,7 @@
 
 # Primitive catalog — the never-stale anti-reinvention inventory
 
-> **GENERATED** from `@tangle-network/agent-runtime@0.173.0` and `@tangle-network/agent-eval@0.170.0` by `scripts/gen-primitive-catalog.mjs`. Do NOT hand-edit — run `pnpm run docs:api`. This is the mechanical companion to the JUDGMENT in `canonical-api.md` (§2 decision table + §1.5 AgentProfile law): that doc says WHICH primitive to reach for and what NOT to build; this catalog proves WHAT exists. Per-symbol signatures + `file:line` live in the per-module pages under `docs/api/`.
+> **GENERATED** from `@tangle-network/agent-runtime@0.174.0` and `@tangle-network/agent-eval@0.170.0` by `scripts/gen-primitive-catalog.mjs`. Do NOT hand-edit — run `pnpm run docs:api`. This is the mechanical companion to the JUDGMENT in `canonical-api.md` (§2 decision table + §1.5 AgentProfile law): that doc says WHICH primitive to reach for and what NOT to build; this catalog proves WHAT exists. Per-symbol signatures + `file:line` live in the per-module pages under `docs/api/`.
 
 ## 1. agent-runtime — own public surface
 
@@ -231,7 +231,7 @@ Import from `@tangle-network/agent-runtime` — 440 exports.
 | `SpendGap` | interface | One journaled node whose usage accounting is incomplete — the named gap behind a `false` |
 | `SqlAdapter` | interface | Minimal SQL driver shape. Implementations forward to whichever client the |
 | `Supervisor` | interface | Owns the conserved pool, the spawn log, the abort cascade, the OTP intensity breaker, |
-| `VerifyResult` | interface | Outcome of verifying a candidate worktree. `feedback` (compiler errors, |
+| `VerifyResult` | interface | Outcome of verifying a candidate worktree. |
 | `AgentCandidateBundleInput` | type | Exact candidate wire shape before the runtime computes its canonical digest. |
 | `AgentCandidateCodeSource` | type | Explicit control/no-op code or one finalized CodeSurface whose bytes must still verify. |
 | `AgentCandidateExecutionClaimResult` | type | Result of atomically claiming one execution attempt. |

docs/canonical-api.md

@@ -4,7 +4,7 @@
 Generated signatures and the complete export list live in docs/api/.
 Run pnpm docs:freshness after editing this file. -->
 
-> **Version 0.173.0.**
+> **Version 0.174.0.**
 > [`docs/api/primitive-catalog.md`](./api/primitive-catalog.md) lists every export and import path.
 > `agent-eval` must satisfy `>=0.163.2 <0.171.0`.
 > `sandbox` must satisfy `>=0.31.0 <0.32.0`.

package.json

@@ -1,6 +1,6 @@
 {
   "name": "@tangle-network/agent-runtime",
-  "version": "0.173.0",
+  "version": "0.174.0",
   "description": "Shared task-lifecycle skeleton for agents: a recursive loop kernel for chat turns, one-shot tasks, and multi-attempt loops, with trace capture and eval-gated self-improvement. Domain behavior lives in adapters; scoring and ship-gates in @tangle-network/agent-eval.",
   "homepage": "https://github.com/tangle-network/agent-runtime#readme",
   "repository": {