Batch mutations into one snapshot, and a DocxDiff-scored puzzle eval for the agent surface - #475
Batch mutations into one snapshot, and a DocxDiff-scored puzzle eval for the agent surface#475JSv4 wants to merge 3 commits into
Conversation
Every DocxSession mutation records a pre-op snapshot, and a snapshot deep clones every projected part — so its cost scales with the DOCUMENT, not with the edit. Measured on TestFiles/NVCA-Model-COI.docx (144 KB on disk), one snapshot retains ~7.5 MB, so a forty-edit sequence paid forty of those AND consumed forty entries of a twenty-deep ring. The sequence a caller had just applied was therefore already only half reversible by the time it finished. Both costs are properties of the loop, not of the work: N edits that form one intent deserve one snapshot and one undo step. Adds Batch(steps, options) for callers who can express steps as Func<EditResult> closures, and BeginBatch/EndBatch for those who cannot — a JSON dispatcher running its own switch, which is exactly the MCP case. Batch is implemented on top of the pair, so there is one mechanism. The 47 `_history.RecordPreOp(TakeSnapshot())` sites now route through RecordPreOpSnapshot(), which suppresses inside a batch. The suppression has to live at the call site rather than in UndoRing: TakeSnapshot() is evaluated before the ring ever sees the entry, so dropping it inside the ring would still pay for the clone. Failure policy cannot be used to escape the guarantee PR #459 established. Atomic (default) reverses on the first failure. Best-effort tolerates only failures that provably did not touch the document; a step that threw partway, or one the validator rejected after it had already written, reverses the whole batch regardless — inside a batch there is no per-step snapshot left to unpick that step alone. The two raw-op validation paths that hand-rolled their own pop-and-restore now use RollbackFailedOp() too, which removes the last non-uniform history call site. docxodus_mutations becomes genuinely atomic on the back of this, with no wire schema change. Its preview mode restores one snapshot instead of issuing N Undo() calls with N tracked by hand — that loop under-reverted whenever a step consumed more than one ring entry, making "nothing is left changed" a promise it could not keep. 20 tests (DS430-DS443) cover undo grain, both failure policies, damage overriding policy, nesting, the scope form, and the retained-memory shape. Full suite: 3476 passed, 0 failed.
The arcade proved the addressing model — anchors survive save/reopen, and the reconciler stays incremental while a host mutates the session behind it. It proved nothing about the surface we ask agents to drive, because it drives raw.replaceXml: the escape hatch for callers who want to hand-author OOXML, which is the opposite of what the tool surface is for. A level is a pair of documents and a budget: transform start into target in at most `par` calls, using only the grouped tools and anchor addressing. A level is solved when DocxDiff between the player's document and the target returns zero revisions, so the scoring function is the comparison engine itself — exact, unarguable, and the differentiator no demo surface touches. Levels declare both documents as paragraph lists rather than shipping binaries: they diff in review, and one builder constructs both sides so a scoring difference can only come from the player's edits. Reference solutions address blocks by content through FindAllByText — the op behind docxodus_search — so a reference cannot accidentally be easier than the task by starting with ids nobody has yet. CI keeps the LEVELS honest rather than testing the session: the reference solves the level (PE001), does so within par (PE002), and the start does not already score as solved (PE003). PE003 is the load-bearing one — a mis-built target would otherwise let an empty solution pass every level and report the pack at 100%. PE004 pins that the scorer rejects a partial solve; PE005 applies a whole level as one batch, since a plan is exactly what batching exists for. L01-clause-order ships first, and already shows the surface something: a two-paragraph clause costs two MoveBlock calls because there is no move-the-section op, which is the kind of affordance gap this pack exists to surface. Also corrects the arcade's og:/twitter: copy, which read as a live WAD parser where wad2cart.mjs rasterizes E1M1 at build time. freedoom-e1m1.js was already accurate; only the social meta overstated it. Full suite: 3481 passed, 0 failed. Library 115 warnings, tests 618 — both unchanged from baseline.
Main gained three PRs that touch the same surfaces: #476 (mutation preconditions + ExecuteMutation and a mutation gate), #478 (canonical table addressing), and #483 (page citations). Resolutions: - DocxSession.cs, 8 conflicts, all the same shape — main added code around a snapshot call site. Took main's side wholesale, then re-applied the call-site rewrite over the merged file so main's two NEW mutating ops are batch-aware too. 49 sites now route through RecordPreOpSnapshot(), up from 47. - Dispatcher.cs, docxodus_mutations. Main independently fixed the preview under-revert by counting version deltas per step instead of steps, then rewinding that many entries. The batch supersedes the counting entirely — there is exactly one snapshot to restore however much each step cost — so the delta accounting and the undo loop are gone. Main's entry precondition check is kept as-is, and so is RestorePreviewVersion: the batch restores the DOCUMENT but not the version counter, and a caller holding a version that describes no state the document was ever in would fail its next precondition check against a mutation that never happened. Version is now restored on any reversal, not just preview. - Batch() takes the mutation gate main introduced, for the whole sequence rather than per step: a batch that released it between steps could interleave with a concurrent mutation and then reverse that mutation's work as part of its own rollback. BeginBatch/EndBatch cannot offer this, since holding a lock across a JSON-RPC round trip waits on a caller that may never close its batch. The asymmetry is documented rather than papered over. - CHANGELOG.md: additive on both sides, kept both. Merged suite: 3518 passed, 1 failed. The failure is HCO081_RenderBlocksHtml_MatchesFullRenderFragments, which fails identically on clean origin/main at 7349259 and whose CI run on main is already red. It is a base-branch failure, not one this branch introduces.
|
Merged One heads-up on CI, since this branch now inherits a failure it did not cause.
The assertion is that batch block rendering matches the full render, and the two paths disagree on attribute emission: Main's own CI run for 7349259 is already red ( I have not fixed it — it is outside what this PR is for, and picking a side between the two render paths wants whoever owns the page-citation / Everything else on the merged tree is green: 3518 passed, 1 failed, 3 skipped, with library warnings at 115 and test warnings at 618 — both at the documented baseline. The pre-merge head (5637383) had all 14 checks green, including Playwright. Generated by Claude Code |
|
Both failing checks on Same four specs fail on
Same four, in both runs: …failing on the same assertion, an extra empty page in the paginated result: When it broke. Playwright on
This branch's own pre-merge head (5637383) had all 14 checks green, Playwright included. The red only appeared once Summary of the two blockers, neither of which this PR touches:
I've left both alone deliberately: fixing table addressing and pagination geometry inside a batching PR would make this unreviewable, and picking a side wants whoever owns that work. Happy to take either as a follow-up — just say which. Generated by Claude Code |
Three commits, from an assessment of what the arcade actually proves.
The arcade proves the addressing model: anchors survive save/reopen, and the reconciler stays incremental while a host mutates the session behind it. It proves nothing about the surface we ask agents to drive, because it drives
raw.replaceXml— the escape hatch for callers who want to hand-author OOXML. That gap is what these two changes close from opposite ends.1.
DocxSession.Batch— one snapshot, one undo stepEvery mutation records a pre-op snapshot, and a snapshot deep-clones every projected part, so its cost scales with the document, not the edit. Measured on
TestFiles/NVCA-Model-COI.docx(144 KB on disk), one snapshot retains ≈7.5 MB — so a forty-edit sequence paid forty of those and consumed forty entries of a twenty-deep ring. The sequence a caller had just applied was already only half reversible by the time it finished. Both costs are properties of the loop, not of the work. Agent callers hit it hardest, because a plan is naturally a list of edits.Batch(steps, options)for callers who can express steps asFunc<EditResult>closures;BeginBatch/EndBatchfor those who cannot — a JSON dispatcher running its own switch, which is exactly the MCP case.Batchis implemented on the pair, so there is one mechanism._history.RecordPreOp(TakeSnapshot())sites now route throughRecordPreOpSnapshot(), which suppresses inside a batch. The suppression has to live at the call site, not inUndoRing—TakeSnapshot()is evaluated before the ring ever sees the entry, so dropping it inside the ring would still pay for the clone.RollbackFailedOp(), removing the last non-uniform history call site.Batch()holds the mutation gate from Add optimistic mutation preconditions (#447) #476 for the whole sequence rather than per step: releasing it between steps could interleave with a concurrent mutation and then reverse that mutation's work during rollback.BeginBatch/EndBatchcannot offer this — holding a lock across a JSON-RPC round trip waits on a caller that may never close its batch — and the asymmetry is documented rather than papered over.docxodus_mutationsbecomes genuinely atomic on the back of this, with no wire schema change. A step that mutated before failing now reportsrolledBack: truewitheditsApplied: 0rather than describing a half-applied document as a partial success.23 tests (DS430–DS443) cover undo grain, both failure policies, damage overriding policy, nesting, the scope form, and the retained-memory shape.
2. The puzzle eval — measuring the surface agents actually get
A level is a pair of documents and a budget: transform
startintotargetin at mostparcalls. A level is solved whenDocxDiffreturns zero revisions against the target, so the scoring function is the comparison engine itself — exact, unarguable, and the differentiator no demo surface touches.FindAllByText— the op behinddocxodus_search— so a reference cannot accidentally be easier than the task by starting with ids nobody has yet.L01-clause-orderships first and already shows the surface something: a two-paragraph clause costs twoMoveBlockcalls because there is no move-the-section op. That is the kind of affordance gap the pack exists to surface.3. Merge of
main(#476, #478, #483)Main moved three PRs ahead while this was open, touching the same surfaces. Resolutions are detailed in the merge commit; the two that matter:
docxodus_mutationspreview. Main independently fixed the under-revert by counting version deltas per step instead of steps. The batch supersedes the counting entirely — there is exactly one snapshot to restore however much each step cost — so the delta accounting and the undo loop are gone. Main's entry precondition check is kept, and so isRestorePreviewVersion: the batch restores the document but not the version counter, and a caller holding a version describing no state the document was ever in would fail its next precondition check against a mutation that never happened. Version is now restored on any reversal, not just preview.Also
The arcade's
og:/twitter:copy read as a live WAD parser, wheretools/wad2cart.mjsrasterizes E1M1's geometry at build time and ships static data.freedoom-e1m1.jswas already accurate; only the social meta overstated it.Not done here
The batch primitive is core + facade + MCP only. The WASM bridge, npm/TS, and the stdio host + Python client do not yet expose it, so the ripple checklist is deliberately incomplete — flagging rather than hiding it. The MCP surface was the one that mattered for the eval, and it needed no new wire schema.
Testing
HCO081_RenderBlocksHtml_MatchesFullRenderFragments, which fails identically on cleanmainat 7349259, whose own CI run is already red. It is a base-branch failure this branch inherits, not one it introduces — see the comment below for the evidence.