fix: harden plugin shutdown and cold-pass isolation - #3
Conversation
…esce/finalize work
- B1: drop the engines.opencode constraint — opencode hard-enforces
engines and would refuse to load on older hosts; dispose is an
optional hook, so older hosts simply never call it. peerDependencies
range unchanged.
- B2: bound dispose(). settleWithin moved to types.ts and wraps both
the distiller stop and the tracked-operations wait (2.5s each, after
the summarizer's own 15s bound: <20s worst case). A timeout does NOT
skip db.close(); detached continuations are fenced by stopped/
finalized/lease guards before every store read/write. New test proves
dispose completes when an SDK request never settles.
- B3: fetchMessagePage strictness is opt-in (strict?: boolean). Only
distiller call sites pass strict:true; the query path keeps the
documented "non-array data = empty page, ok:true" contract. Restored
the original recall_messages/recall_context expectations and added
strict/lenient unit tests.
- B4: owned-worker cleanup is no longer lease-gated. Remote worker
ownership is a separate authority from the SQLite writer lease;
delete/abort of an ownedWorkers member goes through a bounded
ungated runner, while creates, prompts, and orphan sweeps stay
lease-gated. Test: lease lost mid-batch still deletes the worker.
- B5: stale-writer window closed. Every distiller write batch
(full/append/cold-pass replace, rollups, deletes) re-verifies
authoritative ownership via ownsLease() immediately before the write,
one leaseStatus() read per batch. Test simulates a rival takeover
between fetch and write.
- I1: ownsLease() no longer self-demotes on its own stale-looking
heartbeat; ownership is lost only when the row names someone else or
is gone. Heartbeat freshness is the rival's acquire-side concern.
- I2: the deriveCard quarantine catch is narrowed to data-shape errors
(MalformedSessionError/TypeError); anything else aborts the pass and
surfaces in lastError. DistillStatus gains quarantinedCount.
- I3: status() reports the cached lease flag (side-effect-free);
ownsLease() is reserved for write gates.
- I4: bumped @opencode-ai/plugin to 1.18.23, which declares dispose in
Hooks, and deleted the local module augmentation. Test-side fallout
(execute now returns ToolResult) handled with a toolResultText
narrowing helper.
- I5: disposed-tool calls return the JSON { ok:false, error } shape
instead of rejecting.
- I6: ownedWorkers entries are removed once cleanup settles (a timed-out
delete keeps the id owned for the next holder's orphan sweep).
- I7: fetchNewMessages wraps its page walk in the same
MalformedSessionError conversion as fetchSessionMessages.
- I8: CHANGELOG Unreleased entry for the change-set.
Based on PR rmk40#3 by @kernel-oops with maintainer fixes.
…spose Round-2 review fixes on the bounded-shutdown change-set. - Blocker: dispose no longer closes the DB when the tracked-operations wait times out. `operations` tracks FOREGROUND tool executions with no finalized/lease guards, so a recall paused in an SDK fetch could resume into store reads on a closed handle. A distiller-stop timeout still closes (its continuations are finalized/lease-guarded); an operations timeout skips the close — the process is exiting and the derived store is rebuildable, so an unclosed handle is harmless where a use-after-close is not. Comment rewritten to state what protects each path. New tests: a detached distiller fetch resolved AFTER dispose asserts zero post-close DB calls; a foreground tool operation outliving the bound asserts the close was skipped and the late resumption does not crash. - recordProgress (coldpass_cursor) is now gated on authoritative ownsLease() — it was the one distiller store write outside the stale-writer fence — checked only when the progress floor moves, so the hot skip path costs nothing extra. The cold-pass "done" transition and onColdPassDone() are gated the same way, so a demoted instance whose rollup recompute no-oped cannot declare success. - Page-walk catches in fetchSessionMessages/fetchNewMessages narrowed to TypeError (rethrow the rest), matching the quarantine principle. The deriveCard catch keeps MalformedSessionError|TypeError with a comment acknowledging the accepted tradeoff. - status().leaseHeld is now derived from the lease row status() already reads (holder === instanceId): authoritative AND side-effect-free, no extra SQLite read. Stale-writer test expectation updated. - Inter-page politeness sleeps are tracked and cancelled by quiesce(), so a long distillDelayMs cannot keep the runtime alive past dispose. - ownedWorkerSdk returns the SDK response; deleteOwnedWorker prunes the ownedWorkers id only on CONFIRMED success (resp without error) — a failed delete keeps the id for this instance's own retry. Comment corrected: the next holder's sweep deletes by sentinel regardless. - db.close() wrapped in try/catch so a throwing close cannot reject disposePromise into the host; redundant Promise.allSettled around summarizer.stop() replaced with a plain catch. - Orphan sweep comment documents the accepted winner-deletes-loser's- in-flight-worker race (benign: loser's results were lease-gated out). - fetchSessionMeta classifies a recognizable not-found (v2 `_tag` SessionNotFoundError or response.status 404) as absence (null), not a transport error. - CHANGELOG: "opencode >= 1.15.11" softened to "recent opencode versions"; bounded-shutdown entry describes the skip-close behavior; stale-writer entry now truthfully says every distiller store write is fenced (recordProgress included).
Round-3 review fixes. - Blocker: replace skip-close with deferred close. Skipping leaked the SQLite handle when opencode disposes a cached per-directory instance without the process exiting (cache eviction/reload). When the operations wait times out, dispose now schedules the close behind a fresh Promise.allSettled of the stragglers: it fires after disposePromise resolved (cannot hang the host), never under a live reader, and closes eventually. A truly never-settling straggler degrades to the old skip behavior. Test extended: after the parked tool resolves, the deferred close fires (closes === 1) with zero post-close calls; a comment notes recall_messages is a proxy and the sqlite mock proxies every store method, so any tool's post-close store call would trip the counter. - isNotFoundError: session.get's real 404 body is the generic NotFoundError discriminated by name:"NotFoundError" (SessionGetErrors in the v2 typings), not _tag:"SessionNotFoundError" (another endpoint's shape). Added the name check as primary; _tag and response.status 404 remain as fallbacks; docstring corrected. New tests: name:"NotFoundError" → absence (no lastError, no quarantine, card untouched, no retry noise); apiFailure with neither signal → transport path with lastError surfaced. - ownedWorkers retry is now real: drainOwnedWorkers() at the start of each batch best-effort re-deletes leftover ids (bounded — each delete is already settleWithin-capped), pruning on success and keeping on failure. Comment updated to name the actual retry path. Test: a failed delete's id is retried and pruned by the next batch's drain, nothing leaked. - TypeError quarantine breadth held deliberately (gpt5's objection acknowledged, not converted): both page-walk catches now cross- reference the deriveCard accepted-tradeoff comment, which is strengthened to name the residual risk explicitly — a TypeError regression in distillFields/deriveCard quarantines while the pass reports done; bounded by quarantinedCount observability and the 1000-entry cap; accepted because annotating every field access is worse. Perf (RECALL_PERF=1, after the leaseStatus() reads landed): tier-1 rank p95 10.45ms (<50 budget), distill+replace p50 0.85ms (<150), ftsSearch p95 0.96ms (<100), e2e drilled query 20.0ms (<1500), heap 57.9MB (<150).
Round-4 review fix: drainOwnedWorkers was bounded per delete but unbounded in aggregate — persistent failures grew the retained set one id per batch (O(n²) deletes across a pass; with timeouts instead of errors, (k-1)×15s serial waits wedging the pass while holding the lease). - Cap the drain at MAX_DRAIN_PER_BATCH (3) ids per batch. - Give-up counter: after MAX_DELETE_ATTEMPTS (2) failed deletes an id is dropped from ownedWorkers and the attempts map (logged). Safe: the next holder's sentinel-based orphan sweep reaps it, and sentinel-titled sessions are excluded from every recall path meanwhile. Attempts are also cleared on successful delete. - Bounded-backlog policy: a backlog still at/over MAX_DRAIN_PER_BATCH after the drain means deletes are persistently failing — skip the batch (no new worker) instead of adding to the leak; the give-up shrinks the backlog so later batches proceed. - pluginLog on the deferred-close branch in dispose (count + bound, plus a line when the deferred close fires) so a never-settling straggler's open handle is diagnosable instead of silent. - Drain comment states the real bounds (per-batch cap, 2-attempt give-up, sentinel sweep backstop). - Tests: persistent failure → no id delete-attempted more than twice, give-up logged; saturated backlog (give-up delayed via the new maxDeleteAttempts test affordance) → batches skip worker creation (creates capped at 3). Added the missing await summarizer.stop() in the round-3 retry test.
|
Thanks for this PR — the problem selection is exactly right (the cold-pass poisoning by one malformed session and the missing disposal lifecycle were both real gaps), and the test craftsmanship is above what most of this repo had: the I've reviewed it in depth (three independent review passes), merged
Where we deliberately held your design: the quarantine mechanism (keying, cap, invalidation-on-update) is untouched — it's sound. The quiesce-retains-heartbeat sequencing is untouched — that was the right call. The TypeError-as-malformed classification stays, now documented as an explicit accepted tradeoff with its bounds. Final state: 435 tests green, perf budgets verified under |
Summary
disposehook and declareengines.opencode >=1.15.11sessionID + timeUpdatedso one bad historical row cannot abort and retry the entire cold passWhy
A plugin instance currently owns timers, SQLite state, background distillation, and summarizer workers without exposing lifecycle disposal. When OpenCode disposes a per-directory plugin instance, those resources can remain alive. Separately, one legacy session with an obsolete response shape can abort the complete cold pass; the global retry then scans thousands of healthy sessions again roughly once per minute.
Verification
21files passed,2skipped)Residual constraints