feat(automation): durable run history with failure reasons (run observability)#2581
Merged
Conversation
…vability) Mirror every terminal automation run (completed / failed) to the sys_automation_run table so "did it run, and why did it fail?" survives a process restart and the in-memory ring-buffer eviction — previously runs were observable only in memory and failures surfaced no reason. - engine: `recordLog` fire-and-forget `recordTerminal` on terminal runs; `listRuns` merges durable history with the in-memory buffer (freshest wins). - store: `recordTerminal` / `listHistory` on both SuspendedRunStore impls, keyed by a `run_`-prefixed id disjoint from live suspended runs so the ADR-0019 suspend/resume path is untouched and resume sweeps skip history. - schema: sys_automation_run gains `finished_at` / `duration_ms` / `error` (all optional) and a `completed` / `failed` / `running` status alongside `paused`; becomes a durable run-history table, not suspend-only. - best-effort throughout: a history write/read failure is logged and swallowed, never breaking the run that produced it. Verified end-to-end on a clean showcase: completed + failed runs persisted durable rows (failure reason intact), a live paused run coexisted without collision, and the failed row survived a full process restart. New run-history.test.ts covers persistence, read-across-restart, best-effort isolation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
📓 Docs Drift CheckThis PR changes 1 package(s): 5 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:
|
This was referenced Jul 4, 2026
This was referenced Jul 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Automation runs were observable only in memory. The engine kept the last N runs in a ring buffer, so after a process restart (or once the buffer evicted the entry) you could not answer "did this flow run, and why did it fail?" — and a failed run surfaced no reason at all. This was the single biggest silent-trust gap for anyone authoring automations: a flow could stop firing or start failing with nothing durable to inspect.
This is the #1 UX recommendation from the online-automation dogfood: run observability — durable, queryable runs with status + failure reason.
What
sys_automation_run— previously the ADR-0019 store for live suspended runs only — becomes a durable run-history table.recordLogmirrors every terminal run (completed/failed) to the store viarecordTerminal(fire-and-forget).listRuns()merges durable history with the in-memory buffer (in-memory wins on id, newest-first) so the Studio "Runs" surface shows runs that predate the current process.recordTerminal/listHistoryon bothSuspendedRunStoreimpls. Terminal history rows use arun_-prefixed id, disjoint from live suspended runs (rawrunId,status: 'paused', deleted on completion), so the suspend save/load/delete/list path is untouched and resume sweeps (list()filtersstatus: 'paused') never see history rows.finished_at/duration_ms/error(all optional) + thestatusenum gainsrunning/completed/failedalongsidepaused.Safe & additive
Verification
Unit — new
run-history.test.ts: completed persistence, failed persistence with error, read-across-restart (fresh engine over the same store), best-effort isolation (a throwing store never breaks the run). Full suite green (231 passed), DTS typecheck clean.Runtime, end-to-end on a clean showcase instance (the standard that caught the earlier no-op #2560 fix — a fake-service unit test alone can mask a runtime-broken change):
showcase_scheduled_digest+ 7 task-completion flows → durablecompletedrows withfinished_at;showcase_resilient_sync→ afailedrow carrying itstry_catchfailure reason;showcase_task_follow_uppausedsuspended run coexisted without collision;failedrow — reason intact — was still queryable via/api/v1/data/sys_automation_run.🤖 Generated with Claude Code