Live queries for SQLite / Durable Objects#88
Merged
Merged
Conversation
There was a problem hiding this comment.
Pull request overview
This PR refactors the live-query system into per-dialect implementations behind a LiveBackend interface, adds a new synchronous SQLite/Durable Objects live backend, and updates the extractor/matching pipeline so both Postgres and SQLite share the same predicate-matching contract.
Changes:
- Introduces shared live infrastructure (
LiveBackend,Matcher,canonicalText) and refactors the PG live backend to use it. - Adds a SQLite live backend with synchronous, same-tick mutation capture/dispatch backed by a new
Driver.executeSync. - Adds/updates end-to-end tests for SQLite live queries (including Cap’n Web callback streaming) and PG snapshot parsing.
Reviewed changes
Copilot reviewed 25 out of 27 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| src/live/sqlite/subscribe-capnweb.test.ts | E2E Cap’n Web test for .subscribe(conn, cb) over the SQLite live backend. |
| src/live/sqlite/dispatcher.ts | SQLite live dispatcher using integer sequence cursors and synchronous signaling. |
| src/live/sqlite/db-live.test.ts | E2E SQLite live-query test suite (insert/update/delete/join/canonicalization/DO driver). |
| src/live/sqlite/capture.ts | SQLite mutation image capture via RETURNING + pre-SELECT for UPDATE before-images. |
| src/live/sqlite/backend.ts | SQLite LiveBackend implementation + synchronous mutation capture/dispatch path. |
| src/live/pg/test-helpers.ts | Fixes import paths for PG live test helpers after refactor. |
| src/live/pg/snapshot.ts | Adds snapshot parsing + MVCC visibility helper for PG cursors. |
| src/live/pg/snapshot.test.ts | Tests for snapshot parsing and visible() MVCC helper. |
| src/live/pg/iteration.test.ts | Updates tests to use runLiveIteration from the new PG backend module. |
| src/live/pg/events.ts | Updates import paths for PG events (shadow table capture) module. |
| src/live/pg/events.test.ts | Updates import paths for PG events tests. |
| src/live/pg/events-ddl.ts | Updates import paths for PG events DDL. |
| src/live/pg/db-live.test.ts | Updates import paths for PG live E2E tests. |
| src/live/pg/bus.ts | Refactors PG bus to use shared Matcher/Subscription and shared event parsing. |
| src/live/pg/bus.test.ts | Updates import paths for PG bus tests. |
| src/live/pg/backend.ts | New PG LiveBackend implementation and extracted runLiveIteration(). |
| src/live/matcher.ts | New backend-agnostic matcher/subscription + shared parseEventPairs. |
| src/live/extractor.ts | Refactors extractor to be dialect-neutral, adds SQLite literal param support + canonical rendering + decoy aliases. |
| src/live/extractor.test.ts | Updates expected SQL output for extractor changes (aliasing + canonical casts). |
| src/live/canonical.ts | Introduces shared canonical text rendering (SQLite REAL→INTEGER normalization). |
| src/live/backend.ts | Introduces LiveBackend interface + LiveIteration contract used by Connection.live(). |
| src/index.ts | Updates TypegresLiveEvents export path to ./live/pg/events. |
| src/drivers/types.ts | Adds optional Driver.executeSync for sync engines (SQLite/DO). |
| src/drivers/sqlite.ts | Implements executeSync and routes async execute() through it. |
| src/drivers/do.ts | Implements executeSync and routes async execute() through it. |
| src/database.ts | Refactors live queries to use LiveBackend; adds SQLite mutation capture routing and AbortSignal support in live(). |
| src/builder/query.ts | Adds QueryBuilder.subscribe(conn, cb) callback-streaming terminator for Cap’n Web-style by-ref callbacks. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+427
to
+433
| subscribe( | ||
| conn: Connection<any>, | ||
| cb: (rows: RowTypeToTsType<O>[]) => unknown, | ||
| ): { unsubscribe: () => void } { | ||
| // A callback passed by reference over capnweb arrives as a stub that | ||
| // the runtime auto-releases when this call returns. dup() retains it | ||
| // for the subscription's lifetime (this is what pins a Durable Object |
ryanrasti
force-pushed
the
live-sqlite
branch
6 times, most recently
from
July 20, 2026 23:40
557e50a to
75c052c
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 41 out of 43 changed files in this pull request and generated 4 comments.
Comments suppressed due to low confidence (1)
src/live/pg/events.ts:6
- Import spacing is inconsistent (
{ UpdateBuilder}is missing a space before}), which can cause formatter/lint churn.
Comment on lines
+166
to
168
| cancelSubscriptions(): void { | ||
| for (const sub of [...this.#subs]) {sub.cancel();} | ||
| } |
Comment on lines
+151
to
+155
| // Demo stop-button hook: cancels every active subscription (parked | ||
| // consumers wake with AbortError and exit cleanly); the live engine | ||
| // stays up and the next watch subscribes immediately. In a real | ||
| // deployment the wire would have a per-iter abort channel; here we | ||
| // kick the whole bus because the demo only ever has one iter at a time. |
Comment on lines
330
to
346
| @@ -282,17 +334,21 @@ export class Connection<C = undefined> { | |||
| ? RowTypeToTsType<O>[] | |||
| : never | |||
| > { | |||
| if (this.#boundExecute) { | |||
| if (this.#executor.bound) { | |||
| throw new Error("live() can't be called inside a transaction"); | |||
| } | |||
| const bus = this.#bus; | |||
| if (!bus) { throw new Error("Live bus not started — call conn.startLive() first"); } | |||
| const bus = this.#bus!; | |||
| // Lazy engine start: a no-op on sqlite (capture feeds the bus | |||
| // synchronously from attach); on pg this seeds the snapshot watermark | |||
| // and spins the poll loop on first use, so connections that never | |||
| // call .live() never poll. | |||
| await bus.ensureStarted(); | |||
|
|
|||
- Bus is dialect agnostic, event source differs
- PG: a shadow table that is polled (unchanged)
- Sqlite: assumes single thread access (the DO model), so updates are immediately pushed to the bus
with special handling to only flush events at commit in case of a transaction
- Opt-in to live is now through a { live: true } flag on the db.Table meta-class
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
with special handling to only flush events at commit in case of a transaction