feat(query-db): add cached opaque cursor pagination

.changeset/add-cached-cursor-pagination.md

@@ -0,0 +1,7 @@
+---
+'@tanstack/query-db-collection': patch
+---
+
+Add `createCursorPager` to fulfill offset/limit requests from endpoints with opaque continuation tokens. Reuse fresh backend pages through the existing QueryClient, with Query-managed expiry, invalidation and garbage collection, while retaining the existing UI peek-ahead behavior.
+
+Keep manual raw-row writes from overwriting other cache formats or marking them fresh. Preserve wrapped-response writes and seeding of empty row caches.

docs/collections/query-collection.md

@@ -831,16 +831,155 @@ const { data, fetchNextPage, hasNextPage } = useLiveInfiniteQuery(
 ```
 
 Reject failed requests instead of returning partial rows as success. An
-unlimited request must drain until the endpoint reports exhaustion. If the
-endpoint uses opaque cursors instead of page numbers, keep that cursor handling
-inside `queryFn` or its adapter; honoring a new offset may require starting at
-the beginning again. The hook does not maintain remote cursor history.
+unlimited request must drain until the endpoint reports exhaustion. For an
+endpoint with opaque cursors, use `createCursorPager` as shown below. The hook
+does not maintain remote cursor history on its own.
 
 Manually appending rows with `writeUpsert` is a separate, lower-level loading
 strategy. It does not make an eager `queryFn` incremental: a later successful
 refetch still replaces its complete state and can remove appended rows.
 `staleTime: Infinity` does not prevent explicit refetch or invalidation.
 
+#### Endpoints with opaque cursors
+
+Query Collections ask for arbitrary windows, such as rows 40–59. An
+opaque-cursor API cannot jump to row 40. It can only fetch the first page, then
+follow the continuation token returned with each page.
+
+`createCursorPager` bridges those two protocols. It stores complete backend
+pages in your existing QueryClient and follows their cursors until it has enough
+rows for the requested `offset` and `limit`. A later request can reuse that
+fresh run of pages and fetch only the missing suffix. The hook still requests
+an extra row to decide `hasNextPage`; the pager just fulfills the resulting
+window.
+
+```typescript
+import { createCollection } from '@tanstack/db'
+import { createCursorPager, queryCollectionOptions } from '@tanstack/query-db-collection'
+
+type Post = { id: number; createdAt: number; title: string }
+
+const postsCollection = createCollection(
+  queryCollectionOptions({
+    queryClient,
+    queryKey: ['posts', 'rows'],
+    syncMode: 'on-demand',
+    getKey: (post: Post) => post.id,
+    queryFn: async (ctx) => {
+      const { where, orderBy, offset = 0, limit } = ctx.meta?.loadSubsetOptions ?? {}
+
+      // Translate every filter and sort, or reject unsupported expressions.
+      // This stable, serializable value identifies one ordered result sequence.
+      const request = api.translatePostQuery({ where, orderBy })
+
+      const pager = createCursorPager<Post>({
+        queryClient,
+        queryKey: ['posts', 'cursor-pages', request],
+        staleTime: 60_000,
+        gcTime: 5 * 60_000,
+        fetchPage: async (cursor, signal) => {
+          const response = await api.listPosts({ ...request, cursor, signal })
+
+          // Only null means that the source is exhausted.
+          return { rows: response.items, nextCursor: response.nextCursor ?? null }
+        },
+      })
+
+      return pager.read({ offset, limit }, ctx.signal)
+    },
+  }),
+)
+```
+
+##### Give each result sequence its own key
+
+The pager's query key identifies one ordered backend result, not one requested
+window. Leave `offset` and `limit` out of the key so wider reads can extend the
+same cached prefix. Include everything that changes the result—such as the
+tenant, source, filters, and order.
+
+Keep row data and cursor pages under sibling keys:
+
+```text
+['posts', 'rows']                         QueryCollection row array
+['posts', 'cursor-pages', request]        backend pages and cursors
+```
+
+They cannot share a key because the two cache entries have different shapes.
+Do not put cursor pages beneath the row key either: manual collection writes
+target entries under the row prefix. A shared resource prefix such as
+`['posts']` still lets you cancel or invalidate both kinds of data together.
+
+The example creates a small pager object for each `queryFn` call. Those objects
+still share cached pages and in-flight fetches through QueryClient. Each pager
+serializes its own reads. Retain one pager per result sequence only if calls
+must also share that queue or you need to call `pager.reset()` later.
+
+##### What happens when the collection asks for rows
+
+For fresh data, the pager uses every cached page it can and fetches only the
+missing suffix. `nextCursor: null` is the sole exhaustion signal: a short or
+empty page with another cursor does not end the sequence. An omitted `limit`
+drains the source; a zero limit performs no fetch. Repeated continuation tokens
+throw instead of looping forever.
+
+Query owns freshness, garbage collection, retries, and invalidation. Pass
+`staleTime` and `gcTime` to the pager or let it inherit QueryClient defaults.
+Configure retries through QueryClient defaults for the page-key prefix;
+imperative Query fetches do not retry by default. The pager keeps every page in
+its original form, so Query defaults for `maxPages` and `select` do not apply.
+
+When cached pages are stale, the next read refetches every page loaded so far,
+starting with the first. It does this even for a shallow window because old
+continuation tokens belong to the old sequence. Staleness does not start a
+background timer. The page query has no lasting observer, so its `gcTime` may
+expire while the collection's published rows remain visible.
+
+A failed read rejects rather than returning a partial window. Query keeps the
+last successful pages, so a later read can retry. Treat their rows as immutable,
+just like other Query data. `read()` returns references from the cached pages,
+not detached copies, and does not promise the same array or object identity
+across calls.
+
+##### Refresh or cancel the sequence
+
+After a mutation—or whenever you need a forced refresh—cancel the shared
+resource prefix before invalidating it:
+
+```typescript
+await queryClient.cancelQueries({ queryKey: ['posts'] })
+await queryClient.invalidateQueries({ queryKey: ['posts'] })
+```
+
+Cancelling first stops an old page append from completing after invalidation and
+marking the old sequence fresh again. It rejects readers waiting on that fetch
+with an `AbortError` and prevents a late result from entering the cache. The
+backend must honor the signal if you also want to stop its network work.
+
+Aborting the signal passed to `read()` rejects only that reader; it does not
+cancel a page fetch shared with another reader. A cancelling Query refetch moves
+waiting readers to the replacement fetch. `collection.utils.refetch()` refreshes
+the row queries but may reuse fresh cursor pages. `pager.reset()` removes one
+pager's exact page key and cancels its queued reads, but does not refresh the
+collection's row query.
+
+##### Backend contract
+
+The helper handles cursor traversal only. Your adapter must:
+
+- translate every requested `where` and `orderBy`, or reject unsupported
+  expressions;
+- apply filters and ordering before pagination;
+- use a deterministic total order, including a unique tie-breaker;
+- keep that ordered result consistent while its cursor sequence is read; and
+- return `nextCursor: null` only when the result is authoritatively exhausted.
+
+`LoadSubsetOptions.cursor` contains query-expression hints, not the backend's
+opaque token. `createCursorPager` therefore addresses the cursor sequence by
+offset and deliberately does not consume those hints. No client-side pager can
+make a stable snapshot from an endpoint whose ordering changes between pages;
+the endpoint must define its own consistency and cursor-expiry rules.
+
 ## Important Behaviors
 
 ### Full State Sync

docs/contributing/oracle-coverage.md

@@ -35,6 +35,7 @@ comment and the current API/architecture contract before extending its model.
 | Drafts and native values | [proxy](../../packages/db/tests/proxy.test.ts), [detachment](../../packages/db/tests/proxy-detachment-contract.test.ts), [iteration](../../packages/db/tests/proxy-iteration-contract.test.ts) | Native-operation controls, exact patches and actual stored rows; alias/cycle/adversarial-key histories. General native-mutator and symbol-write support is not established by a plain-object oracle. |
 | Query DB and observer | [ownership](../../packages/query-db-collection/tests/ownership-lifecycle.oracle.test.ts), [load lifecycle](../../packages/query-db-collection/tests/load-subset-lifecycle-oracle.test.ts), [observer histories](../../packages/db/tests/live-query-observer-history.property.test.ts) | Real QueryClient boundary and a per-listener eligibility ledger, not a duplicate dispatch queue. Check reentry, peer survival, FIFO and disposal independently of final rows. |
 | Ordered acquisition | [pagination](../../packages/db/tests/query/pagination-oracle.property.test.ts), [ordered work](../../packages/db/tests/query/ordered-work-oracle.property.test.ts), [ordered lifecycle](../../packages/db/tests/query/ordered-lifecycle-oracle.property.test.ts) | Complete finite provider results, pending windows, ties/nulls, ownership and documented repair timing. Request completion is not proof of unrequested source extent. |
+| Opaque backend pagination | [window oracle](../../packages/query-db-collection/tests/cursor-pagination.oracle.test.ts), [cache histories](../../packages/query-db-collection/tests/cursor-pagination.cache-oracle.test.ts), [cache publication](../../packages/query-db-collection/tests/cursor-pagination.publication-oracle.test.ts), [browser acquisition boundaries](../../packages/query-db-collection/tests/cursor-pagination.boundary-oracle.test.ts), [QueryCollection integration](../../packages/query-db-collection/tests/cursor-pagination.integration.test.ts) | Full filter/sort/slice reference, opaque token transport, actual Query cache expiry/invalidation/GC, forced refresh during growth, protocol failure publication/recovery, bounded slice work, nested cancellation/replacement, reader abort, browser retry defaults, manual-write cache isolation, and production window publications. Stable backend sequences; not snapshot guarantees for changing endpoints. Peek-ahead remains enabled. |
 | Electric and TrailBase | [Electric histories](../../packages/electric-db-collection/tests/electric-oracle.property.test.ts), [PostgreSQL semantics](../../packages/electric-db-collection/e2e/sql-predicate-semantics.e2e.test.ts), [TrailBase contract](../../packages/trailbase-db-collection/tests/ORACLE.md) | Installed SDK delivery/framing, independent predicates, exact subscription arguments and late errors. SDK fixtures and a real service test earn different credit. |
 | PowerSync | [tests](../../packages/powersync-db-collection/tests) | Applied receipt positions crossed with held peers, native SQLite/SDK and cleanup evidence. A timeout mutant proves a progress failure, not every value assertion. |
 | SQLite persistence and native hosts | [persisted histories](../../packages/db-sqlite-persistence-core/tests/persisted.test.ts), [driver contracts](../../packages/db-sqlite-persistence-core/tests/contracts/sqlite-driver-contract.ts), [113-law manifest](../../packages/db-collection-e2e/src/fixtures/persisted-conformance-manifest.ts) | Cache/remote rejection/peer/reopen histories and exact driver results. The manifest excludes progressive and move suites; registration and shim runs are not device execution. |

packages/db/tests/oracle-config.ts

@@ -3,6 +3,21 @@ import { oracleReplayReporter } from './oracle-replay-witness.js'
 type OracleEnvironment = Record<string, string | undefined>
 
 const staticOracleProperties = [
+  `cursor-pagination.no-peek`,
+  `cursor-pagination.history`,
+  `cursor-pagination.cache`,
+  `cursor-pagination.nested-cancellation`,
+  `cursor-pagination.reader-abort`,
+  `cursor-pagination.manual-write`,
+  `cursor-pagination.backend-ownership`,
+  `cursor-pagination.refresh-publication`,
+  `cursor-pagination.protocol-publication`,
+  `cursor-pagination.slice-work`,
+  `cursor-pagination.defaults`,
+  `cursor-pagination.cancellation`,
+  `cursor-pagination.partition`,
+  `cursor-pagination.reset`,
+  `cursor-pagination.failure`,
   `oracle-replay.calibration`,
   `trailbase.lifecycle`,
   `electric.bound-descriptor-history`,

packages/query-db-collection/package.json

@@ -21,7 +21,7 @@
     "dev": "vite build --watch",
     "lint": "eslint . --fix",
     "test": "vitest run",
-    "test:oracles": "vitest run tests/includes-work-counter-oracle.test.ts tests/load-subset-lifecycle-oracle.test.ts tests/ownership-lifecycle.oracle.test.ts",
+    "test:oracles": "vitest run tests/includes-work-counter-oracle.test.ts tests/load-subset-lifecycle-oracle.test.ts tests/ownership-lifecycle.oracle.test.ts tests/cursor-pagination.oracle.test.ts tests/cursor-pagination.cache-oracle.test.ts tests/cursor-pagination.publication-oracle.test.ts tests/cursor-pagination.boundary-oracle.test.ts tests/cursor-pagination.integration.test.ts",
     "test:e2e": "vitest run --config vitest.e2e.config.ts"
   },
   "type": "module",