Skip to content

No public barrier for "the fetched rows have been applied": dataUpdatedAt advances while the collection still holds the old rows, and an identical-payload refetch commits with zero observable effect #1828

Description

@cucumber-sp

Reproduction executed on set A (0.7.2 / 1.2.4); set B (0.9.2 / 1.2.15, the latest published)
was installed and verified by source reading, not re-run.

Versions. Two sets, both against @tanstack/query-core@5.102.8:
A = @tanstack/db 0.7.2 + query-db-collection 1.2.4 + react-db 0.2.1;
B = @tanstack/db 0.9.2 + query-db-collection 1.2.15 + react-db 0.4.1.

Being precise about what was run where: the reproduction below was executed on set A. Set B
was installed and its dist/esm read, but not re-run — so every claim about B in this report is
a source reading, and I have marked the line numbers accordingly. react-db is listed only for
completeness; none of this involves a React binding.

Summary

A consumer cannot ask a query collection "are the rows I can read right now the ones the last
successful fetch returned?". Two halves:

  1. Measured — an identical-payload refetch leaves no trace at all: no change events, no
    _stateRevision bump, no ready event, while dataUpdatedAt does advance.
  2. Read from source, not measured — the observer publishes before the collection commits.
    On the async apply path utils.dataUpdatedAt advances and utils.fetchStatus returns to
    idle before applySuccessfulResult runs, so utils would report a settled, successful,
    non-fetching query over the previous rows. collection.status can't close the interval —
    it is sticky ready.

(1) is the concrete request. It also happens to be what would make (2) fixable in user land: the
natural workaround for (2) is a watermark — "trust the rows only once an apply newer than
dataUpdatedAt is observed" — and (1) makes that watermark latch permanently false after the
first unchanged refetch. utils.refetch() is not a barrier either; it resolves independently of
the commit.

(1) Reproduction — measured, self-contained, no persistence adapter

import { createCollection } from "@tanstack/db"
import { QueryClient } from "@tanstack/query-core"
import { queryCollectionOptions } from "@tanstack/query-db-collection"
import { it } from "vitest"

type Row = { id: string; n: number }

it("identical-payload refetch: dataUpdatedAt moves, nothing else does", async () => {
  const queryClient = new QueryClient({ defaultOptions: { queries: { retry: false } } })
  const collection = createCollection(queryCollectionOptions({
    queryKey: ["repro"],
    queryClient,
    queryFn: async () => {
      await new Promise((r) => setTimeout(r, 5))
      return [{ id: "a", n: 1 }] satisfies Row[]   // always identical
    },
    getKey: (row: Row) => row.id,
  }))
  await collection.preload()

  let changeEvents = 0
  const sub = collection.subscribeChanges(() => { changeEvents++ })
  const rev = () => (collection as unknown as { _stateRevision: number })._stateRevision
  const t0 = collection.utils.dataUpdatedAt
  const r0 = rev()

  for (let i = 1; i <= 3; i++) {
    await new Promise((r) => setTimeout(r, 12))   // so the timestamps differ
    await collection.utils.refetch()
    await new Promise((r) => setTimeout(r, 12))
    console.log(i, { dtUpdatedAt: collection.utils.dataUpdatedAt - t0, dRevision: rev() - r0,
                     changeEvents, status: collection.status })
  }
  sub.unsubscribe()
  await collection.cleanup()
})

Actual — identical payload, then the same loop with n += 1 each round for contrast:

identical  1 { dtUpdatedAt: 22,  dRevision: 0, changeEvents: 0, status: 'ready' }
identical  2 { dtUpdatedAt: 58,  dRevision: 0, changeEvents: 0, status: 'ready' }
identical  3 { dtUpdatedAt: 101, dRevision: 0, changeEvents: 0, status: 'ready' }
changing   1 { dtUpdatedAt: 21,  dRevision: 1, changeEvents: 1, status: 'ready' }
changing   2 { dtUpdatedAt: 55,  dRevision: 2, changeEvents: 2, status: 'ready' }
changing   3 { dtUpdatedAt: 89,  dRevision: 3, changeEvents: 3, status: 'ready' }

Expected: a public, monotonic signal that advances once per successful application of a query
result, whether or not the diff was empty, so a consumer can pair "rows I am reading" with "fetch
that produced them".

QueryCollectionUtilsImpl is identical in 1.2.4 and 1.2.15; its non-write members are
clearError, lastError, isError, errorCount, isFetching, isRefetching, isLoading,
dataUpdatedAt, fetchStatus (the class also carries refetch and
writeInsert/writeUpdate/writeDelete/writeUpsert/writeBatch). Nothing there reports
application.

(2) The ordering interval — read from source, NOT measured

Stating this plainly because it matters for how much weight to give it: I could not stage this
one.
On query-db-collection@1.2.4 a plain query collection calls applySuccessfulResult
synchronously inside the observer callback, so there is no interval to observe there — confirmed
by measurement.

On 1.2.4 the interval is one-shot per query key. handleQueryResult branches at :670:
retainedQueriesPendingRevalidation.has(key) takes the async void reconcileSuccessfulResult(...)
path (:682), and everything else falls through to a plain synchronous
applySuccessfulResult(...) (:689) that lands rows in the same tick as the observer
notification. The set's only .add is :507 (gate :506, metadata lookup :503-505), reachable
only from createQueryFromOpts, and :644 deletes the key immediately after commit(). So the
interval opens for the first successful result of a rehydrated persisted query collection and
then closes for good. That is a narrow window, but it lands at app launch — which for us is
exactly when an irreversible decision reads the flag.

On 1.2.15 it is no longer one-shot. Both branches now go through enqueueResultApplication
(:829 retained, :839 otherwise) and applySuccessfulResult is itself async (:627), so the
synchronous fall-through is gone and every successful result applies asynchronously. If I'm
reading that right, the version that adds the apply-serialization layer is also the version that
widens this interval from once-per-key to every refetch — which seems worth a second opinion from
someone who knows the intent.

What I tried (all on 0.7.2 / 1.2.4), sampling from a queryCache.subscribe callback and from
a setTimeout(0) polling loop, looking for any sample with dataUpdatedAt advanced,
isFetching === false, isError === false and the row still at its old value:

  • plain query collection — 0 such samples (expected: synchronous apply);
  • persisted collection with a stub adapter — 0 such samples; the retained path was never armed,
    because retainedQueriesPendingRevalidation is only populated when a persisted query-retention
    entry already exists (dist/esm/query.js:503-508);
  • two sessions over one real node:sqlite-backed adapter, to create that retention entry —
    0 such samples, but inconclusive: session 2 never rehydrated the old row (it went
    undefined → 2), so the precondition "collection holds the previous row" was never set up.

So: not reproduced, and not disproven either — my harness failed to establish the precondition on
the one path where the interval is reachable. In hindsight the branch structure above explains
each miss: the first two never armed retainedQueriesPendingRevalidation at all, and the third
armed it but never rehydrated a row that could go stale. If you think the reading below is wrong, that
would be useful to know; if it's right, a maintainer can stage it far more cheaply than I can. The
source path is short enough to point at directly:

1.2.4 1.2.15
reconcileSuccessfulResult awaits baseline, then applies dist/esm/query.js:648-655 :745-747
applySuccessfulResult (beginwritecommitmarkReady) :580-647 (commit :643, markReady :646) :627-731 (commit :720, markReady :731), now async
only differing rows are written :614-641 same shape
@tanstack/db 0.7.2 0.9.2
markReady() no-ops once already ready collection/lifecycle.js:76-92 :87-120
empty ready event only on loading → ready collection/lifecycle.js:89-91 :113
stateRevision bumps only when changes.length > 0 collection/changes.js:43-44 :58-59
empty batch returns before notifying subscribers collection/changes.js:58-60 :126-129 (publishEvents)

What 1.2.15 already has, and why it doesn't close this

1.2.15 adds a serialized apply-tracking layer that 1.2.4 has no trace of, and it is the closest
thing in the codebase to what this issue asks for — flagging it so this doesn't read as if it had
been missed:

  • pendingResultApplications / failedResultApplications / resultApplicationTokens /
    resultApplicationControllers (dist/esm/query.js:223-226);
  • getResultApplicationSettlement(hashedQueryKey) (:237-244) — returns the in-flight apply
    promise, or true once settled. That is exactly the predicate we want;
  • enqueueResultApplication (:782-798), called synchronously from handleQueryResult
    (:829, :839), chaining each apply onto the previous one;
  • applySuccessfulResult is now async (:627) and awaited at :751, with the
    applicationToken/signal pair at :745.

The reason it doesn't help a consumer: getResultApplicationSettlement is consumed only inside
createQueryFromOpts (:554, :558, :563, :612, :616, :623), which is reachable from
eager sync start (:908) and from the non-eager loadSubsetDedupe (:1087). It is never
consulted per-refetch, and it is not reachable through collection.utils. So it serializes
applies against each other, but does not give a consumer anything to await or observe.

If widening that settlement to the refetch path and exposing it is easier than either option
below, that would work just as well for us.

The ask

The measured half alone is enough to motivate this. Either shape would work; the first looks like
the smaller change:

  1. A "sync applied" signal that fires on every commit, including empty ones — bump
    stateRevision (or a sibling syncRevision) on every commit() regardless of diff size, and
    expose it publicly.
  2. An awaitable publication barrierutils.refetch() resolving only after
    applySuccessfulResult has committed, or a collection.whenApplied() promise.

The property we need either way: for every successful fetch, exactly one observable event, even
when the resulting diff is empty.

Related, not the same

#661 (awaitPersisted()) awaits mutation persistence by key (client → server); this is the
sync direction, server rows → collection. #1657 / #1659 are adjacent but a different concern.

We also see a lost write with serialized whole-row updates on 0.8/0.9 over a persisted collection
(the optimistic delta drops at transaction settle, before the canonical row is published) — likely
downstream of the same window, but a distinct symptom with its own reproduction, so it is kept out
of this report and can be filed separately.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions