Streaming deserialize: add source stream and task

[bjester]

Summary

• Extracts common behavior for streaming source and task classes into new base classes for use with both serialization and deserialization
• Adds new StoreQuerySet methods that apply filtering for selecting records based on whether they have deserialization errors. These can be used in Kolibri later
• Adds method to model registry for querying store records in model dependency order in parity with the method used by serialization
• Creates deserialization stream and task specific classes as foundation for streaming deserialization stage

TODO

• Have tests been written for the new code?
• Has documentation been written/updated?
• New dependencies (if any) added to requirements file

Reviewer guidance

• Do the tests cover the new and refactored code enough?
• Did I properly translate the self-ref-ordering and deserialization error filtering that were added after this PR was opened in draft? (I rebased it and applied

Issues addressed

Closes #317

AI Usage

I used Claude to do the rebase and bring it up-to-date with the upstream changes. It made a rightful mess of it, and so I rewrote some of the changes to better follow the patterns and then had it clean up the mess by updating tests.

[rtibblesbot]

🟡 Waiting for changes

Last updated: 2026-08-26 16:27 UTC

[rtibblesbot]

PR #318 — 12 of 14 prior findings resolved or acknowledged; 2 still open, plus one new suggestion on prefix_conditions (all inline).

HEAD unchanged at b0b1fd4c since the last pass. Test jobs pass on 3.6–3.14 across SQLite and Postgres; the only red check is Handle review requested / Check if author is contributor, a permissions workflow unrelated to the code. No UI files, so Phase 3 did not apply.

Prior-finding status

RESOLVED — morango/sync/stream/source.py:14 — SourceTask declares no __slots__
RESOLVED — morango/sync/stream/source.py:88 — _seen accumulates even when nothing can duplicate
RESOLVED — morango/sync/stream/source.py:75 — "passes thoughts to stream_for_filter"
ACKNOWLEDGED — morango/sync/stream/source.py:80 — Partition-major iteration drops model-dependency ordering
RESOLVED — morango/sync/stream/deserialize.py:69 — Fresh {} per store record gives the FK cache no reuse
RESOLVED — morango/sync/stream/deserialize.py:58 — skip_errored inverted the legacy default
RESOLVED — morango/registry.py:121 — _self_ref_order sort applied to every model
RESOLVED — morango/models/core.py:453 — NULLIF annotation declared BooleanField
ACKNOWLEDGED — morango/models/core.py:444 — Legacy call site in operations.py still hand-rolls the annotation
RESOLVED — tests/testapp/tests/sync/stream/test_deserialize.py:70 — StoreModelSourceTestCase mocked the ORM throughout
RESOLVED — tests/testapp/tests/sync/stream/test_deserialize.py:116 — Both NULL and empty-string deserialization_error against real rows
RESOLVED — tests/testapp/tests/test_registry.py:130 — Asserting returned rows rather than generated SQL
UNADDRESSED — morango/sync/stream/deserialize.py:69 — fk_cache or {} drops a caller-supplied empty dict
UNADDRESSED — morango/sync/stream/source.py:47 — _seen and fk_cache are not reset in begin()

---

@rtibblesbot's comments are generated by an LLM, and should be evaluated accordingly

How was this generated?

Compared the current PR state against findings from a prior review:

• Retrieved prior bot reviews via the GitHub API
• Classified each prior finding as RESOLVED, UNADDRESSED, ACKNOWLEDGED, or CONTESTED
• Only raised NEW findings for newly introduced code
• Ran the same phased review passes as a first review (core, frontend/backend lenses, manual QA when required)
• Synthesized one review from the passes and chose the verdict from the findings, CI status, and QA evidence