Skip to content

moq-mux: decouple importers from the catalog, split byte-parsing into per-codec splitters, and make importers pure frame publishers#1749

Open
kixelated wants to merge 20 commits into
devfrom
claude/moq-mux-import-export-api-dtkqdp
Open

moq-mux: decouple importers from the catalog, split byte-parsing into per-codec splitters, and make importers pure frame publishers#1749
kixelated wants to merge 20 commits into
devfrom
claude/moq-mux-import-export-api-dtkqdp

Conversation

@kixelated

@kixelated kixelated commented Jun 16, 2026

Copy link
Copy Markdown
Collaborator

The import half of the moq-mux refactor. It lets moq-mux fill a single track on demand without a BroadcastProducer/CatalogProducer, separates each codec's byte parsing from its publisher, and removes the manual sync() footgun.

This started as "decouple the importers from the broadcast catalog" and grew (same branch, by request) to land the per-format splitters, Published-owns-decode, and pure-publisher importers that were originally deferred. Ports every single-track importer (opus, H.264, H.265, AV1, VP8, VP9, AAC, and the legacy MP2/AC-3/E-AC-3 importer) plus the dispatchers and the TS container that embeds them. All existing callers keep working.

1. The catalog bridge (publish module)

  • Renditions trait: an importer exposes the hang::Catalog it publishes.
  • Published: mirrors an importer's renditions into a catalog::Producer and retires them on drop. Generic over the catalog extension E so it can attach to a container's extended catalog.
  • unique_track(broadcast, suffix): mints a legacy single-codec track at hang::container::TIMESCALE.

Each importer is new(TrackRequest) (on-demand) / from_track(TrackProducer) (broadcast push / fixed track) with a local hang::Catalog and a lazy catalog() (eager for audio). No catalog::Producer, no Drop hook. There is no "fixed track" concept: a changed codec config re-mirrors the rendition in place rather than erroring.

2. Per-codec splitters + pure-publisher importers

Byte parsing and publishing are fully separated:

  • codec::h264::Split / h265::Split / av1::Split: dumb byte->frame engines. They find access-unit / temporal-unit boundaries, flag keyframes, and stamp wall-clock timestamps for stdin. They own no track, catalog, or config. h264/h265 cache SPS/PPS(/VPS) and re-insert them ahead of each keyframe; h264's Split handles both avc1 (length-prefixed) and avc3 (Annex-B) shapes; AV1 carries the sequence header inline. decode_stream (unknown boundaries) / decode_frame (one AU) / decode_from / seed / reset.
  • codec::{h264,h265,av1}::Import are pure frame publishers: they take already-split frames via decode(impl IntoIterator<Item = Frame>) (the FrameDecode trait) and resolve the catalog config from the inline SPS/sequence-header in the first keyframe (or an out-of-band avcC/av1C via initialize). A keyframe that can't be configured is an error.
  • The Framed/Stream dispatchers, the TS container, moq-cli, moq-rtc, and moq-video own a Split and drive split.decode_X(buf) -> import.decode(frames).
  • Splitters are independently unit-tested (h265/av1 had no tests before; h264 gained avc1 split tests). VP8/VP9 stay one-buffer-per-frame (no streaming, no split).

3. Published owns decode (the sync() footgun is gone)

  • FrameDecode + Published::decode(impl IntoIterator<Item = Frame>): the frames path; it syncs the catalog after decoding.
  • Published::decoding(|inner| ...): the byte-path/edit wrapper (still used by VP8/VP9 and the TS jitter edit), which also syncs.
  • Published::sync is private — the only way to decode is a path that syncs.

4. lenient_start -> MissingKeyframe

The container Producer no longer silently drops pre-keyframe frames. Writing a non-keyframe with no open group returns MissingKeyframe; importers write deltas straight through, and the TS/FLV containers swallow MissingKeyframe so a mid-stream join resumes cleanly at the next keyframe.

5. thiserror everywhere

The single-track importers no longer surface anyhow: vp8/vp9/legacy (and the ac3/eac3/mp2 header parsers) get thiserror Error enums wired into crate::Error via #[from], matching h264/h265/av1.

TS container

H.264/H.265/AAC/legacy per-PID streams build through Published + a per-PID Split. AAC's synthesized description is set on the importer before attach (so Published::new's mirror covers it) and the audio-burst jitter refinement goes through decoding. External behavior is unchanged — the byte-exact roundtrip tests guard it.

Notes

  • Targets dev: breaking changes to moq-mux public APIs (new Split types, FrameDecode, Published::{decode,decoding}, private sync, importers lose their byte methods, removed with_lenient_start/FixedTrackReconfigured), built on dev-only primitives (TrackRequest, TrackInfo::with_timescale, Frame.duration).
  • Also fixes a pre-existing missing .await in rs/moq-ffi/src/test.rs (superseded by dev's tokio::join! fix after the merge).

Test plan

  • cargo test -p moq-mux (269 pass): on-demand + broadcast paths, lazy video catalogs, eager audio, Published sync/retire-on-drop and auto-sync via decode/decoding, the splitter packaging + boundary + keyframe-detection tests (h264 avc1/avc3, h265, av1), in-place reconfiguration, MissingKeyframe on a delta-before-keyframe, TS byte-exact roundtrips (incl. mid-stream / dirty joins), fMP4/MKV roundtrips.
  • cargo clippy --all-targets -- -D warnings, cargo fmt --all --check, RUSTDOCFLAGS=-D warnings cargo doc clean.
  • moq-cli, moq-rtc, moq-video, hang, moq-boy, libmoq, moq-ffi build.
  • moq-gst not built here (missing gstreamer-1.0 system lib); consumes the unchanged Framed API.

(Written by Claude)

claude added 2 commits June 16, 2026 02:40
First slice of the import refactor. The opus importer no longer holds a
`catalog::Producer` and mutates a shared catalog with a `Drop` hook. It now
produces frames on a single track and exposes its own standalone `hang::Catalog`
rendition, which a new `publish` bridge merges into a broadcast catalog
(removing it on drop).

- `codec::opus::Import`: `new(TrackRequest, Config)` (on-demand) and
  `from_track(TrackProducer, Config)` (broadcast push / fixed track). `decode`
  now takes `impl IntoIterator<Item = Frame>`; `decode_buf` is the raw-packet
  convenience that stamps a wall clock when no timestamp is given. `catalog()`
  returns the standalone `hang::Catalog`.
- `publish` module: `Renditions` trait, `Published<I>` (merges renditions into a
  `catalog::Producer`, retires on drop, derefs to the importer), and
  `unique_track` (mints a legacy single-codec track at the microsecond timescale).
- Rewire `import::Framed`, moq-gst, and moq-rtc through the bridge.

Tests cover both construction paths (TrackRequest and broadcast unique_track),
frame delivery, the catalog merge, and retire-on-drop.

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
Second slice of the import refactor. The H.264 importer joins opus on the
request-based core: it no longer holds a `catalog::Producer` or mutates a shared
catalog from a `Drop` hook. It produces frames on a single track and exposes its
own `hang::Catalog`, which the `publish` bridge mirrors into a broadcast catalog.

Unlike opus, H.264's catalog is lazy (avcC for avc1, the first SPS for avc3) and
refines over time (jitter), so `Published` gains a `sync()` that re-mirrors the
importer's renditions and is a no-op when nothing changed. Callers invoke it
after each decode.

- `codec::h264::Import`: drop the `E`/`catalog::Producer`/`TrackProvider`
  coupling. `new(TrackRequest)` (on-demand) and `from_track(TrackProducer)`
  (broadcast push / fixed track), `with_mode`, lazy `catalog() -> Option`,
  `Renditions` impl. avc1 reconfiguration now errors instead of minting a new
  track (a single fixed track can't represent a new init segment); avc3 SPS
  changes still update the rendition in place.
- `publish::Published`: generic over the catalog extension `E` (so it attaches
  to a container's extended catalog), plus `sync()` for lazy/updating catalogs.
- The TS container builds its per-PID H.264 stream through `Published`
  (`unique_track` + `from_track`), syncing after each decode; external behavior
  is unchanged (byte-exact roundtrip tests guard it).
- Rewire `import::{Framed,Stream}`, moq-cli, moq-video, moq-rtc through the
  bridge. Drop the now-unused `TrackProvider::set_suffix`.

Also fixes a pre-existing missing `.await` in the moq-ffi tests
(`dynamic_track_request_can_publish_media`) that broke `cargo check
--all-targets` on dev, unrelated to this change.

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
@kixelated kixelated changed the title moq-mux: decouple the opus importer from the broadcast catalog moq-mux: decouple the single-track importers (opus, H.264) from the broadcast catalog Jun 16, 2026
claude added 2 commits June 16, 2026 03:32
…Option

moq-video's `Producer::track()` now returns `&TrackProducer` (the avc3 track is
always created eagerly), so the `.expect(...)` no longer applies. This broke
`cargo check --all-targets` on moq-boy.

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
Completes the import reshape: h265, av1, vp8, vp9, aac, and the legacy
audio importer (MP2/AC-3/E-AC-3) now follow the same request-based core as
opus/h264. None hold a `catalog::Producer` or mutate a shared catalog from a
`Drop` hook; each produces frames on a single track and reports its own
`hang::Catalog`, attached to a broadcast catalog through `publish::Published`.

- Each importer: `new(TrackRequest)` / `from_track(TrackProducer)`, a local
  catalog, lazy `catalog() -> Option` for the video codecs (config known on the
  first key frame / SPS) and eager `catalog() -> &hang::Catalog` for aac.
  Reconfiguration on the fixed track is an error (no new-track minting).
- `import::{Framed,Stream}`: every codec arm mints via `unique_track` +
  `from_track` + `Published`, syncing the video codecs after each decode.
- TS container: H.265, AAC, and legacy per-PID streams build through
  `Published`. AAC's synthesized `description` and audio-burst `jitter` are set
  via a `pub(crate) aac::Import::rendition_mut` + `sync`, since the rendition now
  lives in the importer's local catalog. External behavior is unchanged (the
  byte-exact roundtrip tests guard it).
- Delete the now-unused `TrackProvider` (every codec mints via `unique_track`).

All codec/TS importers now share one shape; `TrackRequest` is the on-demand
entry point and `from_track` the broadcast-push one.

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
@kixelated kixelated changed the title moq-mux: decouple the single-track importers (opus, H.264) from the broadcast catalog moq-mux: decouple the single-track importers from the broadcast catalog Jun 16, 2026
claude added 7 commits June 16, 2026 13:35
First slice of the splitter / unified-decode work.

- New `codec::h264::Split`: a standalone parser that turns H.264 bytes into
  `container::Frame`s plus a resolved `VideoConfig` (the NAL/AU assembly, avcC
  parsing, SPS/PPS cache, and Annex-B wall-clock all live here, independently
  testable). avc1 still errors on reconfiguration; avc3 still updates in place.
- `codec::h264::Import` now drives `Split` internally for its byte APIs
  (`decode_frame` / `decode_stream` unchanged, so the Framed/Stream/TS
  dispatchers and external callers are untouched and don't regress) and pulls
  the resolved config into its local catalog.
- New `publish::FrameDecode` trait (`decode(impl IntoIterator<Item = Frame>)`),
  the uniform decode entry point a caller drives with frames from its own
  `Split`. `Published::decode` wraps it and syncs, so the catalog re-mirror
  can't be forgotten — the footgun-free path. h264::Import implements it.

The dispatchers still use the byte API + manual `sync()`; migrating them to
`Split` + `Published::decode` (and resolving the avc1-vs-avc3 config flow on
that path) is the next slice, then rolling the same split to the other codecs.

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
Per review: the splitter shouldn't know what a codec config is. It just
takes a single byte stream (no out-of-band init), finds access-unit
boundaries, and packages SPS/PPS into each keyframe so every keyframe is
self-contained.

- `codec::h264::Split` is now an Annex-B stream assembler only: no avc1, no
  `VideoConfig`, no `take_config`. It exposes `decode_stream` (unknown
  boundaries), `decode_frame` (one access unit), `decode_from`, `seed`
  (prime the SPS/PPS cache from an out-of-band parameter-set buffer), and
  `reset`. Wall-clock timestamps for stdin live here.
- `codec::h264::Import` owns all config, from exactly two sources: an avcC
  handed to `initialize` (avc1, required), or the SPS the splitter packages
  into the first keyframe, scanned out of the frame here (avc3, no init
  needed). It also owns the avc1 length-prefixed framed path; the stream
  splitter is Annex-B/avc3 only, matching "if you can init out of band you
  already know frame boundaries, so you don't need the stream splitter".
- A keyframe that can't be configured (no inline SPS, no avcC/seed) is a
  hard error. A non-keyframe before the first config is tolerated: it's a
  mid-stream-join leftover that the producer's lenient start drops ahead of
  the first keyframe (preserves `survives_midstream_join` and the dirty TS
  joins).

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
The lazy-catalog codecs all did `decode_frame(...)?; sync();` by hand, an
easy-to-forget two-step. Make `Published` own the pairing so the catalog
re-mirror can't be skipped.

- New `Published::decoding(|inner| ...)`: runs a decode/edit on the inner
  importer and re-mirrors the catalog in one call. Generic over the
  closure's error so it wraps both the `crate::Result` and `anyhow::Result`
  importers. Pairs with the existing `Published::decode(frames)` (frames in
  hand) as the byte-path equivalent.
- Convert every `decode + sync` site to `decoding`: the Framed and Stream
  dispatchers, the TS H.264/H.265 streams, moq-rtc, moq-video, and moq-cli.
- TS AAC: set the rendition `description` on the importer before
  `Published::new` (its attach-time mirror covers it) and route the jitter
  refinement through `decoding`.
- `Published::sync` is now private: the only way to decode is through a path
  that syncs, so the footgun is gone.

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
`Producer::with_lenient_start` silently dropped non-keyframes that arrived
before the first keyframe. Replace that implicit drop with an explicit
`MissingKeyframe` error the producer returns, and let the one caller that
wants to tolerate a mid-stream join (MPEG-TS) skip it.

- New `container::MissingKeyframe` error: `Producer::write` returns it when a
  non-keyframe arrives with no open group (was a silent drop under
  lenient_start, or a generic ProtocolViolation without it). Wired into
  `crate::Error` and `fmp4::Error` via the `Container::Error` bound.
- Drop `with_lenient_start` entirely.
- Importers now surface MissingKeyframe for a pre-keyframe delta: h264 and
  h265 (and av1) write the delta through to the producer instead of
  pre-empting with a config error, erroring early only on an *unconfigurable
  keyframe* (NotInitialized / MissingSps / MissingSequenceHeader). vp8/vp9
  already wrote straight through.
- The TS importer wraps its H.264/H.265 decode in `skip_missing_keyframe`,
  so a capture that joins mid-GOP drops the leading deltas and resumes at the
  first keyframe (preserves survives_midstream_join + the dirty TS joins).

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
Mirror the h264 split: separate the Annex-B parsing from the publisher.

- New `codec::h265::Split`: a dumb Annex-B stream assembler that finds
  access-unit boundaries, caches VPS/SPS/PPS and re-inserts them ahead of
  each keyframe, and stamps wall-clock timestamps for stdin. No track,
  catalog, or codec config. `decode_stream` / `decode_frame` / `decode_from`
  / `seed` / `reset`, like h264.
- `codec::h265::Import` now drives the splitter and owns the config: it
  scans the SPS the splitter packages into the first keyframe (or a seed
  buffer via `initialize`) to fill the catalog, errors on an unconfigurable
  keyframe, and writes pre-keyframe deltas through to the producer (which
  reports MissingKeyframe for a mid-stream join). It also implements
  `FrameDecode` so a caller with its own splitter can publish frames.
- Adds the first H.265 unit tests (the splitter packaging path).

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
Round out the streaming codecs: separate AV1 OBU parsing from the publisher,
matching h264/h265.

- New `codec::av1::Split`: a dumb OBU stream assembler that finds
  temporal-unit boundaries, flags keyframes (a sequence header or a
  KEY_FRAME), and stamps wall-clock timestamps for stdin. No track, catalog,
  or codec config. AV1 carries the sequence header inline ahead of keyframes,
  so unlike H.264/H.265 there's nothing to cache or re-insert; `seed` just
  prefixes leading metadata OBUs onto the next frame. The `ObuIterator` moves
  here. `decode_stream` keeps the per-OBU wall-clock timestamps.
- `codec::av1::Import` now drives the splitter and owns the config: it scans
  the sequence header the splitter packages into the first keyframe (or an
  av1C / seed buffer via `initialize`) to fill the catalog, falls back to a
  minimal config on a parse failure, errors on an unconfigurable keyframe,
  and writes pre-keyframe deltas through to the producer (MissingKeyframe for
  a mid-stream join). It also implements `FrameDecode`.
- Adds the first AV1 splitter unit tests (boundary + keyframe detection).

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
The lenient-start drop was replaced by the producer's MissingKeyframe; update
the two h264 comments that still described the old behavior.
@kixelated kixelated changed the title moq-mux: decouple the single-track importers from the broadcast catalog moq-mux: decouple importers from the catalog, add per-codec splitters, and let Published own decode Jun 16, 2026
claude added 4 commits June 16, 2026 19:47
Resolved conflicts:
- rs/moq-ffi/src/test.rs: kept dev's tokio::join! fix for the
  dynamic-track-request test (it supersedes this branch's earlier
  sequential subscribe_media fix).
- rs/moq-mux/src/container/flv/import.rs: dev's new FLV importer used
  the removed with_lenient_start(); ported it to the MissingKeyframe
  model (drop the call, swallow MissingKeyframe at the video write so a
  mid-GOP join still works).
Review follow-up. There are no fixed tracks anymore, so a changed codec
config just re-mirrors the catalog rendition instead of erroring.

- Remove the `FixedTrackReconfigured` error variant from the h264, h265, and
  av1 error enums.
- h264: drop `set_config`; avc1 (avcC) and avc3 (inline SPS) both resolve
  through one in-place `apply_config` that no-ops on an unchanged config.
- h265 / av1: `configure_from_sps` / `apply_config` update the rendition in
  place on a change.
- av1 `Split::decode_stream` / `decode_frame` take `impl Into<Option<Timestamp>>`,
  matching the h264 and h265 splitters.

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
Review follow-up: a library crate shouldn't surface anyhow. Port the
single-track codec importers that still used `anyhow::Result` to the crate's
thiserror-based `crate::Result`, mirroring h264/h265/av1.

- New `vp8::Error`, `vp9::Error`, and a `legacy::Error` (covering the
  MP2/AC-3/E-AC-3 header parsers), each wired into `crate::Error` via
  `#[from]` (`Vp8`/`Vp9`/`Legacy`).
- `FrameHeader::parse` (vp8/vp9), the vp9 `BitReader`, the `ac3`/`eac3`/`mp2`
  `parse_header`s, and the vp8/vp9/legacy importer methods all return the
  typed errors now; no `anyhow::ensure!`/`bail!` remain in these modules.
- Dropped the vp8/vp9 "fixed track cannot be reconfigured" bail too, so they
  update the rendition in place like the other codecs. The dispatcher test
  that asserted the old error now asserts in-place reconfiguration.
- With every importer on `crate::Result`, `Published::decoding` drops its
  generic error parameter and just takes a `crate::Result` closure.

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
The importers owned a Split internally and exposed decode_frame/decode_stream;
that duplicated the splitter and kept byte parsing in the publish layer. Move
all byte parsing to dispatcher-owned splits so the importers only take frames.

- `h264`/`h265`/`av1` `Import` lose their `split` field and the
  `decode_frame`/`decode_stream`/`decode_from` methods. They're pure
  publishers now: `decode(impl IntoIterator<Item = Frame>)` (FrameDecode) +
  config resolution (from the inline SPS/sequence-header in keyframes, or an
  avcC/av1C via `initialize`) + catalog + finish/seek/track. `initialize`
  resolves config without consuming the buffer; `seek` no longer resets a
  split.
- `h264::Split` regains avc1: it's the sole h264 byte->frame engine for both
  wire shapes (framing + NALU length size only, config stays in the importer).
  `Mode`/`with_mode`/auto-detect moved here from the importer.
- The Framed/Stream dispatchers, the TS container, moq-cli, moq-rtc, and
  moq-video now own a `Split` and drive `split.decode_X(buf) ->
  import.decode(frames)`. Small `build_h264`/`build_h265`/`build_av1` helpers
  in the dispatcher encode the "import reads config, split consumes" contract.

269 tests pass (incl. the byte-exact TS roundtrips and fMP4/MKV); the split
gained avc1 unit tests.

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
@kixelated kixelated changed the title moq-mux: decouple importers from the catalog, add per-codec splitters, and let Published own decode moq-mux: decouple importers from the catalog, split byte-parsing into per-codec splitters, and make importers pure frame publishers Jun 17, 2026
claude and others added 5 commits June 17, 2026 02:49
A Split is for a raw byte stream (stdin / unknown boundaries); the
known-boundary decode_frame didn't belong on it.

- Rename `decode_stream` -> `decode` on all three splits, add `flush` (emit
  the in-flight access/temporal unit), and drop `decode_frame`. `decode_from`
  flushes at EOF. The Annex-B splits (h264/h265) keep a `tail` buffer so
  `decode` fully consumes the caller's buffer (Framed's contract) while
  retaining the trailing NAL across chunks; `flush` pulls it.
- Drop avc1 from `h264::Split` entirely. avc1 (length-prefixed + out-of-band
  avcC) is not a stream and can't arrive over stdin, so `Mode`/`with_mode`/
  `initialize`/`detect_mode` and the avc1 framing leave the splitter. avc1
  becomes a free `h264::avc1_frame(data, length_size, pts)` helper.
- Framed splits its H264 arm into `Avc1 { length_size, import }` (no split,
  wraps each AU via `avc1_frame`) and `Avc3 { split, import }`. Known-boundary
  callers (Framed avc3/hev1/av01, TS, moq-rtc, moq-video) do `decode + flush`
  per unit; stdin callers (Stream, moq-cli) flush the tail at finish/EOF.

265 tests pass (incl. TS byte-exact + fMP4/MKV roundtrips, moq-rtc bitstream).

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
No caller used the splitters' decode_from async helper (the container
importers have their own). Remove it from h264/h265/av1 Split; a caller that
wants to drive a reader loops decode() + flush() itself.
A Split is just a byte stream; a dedicated "here are out-of-band parameter
sets" hook (seed) contradicts that. Remove it from all three splits.

The dispatchers' init buffer (the codec header passed to Framed::new /
Stream::initialize) is now fed to the splitter via `decode` as the leading
bytes of the stream: the importer still reads it for the catalog config, and
the splitter caches any inline SPS/PPS the same way it would mid-stream, so a
"parameter sets once up front, then bare keyframes" encoder still produces
self-contained keyframes. av1C (the out-of-band 0x81 config record) is not an
OBU stream, so it stays config-only and isn't fed to the splitter.

The two seed unit tests now exercise the same property through `decode`
(leading params + a later bare IDR -> self-contained keyframe).

265 tests pass; dependents (moq-cli/rtc/video/ffi/libmoq) build.

https://claude.ai/code/session_011S7pzcg2XsPP3AExuymac1
Merge the new `publish` module into `import` and rename `Published` to
`import::Track`. The module is now a directory: `import/mod.rs` (the
`Framed`/`Stream` format dispatchers) plus a private `import/track.rs`
(the catalog-bridge `Track`, `Renditions`, `FrameDecode`, `unique_track`),
re-exported flat. The `publish::Published` stutter is gone, and "import"
already names the ingest direction (the mirror of `export`).

Add `moq_net::TrackDemand`, a cloneable, weak-backed watch-only handle
(`name`/`used`/`unused`/`closed`) obtained from `TrackProducer::demand()`.
It can't publish or close the track and doesn't pin the group cache, so
callers can gate on subscriber demand without holding a writable producer.
`TrackProducer: Clone` is left intact for now.

Encapsulate the high-level front door: `Framed` no longer hands out a
`&TrackProducer`. The match is now a private `producer()` helper behind
curated `name()` / `subscribe()` / `demand()` accessors. moq-ffi's
`MediaProducer` holds a `TrackDemand` instead of a cloned producer. The
low-level `Track` and codec importers keep their public `track()`, since
their callers build the track themselves, so moq-video/boy/audio/rtc/cli
need no changes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reshape the import layer so each codec importer owns its catalog rendition
and deals only in whole frames, and so concurrently-produced tracks share
one timeline.

Catalog: add `catalog::VideoTrack` / `catalog::AudioTrack`, scoped handles
(via `Producer::video_track` / `audio_track`) that publish one importer's
rendition and retire it on drop. This replaces the `import::Track` wrapper
plus the `Renditions` / `FrameDecode` traits, which are deleted; the
`Published`-style mirror is gone.

Importers: every codec importer is now `Import<E: CatalogExt = ()>` built
with a single `new(track, catalog, [config])` (the `TrackRequest`
constructor and `from_track` are dropped; the on-demand path accepts the
request at the call site). They expose `demand() -> TrackDemand` instead of
handing out a `TrackProducer`, take whole frames as `&[u8]` (no more `Buf`),
and the per-importer `decode_buf` / `pts` helpers collapse into one
`decode`. `Framed::decode_frame` becomes `Framed::decode(&[u8], pts)`.

Sync: the wall-clock fallback moves to a `Clock` owned by the shared
`catalog::Producer`, so audio and video synthesizing timestamps anchor to
the same epoch (`Producer::timestamp` / `VideoTrack`/`AudioTrack::timestamp`).

Containers and consumers (ffi/cli/rtc/gst/video/boy) are updated to the new
constructors and `demand()`.

Follow-ups: split `Framed` into `Framed` + `FramedTrack`, and give the
TS/MKV/FLV/fMP4 containers their own `Split` modules so they too deal in
frames.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants