Skip to content

feat(moq-net): add OriginDynamic for unannounced fallback broadcasts#1772

Merged
kixelated merged 7 commits into
devfrom
claude/optimistic-elbakyan-f3650f
Jun 17, 2026
Merged

feat(moq-net): add OriginDynamic for unannounced fallback broadcasts#1772
kixelated merged 7 commits into
devfrom
claude/optimistic-elbakyan-f3650f

Conversation

@kixelated

@kixelated kixelated commented Jun 17, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds OriginDynamic, the origin-level analogue of the existing BroadcastDynamic/TrackDynamic on-demand handlers, makes request_broadcast the one public broadcast-lookup, and reworks the FFI surface around it.

Level Container On-demand handler
Origin broadcasts by path OriginDynamic (new)
Broadcast tracks by name BroadcastDynamic
Track groups by sequence TrackDynamic

request_broadcast (the unified lookup)

OriginConsumer::request_broadcast(path) -> Result<kio::Pending<BroadcastRequested>, Error> — a kio::Pending future you await (not an async fn), mirroring TrackConsumer::fetch_group. Registration is synchronous (a handler sees the request immediately); the future then resolves:

  • announced → immediately with the announced broadcast;
  • not announced + live OriginDynamic → once the handler accepts (live broadcast) or rejects (error), coalescing concurrent requests for the same path;
  • not announced + no handler → fails fast, synchronously, with NotFound (Dropped once the origin is gone).

Not announced, by design

Dynamically served broadcasts are never announcedOriginConsumer::announced never observes them. The request queue lives in a shared OriginDynamicState off to the side of the announce tree (ref-counted handlers, coalesced requests, reject-on-last-drop). The result rides a one-shot kio channel; kio checks the value before the closed flag, so the outcome is observed without a close race.

Handler side

OriginProducer::dynamic() -> OriginDynamic; requested_broadcast() yields a BroadcastRequest. accept(broadcast) resolves every awaiting requester with the supplied live broadcast (the handler keeps producing into it, e.g. a relay proxying upstream); reject(err) resolves them with an error.

get_broadcast retired from the public API

OriginConsumer::get_broadcast was a synchronous announced-only peek that races announcement gossip. It's now private — the internal helper backing request_broadcast's announced case (and the tree-state test assertions). Every caller moved to request_broadcast:

  • moq-net serve paths: lite recv_subscribe, run_track_info, run_fetch, ietf subscribe.
  • moq-rtc: WHIP/WHEP gateways.

With no OriginDynamic registered this is behavior-identical (resolve-if-announced, else fail fast); a registered handler adds the fallback. announced_broadcast is unchanged (race-free await-for-announce).

FFI: dynamic broadcast requesting

Neither libmoq nor moq-ffi could previously request a broadcast that hadn't been announced. Now:

  • rs/moq-ffi: MoqOriginConsumer::request_broadcast (async).
  • rs/libmoq: moq_origin_request + moq_origin_request_close (callback style). Removes the synchronous moq_origin_consume (breaking): it was a racy announced-only peek; moq_origin_request resolves an already-announced broadcast immediately (same common case, no race) and adds the dynamic fallback. Callers reacting to an announcement use the broadcast delivered by moq_origin_announced.
  • Wrappers: py (OriginConsumer + Client), swift (OriginConsumer), go (OriginConsumer + Client); kt via the uniffi typealias.
  • Docs: py README + doc/lib/py; libmoq README + doc/lib/c.

FFI: free delivered announcements

moq_origin_announced handed its callback a fresh announce ID per event with no way to release one (they accumulated). Adds moq_origin_announced_free to drop a delivered record once read. Explicit free, not auto-on-unannounce: an unannounce is its own record, and auto-freeing the prior one would race a caller still reading its borrowed path pointer.

Scope / follow-ups

  • Targets dev (new public surface in rs/moq-net + rs/moq-ffi; get_broadcast privatized; moq_origin_consume removed).
  • Follow-up: register an OriginDynamic in the relay/cluster to actually fetch from upstream on a fallback request; js/net mirror; carry the BroadcastConsumer in libmoq's announced record (+ accessor) so announce→consume is fully race-free (now unblocked by moq_origin_announced_free).

Public API changes

rs/moq-net: added OriginProducer::dynamic, OriginConsumer::request_broadcast, OriginDynamic, BroadcastRequest, BroadcastRequested; OriginConsumer::get_broadcast is now private.
rs/moq-ffi: added MoqOriginConsumer::request_broadcast.
rs/libmoq: added moq_origin_request, moq_origin_request_close, moq_origin_announced_free; removed moq_origin_consume.

Test plan

  • moq-net: 385 lib tests (7 new for OriginDynamic incl. "never announced")
  • request_broadcast fail-fast / served / coalesce / reject / handler-dropped / prefers-announced / clone-keeps-alive
  • libmoq announced_free_lifecycle; 7 roundtrip tests migrated onto moq_origin_request
  • cargo test -p moq-ffi -p libmoq (24 + 25); moq-rtc / moq-relay build
  • cargo fmt --check + clippy --all-targets clean for moq-net / moq-ffi / libmoq / moq-rtc (via nix)

(Written by Claude)

kixelated and others added 7 commits June 17, 2026 10:05
Adds the origin-level analogue of BroadcastDynamic. Where BroadcastDynamic
serves tracks on demand within a broadcast and TrackDynamic serves uncached
groups within a track, OriginDynamic serves whole broadcasts on demand within
an origin.

A consumer calls the new OriginConsumer::request_broadcast(path), which first
tries the existing announced lookup and, if nothing is announced, registers a
fallback request for an OriginProducer::dynamic() handler to pick up via
requested_broadcast(). The handler accepts with a BroadcastDynamic to serve the
broadcast's tracks (e.g. a relay proxying upstream).

Dynamically served broadcasts are deliberately never announced: they exist only
as a fallback when a live announcement is absent, so OriginConsumer::announced
never observes them. The request queue lives in a shared OriginDynamicState off
to the side of the announce tree, mirroring the dynamic/requests/request_order
shape of the broadcast and track models (ref-counted handlers, coalesced
requests, reject-on-last-drop).

Purely additive in-process API, no wire change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… through it

Reworks request_broadcast to return a kio::Pending future (like
TrackConsumer::fetch_group) instead of a half-pending BroadcastConsumer:

- request_broadcast(path) -> Result<kio::Pending<BroadcastRequested>, Error>.
  Registration is synchronous (so a handler sees the request immediately); the
  future resolves to the announced broadcast at once, or once an OriginDynamic
  handler accepts (live broadcast) or rejects (error), or NotFound if every
  handler drops first.
- BroadcastRequest::accept now takes the broadcast to serve
  (impl Consume<BroadcastConsumer>) and resolves all awaiting requesters; reject
  resolves them with an error. The result rides a one-shot kio channel; kio polls
  the value before the closed flag, so the final result is observed without a
  close race.
- Coalescing is unchanged: concurrent requests for the same queued path share one
  handler request.

Routes the publisher serve paths through request_broadcast instead of
get_broadcast: lite recv_subscribe / run_track_info / run_fetch and the ietf
subscribe handler. With no OriginDynamic handler registered this is identical to
before (immediate announced lookup or NotFound); a registered handler gains the
dynamic fallback. get_broadcast stays as the synchronous announced-only peek for
tests and external callers (libmoq, moq-rtc).

announced_broadcast is unchanged (still the race-free await-for-announce path).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds dynamic broadcast requesting to the FFI layers so callers aren't gated to
only-announced broadcasts. request_broadcast resolves the announced broadcast
immediately, falls back to an OriginDynamic handler if the origin has one, or
errors; unlike the announce-wait paths it does not block for a future announce.

- rs/moq-ffi: MoqOriginConsumer::request_broadcast (async).
- rs/libmoq: moq_origin_request + moq_origin_request_close (callback style,
  reusing the consume-task slab). Sits between moq_origin_consume (announced-only,
  fails fast) and moq_origin_consume_announced (waits indefinitely).
- Wrappers: py (OriginConsumer + Client), swift (OriginConsumer), go
  (OriginConsumer + Client). kt gets it for free via the uniffi typealias.
- Docs: py README + doc/lib/py, and the libmoq callback-function list in
  doc/lib/c. The go/swift/kt doc pages are prose quickstarts that don't enumerate
  per-method APIs, so request_broadcast flows through their generated/typealiased
  API docs instead.

libmoq's existing get_broadcast-based `consume` is intentionally left as-is: it
must stay a synchronous announced-only peek so a stale announce/unannounce can't
trigger a dynamic fetch. Carrying the announced BroadcastConsumer in libmoq's
announce buffer (to make the announce->consume handoff race-free) is a separate
follow-up; it needs a free lifecycle on that slab and doesn't regress today's path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…free

moq_origin_announced hands its callback a fresh announce ID for every
announce/unannounce event, but there was no way to release one, so they
accumulated for the life of the listener. Adds moq_origin_announced_free (and
Origin::announced_free) to drop a single delivered record once read.

Explicit free rather than auto-cleanup on unannounce: an unannounce is its own
delivered record, and auto-freeing the prior one would race a caller still
reading its info (the path pointer borrows the record's storage). This is
per-record and distinct from moq_origin_announced_close, which stops the listener.

Adds an announced_free_lifecycle test and documents the free in doc/lib/c.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…gin_request

Removes the synchronous moq_origin_consume (and Origin::consume): it was a
point-in-time announced-only peek that raced announcement gossip. moq_origin_request
covers the same case (it resolves an already-announced broadcast immediately) without
the racy footgun, and additionally supports the dynamic fallback.

Callers reacting to an announcement should use the broadcast delivered by
moq_origin_announced; callers that want a specific path use moq_origin_request.

Migrates the libmoq tests to a request_broadcast helper over moq_origin_request, and
updates the README signature list and doc cross-references.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…to request_broadcast

get_broadcast is a synchronous announced-only peek that races announcement gossip.
request_broadcast is now the public lookup (announced-now resolves immediately, with
a dynamic fallback), so the racy peek no longer needs to be public surface.

- moq-net: drop `pub` from OriginConsumer::get_broadcast; it stays as the internal
  helper backing request_broadcast (and the tree-state test assertions).
- moq-rtc: WHIP/WHEP gateways look up the broadcast via request_broadcast. With no
  dynamic handler this is identical (resolve-if-announced, else fail fast); a handler
  would add the fallback.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cargo doc -D warnings (rustdoc::private_intra_doc_links) rejected the public
announced_broadcast / request_broadcast doc comments linking to Self::get_broadcast
after it was made private. Point them at request_broadcast / drop the link.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@kixelated kixelated enabled auto-merge (squash) June 17, 2026 21:52
@kixelated kixelated merged commit 11c3df9 into dev Jun 17, 2026
5 checks passed
@kixelated kixelated deleted the claude/optimistic-elbakyan-f3650f branch June 17, 2026 22:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant