Skip to content

feat(audio): the audio module — credit-based PCM spec, web/sim hosts, a music demo that really plays - #217

Merged
doodlewind merged 7 commits into
mainfrom
inconclusive-alley
Aug 4, 2026
Merged

feat(audio): the audio module — credit-based PCM spec, web/sim hosts, a music demo that really plays#217
doodlewind merged 7 commits into
mainfrom
inconclusive-alley

Conversation

@doodlewind

Copy link
Copy Markdown
Collaborator

What

Audio lands as PocketJS's third module (after the ui surface and OpenStrike's strike), and the first one built module-first: spec before any host code, every host implementing the same pinned boundary. The concrete payoff: apps/music now actually plays WAV music in the web dev host — while every one of its 17 committed pixel goldens (web + psp×3 + vita) stays byte-identical, un-rebaked.

The module (docs/AUDIO.md is the map)

platform audio device (AudioWorklet · sceAudio · …)      Host / substrate
audio core: per-stream PCM ring + credit accounting
audio spec: ops · events · data contract · frame contract (contracts/spec/audio.ts)
SDK: @pocketjs/framework/audio (decodeWav, createWavPlayer)
apps/music                                               Guest
  • Own namespace, own op space. Mounted as globalThis.audio; AUDIO_OP codes append-only; capability id audio.pcm = spec namespace = pak prefix (audio:wav.*). gen-rust emits pub mod audio into engine/core/src/spec.rs, drift-guarded by tests/contract.ts as always.
  • Credit-based flow control. Core grants ring space as credit events (drained via poll(), one JSON line per call — the svcPoll convention); guest writes s16 PCM as borrowed-for-the-call buffers against a guest-side free-frame mirror. Hot path: one poll drain + at most one writePcm per tick, never a query.
  • Multi-clock, honestly. The frame contract declares a native-side audio clock (worklet/thread) that never calls the guest and batches its facts to tick boundaries — law 3 holds unchanged. Virtual-clock hosts consume exactly floor((n+1)·rate/60) − floor(n·rate/60) frames per playing tick, so consumed PCM is a pure function of (tick, ops): the audio equivalent of byte-exact goldens.
  • Rates pinned to 44100/22050/11025 — integer divisors of the PSP's native output, the hardware lesson from hosts/psp/src/audio.rs promoted to the portable contract.

Hosts

host embodiment status
web hosts/web/audio.js + audio-worklet.js (device-clock ring, gesture-deferred AudioContext) audible
sim hosts/sim/audio.ts (formula consumption, PCM FNV-1a + op/event log) deterministic tests
psp / vita / others not mounted; contract constants shipped, ring/thread discipline already exists in their video-plane audio documented seam, capability not advertised

Consoles adopt by implementing the namespace in their FFI table and appending audio.pcm to their target profile — no spec/framework/app change. That's the no-heavy-fork property this PR exists to establish.

The demo

All three framework variants of apps/music wire the same createWavPlayer: load on select/skip, play/pause slaved to UI state, one pump() per frame. Three original 5-second tracks are deterministically synthesized (apps/music/gen-assets.ts, SHA-pinned in tests) and spliced via the pak.json raw-blob route. The tick clock stays authoritative — the player follows the frame counters, never drives them — which is the invariant tests/audio-sim.test.ts pins by running the same journey with the module mounted and absent and asserting identical pixel hashes.

Verification

  • bun run test (full chain, includes new tests/audio.test.ts + tests/audio-sim.test.ts) — green
  • bun tests/golden.ts54/54, music frames untouched
  • bun run gen + bun run contract — drift guard green; cargo check -p pocketjs-core green
  • bun run tape:check — hero tape holds
  • Live headless-Chrome probe of the web host: namespace mounted, worklet consuming (credit free-values oscillating at steady state), zero console errors, demo at 60 FPS

Deliberately deferred (with triggers)

Host-level mixing across runtimes, codec registration, raw-PCM pak entries, native console mounts — all per the delayed-items table in the module discipline.

🤖 Generated with Claude Code

doodlewind and others added 7 commits August 4, 2026 23:59
… a music demo that really plays

The third module-shaped vertical slice (after ui and strike), built
spec-first per the module discipline: contracts/spec/audio.ts pins ops
(createStream/writePcm/play/pause/stop/setVolume/endStream/poll), tick-
batched events (credit/underrun/ended), the PCM + audio:wav.* pak data
contract, and a frame contract that declares the module's native-side
clock — guest stays single-clock, facts batch to tick boundaries, and
virtual-clock hosts consume exactly floor((n+1)r/60)-floor(nr/60)
frames per tick, making consumed PCM byte-reproducible.

- gen-rust emits pub mod audio into engine/core/src/spec.rs (drift-guarded);
  rates pinned to integer divisors of 44.1 kHz (the PSP resampler lesson)
- audio.pcm capability registered; apps/music declares it as an enhancement
- @pocketjs/framework/audio SDK: decodeWav + createWavPlayer (guest-side
  free-frame mirror, pump() once per frame, play deferred until the ring
  is fed so a track switch can't open on an underrun)
- hosts/web: AudioWorklet ring on the device clock + main-thread credit
  mirror, mounted as globalThis.audio beside ui
- hosts/sim: deterministic sink (PCM FNV-1a + op/event log) injected via
  extraGlobals
- apps/music (all three framework variants) streams three original,
  deterministically synthesized 5 s WAVs; the tick clock stays
  authoritative, so all 17 committed music goldens hold without re-baking
  (pinned by tests/audio-sim.test.ts: pixel hashes identical with the
  module mounted and absent)

Verified: bun run test, 54/54 web goldens, tape:check, cargo check, and
a live headless-Chrome probe of the web host (worklet consuming, zero
console errors).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The adoption path docs/AUDIO.md promised, executed literally: spec,
framework and app unchanged — the PSP host implements the namespace and
the target profile advertises audio.pcm.

hosts/psp/src/audio_mod.rs: up to 4 guest streams (spec MAX_STREAMS),
each an SPSC stereo ring at its source rate, mixed by one dedicated
thread into a NORMAL 44.1 kHz channel. Every hardware rule inherited
from audio.rs where it was earned: normal channel (SRC resampler
sizzles), integer-only upsample with interpolation carry, dcache
writeback before submit, reserve/release on the main thread, starved
mixer sleeps instead of queueing silence. Events cross as per-stream
atomic edges; poll() state-diffs them into the spec's JSON lines on the
main thread (ended > underrun > credit, one per call). Guest-swap
teardown resets all streams (launcher-safe); handles are generation-
tagged like ui node ids.

ffi.rs registers globalThis.audio beside ui — the pocket-mod
mount("audio", …) shape in raw QuickJS. hostAbi stays 1: the ui op
table is untouched and the SDK feature-detects the namespace, so old
bundles and old EBOOTs keep composing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ports

Adding a framework subpath used to mean six hand-kept mirrors: three
package.json export groups and three resolution maps inside jsx-plugin
(per-framework subpaths + two fallback tables). The audio module walked
straight into it.

framework/compiler/subpaths.ts is now THE declaration — one row per
public module: per-framework file (availability is load-bearing: pass 1
walks exactly what resolves, so the solid-only rows keep vue-vapor and
octane walk sets — and styles.bin bytes — unchanged), optional npm-only
target (./renderer ships the neutral shim), and which prefixed aliases
npm publishes. jsx-plugin derives its whole resolution table from it;
tools/gen-exports.ts regenerates package.json's exports block (now part
of `bun run gen`); tests/contract.ts byte-guards the block exactly like
engine/core/src/spec.rs.

Behavior-preserving by construction for every import that resolves
today; the only widening is prefixed forms that previously errored.
Verified: full `bun run test` chain and 54/54 web goldens byte-exact
under the derived resolver.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
/docs/concepts/: the five nouns (Host, Module, Spec, Capability,
Substrate) and the one relation (Runtime = Host + mounted Modules +
Guest), the spec's four-part boundary, the three laws with the
O(changes) placement principle, the clock rules, and the audio module
as the worked example — ending on the composition property the PSP
adoption just demonstrated (host-only change, spec/framework/app
untouched). Slots into Introduction between Getting started and
Architecture.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…19-segment one-liner

package.json's "test" had grown into nineteen &&-chained segments of
builds and bun-test invocations nobody could read, reorder, or partially
re-run. The suite is now a Stage[] table in tools/test.ts: prep builds
(quiet unless they fail), self-reporting scripts (the contracts drift
guard), and one bun-test invocation per stage with the browser-conditions
flag as a field instead of a spelling. Fail-fast with the exact repro
command; `--stage=<substr>` runs a slice; `--list` prints the table.
Same commands, same order, same CI entry point (`bun run test`) —
verified 11/11 stages green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Drop the meta framing ("five nouns and one relation", "ontology",
"vertical slice", "natural algebra") for the register the rest of
/docs/ uses: state the mechanism, bold the engineering fact, keep the
examples. Same structure and content otherwise.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…n AGENTS.md

Systematic pass over site/content/docs/ against the register the
concepts page set: state the mechanism, bold engineering facts, drop the
rhetoric. Ten edits across seven pages — "the whole point", "it helps to
think of", "the mental model is familiar", "first-class frameworks",
"worth internalizing" x2, "simply" x2, "Hover IS focus" caps,
"earn a compile error" — each replaced by the flat factual sentence the
passage already contained. api/tailwind/devtools/native-contract/
overview/getting-started/platform-contracts/styling/reactivity and
README audited clean; the blog keeps its separate essay voice on
purpose. docs/AUDIO.md loses its "module-shaped vertical slice" opener
the same way.

AGENTS.md (the CLAUDE.md symlink) now states the rule so future docs
land in-register: mechanism first, concrete bolds, no meta-framing, no
imported jargon, no personification, no empty intensifiers — reference
files named.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@doodlewind
doodlewind marked this pull request as ready for review August 4, 2026 21:18
@doodlewind
doodlewind merged commit c6ab0f5 into main Aug 4, 2026
2 checks passed
@doodlewind
doodlewind deleted the inconclusive-alley branch August 4, 2026 21:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant