feat(audio): the audio module — credit-based PCM spec, web/sim hosts, a music demo that really plays - #217
Merged
Merged
Conversation
… a music demo that really plays The third module-shaped vertical slice (after ui and strike), built spec-first per the module discipline: contracts/spec/audio.ts pins ops (createStream/writePcm/play/pause/stop/setVolume/endStream/poll), tick- batched events (credit/underrun/ended), the PCM + audio:wav.* pak data contract, and a frame contract that declares the module's native-side clock — guest stays single-clock, facts batch to tick boundaries, and virtual-clock hosts consume exactly floor((n+1)r/60)-floor(nr/60) frames per tick, making consumed PCM byte-reproducible. - gen-rust emits pub mod audio into engine/core/src/spec.rs (drift-guarded); rates pinned to integer divisors of 44.1 kHz (the PSP resampler lesson) - audio.pcm capability registered; apps/music declares it as an enhancement - @pocketjs/framework/audio SDK: decodeWav + createWavPlayer (guest-side free-frame mirror, pump() once per frame, play deferred until the ring is fed so a track switch can't open on an underrun) - hosts/web: AudioWorklet ring on the device clock + main-thread credit mirror, mounted as globalThis.audio beside ui - hosts/sim: deterministic sink (PCM FNV-1a + op/event log) injected via extraGlobals - apps/music (all three framework variants) streams three original, deterministically synthesized 5 s WAVs; the tick clock stays authoritative, so all 17 committed music goldens hold without re-baking (pinned by tests/audio-sim.test.ts: pixel hashes identical with the module mounted and absent) Verified: bun run test, 54/54 web goldens, tape:check, cargo check, and a live headless-Chrome probe of the web host (worklet consuming, zero console errors). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The adoption path docs/AUDIO.md promised, executed literally: spec,
framework and app unchanged — the PSP host implements the namespace and
the target profile advertises audio.pcm.
hosts/psp/src/audio_mod.rs: up to 4 guest streams (spec MAX_STREAMS),
each an SPSC stereo ring at its source rate, mixed by one dedicated
thread into a NORMAL 44.1 kHz channel. Every hardware rule inherited
from audio.rs where it was earned: normal channel (SRC resampler
sizzles), integer-only upsample with interpolation carry, dcache
writeback before submit, reserve/release on the main thread, starved
mixer sleeps instead of queueing silence. Events cross as per-stream
atomic edges; poll() state-diffs them into the spec's JSON lines on the
main thread (ended > underrun > credit, one per call). Guest-swap
teardown resets all streams (launcher-safe); handles are generation-
tagged like ui node ids.
ffi.rs registers globalThis.audio beside ui — the pocket-mod
mount("audio", …) shape in raw QuickJS. hostAbi stays 1: the ui op
table is untouched and the SDK feature-detects the namespace, so old
bundles and old EBOOTs keep composing.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ports Adding a framework subpath used to mean six hand-kept mirrors: three package.json export groups and three resolution maps inside jsx-plugin (per-framework subpaths + two fallback tables). The audio module walked straight into it. framework/compiler/subpaths.ts is now THE declaration — one row per public module: per-framework file (availability is load-bearing: pass 1 walks exactly what resolves, so the solid-only rows keep vue-vapor and octane walk sets — and styles.bin bytes — unchanged), optional npm-only target (./renderer ships the neutral shim), and which prefixed aliases npm publishes. jsx-plugin derives its whole resolution table from it; tools/gen-exports.ts regenerates package.json's exports block (now part of `bun run gen`); tests/contract.ts byte-guards the block exactly like engine/core/src/spec.rs. Behavior-preserving by construction for every import that resolves today; the only widening is prefixed forms that previously errored. Verified: full `bun run test` chain and 54/54 web goldens byte-exact under the derived resolver. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
/docs/concepts/: the five nouns (Host, Module, Spec, Capability, Substrate) and the one relation (Runtime = Host + mounted Modules + Guest), the spec's four-part boundary, the three laws with the O(changes) placement principle, the clock rules, and the audio module as the worked example — ending on the composition property the PSP adoption just demonstrated (host-only change, spec/framework/app untouched). Slots into Introduction between Getting started and Architecture. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…19-segment one-liner package.json's "test" had grown into nineteen &&-chained segments of builds and bun-test invocations nobody could read, reorder, or partially re-run. The suite is now a Stage[] table in tools/test.ts: prep builds (quiet unless they fail), self-reporting scripts (the contracts drift guard), and one bun-test invocation per stage with the browser-conditions flag as a field instead of a spelling. Fail-fast with the exact repro command; `--stage=<substr>` runs a slice; `--list` prints the table. Same commands, same order, same CI entry point (`bun run test`) — verified 11/11 stages green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Drop the meta framing ("five nouns and one relation", "ontology",
"vertical slice", "natural algebra") for the register the rest of
/docs/ uses: state the mechanism, bold the engineering fact, keep the
examples. Same structure and content otherwise.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…n AGENTS.md Systematic pass over site/content/docs/ against the register the concepts page set: state the mechanism, bold engineering facts, drop the rhetoric. Ten edits across seven pages — "the whole point", "it helps to think of", "the mental model is familiar", "first-class frameworks", "worth internalizing" x2, "simply" x2, "Hover IS focus" caps, "earn a compile error" — each replaced by the flat factual sentence the passage already contained. api/tailwind/devtools/native-contract/ overview/getting-started/platform-contracts/styling/reactivity and README audited clean; the blog keeps its separate essay voice on purpose. docs/AUDIO.md loses its "module-shaped vertical slice" opener the same way. AGENTS.md (the CLAUDE.md symlink) now states the rule so future docs land in-register: mechanism first, concrete bolds, no meta-framing, no imported jargon, no personification, no empty intensifiers — reference files named. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Audio lands as PocketJS's third module (after the
uisurface and OpenStrike'sstrike), and the first one built module-first: spec before any host code, every host implementing the same pinned boundary. The concrete payoff:apps/musicnow actually plays WAV music in the web dev host — while every one of its 17 committed pixel goldens (web + psp×3 + vita) stays byte-identical, un-rebaked.The module (docs/AUDIO.md is the map)
globalThis.audio;AUDIO_OPcodes append-only; capability idaudio.pcm= spec namespace = pak prefix (audio:wav.*).gen-rustemitspub mod audiointoengine/core/src/spec.rs, drift-guarded bytests/contract.tsas always.creditevents (drained viapoll(), one JSON line per call — the svcPoll convention); guest writes s16 PCM as borrowed-for-the-call buffers against a guest-side free-frame mirror. Hot path: one poll drain + at most onewritePcmper tick, never a query.floor((n+1)·rate/60) − floor(n·rate/60)frames per playing tick, so consumed PCM is a pure function of (tick, ops): the audio equivalent of byte-exact goldens.hosts/psp/src/audio.rspromoted to the portable contract.Hosts
hosts/web/audio.js+audio-worklet.js(device-clock ring, gesture-deferred AudioContext)hosts/sim/audio.ts(formula consumption, PCM FNV-1a + op/event log)Consoles adopt by implementing the namespace in their FFI table and appending
audio.pcmto their target profile — no spec/framework/app change. That's the no-heavy-fork property this PR exists to establish.The demo
All three framework variants of
apps/musicwire the samecreateWavPlayer: load on select/skip, play/pause slaved to UI state, onepump()per frame. Three original 5-second tracks are deterministically synthesized (apps/music/gen-assets.ts, SHA-pinned in tests) and spliced via thepak.jsonraw-blob route. The tick clock stays authoritative — the player follows the frame counters, never drives them — which is the invarianttests/audio-sim.test.tspins by running the same journey with the module mounted and absent and asserting identical pixel hashes.Verification
bun run test(full chain, includes newtests/audio.test.ts+tests/audio-sim.test.ts) — greenbun tests/golden.ts— 54/54, music frames untouchedbun run gen+bun run contract— drift guard green;cargo check -p pocketjs-coregreenbun run tape:check— hero tape holdsDeliberately deferred (with triggers)
Host-level mixing across runtimes, codec registration, raw-PCM pak entries, native console mounts — all per the delayed-items table in the module discipline.
🤖 Generated with Claude Code