Compositions documentation - #278
Open
Gawor270 wants to merge 39 commits into
Open
Conversation
New top-level docs section for the hosted composition service: concept overview (disambiguated from the client-side Smelter library), quick start, template authoring with the composition SDK and CLI, room composition, runtime events, an input/output guide, a scene-model explainer, and an API reference. The REST reference renders a sanitized preview OpenAPI spec via Scalar; the auth model and WebSocket event stream are documented in prose.
Replace the hand-written preview spec with one generated from the
service source, copied by scripts/update_api.sh (the source repo is
private, so it is cloned on demand instead of being a submodule) and
sanitized of internal-only fields. The committed spec comes from the
template-workers branch with the oneOf variant-title improvement
applied, matching what main will contain after the pending merge.
Align the docs with the actual API surface: templated outputs are
registered in a single multipart request to POST
.../output/{output_id}/template (config + template), not registered
first and uploaded after; rescaler takes a single child. Template code
samples are now type-checked by the build, with
@fishjam-cloud/composition linked from the js-server-sdk submodule.
CI's SUBMODULES_TOKEN covers the private source repo (same as the fishjam and room-manager submodules), so clone-on-demand is not needed. The repo has no semver release tags yet, so the submodule is checked out at a branch instead of the latest tag.
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new beta Smelter Cloud documentation section to the Docusaurus site, including concept docs, how-to guides, and a published OpenAPI/Scalar REST API reference.
Changes:
- Introduces new
docs/smelter/docs tree (overview, quick start, scenes, how-tos, API reference) and sidebar categories. - Publishes Smelter Cloud REST API docs via Scalar at
/api/smelter-cloud, backed by a committed OpenAPI JSON spec. - Updates tooling to fetch/copy the Smelter Cloud spec from a new submodule and links the local
@fishjam-cloud/compositionpackage for type-checkable examples.
Reviewed changes
Copilot reviewed 16 out of 18 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| yarn.lock | Adds workspace link entry for @fishjam-cloud/composition. |
| static/api/smelter-cloud-openapi.json | Adds committed Smelter Cloud OpenAPI 3.1 JSON spec for publishing. |
| spelling.txt | Adds rtmp to spellcheck allowlist. |
| scripts/update_api.sh | Adds branch-based submodule checkout and JSON spec sanitization/copy step. |
| package.json | Adds linked dependency on @fishjam-cloud/composition for examples/typechecking. |
| docusaurus.config.ts | Adds a Scalar instance to render the Smelter Cloud REST API reference page. |
| docs/smelter/what-is-smelter-cloud.mdx | Adds Smelter Cloud concept overview and navigation pointers. |
| docs/smelter/scenes.mdx | Documents video/audio scene model and live updates. |
| docs/smelter/quick-start.mdx | Adds end-to-end tutorial for creating/registering inputs/outputs and publishing. |
| docs/smelter/how-to/write-and-deploy-a-template.mdx | Adds template authoring/build/deploy guide including multipart upload endpoint. |
| docs/smelter/how-to/inputs-and-outputs.mdx | Summarizes supported input/output protocols and key fields. |
| docs/smelter/how-to/drive-a-template-with-events.mdx | Documents event delivery to templates via /event endpoint and eventBus. |
| docs/smelter/how-to/compose-a-fishjam-room.mdx | Documents linking a Fishjam room and templating room-driven layouts. |
| docs/smelter/how-to/category.json | Adds How-to Guides generated index under Smelter Cloud section. |
| docs/smelter/api-reference.mdx | Adds entry-point page for REST API, WebSocket auth, and error format. |
| docs/smelter/category.json | Adds top-level “Smelter Cloud” sidebar category at position 1. |
| .gitmodules | Adds api/smelter-cloud submodule entry. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Guard the jq dependency in update_api.sh, use optional chaining on peer metadata in the template example, and fix quick-start wording: a whip_client output publishes to a WHIP endpoint; playback is WHEP on the receiving service.
Gawor270
marked this pull request as ready for review
July 24, 2026 14:37
The reference page now lives at /api/smelter-cloud next to the other API docs, the Scalar page moves to /api/smelter-cloud/rest (mirroring the /api + /api/rest pair of the server API), and the API hub page links both. Smelter section links updated.
Replace the navbar dropdown with a single API Reference link landing on the /api index, where the Composition API (Smelter Cloud) sits as a subsection next to the SDK references and the hub page. The prose page is renamed Composition API to match the spec title shown by Scalar.
Split the single autogenerated sidebar in two: the Docs sidebar drops the api category, and a new apiSidebar (docs and versioned) holds the API index, the SDK references, and the Composition API page. Landing on API Reference now shows only API navigation instead of the full docs tree.
Dissolve the standalone Smelter Cloud section: the concept page (with the scene model) lives in explanation/, the quick start in tutorials/, and the guides in how-to/compositions/. All content now presents compositions as a Fishjam feature named Compositions, without beta notes. The API Reference returns to its earlier shape. The Composition API is published like the other APIs: a Scalar page at /api/compositions, a second document on /api/rest, a navbar dropdown listing both APIs, and a Compositions section (REST, WebSocket, auth, errors) on the API hub page. A presentation overlay (scripts/composition_openapi_overlay.jq) adds operation summaries, tag groups, request examples, and titles for anonymous oneOf variants. The submodule and spec are renamed to api/composition and composition-openapi.json. The room guide now documents the real flow: a single Fishjam track_forwardings call forwards the room, links the composition, and registers inputs automatically. The template guide starts from a simple static example, and the room-aware hooks live in the room guide.
Gawor270
changed the title
The submodule was moved to api/composition earlier but kept its old api/smelter-cloud section name in .gitmodules.
Bump the api/composition submodule to the template-workers merge of fishjam-cloud/foundry#133 and refresh the published spec. The source spec now carries all presentation metadata itself: operation summaries, human tag groups with a new Room forwarding tag, request body examples for every operation, titled union variants, and tagged unions whose examples validate against their schemas.
The overlay existed to add summaries, tags, examples, and variant titles the generated spec lacked, and to strip internal fields and relative links. All of it now lives in the source generator (fishjam-cloud/foundry#133), and the overlay verifiably produces byte-identical output to a plain copy, so update_api.sh copies the spec directly and no longer needs jq. Also quote the paths in checkout_submodule_branch.
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 14 out of 16 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (7)
docs/how-to/compositions/compose-a-fishjam-room.mdx:114
- This curl example uses a placeholder
Authorization: ******header, which is not a valid HTTP Authorization value and omits the requiredBearerprefix for the Management Token.
-H "Authorization: Bearer $TOKEN" \
docusaurus.config.ts:390
- The
/api/restScalar config declares asourcesentry for the Composition API, but the page-levelauthentication.preferredSecuritySchemeis set tomanagement_token, which does not exist incomposition-openapi.json(it usesBearerAuth). This can break or misconfigure the auth UI when switching to the Composition source, and it’s also redundant given the dedicated/api/compositionsScalar page below.
title: "Composition API",
slug: "composition",
url: "/docs/api/composition-openapi.json",
},
],
docs/how-to/compositions/compose-a-fishjam-room.mdx:97
- This curl example uses a placeholder
Authorization: ******header, which is not a valid HTTP Authorization value and omits the requiredBearerprefix for the Management Token.
-H "Authorization: Bearer $TOKEN" \
docs/how-to/compositions/compose-a-fishjam-room.mdx:125
- This curl example uses a placeholder
Authorization: ******header, which is not a valid HTTP Authorization value and omits the requiredBearerprefix for the Management Token.
-H "Authorization: Bearer $TOKEN"
docs/how-to/compositions/compose-a-fishjam-room.mdx:136
- This curl example uses a placeholder
Authorization: ******header, which is not a valid HTTP Authorization value and omits the requiredBearerprefix for the Management Token.
-H "Authorization: Bearer $TOKEN"
docs/api/reference.md:93
- The Composition authentication table contains broken/placeholder header values (e.g. missing
Bearer, missing closing backticks) which will render incorrectly and gives an invalid auth example.
| Control-plane calls (composition, inputs, outputs, renderers, events, room) | `Authorization: Bearer <token>`: your Fishjam **Management Token**, the same token used across Fishjam. |
| Publishing to an input (`/whip/{input_id}`) | The **input's** own bearer token, returned when you register a `whip_server` input (or the one you supplied). This is distinct from your account token. |
| Playing back an output (`/whep/{output_id}`) | Optional. Public playback is allowed; add a Bearer token only if you want to require one. |
docs/how-to/compositions/compose-a-fishjam-room.mdx:88
- This curl example uses a placeholder
Authorization: ******header, which is not a valid HTTP Authorization value and omits the requiredBearerprefix for the Management Token.
This issue also appears in the following locations of the same file:
- line 97
- line 114
- line 125
- line 136
-H "Authorization: Bearer $TOKEN" \
Verified the compositions docs against foundry, fishjam, and the SFU, and fixed the places where they diverged from the implementation: - Drop the claim that a track forwarding can be removed from a live room. Fishjam routes track_forwardings with only: [:create], and the SFU only tears a forwarding down on room stop or endpoint crash. - Describe cleanup as it works: after five minutes with no media on any input, rather than as soon as the composition has no inputs. - Explain cleanup_without_inputs: false as widening the cleanup condition to need inputs and outputs quiet, not as switching cleanup off. - Note that a room forwards to one composition at a time, with the repeat call being a no-op and a different target returning 409. - Cover the start, reset, and delete lifecycle calls, and list every endpoint that can answer 503, not just composition creation.
PiotrWodecki
reviewed
Aug 3, 2026
PiotrWodecki
left a comment
PiotrWodecki
left a comment
Member
There was a problem hiding this comment.
postponing review until internal things are live
PiotrWodecki
self-requested a review
August 6, 2026 09:59
Ran every documented call against rtc.fishjam.io and built every template snippet with the published composition-cli. Fixes what did not hold up: - Scene updates must mirror the sides the output was registered with. The docs claimed the opposite, that omitted sides were left untouched, and the example they gave returns 400 for any output that has audio. - Drop the WHEP playback row from the auth table. Outputs cannot be played back from the Composition API, since whep_server outputs are rejected, so viewers connect to the destination the output pushes to instead. - Show endpoint_route in the input register response, which the service returns but neither the docs nor the OpenAPI document mentioned, and note that the input token is the only one the publish endpoint accepts. - Warn that registering an output connects to endpoint_url right away, so the destination has to exist first. Also point the composition submodule and update_api.sh at foundry main, now that template workers have landed there. The generated spec is unchanged, confirming the published document already matched main.
The diagram sent the composition's WHIP output straight to a WHEP viewer, which skipped the service in between. Nothing plays back from a composition itself: the output publishes to a destination over WHIP, and viewers watch from that destination over WHEP. Name that destination in the diagram, drop the layout hint next to the composition node, and fix the two places in the surrounding prose that described playing back from the composition. Also list the destination as a prerequisite, since registering the output fails unless it exists.
Updating a scene lived in the concepts page, curl and all, which left the tutorial ending on a layout the reader could only set once and never change. Split it by what each page is for: - The tutorial gains a step that swaps the grid for a single full-frame camera while the composition runs, so the reader sees a scene change, and closes by pointing at templates as the way to stop sending those updates by hand. - Choose inputs and outputs gains an Update a scene section holding the mechanics: the request, the rule that an update must carry the same sides the output was registered with, and scheduled updates. - The concepts page keeps only the idea, that a scene can change over time and that you either drive it yourself or hand it to a template. Also trim the wording of the note about registering an output against an unreachable endpoint, and say what the Scenes link actually leads to.
Every link to smelter.dev landed on the home page, which left the reader to find the reference themselves, and the two pages need different ones. The concepts page describes the JSON scene, so it now points at the HTTP API reference, while the template guide describes the React components and points at the TypeScript SDK reference. Sending someone writing JSON to the TypeScript docs shows them the wrong property names. Both stay at section level rather than deep linking each component, since those paths can move and nothing here would catch it if they did. Also shorten the section on changing a scene over time, which had grown past what a concepts page needs to say.
PiotrWodecki
reviewed
Aug 6, 2026
Without one, markdownlint falls back to its own defaults and reports over 2700 problems across docs/, which drowns anything real. Almost all of them are rules Prettier already governs, so they contradict `yarn format`: it never reflows prose, and it owns whitespace, list, and emphasis style. Turn those off, along with the ones that misread our sources: MDX components are not stray inline HTML, Docusaurus frontmatter carries the title, and explanation pages open with an italic subtitle by convention. Skip generated TypeDoc output and released version snapshots too. What is left is 26 real findings, mostly skipped heading levels and undescriptive link text, rather than noise. Matches how cloud and internal-docs already configure this.
A `globs` key overrides the file passed on the command line, so linting a
single file linted all of docs/ instead. Editors pass the current buffer
and got diagnostics for ten other files back.
Without it, a file argument is honoured and `ignores` still applies. For a
repo-wide run, pass the glob explicitly:
markdownlint-cli2 "docs/**/*.{md,mdx}"
Both rules are enforced but were written down nowhere, so the only way to find them was to guess a resolution and be rejected. Verified against the API: each dimension must be even, and 3840x2160 is the maximum.
Regenerated from foundry's fce-composition-openapi-surface rather than main, so the reference stops publishing the WebSocket and room-link endpoints, which Fishjam calls and users do not, and starts documenting the 422 that eight endpoints return for a body that does not match the schema. That branch is approved but held back from main while a client updates, so the submodule still points at main. Running update-api once it lands regenerates the same file and moves the pointer with it.
The section before it linked to the same two pages the list repeats one line later, so the only thing it carried was the reason templates exist. That reads better as the lead-in to the list than as a section of its own, and the tutorial now ends on cleaning up.
The prerequisites said compositions live on rtc.fishjam.io while rooms and livestreams live on the Fishjam API, but the reference calls them the Composition API and the Fishjam Server API. Same wording now, and both link to the reference so the mapping is one click away.
"GPU-backed rendering session" and "bills for GPU time" pinned the docs to an implementation detail that buys the reader nothing: what they act on is that a composition bills continuously while it exists, not what it runs on. It also committed us to one composition per GPU, which we would have to keep true. The claim about being the most expensive thing in Fishjam went too, since pricing belongs on the pricing page and that sentence ages badly. The idle-cleanup warning was repeated almost verbatim in two how-tos and carried the five minute figure, which also appears in two more files. It is now one shared snippet, so the number has one home.
Outputs were described as pushing, and as where the result is sent, which is accurate but leaves the reader looking for the URL to watch a composition on. There isn't one. The concept page now states that a composition produces video rather than serving it, and the how-to leads with the missing URL and shows where viewers actually connect instead.
Saying the output pushes somewhere else, and that viewers connect to that somewhere else, restated the problem rather than answering it. The two destinations are a Fishjam livestream watched over WHEP and an RTMP service such as YouTube, so the diagram and the prose name them.
The shared cleanup warning repeated the paragraph it sat under in both how-tos, so it is now only the part those paragraphs do not say: nothing will delete this composition for you. Both how-tos created a composition and then used $COMPOSITION without ever capturing it, and two of them used $COMPOSITION_URL and $TOKEN with nothing defining them, so following either from the top produced requests to an empty URL. Both now save the id, and both carry the same environment block the tutorial uses. Also: the room how-to now says where to get the livestream URL and token it asks you to paste, its two links to the Server API agree with each other, the reference states that unknown fields are rejected rather than ignored, and the intro no longer says the composed stream can be published anywhere when the point is that it goes to one destination you choose.
Member
Author
|
Updated with new composition api openapi after fixes and other improvements |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes FCE-2924, FCE-2925 and FCE-3137 by publishing the API specification here.
Documents compositions as a core Fishjam feature, integrated into the existing docs structure instead of a standalone section:
explanation/compositions(what compositions are, core concepts, the scene model).tutorials/compositions(create a composition end to end with curl).how-to/compositions/(write and deploy a template, compose a Fishjam room, drive a template with events, choose inputs and outputs).api/reference): a Compositions section covering REST, the WebSocket event stream, authentication, and errors.