Skip to content

docs(server-events): document assistant.speechStarted message#1028

Merged
dhruva-reddy merged 1 commit intomainfrom
dr/docs-events-assistant-speech-started
Apr 22, 2026
Merged

docs(server-events): document assistant.speechStarted message#1028
dhruva-reddy merged 1 commit intomainfrom
dr/docs-events-assistant-speech-started

Conversation

@dhruva-reddy
Copy link
Copy Markdown
Contributor

@dhruva-reddy dhruva-reddy commented Apr 22, 2026

Description

  • Adds a new ### Assistant Speech Started section to server-url/events.mdx documenting the opt-in assistant.speechStarted server/client message
  • Documents the full payload (text, turn, source, optional timing) and the three timing shapes:
    • word-alignment (ElevenLabs) — per-word timestamps at playback cadence
    • word-progress (Minimax with voice.subtitleType: "word") — cursor-based per-segment progress
    • absent — text-only fallback for all other providers
  • Calls out the real limitations honestly so customers can choose the right provider for their use case:
    • Minimax events fire near the end of each synthesis segment (subtitle data only attaches to the final audio chunk per segment), not per-word in real time
    • force-say events always emit text-only, even on ElevenLabs/Minimax
    • No companion assistant.speechStopped event — use speech-update (status: "stopped") or watch turn increment
    • Barge-in stops emission for the interrupted turn — pair with user-interrupted
    • totalWords: 0 is a valid sentinel; guard against divide-by-zero
  • This is the canonical schema page for the event. Two follow-up PRs in this stack (Minimax provider page, Web SDK live captions section) deep-link into the #assistant-speech-started anchor created here.

Testing Steps

  • Verified MDX renders by inspecting the source — no broken code fences, all <Warning> and table syntax matches existing usage on the page
  • Anchor #assistant-speech-started will be auto-generated by Fern from the ### Assistant Speech Started heading; cross-references in PR 2 and PR 3 use this anchor
  • Reviewer: spot-check the Fern preview build for this PR

Copy link
Copy Markdown
Contributor Author

dhruva-reddy commented Apr 22, 2026

@github-actions
Copy link
Copy Markdown
Contributor

Copy link
Copy Markdown
Contributor Author

image.png

@dhruva-reddy dhruva-reddy merged commit aecebff into main Apr 22, 2026
7 checks passed
Copy link
Copy Markdown
Contributor Author

Merge activity

dhruva-reddy added a commit that referenced this pull request Apr 22, 2026
## Description

- Creates `fern/providers/voice/minimax.mdx` — a new voice provider page for Minimax. Minimax was previously absent from the Voices section in the docs nav.
- Registers the new page in `fern/docs.yml` under the existing `Voices (Text-to-speech)` section, alphabetically positioned after LMNT.
- Documents basic Minimax voice configuration (`provider`, `model`, `voiceId`).
- Documents the new `subtitleType` voice param (`"sentence"` default, `"word"` opt-in) and how it interacts with the [`assistant.speechStarted`](/server-url/events#assistant-speech-started) message.
- Honest limitations section explaining that Minimax word-progress events fire near the _end_ of each synthesis segment (because Minimax only attaches subtitle metadata to the final audio chunk per segment), not per-word in real time. Recommends ElevenLabs for true playback-cadence highlighting.
- Documents the other gotchas: `totalWords: 0` sentinel, `force-say` always emits text-only, barge-in cuts emission, CJK languages are word-counted per ideograph/kana/hangul.
- Stacked on [PR #1028](https://app.graphite.com/github/pr/VapiAI/docs/1028) which adds the `#assistant-speech-started` anchor this page links to.

## Testing Steps

- [x] Verified the page registers in `docs.yml` correctly (matches the structure of sibling voice provider pages like ElevenLabs and Cartesia)
- [x] Verified the cross-reference link `/server-url/events#assistant-speech-started` matches the anchor added in the parent PR
- [x] Reviewer: spot-check the Fern preview build to confirm the page renders and shows up in the Voices nav between LMNT and RimeAI
dhruva-reddy added a commit that referenced this pull request Apr 22, 2026
## Description

- Adds a new `### Live captions and word-level timing` section to `fern/quickstart/web.mdx`, slotted between the Web SDK installation tabs and `### Voice widget implementation` so it's discoverable in the natural client-side flow.
- Originally targeted `fern/sdk/web.mdx`, but that page is redirected away (`/sdk/web → /quickstart/web` per `docs.yml` line 1058) and is effectively orphaned. Moved the content to the canonical destination so customers can actually find it.
- Shows how to opt in via `clientMessages` and consume the event in a `vapi.on("message", ...)` handler with concrete code that branches on `timing?.type`.
- Includes a `<Warning>` callout explaining the cadence/granularity differences between providers so customers don't pick the wrong one for their UI:
    - ElevenLabs (`word-alignment`) — true per-word playback cadence
    - Minimax (`word-progress`) — segment-sized jumps near segment end
    - All others — text-only events tied to audio playback
- Notes that `force-say` always emits text-only and that barge-in stops emission for the interrupted turn.
- Cross-links to the new [Minimax provider page](/providers/voice/minimax) and the canonical [Server events → Assistant Speech Started](/server-url/events#assistant-speech-started) reference.
- Stacked on [PR #1028](https://app.graphite.com/github/pr/VapiAI/docs/1028) (adds the events anchor) and [PR #1029](https://app.graphite.com/github/pr/VapiAI/docs/1029) (adds the Minimax page) — the cross-references in this PR resolve once the stack is merged.

## Testing Steps

- [x] Verified `/sdk/web` redirects to `/quickstart/web` per `docs.yml` redirect rule, confirming the original target page was not user-reachable
- [x] Verified both cross-reference links target paths/anchors created in the downstack PRs
- [x] Verified the new `<Warning>` block uses the same component pattern as existing warnings on this page
- [x] Reviewer: spot-check the Fern preview build to confirm the new section renders correctly between "Installation and setup" and "Voice widget implementation"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants