Skip to content

feat(assemblyai): add language code streaming option #1908

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
rosetta-livekit-bot wants to merge 1 commit into
base: main
Choose a base branch
from
+13 −0
Open

feat(assemblyai): add language code streaming option #1908

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/assemblyai-language-code.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@livekit/agents-plugin-assemblyai': patch
---

Add AssemblyAI streaming language steering via the `languageCode` option.
8 changes: 8 additions & 0 deletions plugins/assemblyai/src/stt.ts
View file
Open in desktop

@devin-ai-integration devin-ai-integration Bot Jun 29, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Language code normalization is skipped when options are updated after construction

The language code is stored without normalization ({ ...this.#opts, ...opts } at plugins/assemblyai/src/stt.ts:189) when updated after initial setup, so a value like 'en-US' is sent raw to AssemblyAI instead of being reduced to 'en'.

Impact: Future streaming connections after an option update may send an un-normalized language code to AssemblyAI, potentially causing rejected or misinterpreted language steering.

Constructor normalizes but updateOptions does not

In the constructor at plugins/assemblyai/src/stt.ts:171-172, languageCode is normalized via getBaseLanguage() and then explicitly set after the spread at line 179, ensuring the normalized value wins. However, STT.updateOptions at line 188-189 simply spreads the raw opts over this.#opts without applying getBaseLanguage(). This means:

  1. User calls stt.updateOptions({ languageCode: 'en-US' })
  2. this.#opts.languageCode is set to 'en-US' (raw, un-normalized)
  3. Next stt.stream() call creates a SpeechStream with these opts
  4. #connectWS() at line 335 sends language_code: 'en-US' to AssemblyAI instead of 'en'

The fix should apply getBaseLanguage() to opts.languageCode in updateOptions before merging, consistent with the constructor logic.

(Refers to lines 188-189)

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import {
Task,
createTimedString,
delay,
getBaseLanguage,
log,
normalizeLanguage,
stt,
Expand Down Expand Up @@ -66,6 +67,8 @@ export interface STTOptions {
encoding: STTEncoding;
speechModel: STTModels;
languageDetection?: boolean;
/** Only supported with the Universal-3 Pro model family. Set at connection time only. */
languageCode?: string;
endOfTurnConfidenceThreshold?: number;
/** Minimum silence (ms) before a confident end-of-turn is finalized. */
minTurnSilence?: number;
Expand Down Expand Up @@ -146,6 +149,7 @@ export class STT extends stt.STT {
'voiceFocus',
'voiceFocusThreshold',
'mode',
'languageCode',
] as const) {
if (opts[param] !== undefined) {
throw new Error(
Expand All @@ -164,12 +168,15 @@ export class STT extends stt.STT {

// Minimize latency by default, but let AssemblyAI's mode preset control silence tuning.
const minTurnSilence = opts.minTurnSilence ?? (opts.mode === undefined ? 100 : undefined);
const languageCode =
opts.languageCode !== undefined ? getBaseLanguage(opts.languageCode) : undefined;
Comment on lines +171 to +172

@devin-ai-integration devin-ai-integration Bot Jun 29, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚩 getBaseLanguage strips region subtags, which may be intentional for AssemblyAI but lossy

The constructor normalizes languageCode via getBaseLanguage() at plugins/assemblyai/src/stt.ts:171-172, which strips region subtags (e.g. 'en-US' → 'en', 'pt-BR' → 'pt'). This is consistent with how other plugins (ElevenLabs, Cartesia) use getBaseLanguage. However, if AssemblyAI's API supports or benefits from region-level language codes (e.g. distinguishing pt-BR from pt-PT), this normalization would silently drop useful information. Worth confirming with AssemblyAI's API docs whether region subtags are supported for the language_code parameter.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.


this.#opts = {
...defaultSTTOptions,
...opts,
apiKey,
minTurnSilence,
languageCode,
};
}

Expand Down Expand Up @@ -325,6 +332,7 @@ export class SpeechStream extends stt.SpeechStream {
? JSON.stringify(this.#opts.keytermsPrompt)
: undefined,
language_detection: languageDetection,
language_code: this.#opts.languageCode,
prompt: this.#opts.prompt,
agent_context: this.#opts.agentContext,
previous_context_n_turns: this.#opts.previousContextNTurns,
Expand Down
Loading