fix(soniox): send per-stream keepalive to prevent 408 timeout mid-utterance#6237
Open
tsushanth wants to merge 1 commit into
Open
fix(soniox): send per-stream keepalive to prevent 408 timeout mid-utterance#6237tsushanth wants to merge 1 commit into
tsushanth wants to merge 1 commit into
Conversation
…erance
In agents 1.6.x the agent drives a single long-lived TTS stream per turn
and feeds text incrementally as LLM tokens arrive. Between token chunks
(and especially during tool calls) the stream sits idle for many seconds.
Soniox enforces a per-stream idle timeout and responds with a 408 error
when it fires, which tears down the stream and crashes the agent's turn.
The existing connection-level keepalive ({"keep_alive": true} without a
stream_id) keeps the WebSocket alive but does not reset the per-stream
idle counter on the server.
This adds a _KeepAliveStream outbound message type sent through the same
_input_queue used for text and cancel messages. The _keepalive_loop now
runs on a 5 s tick and, every 15 s, enqueues a per-stream keepalive
({"stream_id": "...", "keep_alive": true}) for every stream that has
already sent its config but has not yet seen audio_end. 15 s is well
within Soniox's ~30 s idle limit so the timeout no longer fires during
normal inter-token pauses.
Fixes livekit#6225
Member
|
@tsushanth could you run |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #6225
Root cause
In agents 1.6.x the framework drives a single long-lived
SynthesizeStreamper turn and feeds text incrementally as LLM tokens arrive. Between token chunks — and especially during tool calls — the stream can sit idle for many seconds. Soniox enforces a per-stream idle timeout and responds with error code 408 when it fires. That exception propagates up through_main_taskand tears down the agent's TTS turn entirely.The existing connection-level keepalive (
{"keep_alive": true}without astream_id, sent every 10 s) keeps the WebSocket alive between streams but does not reset the per-stream idle timer on the Soniox server — the server tracks each stream independently.Fix
Add a
_KeepAliveStreamoutbound message type that carries astream_id. The_keepalive_loopnow runs on a 5 s tick and, every 15 s, enqueues{"stream_id": "...", "keep_alive": true}for every active stream that has already sent its config but has not yet seenaudio_end. The message is routed through the same_input_queuethe send loop already serialises, so there is no new locking surface.15 s is well within Soniox's ~30 s per-stream idle limit, so the timeout no longer fires during normal inter-token pauses or tool-call gaps.
Changes
livekit-plugins/livekit-plugins-soniox/livekit/plugins/soniox/tts.py_STREAM_KEEPALIVE_INTERVAL = 15constant with explanatory comment_KeepAliveStreamdataclass and include it in_OutboundMsg_KeepAliveStreamin_send_loopby emitting{"stream_id": ..., "keep_alive": true}_keepalive_loopto tick every 5 s; fire the connection-level keepalive on the existing 10 s cadence and the per-stream keepalive on the 15 s cadence