Skip to content

fix(openai): expose azure_deployment on LLM and fix model property fallback#6221

Open
tsushanth wants to merge 1 commit into
livekit:mainfrom
tsushanth:fix/azure-llm-model-default
Open

fix(openai): expose azure_deployment on LLM and fix model property fallback#6221
tsushanth wants to merge 1 commit into
livekit:mainfrom
tsushanth:fix/azure-llm-model-default

Conversation

@tsushanth

Copy link
Copy Markdown

Closes #6209

When calling LLM.with_azure() without an explicit model argument, the model property on the returned instance returned "unknown" even when azure_deployment was set. This happened because the model fallback in with_azure() previously hard-coded a default and the model property had no awareness of azure_deployment.

This change adds azure_deployment to _LLMOptions so it is tracked independently of model. The with_azure() factory now stores the azure_deployment value in opts rather than using it to silently fill in model. The model property is updated to fall back to azure_deployment when model is None, and a new azure_deployment property is exposed so callers can inspect the value directly.

The LLM.__init__ signature is updated to accept None for model to support the Azure path cleanly without overloading the default.

…llback

When using LLM.with_azure() without an explicit model, the model property
returned "unknown" rather than reflecting the configured azure_deployment.
This adds azure_deployment to _LLMOptions, stores it from with_azure(),
exposes it as a public property, and updates the model property to fall
back to azure_deployment when model is None.

Fixes livekit#6209
@tsushanth tsushanth requested a review from a team as a code owner June 25, 2026 00:42

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 3 potential issues.

Open in Devin Review

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

πŸ”΄ Passing None model to LLMStream causes TypeError crash in Azure path

When LLM.with_azure() is called without specifying model (which now defaults to None), the chat() method at line 1031 passes self._opts.model (which is None) directly to LLMStream. This propagates through the openai plugin's LLMStream.__init__ into the inference LLMStream.__init__ (livekit-agents/livekit/agents/inference/llm.py:378), which calls drop_unsupported_params(model, ...). Inside that function (livekit-agents/livekit/agents/inference/llm.py:85), the expression "/" in model raises TypeError: argument of type 'NoneType' is not iterable because model is None.

The model property (line 186) already handles the None case correctly by falling back to azure_deployment or "unknown", but the chat() method bypasses the property and reads self._opts.model directly.

(Refers to line 1031)

Open in Devin Review

Was this helpful? React with πŸ‘ or πŸ‘Ž to provide feedback.

def with_azure(
*,
model: str | ChatModels = "gpt-4o",
model: str | ChatModels | None = None,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚩 Breaking change: with_azure model default changed from "gpt-4o" to None

The model parameter default in with_azure changed from "gpt-4o" to None (line 199). This is a behavioral change for existing callers who relied on the implicit default model. Previously, LLM.with_azure(azure_deployment="my-deployment") would still send model="gpt-4o" to the API. Now it sends None (or after the bug fix, sends the azure_deployment value). This is likely the intended behavior since Azure users typically specify deployment names rather than model names, but it's a breaking change for users who relied on the old default.

Open in Devin Review

Was this helpful? React with πŸ‘ or πŸ‘Ž to provide feedback.

verbosity=verbosity,
max_completion_tokens=max_completion_tokens,
)
llm._opts.azure_deployment = azure_deployment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚩 Setting azure_deployment after construction bypasses validation

At line 261, azure_deployment is set directly on _opts after the LLM instance is already constructed (llm._opts.azure_deployment = azure_deployment). This means the __init__ method doesn't know about the deployment during construction (it sees azure_deployment=None). This pattern works but is fragile β€” any future logic in __init__ that needs to consider azure_deployment would be bypassed. Additionally, azure_deployment could itself be None if the caller doesn't pass it, making both model and azure_deployment None, resulting in the model property returning "unknown".

Open in Devin Review

Was this helpful? React with πŸ‘ or πŸ‘Ž to provide feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove default for "model" parameter of Azure OpenAI LLM

1 participant