Skip to content

[VoiceAgents][Do not merge] Voice Agents private preview#48249

Open
xitzhang wants to merge 12 commits into
mainfrom
xitzhang/prompt-voice-agent-private-preview
Open

[VoiceAgents][Do not merge] Voice Agents private preview#48249
xitzhang wants to merge 12 commits into
mainfrom
xitzhang/prompt-voice-agent-private-preview

Conversation

@xitzhang

Copy link
Copy Markdown
Member

Description

Please add an informative description that covers that changes made by the pull request and link all relevant issues.

If an SDK is being regenerated based on a new API spec, a link to the pull request containing these API spec changes should be included above.

All SDK Contribution checklist:

  • The pull request does not introduce [breaking changes]
  • CHANGELOG is updated for new features, bug fixes or other significant changes.
  • I have read the contribution guidelines.

General Guidelines and Best Practices

  • Title of the pull request is clear and informative.
  • There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

  • Pull request includes test coverage for the included changes.

Xiting Zhang added 2 commits July 24, 2026 12:37
Initial voice agents client: sync/async operations, models, realtime interface (connect raises NotImplementedError), samples, unit tests, and CI/tests pipelines.
Copilot AI review requested due to automatic review settings July 24, 2026 19:56
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
9 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Introduces the private-preview azure-ai-voiceagents SDK for managing voice agents and persisted conversations.

Changes:

  • Adds generated synchronous/async clients, models, and metadata.
  • Adds an interface-only realtime namespace, samples, and unit tests.
  • Configures packaging and CI pipelines.

Reviewed changes

Copilot reviewed 51 out of 54 changed files in this pull request and generated 12 comments.

Show a summary per file
File Description
sdk/voiceagents/tests.yml Adds weekly analysis pipeline.
sdk/voiceagents/ci.yml Adds package CI configuration.
sdk/voiceagents/azure-ai-voiceagents/tsp-location.yaml Defines TypeSpec source locations.
sdk/voiceagents/azure-ai-voiceagents/tests/unit/test_unit_realtime.py Tests the realtime preview interface.
sdk/voiceagents/azure-ai-voiceagents/tests/unit/test_unit_models.py Tests model serialization.
sdk/voiceagents/azure-ai-voiceagents/tests/unit/test_unit_enums.py Tests enum wire values.
sdk/voiceagents/azure-ai-voiceagents/tests/unit/test_unit_client.py Tests client defaults.
sdk/voiceagents/azure-ai-voiceagents/tests/unit/__init__.py Initializes unit tests.
sdk/voiceagents/azure-ai-voiceagents/tests/conftest.py Documents offline test configuration.
sdk/voiceagents/azure-ai-voiceagents/tests/__init__.py Initializes the test package.
sdk/voiceagents/azure-ai-voiceagents/samples/sample_read_conversation.py Demonstrates reading conversations.
sdk/voiceagents/azure-ai-voiceagents/samples/sample_read_conversation_audio.py Demonstrates audio retrieval.
sdk/voiceagents/azure-ai-voiceagents/samples/sample_manage_voice_agent_versions.py Demonstrates version management.
sdk/voiceagents/azure-ai-voiceagents/samples/sample_generate_voice_agent.py Demonstrates guided agent generation.
sdk/voiceagents/azure-ai-voiceagents/samples/sample_create_voice_agent_with_tools.py Demonstrates tools and audio configuration.
sdk/voiceagents/azure-ai-voiceagents/samples/sample_create_and_manage_voice_agent.py Demonstrates the management lifecycle.
sdk/voiceagents/azure-ai-voiceagents/samples/README.md Documents available samples.
sdk/voiceagents/azure-ai-voiceagents/samples/async_samples/sample_create_and_manage_voice_agent_async.py Adds the async lifecycle sample.
sdk/voiceagents/azure-ai-voiceagents/README.md Adds package documentation.
sdk/voiceagents/azure-ai-voiceagents/pyproject.toml Defines package metadata and dependencies.
sdk/voiceagents/azure-ai-voiceagents/MANIFEST.in Configures source distribution contents.
sdk/voiceagents/azure-ai-voiceagents/LICENSE Adds the MIT license.
sdk/voiceagents/azure-ai-voiceagents/dev_requirements.txt Defines development dependencies.
sdk/voiceagents/azure-ai-voiceagents/CHANGELOG.md Adds initial release history.
sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/types.py Defines generated typed dictionaries.
sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/py.typed Marks the package as typed.
sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/operations/_patch.py Adds operation customization hook.
sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/operations/__init__.py Exports synchronous operations.
sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/models/_patch.py Adds model customization hook.
sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/models/_enums.py Defines generated enums.
sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/models/__init__.py Exports public models.
sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/aio/operations/_patch.py Adds async operation customization hook.
sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/aio/operations/__init__.py Exports asynchronous operations.
sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/aio/_realtime.py Implements the interface-only realtime surface.
sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/aio/_patch.py Adds realtime to the async client.
sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/aio/_configuration.py Configures the async client.
sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/aio/_client.py Implements the generated async client.
sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/aio/__init__.py Exports async APIs.
sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/_version.py Defines package version.
sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/_utils/__init__.py Initializes generated utilities.
sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/_unions.py Defines model unions.
sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/_patch.py Adds package customization hook.
sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/_configuration.py Configures the synchronous client.
sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/_client.py Implements the generated sync client.
sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/__init__.py Exports package APIs and version.
sdk/voiceagents/azure-ai-voiceagents/azure/ai/__init__.py Extends the namespace package.
sdk/voiceagents/azure-ai-voiceagents/azure/__init__.py Extends the Azure namespace.
sdk/voiceagents/azure-ai-voiceagents/apiview-properties.json Maps cross-language API identities.
sdk/voiceagents/azure-ai-voiceagents/_metadata.json Records generated API versions.
Comments suppressed due to low confidence (1)

sdk/voiceagents/azure-ai-voiceagents/samples/sample_create_voice_agent_with_tools.py:105

  • example_toolbox_tool is also never used and triggers pylint's unused-variable check. Mark this illustrative value as intentionally unused.

Comment thread sdk/voiceagents/azure-ai-voiceagents/tsp-location.yaml
Comment thread sdk/voiceagents/azure-ai-voiceagents/samples/sample_read_conversation_audio.py Outdated
Comment thread sdk/voiceagents/azure-ai-voiceagents/README.md Outdated
Comment thread sdk/voiceagents/azure-ai-voiceagents/samples/README.md Outdated
Comment thread sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/aio/_patch.py Outdated
Comment thread sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/aio/_patch.py Outdated
@github-actions

This comment has been minimized.

- Generate api.md + api.metadata.yml (parserVersion 0.3.30)
- Add package-local cspell.json
- Convert sample README links to absolute GitHub URLs
- mypy: Final-typed preview enum in samples; type-ignore aiohttp ws generic
- pylint: drop __init__ override in aio/_patch.py (client-accepts-api-version-keyword)
- README descriptions and sample lifecycle/404 fixes
Copilot AI review requested due to automatic review settings July 24, 2026 21:20

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 53 out of 57 changed files in this pull request and generated 5 comments.

Comments suppressed due to low confidence (5)

sdk/voiceagents/azure-ai-voiceagents/tsp-location.yaml:3

  • The committed regeneration manifest still contains template placeholders. azpysdk generate delegates to tsp-client update, which needs a real repository and pinned spec commit; sibling TypeSpec packages use values such as repo: Azure/azure-rest-api-specs plus a SHA (sdk/ai/azure-ai-projects/tsp-location.yaml:1-3). Pin the exact source before merge so this generated SDK is reproducible.
    sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/aio/_patch.py:20
  • Subclassing the generated client without redeclaring its constructor degrades the published API surface: the generated api.md exposes this constructor as credential: Any and omits the api_version keyword, whereas the generated base client has an AsyncTokenCredential and explicit API-version contract. Add an explicitly typed forwarding __init__, following the customized client pattern in sdk/discovery/azure-ai-discovery/.../aio/_patch.py:30-59, then regenerate the API snapshot.
class VoiceAgentsClient(_GeneratedVoiceAgentsClient):

sdk/voiceagents/azure-ai-voiceagents/README.md:53

  • This HTML placeholder is not valid Python, so the primary README example raises SyntaxError when copied. Replace it with a real voice-agent operation that includes the required preview opt-in.
        <!-- write test code here -->

sdk/voiceagents/azure-ai-voiceagents/README.md:13

  • Correct the misspelled section heading.
#### Prequisites

sdk/voiceagents/azure-ai-voiceagents/README.md:17

  • The client accepts an Azure AI Foundry project endpoint; there is no separate “Voiceagents instance” resource in the setup described by this SDK. State the actual prerequisite and use the product's official capitalization.
- An existing Azure Ai Voiceagents instance.

Comment thread sdk/voiceagents/azure-ai-voiceagents/samples/sample_read_conversation_audio.py Outdated
Comment thread sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/aio/_realtime.py Outdated
Comment thread sdk/voiceagents/azure-ai-voiceagents/README.md Outdated
Comment thread sdk/voiceagents/azure-ai-voiceagents/pyproject.toml Outdated
Comment thread sdk/voiceagents/azure-ai-voiceagents/CHANGELOG.md Outdated
@github-actions

This comment has been minimized.

Xiting Zhang added 2 commits July 24, 2026 17:07
Removing the __init__ override in aio/_patch.py changed the public API surface (credential now typed AsyncTokenCredential). Regenerated api.md + api.metadata.yml with apistub 0.3.30 to fix the API.md consistency check.
…nnotations

- README.md: add Key concepts, Troubleshooting, Next steps sections (Verify Readmes check)
- operations/_operations.py + aio/operations/_operations.py: remove unused urllib.parse import (Pylint W0611)
- aio/_realtime.py: annotate AsyncRealtimeConnection sub-resource attrs and ConversationResource.item to improve verifytypes completeness (98.8% -> 99%)
Copilot AI review requested due to automatic review settings July 25, 2026 00:37
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Xiting Zhang and others added 5 commits July 24, 2026 17:38
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 53 out of 57 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (7)

sdk/voiceagents/azure-ai-voiceagents/tsp-location.yaml:3

  • Replace these placeholders with the exact azure-rest-api-specs repository and commit used for generation. As committed, the package has no reproducible source provenance and the standard regeneration workflow cannot consume this file.
    sdk/voiceagents/azure-ai-voiceagents/samples/README.md:39
  • The source says the MCP and toolbox objects are illustrative and are not attached to the agent, but this table claims the sample creates an agent with all four tool kinds. Clarify that only function and system are attached so the sample index accurately describes its behavior.
    sdk/voiceagents/azure-ai-voiceagents/CHANGELOG.md:3
  • The 1970-01-01 generator placeholder is not a meaningful release date. Set this to the planned initial-preview release date before publishing the package.
## 1.0.0b1 (Unreleased)

sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/aio/_realtime.py:10

  • This says realtime is a standalone client, but the implementation exposes it as VoiceAgentsClient.realtime. Describe the actual entry point to avoid directing users toward a client class that does not exist.
Realtime uses a fundamentally different transport (a persistent WebSocket)
than the request/response HTTP surface, so it is exposed as a **dedicated
standalone client** rather than a member of the HTTP ``VoiceAgentsClient``.

sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/aio/_realtime.py:14

  • VoiceAgentsRealtimeClient is not defined or exported by this package; the method belongs to AsyncRealtime. This Sphinx reference will not resolve to the public API.
* :meth:`AsyncRealtime.connect` returns an async context manager.

sdk/voiceagents/azure-ai-voiceagents/README.md:1

  • Use the product's canonical capitalization and spacing in the package title.
# Azure AI Voice Agents client library for Python

sdk/voiceagents/azure-ai-voiceagents/README.md:68

  • This public README example is not valid Python because it contains an HTML placeholder, and it never closes the client. Replace it with an executable preview operation so users can copy and run the documented example.
>>> client = VoiceAgentsClient(endpoint='<endpoint>', credential=DefaultAzureCredential())
>>> try:
        <!-- write test code here -->
    except HttpResponseError as e:
        print('service responds error: {}'.format(e.response.json()))

Comment thread sdk/voiceagents/azure-ai-voiceagents/README.md Outdated
Copilot AI review requested due to automatic review settings July 25, 2026 00:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 53 out of 57 changed files in this pull request and generated 5 comments.

Comments suppressed due to low confidence (3)

sdk/voiceagents/azure-ai-voiceagents/README.md:13

  • Correct the misspelled section heading so documentation checks and the published README use the standard term.
#### Prequisites

sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/aio/_realtime.py:362

  • This class-level example presents connect() as usable, but the method unconditionally raises NotImplementedError. Since this text is part of the public API documentation, replace the guaranteed-to-fail walkthrough with an explicit interface-only notice until streaming is enabled.
        client = VoiceAgentsClient(endpoint, DefaultAzureCredential())
        async with client.realtime.connect(agent_name="my-agent") as conn:

sdk/voiceagents/azure-ai-voiceagents/README.md:66

  • This fenced Python example contains an HTML comment and incomplete doctest-style indentation, so copying or executing it raises a SyntaxError. Replace the placeholder with a real SDK operation and make the whole block valid Python.
        <!-- write test code here -->

Comment thread sdk/voiceagents/azure-ai-voiceagents/README.md Outdated
Comment thread sdk/voiceagents/azure-ai-voiceagents/azure/ai/voiceagents/aio/_realtime.py Outdated
Comment thread sdk/voiceagents/azure-ai-voiceagents/samples/README.md Outdated
Comment thread sdk/voiceagents/azure-ai-voiceagents/samples/README.md Outdated
- Fix README 'Prequisites' typo and replace non-existent resource prerequisite with an Azure AI Foundry project + endpoint (add azure_ai_foundry link)

- Reword aio/_realtime.py module docstring to describe the interface-only client.realtime namespace

- Reword samples/README realtime NOTE and fix reversed BYOS advice in the 409 troubleshooting row

- Reword sample_create_voice_agent_with_tools docstring for client.realtime.connect(...)

- Add sync and async HTTP operation tests via mock transport covering routes, Foundry-Features header, serialization, paging, audio streaming, and error mapping
Copilot AI review requested due to automatic review settings July 25, 2026 01:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 57 out of 61 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (3)

sdk/voiceagents/azure-ai-voiceagents/tsp-location.yaml:3

  • Replace these placeholders with the actual azure-rest-api-specs repository URL and source commit before merging. azpysdk generate delegates this file to tsp-client update, so the current values make the generated SDK non-reproducible and also record invalid source provenance in package metadata.
    sdk/voiceagents/azure-ai-voiceagents/samples/README.md:40
  • This description says the sample creates an agent with MCP and toolbox tools, but those objects are explicitly illustrative and are not attached to the agent (sample_create_voice_agent_with_tools.py:96-107,133-135). Clarify that only function and system tools are configured so users do not expect the created resource to contain all four kinds.
    sdk/voiceagents/azure-ai-voiceagents/README.md:66
  • This placeholder is not Python, so the package's primary example cannot be copied or validated, and it also omits the preview opt-in required by every operation. Replace it with a real request that passes VOICE_AGENTS_V1_PREVIEW.
        <!-- write test code here -->

@github-actions

Copy link
Copy Markdown
Contributor
[Pilot] PR Pipeline Failure Analysis

A CI pipeline failed on this pull request. Here is an automated analysis of what went wrong and how to get the build green.

What failed

The CSpell (spell-check) validation step failed in the pipeline for the azure-ai-voiceagents package. 8 spelling errors were detected across 3 files:

  • sdk/voiceagents/azure-ai-voiceagents/tests/unit/_mock_transport.py — the word hdrs (lines 41, 44, 49, 57, 58) is flagged as unknown.
  • sdk/voiceagents/azure-ai-voiceagents/tests/unit/test_unit_operations.py — the word Efmt (lines 218, 240) is flagged as unknown.
  • sdk/voiceagents/azure-ai-voiceagents/tests/unit/test_unit_operations_async.py — the word Efmt (line 206) is flagged as unknown.

Azure DevOps build

Recommended next steps

  • Fix hdrs: This is likely an abbreviation for "headers". Either rename the variable/attribute to headers, or add hdrs to the CSpell allow-list for the package (.vscode/cspell.json or the repo-level word list). See spell-check docs.
  • Fix Efmt: Review the usage at test_unit_operations.py:218,240 and test_unit_operations_async.py:206. If this is a valid identifier (e.g. an enum value or format string), add it to the CSpell allow-list; otherwise rename it to a recognizable word.
  • See the CI troubleshooting guide: https://aka.ms/ci-fix
  • Push new commits to address the failures; this comment updates automatically on the next failing run.
Raw pipeline analysis (azsdk ci analyze)
Failed Tasks
### Errors:
--> sdk/voiceagents/azure-ai-voiceagents/tests/unit/_mock_transport.py:41:5 - Unknown word (hdrs)
--> sdk/voiceagents/azure-ai-voiceagents/tests/unit/_mock_transport.py:44:9 - Unknown word (hdrs)
--> sdk/voiceagents/azure-ai-voiceagents/tests/unit/_mock_transport.py:49:9 - Unknown word (hdrs)
--> sdk/voiceagents/azure-ai-voiceagents/tests/unit/_mock_transport.py:57:22 - Unknown word (hdrs)
--> sdk/voiceagents/azure-ai-voiceagents/tests/unit/_mock_transport.py:58:17 - Unknown word (hdrs)
--> sdk/voiceagents/azure-ai-voiceagents/tests/unit/test_unit_operations.py:218:28 - Unknown word (Efmt)
--> sdk/voiceagents/azure-ai-voiceagents/tests/unit/test_unit_operations.py:240:28 - Unknown word (Efmt)
--> sdk/voiceagents/azure-ai-voiceagents/tests/unit/test_unit_operations_async.py:206:32 - Unknown word (Efmt)

CSpell: Files checked: 58, Issues found: 8 in 3 files.
Spelling errors detected. To correct false positives or learn about spell checking see: https://aka.ms/azsdk/engsys/spellcheck

Pipeline: https://dev.azure.com/azure-sdk/public/_build/results?buildId=6616178

Copilot detected the failing pipeline and generated the analysis above. To have it attempt a fix automatically, reply with @copilot please fix the failing pipeline on this PR.

Generated by Pipeline Analysis - Next Steps · 20.7 AIC · ⌖ 6.24 AIC · ⊞ 6.6K ·

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants