Restore polling in the sample in LRO operations by dargilco · Pull Request #48343 · Azure/azure-sdk-for-python

dargilco · 2026-07-29T15:41:13Z

The samples associated with version 2.3.0 for agent optimization, dataset generation and rubric evaluator all showed how an application can do polling during a long running operation. In version 2.4.0 we switched to using standard Azure SDK LRO operations, and as part of sample update to account for SDK API changes, I made a decision to switch all those samples to use SDK's built-in polling (such that what you see in the sample is just the .result() blocking call). I did not give enough though to this, just wanted to unblock an urgent release. Since then I changed my mind about that. I think it's more useful for the samples to show how to do polling outside the SDK (while still adding a comment about the easy option to let the SDK do the polling). Also note that existing learn.microsoft docs show how the application can do polling. And I believe production call will not make long blocking calls... it will use application polling.
As part of the 2.4.0 release, for the Agent Optimization case only, I made sure we have both sample-polling and SDK-polling for the Agent Optimization case (see sample_optimization_job_basic.py and sample_optimization_job_basic_polling.py and their equivalent async version). If we want to explicitly show samples of built-in SDK polling for dataset generation and rubric evaluators, we can add those samples.
But for now I would like to return to showing sample polling as was before.

Also add async sample: samples\datasets\sample_dataset_generation_job_simpleqna_for_finetuning_async.py

azure-pipelines · 2026-07-29T15:41:47Z

Azure Pipelines: Successfully started running 1 pipeline(s). 9 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run.

Copilot

Pull request overview

Updates Azure AI Projects samples to expose standard LRO polling progress and capture service error details.

Changes:

Reports polling and terminal statuses.
Captures LRO responses for failure messages.
Converts optimization polling samples to standard pollers.

Reviewed changes

Copilot reviewed 14 out of 14 changed files in this pull request and generated 15 comments.

Show a summary per file

File	Description
`samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py`	Adds polling for lifecycle and replay jobs.
`samples/evaluations/sample_rubric_evaluator_generation_iterate.py`	Adds polling for iterative generation.
`samples/evaluations/sample_rubric_evaluator_generation_basic.py`	Adds polling for basic generation.
`samples/evaluations/sample_rubric_evaluator_generation_all_sources.py`	Adds polling for multi-source and trace jobs.
`samples/datasets/sample_dataset_generation_job_traces_for_finetuning.py`	Adds polling for fine-tuning trace generation.
`samples/datasets/sample_dataset_generation_job_traces_for_evaluation.py`	Adds polling for evaluation trace generation.
`samples/datasets/sample_dataset_generation_job_simpleqna_with_prompt_source.py`	Adds polling for prompt-based generation.
`samples/datasets/sample_dataset_generation_job_simpleqna_with_file_source.py`	Adds polling for file-based generation.
`samples/datasets/sample_dataset_generation_job_simpleqna_with_agent_source.py`	Adds polling for agent-based generation.
`samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning.py`	Adds polling for fine-tuning data generation.
`samples/agents/optimization/sample_optimization_job_basic.py`	Updates built-in polling documentation.
`samples/agents/optimization/sample_optimization_job_basic_polling.py`	Replaces manual polling with an LRO poller.
`samples/agents/optimization/sample_optimization_job_basic_polling_async.py`	Replaces async manual polling with an LRO poller.
`samples/agents/optimization/sample_optimization_job_basic_async.py`	Updates async polling documentation.

Comments suppressed due to low confidence (4)

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:159

When background polling fails, done() only indicates that the poller thread stopped; it does not guarantee a terminal service response. Since this branch never calls result(), it hides the poller's HTTP/transport exception and may claim the replay ended in a stale nonterminal status. Observe and chain that exception before raising the custom message.

    if replay_status.lower() != "succeeded":
        error = latest_replay_lro_response.get("error")
        message = error.get("message", "<no error message>") if isinstance(error, dict) else "<no error message>"
        raise RuntimeError(f"Replay job ended with status `{replay_status}`: {message}")

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_all_sources.py:228

done() becomes true if the synchronous poller thread exits on an HTTP/transport exception as well as on normal completion. Because this branch never calls result(), it loses that exception and may misleadingly report a stale nonterminal status. Observe and chain the poller exception before raising the custom message.

            if status.lower() != "succeeded":
                error = latest_lro_response.get("error")

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic_polling.py:117

This conversion drops the sample's previous reporting of OptimizationJob.warnings. The LRO returns only OptimizationJobResult, which has no warnings field, so non-fatal service advisories are silently discarded even though the raw terminal job response is already captured. Preserve and print its warnings before moving on to the result.

    result = poller.result()

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic_polling_async.py:124

This conversion drops the sample's previous reporting of OptimizationJob.warnings. The LRO returns only OptimizationJobResult, which has no warnings field, so non-fatal service advisories are silently discarded even though the raw terminal job response is already captured. Preserve and print its warnings before moving on to the result.

        result = await result_task

Copilot

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (16)

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic_polling_async.py:107

If the polling coroutine raises (for example, when the service reports Failed or Canceled), AsyncLROPoller.wait() exits before setting its internal _done flag. The task is then complete with an exception, but this loop continues forever and never reaches the code that awaits and wraps that exception. Poll the task's completion instead.

        while not poller.done():

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:153

This replay loop also only observes a sync LROPoller whose Azure Core background thread is doing the actual polling, so it does not restore application-managed polling as described. Disable SDK polling and explicitly fetch the replay job status instead.

    while not replay_poller.done():
        print(f"Replay job status: {replay_poller.status()}")
        time.sleep(poll_interval_seconds)

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_all_sources.py:224

This traces loop likewise only watches a sync LROPoller while its Azure Core background thread performs the requests. It does not demonstrate polling outside the SDK as described. Disable SDK polling and explicitly fetch the traces job status.

            while not poller.done():
                print(f"Traces job status: {poller.status()}")
                time.sleep(poll_interval_seconds)

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:14

The continuation text is over-indented relative to the bullet, so the rendered sample description no longer aligns with the other list entries.

                returns `LROPoller[EvaluatorVersion]`, whose status is reported until
                the job reaches a terminal state.

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:137

This loop only observes the sync LROPoller; constructing that poller has already started Azure Core's background polling thread. It therefore does not demonstrate the application-managed polling described in the PR. Disable SDK polling and explicitly fetch the generation job status if this sample is meant to restore polling outside the SDK.

This issue also appears on line 151 of the same file.

    while not poller.done():
        print(f"Generation job status: {poller.status()}")
        time.sleep(poll_interval_seconds)

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_iterate.py:110

A sync LROPoller begins polling on an Azure Core background thread as soon as it is constructed; this loop merely reports that thread's status. Consequently this sample still uses SDK polling rather than the application-managed polling promised by the PR description. Disable SDK polling and explicitly retrieve the job status.

    while not poller.done():
        print(f"Generation job status: {poller.status()}")
        time.sleep(poll_interval_seconds)

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_basic.py:134

A sync LROPoller starts Azure Core's polling thread during construction, so this loop only observes SDK polling; it does not perform polling outside the SDK as the PR description says. Disable SDK polling and explicitly retrieve the generation job status for an application-polling example.

    while not poller.done():
        print(f"Generation job status: {poller.status()}")
        time.sleep(poll_interval_seconds)

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_all_sources.py:156

This loop only monitors a sync LROPoller; Azure Core starts a background polling thread when that poller is constructed. The sample therefore still relies on SDK polling instead of demonstrating the application-managed polling stated in the PR description. Disable SDK polling and fetch the multi-source job status explicitly.

This issue also appears on line 222 of the same file.

        while not poller.done():
            print(f"Multi-source job status: {poller.status()}")
            time.sleep(poll_interval_seconds)

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_traces_for_finetuning.py:177

Constructing this sync LROPoller already starts Azure Core's polling thread, so this loop merely reports SDK-managed polling. That contradicts the PR's stated goal of restoring polling outside the SDK. Disable SDK polling and explicitly fetch the data-generation job status.

                while not poller.done():
                    print(f"Data generation job status: {poller.status()}")
                    time.sleep(POLL_INTERVAL_SECONDS)

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_traces_for_evaluation.py:174

Constructing this sync LROPoller already starts Azure Core's polling thread, so this loop merely reports SDK-managed polling. That contradicts the PR's stated goal of restoring polling outside the SDK. Disable SDK polling and explicitly fetch the data-generation job status.

                while not poller.done():
                    print(f"Data generation job status: {poller.status()}")
                    time.sleep(POLL_INTERVAL_SECONDS)

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_prompt_source.py:138

This loop does not itself poll the service: the sync LROPoller started an Azure Core background polling thread when it was created. The sample therefore still demonstrates SDK polling, contrary to the PR description's application-polling goal. Disable SDK polling and explicitly retrieve the job status.

        while not poller.done():
            print(f"Data generation job status: {poller.status()}")
            time.sleep(poll_interval_seconds)

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_file_source.py:200

This loop does not itself poll the service: the sync LROPoller started an Azure Core background polling thread when it was created. The sample therefore still demonstrates SDK polling, contrary to the PR description's application-polling goal. Disable SDK polling and explicitly retrieve the job status.

    while not poller.done():
        print(f"Data generation job status: {poller.status()}")
        time.sleep(poll_interval_seconds)

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_agent_source.py:180

This loop only observes the sync LROPoller; Azure Core is already polling in its background thread. As written, the sample does not restore application-managed polling as claimed in the PR description. Disable SDK polling and explicitly retrieve the data-generation job status.

        while not poller.done():
            print(f"Data generation job status: {poller.status()}")
            time.sleep(poll_interval_seconds)

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning.py:189

The sync LROPoller starts Azure Core's background polling thread during construction, so this loop only reports SDK polling. It does not implement the application-managed polling described by the PR. Disable SDK polling and explicitly fetch the fine-tuning generation job status.

    while not poller.done():
        print(f"Data generation job status: {poller.status()}")
        time.sleep(poll_interval_seconds)

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic_polling.py:106

This loop only monitors the sync LROPoller; Azure Core has already started polling on a background thread. The change therefore replaces the prior application-managed job polling with SDK polling, opposite to the PR description. Keep polling disabled and explicitly call get_optimization_job until terminal.

    while not poller.done():
        print(f"Optimization job status: {poller.status()}")
        time.sleep(poll_interval)

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic_polling_async.py:109

Creating a task from poller.result() explicitly starts Azure Core's async polling loop; the surrounding loop only observes it. This changes the previous application-managed optimization-job polling into SDK-managed polling, contrary to the PR description. Keep SDK polling disabled and explicitly retrieve the job status.

        result_task = asyncio.create_task(poller.result())
        while not poller.done():
            print(f"Optimization job status: {poller.status()}")
            await asyncio.sleep(poll_interval)

Copilot

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (16)

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic_polling_async.py:109

AsyncLROPoller.done() is set only after wait() returns successfully. If the LRO fails, result_task completes with an exception but poller.done() remains false, so this loop sleeps forever and the failure handling below is never reached. Use the task's completion state as the loop condition.

        result_task = asyncio.create_task(poller.result())
        while not poller.done():
            print(f"Optimization job status: {poller.status()}")
            await asyncio.sleep(poll_interval)

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:14

The added indentation makes these continuation lines render far to the right of the surrounding bullet text instead of aligning with it.

                returns `LROPoller[EvaluatorVersion]`, whose status is reported until
                the job reaches a terminal state.

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:153

This replay loop also only observes the synchronous LROPoller's background SDK polling; calling done() does not perform an application poll. It therefore does not demonstrate the application-managed polling promised by the PR.

    while not replay_poller.done():
        print(f"Replay job status: {replay_poller.status()}")
        time.sleep(poll_interval_seconds)

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_all_sources.py:224

This second synchronous loop likewise only observes the LROPoller's background SDK polling. It does not issue application-level status requests and therefore does not demonstrate the polling mode described in the PR.

            while not poller.done():
                print(f"Traces job status: {poller.status()}")
                time.sleep(poll_interval_seconds)

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:137

This does not implement the application-managed polling described by the PR. A synchronous LROPoller starts its SDK polling thread during construction (azure-core/.../_poller.py:228-240), so done() and status() only observe SDK-managed polling. Use polling=False plus explicit get_generation_job calls between sleeps, or revise the sample's stated purpose.

This issue also appears on line 151 of the same file.

    while not poller.done():
        print(f"Generation job status: {poller.status()}")
        time.sleep(poll_interval_seconds)

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_iterate.py:110

A synchronous LROPoller begins SDK polling in a background thread as soon as it is created (azure-core/.../_poller.py:228-240). This loop only observes that thread, so it does not restore the application-managed polling described by the PR; use polling=False and explicit status retrieval between sleeps.

    while not poller.done():
        print(f"Generation job status: {poller.status()}")
        time.sleep(poll_interval_seconds)

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_basic.py:134

A synchronous LROPoller begins SDK polling in a background thread as soon as it is created (azure-core/.../_poller.py:228-240). This loop only observes that thread, so it does not restore the application-managed polling described by the PR; use polling=False and explicit status retrieval between sleeps.

    while not poller.done():
        print(f"Generation job status: {poller.status()}")
        time.sleep(poll_interval_seconds)

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_all_sources.py:156

A synchronous LROPoller begins SDK polling in a background thread as soon as it is created (azure-core/.../_poller.py:228-240). This loop only observes that thread, so it does not restore the application-managed polling described by the PR; use polling=False and explicit status retrieval between sleeps.

This issue also appears on line 222 of the same file.

        while not poller.done():
            print(f"Multi-source job status: {poller.status()}")
            time.sleep(poll_interval_seconds)

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_traces_for_finetuning.py:177

A synchronous LROPoller begins SDK polling in a background thread as soon as it is created (azure-core/.../_poller.py:228-240). This loop only observes that thread, so it does not restore the application-managed polling described by the PR; use polling=False and explicit status retrieval between sleeps.

                while not poller.done():
                    print(f"Data generation job status: {poller.status()}")
                    time.sleep(POLL_INTERVAL_SECONDS)

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_traces_for_evaluation.py:174

A synchronous LROPoller begins SDK polling in a background thread as soon as it is created (azure-core/.../_poller.py:228-240). This loop only observes that thread, so it does not restore the application-managed polling described by the PR; use polling=False and explicit status retrieval between sleeps.

                while not poller.done():
                    print(f"Data generation job status: {poller.status()}")
                    time.sleep(POLL_INTERVAL_SECONDS)

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_prompt_source.py:138

A synchronous LROPoller begins SDK polling in a background thread as soon as it is created (azure-core/.../_poller.py:228-240). This loop only observes that thread, so it does not restore the application-managed polling described by the PR; use polling=False and explicit status retrieval between sleeps.

        while not poller.done():
            print(f"Data generation job status: {poller.status()}")
            time.sleep(poll_interval_seconds)

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_file_source.py:200

A synchronous LROPoller begins SDK polling in a background thread as soon as it is created (azure-core/.../_poller.py:228-240). This loop only observes that thread, so it does not restore the application-managed polling described by the PR; use polling=False and explicit status retrieval between sleeps.

    while not poller.done():
        print(f"Data generation job status: {poller.status()}")
        time.sleep(poll_interval_seconds)

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_agent_source.py:180

A synchronous LROPoller begins SDK polling in a background thread as soon as it is created (azure-core/.../_poller.py:228-240). This loop only observes that thread, so it does not restore the application-managed polling described by the PR; use polling=False and explicit status retrieval between sleeps.

        while not poller.done():
            print(f"Data generation job status: {poller.status()}")
            time.sleep(poll_interval_seconds)

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning.py:189

A synchronous LROPoller begins SDK polling in a background thread as soon as it is created (azure-core/.../_poller.py:228-240). This loop only observes that thread, so it does not restore the application-managed polling described by the PR; use polling=False and explicit status retrieval between sleeps.

    while not poller.done():
        print(f"Data generation job status: {poller.status()}")
        time.sleep(poll_interval_seconds)

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic_polling.py:106

This replaces the prior polling=False plus get_optimization_job application polling with the default synchronous LROPoller, which starts SDK polling in a background thread during construction. The loop therefore only observes SDK polling and makes this sample another built-in-polling example, contrary to the PR's stated separation between the basic and polling samples.

    while not poller.done():
        print(f"Optimization job status: {poller.status()}")
        time.sleep(poll_interval)

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic_polling_async.py:109

Creating a task for poller.result() starts the async SDK polling loop (azure-core/.../_async_poller.py:230-258); the surrounding loop merely reports its status. This converts the polling sample into another SDK-managed-polling example instead of preserving the application-managed polling that the PR description says should remain distinct.

        result_task = asyncio.create_task(poller.result())
        while not poller.done():
            print(f"Optimization job status: {poller.status()}")
            await asyncio.sleep(poll_interval)

Copilot

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (28)

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:130

The replay call also leaves polling=True, so this second synchronous LROPoller starts another SDK polling thread immediately. The subsequent done()/status() loop observes SDK polling rather than performing the application-controlled polling described by the PR.

    replay_poller = project_client.beta.evaluators.begin_create_generation_job(
        job=job_body,
        operation_id=operation_id,
        polling_interval=poll_interval_seconds,
        raw_response_hook=capture_replay_lro_response,

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_all_sources.py:197

The traces path likewise leaves SDK polling enabled. The synchronous LROPoller starts polling as soon as it is constructed, so this path only reports SDK-managed progress instead of demonstrating application-controlled polling.

            poller = project_client.beta.evaluators.begin_create_generation_job(

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:143

done() also becomes true when the background polling thread exits with an exception (azure/core/polling/_poller.py:242-261). If a polling request fails while the status is still InProgress, this branch masks the stored SDK exception as “ended with status InProgress: ”; only result()/wait() propagates the original exception. Surface the poller exception before interpreting a non-success status.

    if status.lower() != "succeeded":
        error = latest_lro_response.get("error")
        message = error.get("message", "<no error message>") if isinstance(error, dict) else "<no error message>"
        raise RuntimeError(f"Generation job ended with status `{status}`: {message}")

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_iterate.py:116

done() is also true when the polling thread exits with an exception. If a request fails while status remains InProgress, this branch masks the stored SDK exception with “ended with status InProgress”; result() is the call that would propagate the real failure.

    if status.lower() != "succeeded":
        error = latest_lro_response.get("error")
        message = error.get("message", "<no error message>") if isinstance(error, dict) else "<no error message>"
        raise RuntimeError(f"Generation job ended with status `{status}`: {message}")

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:159

The replay loop has the same exception path: done() is true after the polling thread fails, while status() may still be InProgress. Raising this custom error before calling result() masks the underlying SDK polling exception and reports a false terminal status.

    if replay_status.lower() != "succeeded":
        error = latest_replay_lro_response.get("error")
        message = error.get("message", "<no error message>") if isinstance(error, dict) else "<no error message>"
        raise RuntimeError(f"Replay job ended with status `{replay_status}`: {message}")

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_basic.py:140

done() is also true when the polling thread exits with an exception. If a request fails while status remains InProgress, this branch masks the stored SDK exception with “ended with status InProgress”; result() is the call that would propagate the real failure.

    if status.lower() != "succeeded":
        error = latest_lro_response.get("error")
        message = error.get("message", "<no error message>") if isinstance(error, dict) else "<no error message>"
        raise RuntimeError(f"Generation job ended with status `{status}`: {message}")

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_all_sources.py:162

A failed polling thread also makes done() true. When a request exception leaves status as InProgress, this branch replaces the stored SDK exception with a misleading terminal-status RuntimeError; calling result() is what propagates the actual polling failure.

        if status.lower() != "succeeded":
            error = latest_lro_response.get("error")
            message = error.get("message", "<no error message>") if isinstance(error, dict) else "<no error message>"
            raise RuntimeError(f"Multi-source job ended with status `{status}`: {message}")

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_prompt_source.py:144

done() is also true after the polling thread fails. If the failure occurs while status remains InProgress, this branch masks the stored SDK exception with a false terminal-status error; result() would otherwise propagate the actual polling failure.

        if status.lower() != "succeeded":
            error = latest_lro_response.get("error")
            message = error.get("message", "<no error message>") if isinstance(error, dict) else "<no error message>"
            raise RuntimeError(f"Data generation job ended with status `{status}`: {message}")

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_file_source.py:206

done() is also true after the polling thread fails. If the failure occurs while status remains InProgress, this branch masks the stored SDK exception with a false terminal-status error; result() would otherwise propagate the actual polling failure.

    if status.lower() != "succeeded":
        error = latest_lro_response.get("error")
        message = error.get("message", "<no error message>") if isinstance(error, dict) else "<no error message>"
        raise RuntimeError(f"Data generation job ended with status `{status}`: {message}")

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_agent_source.py:186

done() is also true after the polling thread fails. If the failure occurs while status remains InProgress, this branch masks the stored SDK exception with a false terminal-status error; result() would otherwise propagate the actual polling failure.

        if status.lower() != "succeeded":
            error = latest_lro_response.get("error")
            message = error.get("message", "<no error message>") if isinstance(error, dict) else "<no error message>"
            raise RuntimeError(f"Data generation job ended with status `{status}`: {message}")

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning.py:195

done() is also true after the polling thread fails. If the failure occurs while status remains InProgress, this branch masks the stored SDK exception with a false terminal-status error; result() would otherwise propagate the actual polling failure.

    if status.lower() != "succeeded":
        error = latest_lro_response.get("error")
        message = error.get("message", "<no error message>") if isinstance(error, dict) else "<no error message>"
        raise RuntimeError(f"Data generation job ended with status `{status}`: {message}")

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:14

These continuation lines are over-indented relative to the surrounding bullet list, so the module description renders as a malformed list item.

                returns `LROPoller[EvaluatorVersion]`, whose status is reported until
                the job reaches a terminal state.

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_all_sources.py:228

A failed polling thread also makes done() true. When a request exception leaves status as InProgress, this branch replaces the stored SDK exception with a misleading terminal-status RuntimeError; calling result() is what propagates the actual polling failure.

            if status.lower() != "succeeded":
                error = latest_lro_response.get("error")

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_traces_for_finetuning.py:181

done() becomes true when polling exits with an exception as well as on service completion. If a polling request fails while status is still InProgress, this branch masks the SDK exception as a misleading terminal-status error instead of letting result() propagate the real cause.

                if status.lower() != "succeeded":
                    error = latest_lro_response.get("error")

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_traces_for_evaluation.py:178

done() becomes true when polling exits with an exception as well as on service completion. If a polling request fails while status is still InProgress, this branch masks the SDK exception as a misleading terminal-status error instead of letting result() propagate the real cause.

                if status.lower() != "succeeded":
                    error = latest_lro_response.get("error")

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:111

This still uses SDK-managed polling: the default polling=True constructs a synchronous LROPoller, which starts its polling thread immediately (sdk/core/azure-core/azure/core/polling/_poller.py:225-240). done() and status() only observe that background poller, so this does not restore the application-controlled polling described by the PR. Disable SDK polling and fetch job status explicitly, as the optimization polling sample does, or revise the sample's stated purpose.

This issue also appears on line 126 of the same file.

    poller = project_client.beta.evaluators.begin_create_generation_job(
        job=job_body,
        operation_id=operation_id,
        polling_interval=poll_interval_seconds,
        raw_response_hook=capture_lro_response,

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_iterate.py:84

This still uses SDK-managed polling: the default synchronous LROPoller starts polling in a background thread at construction (sdk/core/azure-core/azure/core/polling/_poller.py:225-240). The new loop only reports that poller's state, rather than restoring the application-controlled polling described by the PR.

    poller = project_client.beta.evaluators.begin_create_generation_job(

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_basic.py:101

This still uses SDK-managed polling: the default synchronous LROPoller starts polling in a background thread at construction (sdk/core/azure-core/azure/core/polling/_poller.py:225-240). The new loop only reports that poller's state, rather than restoring the application-controlled polling described by the PR.

    poller = project_client.beta.evaluators.begin_create_generation_job(

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_all_sources.py:140

This call leaves SDK polling enabled. A synchronous LROPoller starts its polling thread immediately (sdk/core/azure-core/azure/core/polling/_poller.py:225-240), so the added done()/status() loop merely observes SDK polling and does not demonstrate the application-controlled polling described by the PR.

This issue also appears on line 197 of the same file.

        poller = project_client.beta.evaluators.begin_create_generation_job(

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_traces_for_finetuning.py:152

This call still defaults to SDK polling. The synchronous LROPoller starts a background polling thread on construction (sdk/core/azure-core/azure/core/polling/_poller.py:225-240), so the new loop observes SDK polling rather than performing the application-controlled polling described by the PR.

                poller = project_client.beta.datasets.begin_create_generation_job(

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_traces_for_evaluation.py:151

This call still defaults to SDK polling. The synchronous LROPoller starts a background polling thread on construction (sdk/core/azure-core/azure/core/polling/_poller.py:225-240), so the new loop observes SDK polling rather than performing the application-controlled polling described by the PR.

                poller = project_client.beta.datasets.begin_create_generation_job(

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_prompt_source.py:131

This call still defaults to SDK polling. A synchronous LROPoller begins polling in its background thread immediately, and done()/status() only observe it. Therefore this does not demonstrate the application-controlled polling described by the PR.

        poller = project_client.beta.datasets.begin_create_generation_job(

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_file_source.py:167

This call still defaults to SDK polling. A synchronous LROPoller begins polling in its background thread immediately, and done()/status() only observe it. Therefore this does not demonstrate the application-controlled polling described by the PR.

    poller = project_client.beta.datasets.begin_create_generation_job(

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_agent_source.py:173

This call still defaults to SDK polling. A synchronous LROPoller begins polling in its background thread immediately, and done()/status() only observe it. Therefore this does not demonstrate the application-controlled polling described by the PR.

        poller = project_client.beta.datasets.begin_create_generation_job(

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning.py:182

This call still defaults to SDK polling. A synchronous LROPoller begins polling in its background thread immediately, and done()/status() only observe it. Therefore this does not demonstrate the application-controlled polling described by the PR.

    poller = project_client.beta.datasets.begin_create_generation_job(

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic_polling.py:10

This description says the sample polls the standard LRO, but the implementation passes polling=False and repeatedly calls get_optimization_job; it polls the job resource outside the SDK poller. Keep the description aligned with the behavior.

    optimization job and poll its standard LRO to completion.

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic_polling.py:78

response is a PipelineResponse, not an OptimizationJob, as shown by the immediate response.http_response.json() access. The comment gives sample readers the wrong raw-response-hook callback contract.

        # Since `polling=False` is set below, it is guaranteed that `capture_created_job` will be
        # invoked once on the initial "201 Created" response, and `response` is of type `OptimizationJob`.

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic_polling_async.py:81

response is a PipelineResponse, not an OptimizationJob, as shown by the immediate response.http_response.text() access. The comment gives sample readers the wrong raw-response-hook callback contract.

            # Since `polling=False` is set below, it is guaranteed that `capture_created_job` will be
            # invoked once on the initial "201 Created" response, and `response` is of type `OptimizationJob`.

Copilot

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (19)

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:118

This no longer performs a second create call with the same operation_id: it fetches job.id and then compares that ID with itself, so the assertion is tautological and the sample no longer demonstrates the idempotent re-submit described by the comment. Restore the second begin_create_generation_job call (with polling=False) and compare the captured replay job with the first job.

    # Idempotency: a second call with the same operation_id returns the same job.
    replay_job = project_client.beta.evaluators.get_generation_job(job.id)
    assert replay_job.id == job.id

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_all_sources.py:200

Appending .result() as instructed would still leave polling=False, so the poller is NoPolling and returns the initial response immediately rather than waiting. The alternative must also tell users to remove polling=False.

            # Alternatively, append `.result()` to block while the SDK handles polling.

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_prompt_source.py:129

Appending .result() as instructed would still leave polling=False, so the poller is NoPolling and returns the initial response immediately rather than waiting. The alternative must also tell users to remove polling=False.

        # Alternatively, append `.result()` to block while the SDK handles polling.

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_iterate.py:82

Appending .result() as instructed would still leave polling=False, so the poller is NoPolling and returns the initial response immediately rather than waiting. The alternative must also tell users to remove polling=False.

    # Alternatively, append `.result()` to block while the SDK handles polling.

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:104

Appending .result() as instructed would still leave polling=False, so the poller is NoPolling and returns the initial response immediately rather than waiting. The alternative must also tell users to remove polling=False.

This issue also appears on line 116 of the same file.

    # Alternatively, append `.result()` to block while the SDK handles polling.

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_basic.py:99

Appending .result() as instructed would still leave polling=False, so the poller is NoPolling and returns the initial response immediately rather than waiting. The alternative must also tell users to remove polling=False.

    # Alternatively, append `.result()` to block while the SDK handles polling.

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_all_sources.py:138

Appending .result() as instructed would still leave polling=False, so the poller is NoPolling and returns the initial response immediately rather than waiting. The alternative must also tell users to remove polling=False.

This issue also appears on line 200 of the same file.

        # Alternatively, append `.result()` to block while the SDK handles polling.

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_traces_for_finetuning.py:149

Appending .result() as instructed would still leave polling=False, so the poller is NoPolling and returns the initial response immediately rather than waiting. The alternative must also tell users to remove polling=False.

                # Alternatively, append `.result()` to block while the SDK handles polling.

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_traces_for_evaluation.py:148

Appending .result() as instructed would still leave polling=False, so the poller is NoPolling and returns the initial response immediately rather than waiting. The alternative must also tell users to remove polling=False.

                # Alternatively, append `.result()` to block while the SDK handles polling.

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_file_source.py:165

Appending .result() as instructed would still leave polling=False, so the poller is NoPolling and returns the initial response immediately rather than waiting. The alternative must also tell users to remove polling=False.

    # Alternatively, append `.result()` to block while the SDK handles polling.

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_agent_source.py:171

Appending .result() as instructed would still leave polling=False, so the poller is NoPolling and returns the initial response immediately rather than waiting. The alternative must also tell users to remove polling=False.

        # Alternatively, append `.result()` to block while the SDK handles polling.

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning.py:180

Appending .result() as instructed would still leave polling=False, so the poller is NoPolling and returns the initial response immediately rather than waiting. The alternative must also tell users to remove polling=False.

    # Alternatively, append `.result()` to block while the SDK handles polling.

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic_polling_async.py:81

The raw-response hook receives a PipelineResponse, not an OptimizationJob (as the following line's response.http_response access also shows). This comment gives users the wrong hook contract.

            # Since `polling=False` is set below, it is guaranteed that `capture_created_job` will be
            # invoked once on the initial "201 Created" response, and `response` is of type `OptimizationJob`.

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic_polling.py:78

The raw-response hook receives a PipelineResponse, not an OptimizationJob (as the following line's response.http_response access also shows). This comment gives users the wrong hook contract.

        # Since `polling=False` is set below, it is guaranteed that `capture_created_job` will be
        # invoked once on the initial "201 Created" response, and `response` is of type `OptimizationJob`.

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:14

The sample does not report the LROPoller status; it disables SDK polling, discards the poller, and reports statuses from get_generation_job. The overview should describe the job-resource polling that users will actually see.

      * `begin_create_generation_job` with `operation_id` for idempotent re-submits;
         returns `LROPoller[EvaluatorVersion]`, whose status is reported until
         the job reaches a terminal state.

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_basic.py:17

The sample does not report the standard poller's status; it disables SDK polling and repeatedly fetches the generation job. This overview currently teaches the opposite mechanism from the code.

      2. Calls `begin_create_generation_job` and reports the standard LRO
         poller's status until it returns the generated `EvaluatorVersion`.

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_prompt_source.py:15

The sample does not report the standard poller's status; it disables SDK polling and repeatedly fetches the generation job. This overview currently teaches the opposite mechanism from the code.

This issue also appears on line 129 of the same file.

         to a new versioned Dataset. Uses `begin_create_generation_job` and
         reports the standard LRO poller's status until the operation completes.

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic_polling.py:10

This sample disables the SDK's LRO poller and manually fetches the optimization job until it reaches a terminal status, so describing it as polling the standard LRO obscures the distinction from the built-in-polling sample.

    optimization job and poll its standard LRO to completion.

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_all_sources.py:198

This defines capture_created_job a second time in the same module scope (the earlier definition is at line 135). Because Python blocks do not introduce a scope, the package's sample MyPy pass reports this as [no-redef]; the repository runner checks the entire samples directory (eng/tools/azure-sdk-tools/azpysdk/mypy.py:121-129). Reuse the first callback after resetting created_jobs, or give this callback a distinct name and update the hook.

            def capture_created_job(response):
                created_jobs.append(EvaluatorGenerationJob(response.http_response.json()))

Copilot

Pull request overview

Copilot reviewed 15 out of 15 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (19)

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:119

This no longer verifies the documented idempotent re-submit: it performs a GET using the already returned ID, which will trivially return the same job regardless of operation_id. Re-submit begin_create_generation_job with the same operation ID (with polling disabled) and compare the returned job instead, or remove the idempotency claim.

    # Idempotency: a second call with the same operation_id returns the same job.
    replay_job = project_client.beta.evaluators.get_generation_job(job.id)
    assert replay_job.id == job.id

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic_polling_async.py:79

This empty collection no longer has the explicit element type that the previous initial_responses declaration provided. Because main is typed and package CI runs mypy over samples, mypy cannot infer the type from appends inside the untyped nested hook and reports Need type annotation for "pipeline_responses" [var-annotated]. Preserve a list[PipelineResponse[HttpRequest, AsyncHttpResponse]] annotation (and its imports).

        pipeline_responses = []

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_iterate.py:83

Appending .result() here will not enable SDK polling because this call still passes polling=False; the resulting NoPolling poller resolves the initial response immediately. The guidance must also tell users to remove polling=False (and adapt the raw-hook/job handling).

    # Alternatively, append `.result()` to block while the SDK handles polling.

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:105

Appending .result() here will not enable SDK polling because this call still passes polling=False; the resulting NoPolling poller resolves the initial response immediately. The guidance must also tell users to remove polling=False (and adapt the raw-hook/job handling).

    # Alternatively, append `.result()` to block while the SDK handles polling.

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_basic.py:100

Appending .result() here will not enable SDK polling because this call still passes polling=False; the resulting NoPolling poller resolves the initial response immediately. The guidance must also tell users to remove polling=False (and adapt the raw-hook/job handling).

    # Alternatively, append `.result()` to block while the SDK handles polling.

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_all_sources.py:139

Appending .result() here will not enable SDK polling because this call still passes polling=False; the resulting NoPolling poller resolves the initial response immediately. The guidance must also tell users to remove polling=False (and adapt the raw-hook/job handling).

        # Alternatively, append `.result()` to block while the SDK handles polling.

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_all_sources.py:202

Appending .result() here will not enable SDK polling because this call still passes polling=False; the resulting NoPolling poller resolves the initial response immediately. The guidance must also tell users to remove polling=False (and adapt the raw-hook/job handling).

            # Alternatively, append `.result()` to block while the SDK handles polling.

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_traces_for_finetuning.py:150

Appending .result() here will not enable SDK polling because this call still passes polling=False; the resulting NoPolling poller resolves the initial response immediately. The guidance must also tell users to remove polling=False (and adapt the raw-hook/job handling).

                # Alternatively, append `.result()` to block while the SDK handles polling.

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_traces_for_evaluation.py:149

Appending .result() here will not enable SDK polling because this call still passes polling=False; the resulting NoPolling poller resolves the initial response immediately. The guidance must also tell users to remove polling=False (and adapt the raw-hook/job handling).

                # Alternatively, append `.result()` to block while the SDK handles polling.

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_prompt_source.py:130

Appending .result() here will not enable SDK polling because this call still passes polling=False; the resulting NoPolling poller resolves the initial response immediately. The guidance must also tell users to remove polling=False (and adapt the raw-hook/job handling).

        # Alternatively, append `.result()` to block while the SDK handles polling.

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_file_source.py:166

Appending .result() here will not enable SDK polling because this call still passes polling=False; the resulting NoPolling poller resolves the initial response immediately. The guidance must also tell users to remove polling=False (and adapt the raw-hook/job handling).

    # Alternatively, append `.result()` to block while the SDK handles polling.

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_agent_source.py:172

Appending .result() here will not enable SDK polling because this call still passes polling=False; the resulting NoPolling poller resolves the initial response immediately. The guidance must also tell users to remove polling=False (and adapt the raw-hook/job handling).

        # Alternatively, append `.result()` to block while the SDK handles polling.

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning.py:181

Appending .result() here will not enable SDK polling because this call still passes polling=False; the resulting NoPolling poller resolves the initial response immediately. The guidance must also tell users to remove polling=False (and adapt the raw-hook/job handling).

    # Alternatively, append `.result()` to block while the SDK handles polling.

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_basic.py:17

The sample does not report an LRO poller's status: it disables SDK polling and repeatedly retrieves the generation-job resource. This overview therefore describes the opposite polling mechanism and should say that the application polls the job status outside the SDK.

      2. Calls `begin_create_generation_job` and reports the standard LRO
         poller's status until it returns the generated `EvaluatorVersion`.

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_prompt_source.py:15

The sample does not report an LRO poller's status: it disables SDK polling and repeatedly retrieves the generation-job resource. This overview therefore describes the opposite polling mechanism and should say that the application polls the job status outside the SDK.

         to a new versioned Dataset. Uses `begin_create_generation_job` and
         reports the standard LRO poller's status until the operation completes.

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic_polling.py:10

This sample disables the LRO poller with polling=False and manually polls the optimization-job resource, so describing it as polling the standard LRO is inaccurate.

    optimization job and poll its standard LRO to completion.

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic_polling.py:78

response is the pipeline response passed to the raw-response hook, not an OptimizationJob; the job is only constructed from its HTTP body on the next line. The new type claim is misleading for users copying this sample.

        # Since `polling=False` is set below, it is guaranteed that `raw_response_hook` will be
        # invoked once on the initial "201 Created" response, and `response` is of type `OptimizationJob`.

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:14

The sample disables SDK polling and polls the generation-job resource, so this wording should identify the job status rather than implying the returned LRO poller's status is being reported.

         returns `LROPoller[EvaluatorVersion]`, whose status is reported until
         the job reaches a terminal state.

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_all_sources.py:200

This second module-scope definition of raw_response_hook triggers Pylint's function-redefined error (the surrounding try/else blocks do not create Python scopes), and sample linting does not disable that diagnostic. Give the traces callback a distinct name and pass that name to the second create call.

            def raw_response_hook(response):
                response.http_response.read()
                created_jobs.append(EvaluatorGenerationJob(response.http_response.json()))

Copilot

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (13)

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning_async.py:190

This built-in-polling instruction is invalid for the async API: appending .result() to the call attempts to access it on the coroutine, and AsyncLROPoller.result() must itself be awaited. Show the two awaited steps used by the async basic sample instead.

        # Alternatively, have the SDK handle polling by removing `polling=False` and appending `.result()` to the above call.

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning_async.py:132

The input file is created before multiple reachable failure branches (file processing, job failure, missing result/output), but deletion occurs only on the happy path. Any such exception leaves the uploaded Azure OpenAI file—and possibly generated output files—behind. Put resource deletion in a best-effort finally block after creation.

        seed_file = await openai_client.files.create(
            file=(seed_filename, io.BytesIO(SEED_REFERENCE_DOCUMENT.encode("utf-8"))),
            purpose="user_data",
        )

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning_async.py:177

The repository's mypy sample check uses --check-untyped-defs; because this empty list is populated only from an untyped callback argument, its element type cannot be inferred and mypy reports var-annotated. Add a list[PipelineResponse[HttpRequest, AsyncHttpResponse]] annotation and the corresponding imports.

        pipeline_responses = []

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning_async.py:117

The async sample tests auto-discover every *_async.py file under samples/datasets (tests/samples/test_samples_async.py:164-179), but this new live-resource sample is not in that test's skip list and has no recording. Playback CI will therefore execute this workflow without matching recorded requests. Add the exact filename to samples_to_skip until a recording is committed, or add the recording with this sample.

This issue also appears in the following locations of the same file:

line 129
line 177
line 190

async def main() -> None:

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic_polling.py:78

raw_response_hook receives a PipelineResponse, not an OptimizationJob (as the following response.http_response access itself demonstrates). This type claim is misleading in a sample intended to teach raw-response handling.

        # Since `polling=False` is set below, it is guaranteed that `raw_response_hook` will be
        # invoked once on the initial "201 Created" response, and `response` is of type `OptimizationJob`.

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:119

This no longer tests the documented idempotent re-submit: get_generation_job only fetches the already-known ID and never uses operation_id. Re-submit begin_create_generation_job with the same operation ID and compare the captured job so the lifecycle sample still demonstrates its stated behavior.

    # Idempotency: a second call with the same operation_id returns the same job.
    replay_job = project_client.beta.evaluators.get_generation_job(job.id)
    assert replay_job.id == job.id

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:14

The sample disables the LRO poller and reports the EvaluatorGenerationJob.status returned by get_generation_job; it does not report the poller's status. The overview should describe application-level job polling, which is the behavior this change is intended to demonstrate.

         returns `LROPoller[EvaluatorVersion]`, whose status is reported until
         the job reaches a terminal state.

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_basic.py:17

This description says the standard LRO poller's status is reported, but the implementation passes polling=False, discards the poller, and polls the generation-job resource. Describe that application polling instead so users are not taught the opposite mechanism.

      2. Calls `begin_create_generation_job` and reports the standard LRO
         poller's status until it returns the generated `EvaluatorVersion`.

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_prompt_source.py:15

The implementation disables and ignores the standard LRO poller, then polls get_generation_job; therefore this overview inaccurately describes poller-status reporting. State that it reports the job resource's status instead.

         to a new versioned Dataset. Uses `begin_create_generation_job` and
         reports the standard LRO poller's status until the operation completes.

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic_polling.py:10

This sample passes polling=False and polls the optimization-job resource with get_optimization_job; it does not poll the standard LRO. The description should retain the distinction from the built-in-polling companion sample.

    optimization job and poll its standard LRO to completion.

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning_async.py:30

This sample uses begin_create_generation_job, which was introduced by the 2.4.0 rename according to CHANGELOG.md; installing the documented minimum 2.2.0 leaves only the older API and the sample fails before polling. Match the minimum version used by the other LRO generation samples.

    pip install "azure-ai-projects>=2.2.0" azure-identity openai python-dotenv aiohttp

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic_polling_async.py:76

Samples are checked by mypy with --check-untyped-defs, and this empty list receives only an Any value from an untyped callback, so mypy cannot infer its element type and reports var-annotated. Restore a PipelineResponse[HttpRequest, AsyncHttpResponse] element annotation (and its type imports), as this sample previously did.

        pipeline_responses = []

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_all_sources.py:200

This is the second raw_response_hook definition in the same module scope (the first is at line 135), which causes the package's mypy sample check to report no-redef. Give the traces callback a distinct name and pass that name at line 225.

            def raw_response_hook(response):
                response.http_response.read()
                created_jobs.append(EvaluatorGenerationJob(response.http_response.json()))

Copilot

Pull request overview

Copilot reviewed 18 out of 18 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (8)

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_advanced_app_polling.py:10

The code below sets polling=False and polls the optimization job resource via get_optimization_job; it does not poll the standard LRO as this description states.

This issue also appears on line 77 of the same file.
sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_advanced_app_polling.py:78

response is a pipeline response, not an OptimizationJob (the next line accesses response.http_response). The comment gives readers the wrong hook contract.
sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning_async.py:132
The input file is created before any try/finally, while deletion occurs only at the end of the happy path. A processing failure, job failure, missing output, or file-retrieval error therefore leaves the uploaded Azure OpenAI file (and possibly generated output files) behind despite the sample promising cleanup.

        seed_file = await openai_client.files.create(
            file=(seed_filename, io.BytesIO(SEED_REFERENCE_DOCUMENT.encode("utf-8"))),
            purpose="user_data",
        )

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:14

Although begin_create_generation_job returns a poller, the sample discards it and reports the separately fetched job resource's status. The overview therefore misidentifies what is being observed.

      * `begin_create_generation_job` with `operation_id` for idempotent re-submits;
         returns `LROPoller[EvaluatorVersion]`, whose status is reported until
         the job reaches a terminal state.

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:119

This no longer tests the documented idempotent re-submit: get_generation_job(job.id) only fetches the job that was just created, so the assertion is tautological. Re-submit begin_create_generation_job with the same operation_id and compare the returned job instead.

    # Idempotency: a second call with the same operation_id returns the same job.
    replay_job = project_client.beta.evaluators.get_generation_job(job.id)
    assert replay_job.id == job.id

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_basic.py:17

The sample disables the LRO poller with polling=False and polls get_generation_job, so it never reports the standard poller's status as this description claims.

      2. Calls `begin_create_generation_job` and reports the standard LRO
         poller's status until it returns the generated `EvaluatorVersion`.

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_prompt_source.py:15

The implementation disables the standard LRO poller and polls the generation job resource, so this description documents a different polling mechanism than the sample demonstrates.

         to a new versioned Dataset. Uses `begin_create_generation_job` and
         reports the standard LRO poller's status until the operation completes.

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning_async.py:30

This sample calls begin_create_generation_job, which was introduced by the 2.4.0 LRO API change; azure-ai-projects 2.2.0 only exposes create_generation_job. Following this install command with the minimum allowed version raises AttributeError when the sample reaches line 185.

This issue also appears on line 129 of the same file.

    pip install "azure-ai-projects>=2.2.0" azure-identity openai python-dotenv aiohttp

+        print("SDK is polling the optimization job to completion")
+        while not poller.done():
+            await asyncio.sleep(poll_interval)
+            print(f"status=`{poller.status()}`")
+
+        # done() is true, so awaiting result() returns without waiting further and
+        # propagates any exception raised by the SDK's LRO polling operation.
+        result = await poller.result()


Copilot

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (15)

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_advanced_app_polling.py:10

This sample disables SDK polling and polls the optimization resource through get_optimization_job, so it is not polling the standard LRO as claimed. Calling out application-managed job polling is important because that distinction is the sample's purpose.
sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_advanced_app_polling.py:78
raw_response_hook receives a PipelineResponse, not an OptimizationJob; the code itself accesses response.http_response and then constructs the model from its JSON. The type claim is misleading for users copying this callback pattern.
sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_advanced_app_polling_async.py:98
The nested multiline constructor is missing the trailing commas Black requires for this expanded layout, so the formatting check will fail/rewrite these changed lines.
sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_advanced_app_polling.py:96
This newly expanded constructor is not Black-formatted: the nested multiline calls need trailing commas on their closing lines. The repository's Black check will reformat this block, so commit the formatted form.
sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic.py:18
The usage command names sample_optimization_job_app_polling.py, but this file is sample_optimization_job_basic.py, so copying the documented command fails with a file-not-found error.

    python sample_optimization_job_app_polling.py

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic_async.py:18

The usage command points to a nonexistent sample_optimization_job_app_polling_async.py instead of this sample's filename, so users cannot run the documented command.

    python sample_optimization_job_app_polling_async.py

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:119

This no longer performs the documented second submission with the same operation_id; get_generation_job only fetches the already-known ID and therefore cannot demonstrate or verify idempotent re-submission. Restore a second begin_create_generation_job call (with the same operation ID and polling disabled) before comparing the returned job ID.

    # Idempotency: a second call with the same operation_id returns the same job.
    replay_job = project_client.beta.evaluators.get_generation_job(job.id)
    assert replay_job.id == job.id

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning_async.py:30

These generation-job LRO methods were introduced/renamed in azure-ai-projects 2.4.0 (the 2.4.0 changelog lists begin_create_generation_job as a breaking beta change). With the documented >=2.2.0, users can install an older compatible version where this sample fails because the method is unavailable.

    pip install "azure-ai-projects>=2.2.0" azure-identity openai python-dotenv aiohttp

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning_async.py:221

Cleanup runs only on the success path. Any file-processing, LRO, output-validation, or retrieval error after the input file is uploaded skips these deletes and leaks Azure OpenAI files. Track created IDs and perform best-effort deletion in a finally block, as the trace-generation samples do.

        for output in file_outputs:
            print(f"Delete the generated Azure OpenAI file `{output.id}`.")
            await openai_client.files.delete(file_id=output.id)

        print(f"Delete the Azure OpenAI input file `{seed_file.id}`.")
        await openai_client.files.delete(file_id=seed_file.id)

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_prompt_source.py:15

The implementation disables SDK polling and reports statuses from get_generation_job; it never reports the returned LROPoller's status. The overview should describe application-managed job polling instead.

         to a new versioned Dataset. Uses `begin_create_generation_job` and
         reports the standard LRO poller's status until the operation completes.

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_basic.py:17

This description says the standard LRO poller's status is reported, but the sample passes polling=False, ignores the returned poller, and polls get_generation_job instead. Describe this as application-managed generation-job polling so readers are not taught the wrong mechanism.

      2. Calls `begin_create_generation_job` and reports the standard LRO
         poller's status until it returns the generated `EvaluatorVersion`.

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:14

The overview says the returned LROPoller status is reported, but the code disables SDK polling, ignores that poller, and fetches EvaluatorGenerationJob status explicitly. Update the overview to match the application-managed polling demonstrated below.

         returns `LROPoller[EvaluatorVersion]`, whose status is reported until
         the job reaches a terminal state.

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning.py:180

The PR description says dataset-generation samples should demonstrate polling outside the SDK, but this starts the normal LROPoller; its background thread performs all service polling while this loop only observes done()/status(). Use polling=False, capture the initial DataGenerationJob, and call get_generation_job in the application loop, as the other updated dataset samples do.

    poller = project_client.beta.datasets.begin_create_generation_job(
        job=job,
        polling_interval=poll_interval_seconds,
    )

    print("Optional: While SDK is polling, periodically print the job status until the job is complete")
    while not poller.done():
        time.sleep(poll_interval_seconds)
        print(f"\tstatus=`{poller.status()}`")

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning_async.py:183

This new dataset sample also delegates polling to AsyncLROPoller, contrary to the PR's stated goal of showing application-managed polling outside the SDK. After fixing the non-progressing done() loop, it would still only observe SDK polling; instead disable polling, capture the initial job, and poll get_generation_job asynchronously.

        poller = await project_client.beta.datasets.begin_create_generation_job(
            job=job,
            polling_interval=poll_interval_seconds,
        )

        print("Optional: While SDK is polling, periodically print the job status until the job is complete")
        while not poller.done():
            await asyncio.sleep(poll_interval_seconds)
            print(f"\tstatus=`{poller.status()}`")

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_all_sources.py:200

This redefines the module-level raw_response_hook declared for the first job, which triggers Pylint's function-redefined check and duplicates identical parsing logic. The existing callback reads the current created_jobs binding, so after resetting that list it can be reused for the traces request.

            def raw_response_hook(response):
                response.http_response.read()
                created_jobs.append(EvaluatorGenerationJob(response.http_response.json()))

+        while not poller.done():
+            await asyncio.sleep(poll_interval)
+            print(f"status=`{poller.status()}`")
+
+        # Since done() is true, result() returns the final deserialized job result without
+        # waiting further. It also propagates any LRO polling exception.
        result = await poller.result()


+        while not poller.done():
+            await asyncio.sleep(poll_interval_seconds)
+            print(f"\tstatus=`{poller.status()}`")
+
+        # Since done() is true, result() returns the final deserialized job result without
+        # waiting further. It also propagates any LRO polling exception.
+        job_result = await poller.result()


Copilot

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (10)

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning_async.py:187

AsyncLROPoller does not poll in the background: done() remains False until wait() or result() runs the polling method. This loop therefore sleeps forever and never reaches poller.result(). Start poller.result() as a task before observing its status, then await that task.

        print("Optional: While SDK is polling, periodically print the job status until the job is complete")
        while not poller.done():
            await asyncio.sleep(poll_interval_seconds)
            print(f"\tstatus=`{poller.status()}`")

        # Since done() is true, result() returns the final deserialized job result without
        # waiting further. It also propagates any LRO polling exception.
        job_result = await poller.result()

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic_async.py:102

Unlike the synchronous poller, AsyncLROPoller starts polling only when wait() or result() is awaited. Since this loop checks done() before starting either operation, it never terminates. Run poller.result() in a task while printing status and await the task afterward.

        print("Optional: While SDK is polling, periodically print the job status until the job is complete")
        while not poller.done():
            await asyncio.sleep(poll_interval)
            print(f"status=`{poller.status()}`")

        # Since done() is true, result() returns the final deserialized job result without
        # waiting further. It also propagates any LRO polling exception.
        result = await poller.result()

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_advanced_app_polling.py:10

This is application polling of the OptimizationJob resource after passing polling=False, not polling of the standard LRO. The description currently obscures the distinction this sample is intended to demonstrate.
sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning_async.py:129
Once the input file is uploaded, any processing, LRO, output validation, or retrieval failure exits before lines 216-221, leaving the input and possibly generated files behind. Wrap resource tracking and deletion in try/finally so this live sample does not leak cloud resources on its error paths.

        seed_filename = f"widgets-gizmos-seed-{run_id}.md"
        print(f"Upload the seed reference document as Azure OpenAI file `{seed_filename}`.")
        seed_file = await openai_client.files.create(
            file=(seed_filename, io.BytesIO(SEED_REFERENCE_DOCUMENT.encode("utf-8"))),
            purpose="user_data",
        )

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic.py:18

The usage command names sample_optimization_job_app_polling.py, but this file is sample_optimization_job_basic.py and no file with the documented name exists, so copying the command fails.

    python sample_optimization_job_app_polling.py

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic_async.py:18

The documented script name does not exist; this sample is named sample_optimization_job_basic_async.py, so the usage command currently fails when copied.

    python sample_optimization_job_app_polling_async.py

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:14

The implementation sets polling=False, discards the returned LROPoller, and reports EvaluatorGenerationJob.status from explicit GET requests. Describing this as reporting the poller's status teaches the opposite polling model from the sample.

         returns `LROPoller[EvaluatorVersion]`, whose status is reported until
         the job reaches a terminal state.

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_prompt_source.py:15

The code uses polling=False and reports DataGenerationJob.status from explicit get_generation_job calls; no standard LRO poller status is observed. The description should distinguish application polling from SDK polling.

         to a new versioned Dataset. Uses `begin_create_generation_job` and
         reports the standard LRO poller's status until the operation completes.

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:119

This no longer performs an idempotent re-submit: get_generation_job only fetches by job ID and never sends operation_id. As written, the sample's advertised idempotency check cannot detect whether repeating the create request returns the same job.

    # Idempotency: a second call with the same operation_id returns the same job.
    replay_job = project_client.beta.evaluators.get_generation_job(job.id)
    assert replay_job.id == job.id

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_basic.py:17

This sample disables SDK polling and manually fetches the generation job, so it never reports the standard LRO poller's status as documented.

      2. Calls `begin_create_generation_job` and reports the standard LRO
         poller's status until it returns the generated `EvaluatorVersion`.

+"""
+
+
+async def main() -> None:


Copilot

Pull request overview

Copilot reviewed 16 out of 16 changed files in this pull request and generated no new comments.

Suppressed comments (27)

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning_async.py:183

AsyncLROPoller does not poll in the background: its done() flag is only set by wait()/result(). This loop therefore sleeps forever because poller.result() is never reached. Start result() in an asyncio task and observe that task while reporting poller.status().

        print("Optional: While SDK is polling, periodically print the job status until the job is complete")
        while not poller.done():
            await asyncio.sleep(poll_interval_seconds)
            print(f"\tstatus=`{poller.status()}`")

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic_async.py:98

AsyncLROPoller does not poll in the background: its done() flag is only set by wait()/result(). This loop therefore sleeps forever because poller.result() is never reached. Start result() in an asyncio task and observe that task while reporting poller.status().

        print("Optional: While SDK is polling, periodically print the job status until the job is complete")
        while not poller.done():
            await asyncio.sleep(poll_interval)
            print(f"status=`{poller.status()}`")

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning_async.py:114

This new _async.py sample is automatically collected by tests/samples/test_samples_async.py::test_datasets_samples, but that test's skip list does not include it and this PR adds no recording. The analogous synchronous generation samples are explicitly skipped because recordings are unavailable, so playback CI will attempt to execute this live-only workflow and fail. Add this exact filename to the async datasets skip list (or add a recording).

async def main() -> None:

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_advanced_app_polling.py:80

response is the pipeline response passed to the raw hook, not an OptimizationJob; the code itself confirms this by deserializing response.http_response.json(). The type statement is misleading for users copying this hook pattern.

This issue also appears on line 95 of the same file.
sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning_async.py:183

Even after fixing the infinite loop, awaiting poller.result() in a task delegates network polling to the SDK; observing status() is not the outside-SDK application polling promised in the PR description. Use polling=False plus get_generation_job, or align the description with SDK-managed polling.

        print("Optional: While SDK is polling, periodically print the job status until the job is complete")
        while not poller.done():
            await asyncio.sleep(poll_interval_seconds)
            print(f"\tstatus=`{poller.status()}`")

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic.py:96

This loop only observes the synchronous poller's SDK-owned background thread, so it is not the outside-SDK application polling described by the PR. Use polling=False and get_optimization_job to drive polling from the application, or revise the PR description.

    print("Optional: While SDK is polling, periodically print the job status until the job is complete")
    while not poller.done():
        time.sleep(poll_interval)
        print(f"status=`{poller.status()}`")

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic_async.py:98

Even after fixing the infinite loop, awaiting poller.result() in a task delegates network polling to the SDK; observing status() is not the outside-SDK application polling promised in the PR description. Use polling=False plus get_optimization_job, or align the description with SDK-managed polling.

        print("Optional: While SDK is polling, periodically print the job status until the job is complete")
        while not poller.done():
            await asyncio.sleep(poll_interval)
            print(f"status=`{poller.status()}`")

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:122

replay_job is no longer the result of an idempotent replay; it is fetched with get_generation_job. The stale name makes the lifecycle step look like it still verifies the removed resubmission behavior.

    replay_job = project_client.beta.evaluators.get_generation_job(evaluator.generation_job_id)
    assert replay_job.id == evaluator.generation_job_id

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_advanced_app_polling_async.py:98

This multiline call is not Black-formatted: Black adds the trailing comma after the nested OptimizationJobInputs(...) argument. The package's formatting check will report this file.
sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_advanced_app_polling.py:96
This multiline call is not Black-formatted: Black adds the trailing comma after the nested OptimizationJobInputs(...) argument. The package's formatting check will report this file.
sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic_async.py:18
The usage command names sample_optimization_job_app_polling_async.py, but this file is still named sample_optimization_job_basic_async.py; copying the documented command fails because that sample does not exist.

This issue also appears on line 95 of the same file.

    python sample_optimization_job_app_polling_async.py

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning_async.py:129

The uploaded input file is only deleted on the success path. Any exception while waiting for processing, creating/polling the generation job, inspecting outputs, or deleting an output leaves the Azure OpenAI file (and possibly generated files) behind, despite this sample promising cleanup. Wrap resource use in try/finally and perform best-effort deletion there, as the related trace-generation samples do.

This issue also appears on line 180 of the same file.

        seed_file = await openai_client.files.create(
            file=(seed_filename, io.BytesIO(SEED_REFERENCE_DOCUMENT.encode("utf-8"))),
            purpose="user_data",
        )

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_basic.py:18

The usage command names sample_optimization_job_app_polling.py, but this file is still named sample_optimization_job_basic.py; copying the documented command fails because that sample does not exist.

This issue also appears on line 93 of the same file.

    python sample_optimization_job_app_polling.py

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning_async.py:30

This sample calls beta.datasets.begin_create_generation_job, which was introduced by the 2.4.0 rename to the standard LRO API; azure-ai-projects 2.2.0 does not provide this method. Following the install command can therefore fail with AttributeError.

    pip install "azure-ai-projects>=2.2.0" azure-identity openai python-dotenv aiohttp

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:106

This remains SDK-managed polling: a synchronous LROPoller starts its polling thread at construction, and done()/status() only observe that thread. It therefore does not implement the PR description's stated goal of polling outside the SDK. Use polling=False plus get_generation_job application polling, or align the PR description with SDK polling.

This issue also appears on line 121 of the same file.

    print("Optional: While SDK is polling, periodically print the job status until the job is complete")
    while not poller.done():
        time.sleep(poll_interval_seconds)
        print(f"\tstatus=`{poller.status()}`")

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_iterate.py:101

This remains SDK-managed polling: a synchronous LROPoller starts its polling thread at construction, and done()/status() only observe that thread. It therefore does not implement the PR description's stated goal of polling outside the SDK. Use polling=False plus get_generation_job application polling, or align the PR description with SDK polling.

    print("Optional: While SDK is polling, periodically print the job status until the job is complete")
    while not poller.done():
        time.sleep(poll_interval_seconds)
        print(f"\tstatus=`{poller.status()}`")

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_basic.py:125

This remains SDK-managed polling: a synchronous LROPoller starts its polling thread at construction, and done()/status() only observe that thread. It therefore does not implement the PR description's stated goal of polling outside the SDK. Use polling=False plus get_generation_job application polling, or align the PR description with SDK polling.

    print("Optional: While SDK is polling, periodically print the job status until the job is complete")
    while not poller.done():
        time.sleep(poll_interval_seconds)
        print(f"\tstatus=`{poller.status()}`")

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_all_sources.py:147

This remains SDK-managed polling: a synchronous LROPoller starts its polling thread at construction, and done()/status() only observe that thread. It therefore does not implement the PR description's stated goal of polling outside the SDK. Use polling=False plus get_generation_job application polling, or align the PR description with SDK polling.

        print("Optional: While SDK is polling, periodically print the job status until the job is complete")
        while not poller.done():
            time.sleep(poll_interval_seconds)
            print(f"\tstatus=`{poller.status()}`")

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_all_sources.py:205

This second operation also leaves polling to the SDK background thread; done()/status() merely observe it. It does not demonstrate the outside-SDK application polling promised by the PR description. Use polling=False and repeatedly call get_generation_job, or revise the stated goal.

            print("Optional: While SDK is polling, periodically print the job status until the job is complete")
            while not poller.done():
                time.sleep(poll_interval_seconds)
                print(f"\tstatus=`{poller.status()}`")

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_traces_for_finetuning.py:169

This loop only observes the synchronous poller's SDK-owned background thread, so it is not the outside-SDK application polling described by the PR. Use polling=False and get_generation_job to drive polling from the application, or revise the PR description.

                print("Optional: While SDK is polling, periodically print the job status until the job is complete")
                while not poller.done():
                    time.sleep(POLL_INTERVAL_SECONDS)
                    print(f"\tstatus=`{poller.status()}`")

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_traces_for_evaluation.py:166

This loop only observes the synchronous poller's SDK-owned background thread, so it is not the outside-SDK application polling described by the PR. Use polling=False and get_generation_job to drive polling from the application, or revise the PR description.

                print("Optional: While SDK is polling, periodically print the job status until the job is complete")
                while not poller.done():
                    time.sleep(POLL_INTERVAL_SECONDS)
                    print(f"\tstatus=`{poller.status()}`")

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_prompt_source.py:129

This loop only observes the synchronous poller's SDK-owned background thread, so it is not the outside-SDK application polling described by the PR. Use polling=False and get_generation_job to drive polling from the application, or revise the PR description.

        print("Optional: While SDK is polling, periodically print the job status until the job is complete")
        while not poller.done():
            time.sleep(poll_interval_seconds)
            print(f"\tstatus=`{poller.status()}`")

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_file_source.py:193

This loop only observes the synchronous poller's SDK-owned background thread, so it is not the outside-SDK application polling described by the PR. Use polling=False and get_generation_job to drive polling from the application, or revise the PR description.

    print("Optional: While SDK is polling, periodically print the job status until the job is complete")
    while not poller.done():
        time.sleep(poll_interval_seconds)
        print(f"\tstatus=`{poller.status()}`")

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_with_agent_source.py:171

This loop only observes the synchronous poller's SDK-owned background thread, so it is not the outside-SDK application polling described by the PR. Use polling=False and get_generation_job to drive polling from the application, or revise the PR description.

        print("Optional: While SDK is polling, periodically print the job status until the job is complete")
        while not poller.done():
            time.sleep(poll_interval_seconds)
            print(f"\tstatus=`{poller.status()}`")

sdk/ai/azure-ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning.py:181

This loop only observes the synchronous poller's SDK-owned background thread, so it is not the outside-SDK application polling described by the PR. Use polling=False and get_generation_job to drive polling from the application, or revise the PR description.

    print("Optional: While SDK is polling, periodically print the job status until the job is complete")
    while not poller.done():
        time.sleep(poll_interval_seconds)
        print(f"\tstatus=`{poller.status()}`")

sdk/ai/azure-ai-projects/samples/agents/optimization/sample_optimization_job_cancel.py:111

This change switches the cancel sample from polling=False to an SDK-managed background poller, which contradicts the PR description's goal of restoring outside-SDK application polling. Since the sample already has the created job ID, poll with get_optimization_job after cancellation, or revise the stated goal.

    print("Wait for the SDK poller to observe the cancellation.")
    while not poller.done():
        time.sleep(poll_interval)
        print(f"status=`{poller.status()}`")

sdk/ai/azure-ai-projects/samples/evaluations/sample_rubric_evaluator_generation_lifecycle.py:14

The module says it exercises idempotent re-submits, but the replay begin_create_generation_job call was removed; the sample now submits only once and performs a GET. Either restore the duplicate submit or update this description so it does not promise behavior the sample no longer demonstrates.

      * `begin_create_generation_job` with `operation_id` for idempotent re-submits;
         returns `LROPoller[EvaluatorVersion]`, whose status is reported until
         the job reaches a terminal state.

github-actions · 2026-07-31T03:54:18Z

[Pilot] PR Pipeline Failure Analysis

A CI pipeline failed on this pull request. Here is an automated analysis of what went wrong and how to get the build green.

What failed

A single test case failed consistently across all platforms (macOS, Ubuntu, Windows) and all install modes (sdist, whl, mindependency):

Test: sdk.ai.azure-ai-projects.tests.samples.test_samples_async.TestSamplesAsync.test_datasets_samples[sample_dataset_generation_job_simpleqna_for_finetuning_async]

This is a test failure. The same test fails in every matrix combination (5 platforms × 3 install modes = 15 failures), which strongly indicates the failure is in the test/sample logic itself rather than a platform- or environment-specific issue. Given the PR description mentions adding a new async sample (sample_dataset_generation_job_simpleqna_for_finetuning_async.py) and switching samples back to application-level polling, the failure is most likely in that newly added async sample or its test recording.

Recommended next steps

Investigate samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning_async.py and the corresponding test in tests/samples/test_samples_async.py for issues with the new polling logic.
Check whether a test recording exists and is valid for test_datasets_samples[sample_dataset_generation_job_simpleqna_for_finetuning_async]; if it is missing or stale, regenerate it.
Run the failing test locally (pytest tests/samples/test_samples_async.py -k sample_dataset_generation_job_simpleqna_for_finetuning_async) to reproduce and debug.
See the CI troubleshooting guide: https://aka.ms/ci-fix
Push new commits to address the failures; this comment updates automatically on the next failing run.

Raw pipeline analysis (azsdk ci analyze)

Analyzing pipeline https://github.com/Azure/azure-sdk-for-python/pull/48343...
--------------------------------------------------------------------------------
Failed Tests
--------------------------------------------------------------------------------
{
  "LLM Artifacts - macos311 - 1/ai-azure-ai-projects-test-junit-sdist.xml": [
    "sdk.ai.azure-ai-projects.tests.samples.test_samples_async.TestSamplesAsync.test_datasets_samples[sample_dataset_generation_job_simpleqna_for_finetuning_async]"
  ],
  "LLM Artifacts - macos311 - 1/ai-azure-ai-projects-test-junit-mindependency.xml": [
    "sdk.ai.azure-ai-projects.tests.samples.test_samples_async.TestSamplesAsync.test_datasets_samples[sample_dataset_generation_job_simpleqna_for_finetuning_async]"
  ],
  "LLM Artifacts - macos311 - 1/ai-azure-ai-projects-test-junit-whl.xml": [
    "sdk.ai.azure-ai-projects.tests.samples.test_samples_async.TestSamplesAsync.test_datasets_samples[sample_dataset_generation_job_simpleqna_for_finetuning_async]"
  ],
  "LLM Artifacts - Ubuntu2404_313 - 1/...": [ "...same test..." ],
  "LLM Artifacts - ubuntu2404_310 - 1/...": [ "...same test..." ],
  "LLM Artifacts - ubuntu2404_310_coverage - 1/...": [ "...same test..." ],
  "LLM Artifacts - Ubuntu2404_314 - 1/...": [ "...same test..." ],
  "LLM Artifacts - windows2022_312 - 1/...": [ "...same test..." ]
  (15 total failures across all platforms and install modes)
}

Copilot detected the failing pipeline and generated the analysis above. To have it attempt a fix automatically, reply with @copilot please fix the failing pipeline on this PR.

Generated by Pipeline Analysis - Next Steps · 20.9 AIC · ⌖ 6.25 AIC · ⊞ 6.6K · ◷

First

f668889

dargilco self-assigned this Jul 29, 2026

Copilot AI review requested due to automatic review settings July 29, 2026 15:41

dargilco requested review from bobogogo1990, glharper, howieleung, kingernupur, nick863, trangevi and trrwilson as code owners July 29, 2026 15:41

github-actions Bot added the AI Projects label Jul 29, 2026

Copilot started reviewing on behalf of dargilco July 29, 2026 15:41 View session

Copilot AI reviewed Jul 29, 2026

View reviewed changes

Address some CoPilot code review comments

b20ec59

Copilot AI review requested due to automatic review settings July 29, 2026 16:02

Copilot started reviewing on behalf of dargilco July 29, 2026 16:02 View session

Copilot AI reviewed Jul 29, 2026

View reviewed changes

This comment has been minimized.

Sign in to view

More

94e131f

Copilot AI review requested due to automatic review settings July 29, 2026 17:05

Copilot started reviewing on behalf of dargilco July 29, 2026 17:06 View session

Copilot AI reviewed Jul 29, 2026

View reviewed changes

This comment has been minimized.

Sign in to view

Update Agent Optimization polling samples

6430486

Copilot AI review requested due to automatic review settings July 29, 2026 18:03

Copilot started reviewing on behalf of dargilco July 29, 2026 18:04 View session

Copilot AI reviewed Jul 29, 2026

View reviewed changes

More

5503b86

Copilot AI review requested due to automatic review settings July 29, 2026 18:24

Copilot AI reviewed Jul 29, 2026

View reviewed changes

more fixes

392f5b6

Copilot AI review requested due to automatic review settings July 29, 2026 22:44

Copilot started reviewing on behalf of dargilco July 29, 2026 22:45 View session

Copilot AI reviewed Jul 29, 2026

View reviewed changes

More

b050caf

Copilot AI review requested due to automatic review settings July 29, 2026 23:23

Copilot started reviewing on behalf of dargilco July 29, 2026 23:23 View session

Copilot AI reviewed Jul 29, 2026

View reviewed changes

This comment has been minimized.

Sign in to view

More

1b3e83a

Copilot AI review requested due to automatic review settings July 30, 2026 16:13

Copilot started reviewing on behalf of dargilco July 30, 2026 16:14 View session

Copilot AI reviewed Jul 30, 2026

View reviewed changes

This comment has been minimized.

Sign in to view

More

8a340df

Copilot AI review requested due to automatic review settings July 31, 2026 00:33

Copilot started reviewing on behalf of dargilco July 31, 2026 00:34 View session

MOre

01f1715

Copilot AI reviewed Jul 31, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings July 31, 2026 00:42

Copilot started reviewing on behalf of dargilco July 31, 2026 00:43 View session

Copilot AI reviewed Jul 31, 2026

View reviewed changes

Comment thread ...ai-projects/samples/datasets/sample_dataset_generation_job_simpleqna_for_finetuning_async.py

"""

async def main() -> None:

This comment has been minimized.

Sign in to view

dargilco added 2 commits July 30, 2026 20:16

More

500616e

More

d3f91ac

Copilot AI review requested due to automatic review settings July 31, 2026 03:20

Copilot started reviewing on behalf of dargilco July 31, 2026 03:21 View session

Copilot AI reviewed Jul 31, 2026

View reviewed changes

Uh oh!

Conversation

dargilco commented Jul 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

azure-pipelines Bot commented Jul 29, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

This comment has been minimized.

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

This comment has been minimized.

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

This comment has been minimized.

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

This comment has been minimized.

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

This comment has been minimized.

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

github-actions Bot commented Jul 31, 2026

What failed

Recommended next steps

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

dargilco commented Jul 29, 2026 •

edited

Loading