Skip to content

Re-enable recorded azure-ai-ml e2e tests blocked by stale live_test_only markers - #48393

Draft
lavakumarrepala with Copilot wants to merge 2 commits into
mainfrom
copilot/implement-reliability-hardening-azure-ai-ml
Draft

Re-enable recorded azure-ai-ml e2e tests blocked by stale live_test_only markers#48393
lavakumarrepala with Copilot wants to merge 2 commits into
mainfrom
copilot/implement-reliability-hardening-azure-ai-ml

Conversation

Copilot AI commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

This change targets the azure-ai-ml gating gap where several e2e tests were excluded from playback-only PR coverage via @pytest.mark.live_test_only(...) markers that explicitly cited stale recordings / sanitizer migration. The intent is to let already-recorded tests participate in normal playback gating again instead of silently bypassing CI.

  • Re-enable playback for stale-recording cases

    • Removed live_test_only markers whose only stated reason was re-recording or sanitizer drift.
    • Affected areas:
      • datastore/e2etests/test_datastore.py
      • environment/e2etests/test_environment.py
      • connection/e2etests/test_connections.py
      • component/e2etests/test_component.py
      • internal/e2etests/test_pipeline_job.py
  • Datastore coverage brought back into gate

    • Re-enabled the datastore e2e cases that were most directly relevant to the regression class called out in the issue:
      • test_blob_store
      • test_blob_store_credential_less
      • test_file_store
      • test_credential_less_adls_gen2_store
  • Guardrail against regression of the test posture

    • Added tests/test_e2e_recording_marker_audit.py, a lightweight AST-based audit that fails if live_test_only is reintroduced for reasons like:
      • “needs re-recording”
      • “new common sanitizers”
      • “new test proxy sanitizers”
@pytest.mark.live_test_only("Needs re-recording to work with new common sanitizers")
def test_blob_store(...):
    ...

becomes:

def test_blob_store(...):
    ...
  • Scope intentionally limited
    • Did not touch skips that still document non-recording reasons (preview-only behavior, service limitations, known product bugs, environment/platform constraints).
    • Did not change test logic or intent; only the gating eligibility and a repo-level audit around stale recording markers.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
10 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Co-authored-by: lavakumarrepala <221403938+lavakumarrepala@users.noreply.github.com>
Copilot AI changed the title [WIP] Implement reliability hardening for azure-ai-ml e2e CI gating Re-enable recorded azure-ai-ml e2e tests blocked by stale live_test_only markers Jul 31, 2026
Copilot AI requested a review from lavakumarrepala July 31, 2026 21:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants