Skip to content

test(proxy): fixme "should exclude patterns" on Firefox/macOS#41770

Open
microsoft-playwright-automation[bot] wants to merge 1 commit into
mainfrom
fix-flakes/macos-15-xlarge-29313037253
Open

test(proxy): fixme "should exclude patterns" on Firefox/macOS#41770
microsoft-playwright-automation[bot] wants to merge 1 commit into
mainfrom
fix-flakes/macos-15-xlarge-29313037253

Conversation

@microsoft-playwright-automation

Copy link
Copy Markdown
Contributor

Times out ~35% of runs on firefox-macos-15-large (19/55) - navigating
bypassed non-existent domains hangs on the Firefox/macOS CI network.
Only this bot fails the test; not reproducible on other OSes or locally
(0/25 with --repeat-each on macos-15-xlarge), so it is environment-specific.
Scoped the fixme to firefox+macOS; the test still runs on chromium/webkit.

Verified locally on macos-15-xlarge: firefox skipped, webkit still passes;
npm run flint green.

Fix-flakes run: https://github.com/microsoft/playwright/actions/runs/29313037253

Suggested-reviewer: dgozman

Times out ~35% of runs on firefox-macos-15-large (19/55) - navigating
bypassed non-existent domains hangs on the Firefox/macOS CI network.
Only this bot fails the test; not reproducible on other OSes or locally
(0/25 with --repeat-each on macos-15-xlarge), so it is environment-specific.
Scoped the fixme to firefox+macOS; the test still runs on chromium/webkit.

Verified locally on macos-15-xlarge: firefox skipped, webkit still passes;
npm run flint green.

Fix-flakes run: https://github.com/microsoft/playwright/actions/runs/29313037253

Suggested-reviewer: dgozman
@github-actions

Copy link
Copy Markdown
Contributor

Test results for "MCP"

1 failed
❌ [webkit] › mcp/annotate.spec.ts:137 › should abort MCP annotation when last screenshot is removed @mcp-windows-latest-webkit

7759 passed, 1249 skipped


Merge workflow run.

@github-actions

Copy link
Copy Markdown
Contributor

Test results for "tests 1"

8 flaky ⚠️ [chromium-library] › library/browsercontext-page-event.spec.ts:173 › should work with Ctrl-clicking `@chromium-ubuntu-22.04-arm-node20`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:434 › should produce network events, routing, and annotations for Service Worker (advanced) `@chromium-ubuntu-22.04-arm-node20`
⚠️ [chromium-library] › library/chromium/oopif.spec.ts:282 › should click `@chromium-ubuntu-22.04-arm-node20`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:434 › should produce network events, routing, and annotations for Service Worker (advanced) `@realtime-time-library-chromium-linux`
⚠️ [chromium-library] › library/har-websocket.spec.ts:231 › should embed websocket messages `@chromium-ubuntu-22.04-node24`
⚠️ [chromium-library] › library/video.spec.ts:717 › screencast › should work with video+trace `@chromium-ubuntu-22.04-node20`
⚠️ [chromium-library] › library/inspector/recorder-api.spec.ts:120 › should type `@chromium-ubuntu-22.04-node22`
⚠️ [firefox-library] › library/heap.spec.ts:223 › should not leak workers `@firefox-ubuntu-22.04-node20`

49569 passed, 1156 skipped


Merge workflow run.

@dgozman

dgozman commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

@Skn0tt It would be great to get some "preview" of the flakiness of the test we are fixing. For example, in this PR marking as "fixme" would just disable the test forever. Having some visual representation would help me to make a call whether this test adds some viable signal, or whether it is noise and should be fixmed.

@Skn0tt

Skn0tt commented Jul 14, 2026

Copy link
Copy Markdown
Member

@dgozman How do you like this?


Run history for should exclude patterns on firefox-macos-15-large (oldest → newest, 🟩 pass / 🟧 rescued by retry / 🟥 fail — 20/58 failed):

🟩🟥🟥🟩🟩🟩🟩🟥🟩🟩🟩🟧🟥🟩🟥🟩🟩🟩🟩🟩🟩🟥🟩🟩🟩🟩🟩🟥🟩🟥🟩🟥🟥🟥🟥🟩🟥🟩🟩🟩🟥🟩🟩🟥🟩🟥🟩🟥🟥🟥🟩🟩🟩🟩🟩🟩🟩🟥

@dgozman

dgozman commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

@dgozman How do you like this?

Looks great, except for it does not fit on a single line 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants