feat(test-runner): add httpCache config option by pavelfeldman · Pull Request #41687 · microsoft/playwright

pavelfeldman · 2026-07-08T04:30:21Z

Summary

Adds a top-level httpCache: { dir, match } test config option. When set, the runner starts a single caching MITM proxy and routes every browser through it: matching responses are recorded to dir on the first run and replayed from disk on later runs.
One proxy owns the cache for the whole run, so all workers share it and concurrent identical misses coalesce into one upstream fetch. match (glob/RegExp) selects which request URLs are cached; https is intercepted with a self-signed cert.

Skn0tt

I'm not seeing typical caching headers like Vary or Accept-Encoding be mentioned, which seems fishy. Am I misunderstanding something?

pavelfeldman · 2026-07-08T18:44:13Z

I'm not seeing typical caching headers like Vary or Accept-Encoding be mentioned, which seems fishy. Am I misunderstanding something?

This is intentional, typical caching headers in dev / testing flow are not aggressive enough. We might want to have some finer grained controls for that though.

Copilot

Pull request overview

Adds a new testConfig.httpCache option to Playwright Test that spins up a caching MITM proxy for the run, recording matching GET responses to disk and replaying them on subsequent runs (shared across workers), with documentation and coverage tests.

Changes:

Introduce httpCache config (single entry or array) with dir and match (glob/RegExp/policy callback) and resolve cache directories relative to the config file.
Add a cache-proxy test-runner plugin that starts one (or more, per distinct upstream proxy) caching proxy server(s) and routes browsers/contexts through them.
Add ResponseCache on-disk format + runner tests + API docs/type updates.

Reviewed changes

Copilot reviewed 11 out of 13 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
utils/generate_types/overrides-test.d.ts	Adds `HttpCachePolicy` type override for generated test types.
tests/playwright-test/cache-proxy.spec.ts	New integration tests covering record/replay, HTTPS MITM, policy matching, coalescing, proxy chaining, and multi-cache routing.
packages/utils/network.ts	Exports `shouldBypassProxy` and allows passing `agentOptions` into `createProxyAgent`.
packages/playwright/types/test.d.ts	Adds `httpCache` config typing and `HttpCachePolicy` export; adds `FullConfig.httpCache`.
packages/playwright/src/runner/testRunner.ts	Wires cache-proxy plugin into runner plugin initialization.
packages/playwright/src/plugins/DEPS.list	Updates allowed imports for the plugins folder (cache proxy + common config).
packages/playwright/src/plugins/cacheProxyPlugin.ts	Implements runner plugin to initialize caches, start proxy servers, and publish them via env var.
packages/playwright/src/plugins/cacheProxy/server.ts	Implements caching proxy server with MITM HTTPS, upstream fetching, and request coalescing.
packages/playwright/src/plugins/cacheProxy/cache.ts	Implements on-disk response cache (index + blobs) with versioning.
packages/playwright/src/index.ts	Routes browser/context traffic through the cache proxy when enabled; enables `ignoreHTTPSErrors` for MITM.
packages/playwright/src/common/config.ts	Resolves `httpCache` config entries (dir resolution) and adds `proxySettingsKey` export.
docs/src/test-api/class-testconfig.md	Documents `TestConfig.httpCache` option and usage patterns.
docs/src/test-api/class-fullconfig.md	Documents `FullConfig.httpCache` as a resolved form of `TestConfig.httpCache`.

Records browser responses to disk and replays them on later runs. When `httpCache: { dir, match }` is set, the runner starts a single caching MITM proxy and routes all browsers through it. GET/2xx responses are stored verbatim (raw bytes + headers) and matched by an optional URL glob/regex; https is intercepted with a self-signed cert. One proxy owns the cache for the whole run, so all workers share it and concurrent identical misses coalesce into one upstream fetch.

github-actions · 2026-07-15T22:49:23Z

Test results for "MCP"

5 failed
❌ [chrome] › mcp/annotate.spec.ts:173 › user-initiated annotate downloads zip with feedback.md @mcp-macos-latest-chrome
❌ [chrome] › mcp/annotate.spec.ts:230 › should capture annotations via show --annotate @mcp-macos-latest-chrome
❌ [webkit] › mcp/annotate.spec.ts:230 › should capture annotations via show --annotate @mcp-macos-latest-webkit
❌ [webkit] › mcp/annotate.spec.ts:497 › should disengage annotate mode when --annotate client disconnects @mcp-macos-latest-webkit
❌ [webkit] › mcp/http.spec.ts:104 › http transport browser lifecycle (isolated) @mcp-ubuntu-latest-webkit

7755 passed, 1249 skipped

Merge workflow run.

github-actions · 2026-07-15T22:51:43Z

Test results for "tests 1"

1 flaky

⚠️ [chromium-library] › library/video.spec.ts:736 › screencast › should work with video+trace `@realtime-time-library-chromium-linux`

49808 passed, 1156 skipped

Merge workflow run.

github-actions · 2026-07-15T22:52:27Z

Hi, I'm the Playwright bot and I took a first look at the CI failures.

🟢 All 5 failures are pre-existing MCP flakes — this PR is clear

The latest report shows 5 failed tests, all in mcp/annotate.spec.ts and mcp/http.spec.ts. This PR only adds a test-runner httpCache config option (plugins/cacheProxy/*, config.ts, index.ts, testRunner.ts, network.ts) — nothing in the MCP surface these tests exercise. Every one of them flips verdict on many unrelated PRs and on push runs, so the PR can't be responsible.

Details

Overall: none of the failures reach code this PR touches. The diff is scoped to the caching proxy plugin and test-runner config; the failing tests all live under tests/mcp/ and exercise the MCP annotate flow and HTTP transport lifecycle, which this PR doesn't wire into.

Pre-existing flake / infra (counts from the aggregated test-results DB, 871 runs in window):

[chrome]/[webkit] › mcp/annotate.spec.ts:230 › should capture annotations via show --annotate — failed 21/871 runs across 10 distinct PRs (41756, 41741, 41734, 41763, 41731, 41752, 41727, 41463, 41710) plus push. Passed 850.
[chrome] › mcp/annotate.spec.ts:173 › user-initiated annotate downloads zip with feedback.md — failed 22/871 across 11 PRs (41786, 41728, 41669, 41755, 41739, 41783, 41673, 41731, 41685, 41752) plus push. Passed 849.
[webkit] › mcp/annotate.spec.ts:497 › should disengage annotate mode when --annotate client disconnects — failed 22/871 across 14 PRs (41695, 41715, 41463, 41751, 41691, 41739, 41646, 41707, 41675, 41712, 41704, 41782, 41716) plus push. Passed 849.
[webkit] › mcp/http.spec.ts:104 › http transport browser lifecycle (isolated) — failed 7/871 across 4 PRs (41781, 41703, 41704, 41712) plus push. Passed 864.

Each of these fails on SHAs this PR has no bearing on (including other people's PRs and mainline pushes), which is the proof they're flakes rather than caused here.

_{Triaged by the Playwright bot - agent run}

yury-s

Some AI findings:

1. Every GET is store-and-forwarded. With no match, _select() claims every GET, so _fetchUpstream() buffers the whole body before responding. SSE/long-polling GETs hang forever, large downloads sit fully in runner memory, TTFB becomes TTLB for uncacheable responses. Storability is decidable from response headers — stream through when the response isn't storable.

2. Miss-coalescing ignores request headers. _fetchCoalesced() keys on method+URL+identity only. Worker A revalidates with If-None-Match while worker B requests the same URL cold → B gets a bodiless 304 it can't satisfy. Concurrent Range requests cross-serve one 206. Exclude Range/If-* requests from coalescing, or strip If-* from the shared fetch.

3. Plain ws:// breaks. Browsers CONNECT for both ws:// and wss:// through an HTTP proxy (RFC 6455 §4.1), and _onConnect() always pipes the tunnel into the TLS server, so a plaintext WS handshake dies. Sniff the first byte (0x16 = TLS) to route. Only wss:// is tested.

4. Proxy errors crash the whole run. The proxy lives in the runner process: proxyRes in _passThrough() has no 'error' handler (upstream ECONNRESET mid-body → uncaught exception), _onRequest() is fire-and-forget so a throwing match callback → unhandled rejection, and there's no req.on('error') for client aborts.

5. TLS verification is disabled end-to-end. rejectUnauthorized: false upstream plus forced ignoreHTTPSErrors: true (overriding an explicit user false). A spoofed response isn't a one-off — it's persisted and replayed on every later run. Verify upstream certs by default; at minimum document this loudly.

6. Routing gaps. _onUpgrade() dials origins directly, bypassing the configured upstream proxy — WS fails in proxy-only networks. test.use({ proxy }) is silently dropped (cacheProxySettings() keys off project.use.proxy). Remote browsers via connectOptions get an unreachable 127.0.0.1 proxy and 502 everything.

7. No non-Chromium coverage. <-loopback> is Chromium-only bypass syntax; Firefox works by accident (allow_hijacking_localhost=true in playwright.cfg), WebKit gets the literal in --proxy-bypass-list. Failure mode is a silent no-op (cache never hit). Needs at least one Firefox and one WebKit test.

yury-s · 2026-07-15T22:45:40Z

+
+      const upstreams = new Map<string, ProxySettings | undefined>();
+      for (const project of config.projects)
+        upstreams.set(proxySettingsKey(project.project.use.proxy), project.project.use.proxy);


maybe use one global proxy per cache if needed?

yury-s · 2026-07-15T22:54:08Z

+    return undefined;
+  cacheProxyServers ??= JSON.parse(process.env.PLAYWRIGHT_TEST_CACHE_PROXY) as Record<string, string>;
+  const server = cacheProxyServers[config.proxySettingsKey((project.use as PlaywrightTestOptions).proxy)];
+  return server ? { server, bypass: '<-loopback>' } : undefined;


yury-s · 2026-07-15T23:35:20Z

+      return; // No cache yet.
+    }
+    // Discard an incompatible cache instead of mis-parsing it.
+    if (await this._onDiskVersion() !== CACHE_VERSION) {


nit: run this before reading index file content

pavelfeldman force-pushed the http-cache-proxy branch from e9ef052 to 1ff8228 Compare July 8, 2026 04:36

This comment has been minimized.

Sign in to view

Skn0tt reviewed Jul 8, 2026

View reviewed changes

dgozman reviewed Jul 9, 2026

View reviewed changes

pavelfeldman force-pushed the http-cache-proxy branch from 1ff8228 to 0d1d6c4 Compare July 9, 2026 23:31

This comment has been minimized.

Sign in to view

pavelfeldman force-pushed the http-cache-proxy branch from 0d1d6c4 to 0ebab49 Compare July 14, 2026 23:48

pavelfeldman requested a review from Copilot July 14, 2026 23:49

Copilot started reviewing on behalf of pavelfeldman July 14, 2026 23:50 View session

Copilot AI reviewed Jul 14, 2026

View reviewed changes

Comment thread packages/playwright/src/plugins/cacheProxy/server.ts

Comment thread packages/playwright/src/plugins/cacheProxy/server.ts

Comment thread packages/playwright/src/index.ts

This comment has been minimized.

Sign in to view

pavelfeldman force-pushed the http-cache-proxy branch from 0ebab49 to 9a13faa Compare July 15, 2026 21:54

pavelfeldman force-pushed the http-cache-proxy branch from 9a13faa to 12edbc0 Compare July 15, 2026 22:10

Uh oh!

Conversation

pavelfeldman commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Uh oh!

This comment has been minimized.

This comment has been minimized.

Skn0tt left a comment

Choose a reason for hiding this comment

Uh oh!

pavelfeldman commented Jul 8, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions Bot commented Jul 15, 2026

Test results for "MCP"

Uh oh!

github-actions Bot commented Jul 15, 2026

Test results for "tests 1"

Uh oh!

github-actions Bot commented Jul 15, 2026

🟢 All 5 failures are pre-existing MCP flakes — this PR is clear

Uh oh!

yury-s left a comment

Choose a reason for hiding this comment

Uh oh!

yury-s Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

yury-s Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

yury-s Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

pavelfeldman commented Jul 8, 2026 •

edited

Loading