From 164952ea23586193c80c885192c82546dc97b7a7 Mon Sep 17 00:00:00 2001 From: adibarra <93070681+adibarra@users.noreply.github.com> Date: Wed, 5 Aug 2026 12:46:19 -0500 Subject: [PATCH] chore: gate full e2e suite in ci MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 中文：将完整 E2E 套件交给 CI 门禁。\n\n默认 bun run test:e2e 运行 83 个 smoke 测试，bun run test:e2e:full 保留完整本地套件。所有 unit tests 和现有 E2E specs 保持不变。 --- AGENTS.md | 13 +++++++------ docs/adding-entities.md | 2 +- docs/testing.md | 6 +++--- package.json | 1 + packages/app/package.json | 8 ++++++-- 5 files changed, 18 insertions(+), 12 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 1bedca93..31712145 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -34,7 +34,8 @@ bun run lint:fix # Auto-fix lint issues bun run fmt # Format check with oxfmt bun run fmt:fix # Auto-fix formatting bun run test:unit # Vitest unit tests -bun run test:e2e # Cypress E2E tests +bun run test:e2e # Cypress smoke suite, the default local check +bun run test:e2e:full # Full Cypress suite, normally covered by CI ``` ## Monorepo Structure @@ -112,10 +113,10 @@ See [Testing](./docs/testing.md) for full requirements, quality standards, and p ### E2E Runtime and PR Workflow -- Prefer focused Cypress specs while iterating, and create the draft PR before starting a full local E2E run so CI and review setup can proceed in parallel. -- A warm local `bun run test:e2e` typically takes about **4–6 minutes** because the component and integration suites run sequentially on one machine; cold dependency/browser setup can take longer. -- GitHub Actions is usually faster in wall-clock time: integration specs run across **four shards per browser** for Chrome and Firefox (eight parallel E2E jobs), while component tests run in a separate job. Recent successful workflows complete in roughly **3–5 minutes**. -- Local targeted tests remain the fastest feedback loop. The sharded GitHub run is the authoritative broad browser check; do not spend time emulating all CI shards serially before opening the PR. +- Prefer `bun run test:e2e` while iterating. It runs the local smoke suite and avoids blocking development on the full browser matrix. +- Use `bun run test:e2e:full` only when a local full-suite run is useful. The merge gate runs the full suite in GitHub Actions. +- A warm local full E2E run typically takes about **4–6 minutes** because the component and integration suites run sequentially on one machine; cold dependency/browser setup can take longer. +- GitHub Actions runs integration specs across **four shards per browser** for Chrome and Firefox (eight parallel E2E jobs), while component tests run in a separate job. Recent successful workflows complete in roughly **3–5 minutes**. ## Analytics Requirement @@ -296,7 +297,7 @@ Detailed design rationale (the "why" and "how", not the "what") lives in [docs/] Three jobs: a lightweight Haiku **`route`** classifier runs on any `@claude` mention in an issue/comment and emits a `profile`; its output gates **`implement`** or **`review`**. (The `review` job also triggers directly on PR open/sync, with no comment to route.) - `@claude ` — `route` picks a **profile** (`ui` / `code` / `docs` / `question` / `review`) and, for implement profiles, a browser (`playwright` / `chrome` / `none`). - - **implement** job (`ui` / `code` / `docs` / `question`): provisions only what's needed — dev server, Playwright browser, and Cypress binary install **on demand** only for browser/UI work, so docs/DB/backend/question tasks stay fast. `ui` gets full browser verification (render real data, check the `?unofficialrun=` overlay, add `track()` + tests, pass `bun run test:e2e`); the rest get scoped checks. Creates `claude/issue-{N}-*` branches and can push. + - **implement** job (`ui` / `code` / `docs` / `question`): provisions only what's needed — dev server, Playwright browser, and Cypress binary install **on demand** only for browser/UI work, so docs/DB/backend/question tasks stay fast. `ui` gets focused browser verification plus the `?unofficialrun=` overlay, then passes `bun run test:e2e`; the full suite remains covered by the merge checks. Creates `claude/issue-{N}-*` branches and can push. - **review** job (`review` profile, or any PR open/sync): a **read-only**, **verifying** review. It checks out the PR head, starts a local dev server backed by the real read-only DB, and uses the **Playwright MCP** on `http://localhost:3000` to confirm the changed UI actually works (renders real data, interactions behave, no console errors). It does **not** re-run the test suite — `typecheck`/`lint`/`test:unit` and the fixtures-based e2e are already covered by the dedicated `tests-*`/`lint` workflows; the review reads their status and folds failures into the review as 🔴 BLOCKING — plus the static diff review (bugs, security, missing tests). Never edits or pushes. A review-phrased ask in **any** wording (e.g. "@claude take a look at this PR") routes here, not just the exact `@claude review`. Prompt: `.github/claude/review-prompt.md`. - **Explicit overrides** (skip the classifier): `@claude review` → review; `@claude chrome` → Chrome DevTools MCP; `@claude frontend` → full Playwright + dev server; `@claude general` (or `lite`) → lean no-browser. If the router guesses wrong, re-run with the override. - `implement` and `review` share a `claude-` concurrency group, so reviews and implementation on the same PR serialize instead of clobbering each other. diff --git a/docs/adding-entities.md b/docs/adding-entities.md index 1feca1a7..d26abee5 100644 --- a/docs/adding-entities.md +++ b/docs/adding-entities.md @@ -168,7 +168,7 @@ From the model name, derive (MiniMax M3 shown as the worked example): ### Verify -`bun run typecheck && bun run lint && bun run fmt && bun run test:unit`, then `rg` for the old slug to confirm only the intentional hidden preset + blog links remain. Final gate: `bun run test:e2e` and a manual `bun run dev` check that the banner/modal/preset read `DISPLAY` and `/inference?preset=SLUG-launch` renders data. +`bun run typecheck && bun run lint && bun run fmt && bun run test:unit`, then `rg` for the old slug to confirm only the intentional hidden preset + blog links remain. Final gate: `bun run test:e2e:full` and a manual `bun run dev` check that the banner/modal/preset read `DISPLAY` and `/inference?preset=SLUG-launch` renders data. --- diff --git a/docs/testing.md b/docs/testing.md index 78eb076b..b93cbef5 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -12,7 +12,7 @@ Enforced by `@pr-claude` — missing/low-quality tests are flagged 🔴 BLOCKING 1. New utility functions → colocated unit test 2. New UI features → E2E test in `cypress/e2e/.cy.ts` 3. Bug fixes → regression test reproducing the bug -4. Run `bun run test:unit` and `bun run test:e2e` before considering task complete +4. Run `bun run test:unit` and the local smoke suite, `bun run test:e2e`, before considering a task complete. The full E2E suite runs in CI and is available locally as `bun run test:e2e:full`. ## Pre-commit Checklist @@ -25,9 +25,9 @@ bun run test:e2e ## Runtime and CI Sharding -A warm local `bun run test:e2e` typically takes about **4–6 minutes** because Cypress component tests and integration tests run sequentially on one machine. Cold dependency or browser setup can take longer, so use focused `--spec` runs while iterating and open the draft PR before starting the full local suite. +The local `bun run test:e2e` command is a smoke suite with 83 Cypress tests across the core page, chart, overlay, localization, and component paths. It is the default agent and developer check and is intended to stay under one minute once the app is running. -GitHub Actions runs integration specs across **four shards per browser** for Chrome and Firefox, for eight parallel E2E jobs, with component tests in a separate job. Recent successful workflows finish in roughly **3–5 minutes** wall-clock. The sharded CI run is the authoritative broad browser check and is substantially faster than reproducing both browsers and every shard serially on a local machine. +The complete suite is `bun run test:e2e:full`. It runs all Cypress component and integration specs. GitHub Actions runs that same coverage as one component job plus four integration shards per browser, Chrome and Firefox. The CI workflow is the merge gate for the full E2E suite. ## Quality Standards diff --git a/package.json b/package.json index d4731b6b..55f56d8f 100644 --- a/package.json +++ b/package.json @@ -26,6 +26,7 @@ "fmt:fix": "oxfmt --write --no-error-on-unmatched-pattern", "test": "bun run --cwd packages/app test", "test:e2e": "bun run --cwd packages/app test:e2e", + "test:e2e:full": "bun run --cwd packages/app test:e2e:full", "test:e2e:component": "bun run --cwd packages/app test:e2e:component", "test:e2e:integration": "bun run --cwd packages/app test:e2e:integration", "test:unit": "bun scripts/run-workspace-script.ts test:unit", diff --git a/packages/app/package.json b/packages/app/package.json index 77293767..d0a93e2d 100644 --- a/packages/app/package.json +++ b/packages/app/package.json @@ -16,8 +16,12 @@ "build": "bun --env-file=../../.env next build --turbopack", "start": "bun --env-file=../../.env next start", "preview": "bun run build && bun run start --port 3001", - "test": "bun run test:unit && bun run test:e2e", - "test:e2e": "cypress run --component && cypress run", + "test": "bun run test:unit && bun run test:e2e:full", + "test:e2e": "bun run test:e2e:quick", + "test:e2e:quick": "bun run test:e2e:quick:component && bun run test:e2e:quick:integration", + "test:e2e:quick:component": "cypress run --component --spec cypress/component/chart-buttons.cy.tsx,cypress/component/scatter-graph.cy.tsx", + "test:e2e:quick:integration": "cypress run --spec cypress/e2e/sanity.cy.ts,cypress/e2e/inference-chart.cy.ts,cypress/e2e/evaluation-chart.cy.ts,cypress/e2e/reliability-chart.cy.ts,cypress/e2e/zh-pages.cy.ts,cypress/e2e/csv-export-overlay.cy.ts,cypress/e2e/overlay-optimal-only.cy.ts", + "test:e2e:full": "cypress run --component && cypress run", "test:e2e:component": "cypress run --component", "test:e2e:integration": "cypress run", "test:unit": "bun --env-file=../../.env vitest run",