|
| 1 | +--- |
| 2 | +name: add-settings-e2e-test |
| 3 | +description: Add or update durable Playwright coverage for Sim settings using the correct literal contract or owning spec, existing personas, guarded orchestration, semantic assertions, cleanup, and secret-safe diagnostics. Use when asked to add a settings browser test, cover a settings regression, or update settings E2E acceptance behavior. |
| 4 | +argument-hint: <settings-behavior-or-regression> |
| 5 | +--- |
| 6 | + |
| 7 | +# Add Settings E2E Test |
| 8 | + |
| 9 | +Add the smallest durable browser proof for the requested observable behavior. |
| 10 | +Do not begin by copying a nearby test: first identify which acceptance dataset |
| 11 | +or spec owns the behavior. |
| 12 | + |
| 13 | +## Read first |
| 14 | + |
| 15 | +1. `apps/sim/e2e/MAINTENANCE.md` — change type to contract/spec ownership. |
| 16 | +2. `apps/sim/e2e/README.md` — orchestrator, project boundaries, focused commands, |
| 17 | + diagnostics, and CI policy. |
| 18 | +3. `apps/sim/e2e/STABILIZATION.md` — non-waivable acceptance and security |
| 19 | + boundaries. |
| 20 | + |
| 21 | +Those files are canonical. This skill is the authoring procedure, not a second |
| 22 | +copy of their inventories or commands. If this procedure ever conflicts with a |
| 23 | +canonical document, the canonical document wins. |
| 24 | + |
| 25 | +## Procedure |
| 26 | + |
| 27 | +### 1. Define the observable contract |
| 28 | + |
| 29 | +Write down: |
| 30 | + |
| 31 | +- the user persona and resource plane (account, organization, or workspace); |
| 32 | +- the action or direct URL; |
| 33 | +- the exact observable result: path, semantic readiness, visible/hidden item, |
| 34 | + access outcome, enabled/disabled control, warning, or persisted state; |
| 35 | +- whether product behavior is intentionally changing or an implementation |
| 36 | + change must preserve the existing contract. |
| 37 | + |
| 38 | +Intended behavior changes update product code and literal acceptance |
| 39 | +expectations together. Behavior-preserving fixes add or strengthen proof without |
| 40 | +rewriting unrelated expectations. |
| 41 | + |
| 42 | +### 2. Choose the owner before writing code |
| 43 | + |
| 44 | +- **Canonical section, route, copy, readiness, or persona visibility:** update |
| 45 | + `apps/sim/e2e/settings/navigation/contracts.ts`; its specs generate cases from |
| 46 | + the literal dataset, including unauthenticated redirect cases. |
| 47 | +- **Browser Back, app Back, direct-entry fallback, or return destination:** edit |
| 48 | + `apps/sim/e2e/settings/navigation/history.spec.ts` directly. |
| 49 | +- **Direct access, plan/role/entitlement gate, or mutation availability:** update |
| 50 | + `apps/sim/e2e/settings/authorization/contracts.ts`; its access and mutation |
| 51 | + specs generate cases from that dataset. |
| 52 | + - If an existing navigation case already owns the sidebar proof, add or reuse |
| 53 | + an `existingNavigationProofs` entry and set |
| 54 | + `sidebar.existingProofId`. Do not execute the same sidebar assertion again |
| 55 | + in the authorization case. |
| 56 | +- **Unsaved-change behavior:** edit |
| 57 | + `apps/sim/e2e/settings/authorization/unsaved-changes.spec.ts` directly. |
| 58 | +- **Secrets or API key lifecycle:** edit the owning credential spec directly. |
| 59 | + Change `apps/sim/e2e/settings/credentials/contracts.ts` only when reused |
| 60 | + authorization proof IDs or boundaries change. |
| 61 | +- **People, access control, SSO, data retention, or MCP lifecycle:** edit the |
| 62 | + owning workflow spec. Change `apps/sim/e2e/settings/workflows/contracts.ts` |
| 63 | + when stable lifecycle or cross-contract proof references change. |
| 64 | +- **Persona or seeded relationship:** update |
| 65 | + `apps/sim/e2e/settings/personas.ts` and the relevant |
| 66 | + scenario/factory/integrity proof. |
| 67 | + |
| 68 | +The categories are non-exclusive. Prefer extending an existing literal row or |
| 69 | +owning workflow over adding a duplicate standalone test. |
| 70 | + |
| 71 | +### 3. Use the owning fixture wrapper |
| 72 | + |
| 73 | +- Browser-touching settings specs must not import runtime `test` or `expect` |
| 74 | + from `@playwright/test`. Type-only imports such as `Page` or `Response` are |
| 75 | + allowed. |
| 76 | +- Navigation, authorization, persona-contract, and persona-isolation browser |
| 77 | + specs use `apps/sim/e2e/fixtures/persona-test.ts`. |
| 78 | +- Credential specs use the local `credential-test.ts`; workflow specs use the |
| 79 | + local `workflow-test.ts`; authenticated or unauthenticated smoke and |
| 80 | + browser-touching harness-level specs use |
| 81 | + `apps/sim/e2e/fixtures/browser-test.ts`. |
| 82 | +- Pure dataset-only contract-integrity specs and non-browser foundation policy |
| 83 | + specs may use the Playwright test runner directly. If they begin creating a |
| 84 | + browser, context, or page, move them to the owning wrapper first. |
| 85 | +- Persona-based tests must create contexts through `contextForPersona` or an |
| 86 | + owning helper that calls it. The inherited `page` and `context` fixtures on |
| 87 | + `persona-test` are not persona-authenticated or network-guarded. |
| 88 | +- `browser-test` guards its default context. Every manual |
| 89 | + `browser.newContext()` must install `installBrowserNetworkGuard()` immediately, |
| 90 | + then close the context and assert the guard even when the test fails; follow |
| 91 | + the aggregate-cleanup pattern in authenticated smoke coverage. Do not use |
| 92 | + `browser.newPage()`: it creates an implicit unguarded context; create and guard |
| 93 | + an explicit context, then call `context.newPage()`. |
| 94 | +- Reuse the owning directory's helpers. Credential tests use their local |
| 95 | + `newPersonaPage` helper so the page is registered for failure sanitization. |
| 96 | + Wrapper selection alone does not install every context, cleanup, attachment, |
| 97 | + redaction, or artifact safeguard; bypassing the guarded fixture/helper path is |
| 98 | + a test-safety bug. |
| 99 | + |
| 100 | +### 4. Reuse the existing world |
| 101 | + |
| 102 | +- Reuse an existing persona and seeded resource when it expresses the required |
| 103 | + role, plan, entitlement, and ownership boundary. |
| 104 | +- Add a persona or scenario edge only when no existing driver can prove the |
| 105 | + contract without changing its meaning. |
| 106 | +- Use run-namespaced factories and unique resources. Do not introduce shared |
| 107 | + mutable fixture state. |
| 108 | +- Keep sensitive fixture values browser-resident through the existing helpers. |
| 109 | + The Playwright process must not receive database credentials, admin keys, |
| 110 | + persona passwords, or plaintext values that the owning spec keeps in-page. |
| 111 | + |
| 112 | +### 5. Assert semantics, not implementation details |
| 113 | + |
| 114 | +- Use accessible roles, labels, names, and visible text. |
| 115 | +- Assert exact paths and user-facing contract copy when those are the behavior |
| 116 | + under test. |
| 117 | +- Wait on semantic readiness, relevant same-origin responses, or explicit |
| 118 | + authorization state—not arbitrary sleeps or CSS classes. |
| 119 | +- For access controls, prove direct URL behavior as well as sidebar visibility. |
| 120 | +- Do not import production navigation or authorization implementations to |
| 121 | + generate expected values. |
| 122 | + |
| 123 | +### 6. Make mutations reversible |
| 124 | + |
| 125 | +- Register cleanup before the first mutation. |
| 126 | +- Create uniquely named resources and restore the exact captured baseline. |
| 127 | +- Use the production UI and same-origin APIs for behavior under test; use a |
| 128 | + trusted database probe only where the existing suite defines one. |
| 129 | +- Keep cleanup LIFO and safe after partial failure. Never weaken fixture |
| 130 | + invariants merely to make a test pass. |
| 131 | + |
| 132 | +### 7. Respect external and diagnostic boundaries |
| 133 | + |
| 134 | +- Use the existing strict loopback Stripe and MCP fakes, the mail mock-success |
| 135 | + path with provider credentials absent, and the other reviewed test |
| 136 | + boundaries. An unexpected external request must fail rather than silently |
| 137 | + egress. |
| 138 | +- Credentials, tokens, certificates, verification values, storage state, and |
| 139 | + sensitive response bodies may be used only through existing reviewed paths; |
| 140 | + never log, attach, or retain them in diagnostics. |
| 141 | +- The SSO workflow's public-certificate browser input is one reviewed example; |
| 142 | + preserve its input clearing and diagnostic suppression. |
| 143 | +- Preserve the owning spec's trace, screenshot, video, and attachment policy. |
| 144 | + New suppression requires a security reason and corresponding safety coverage. |
| 145 | + |
| 146 | +### 8. Keep unit and integrity proof aligned |
| 147 | + |
| 148 | +- Update affected production unit tests for navigation, route gates, |
| 149 | + authorization, billing, or business rules. |
| 150 | +- Update the owning `contract-integrity.spec.ts` when adding IDs, references, or |
| 151 | + a new contract axis. |
| 152 | +- Preserve stable proof IDs; do not rerun a browser case merely because another |
| 153 | + contract can reference its existing proof. |
| 154 | + |
| 155 | +### 9. Verify through the orchestrator |
| 156 | + |
| 157 | +- Run the affected unit and contract-integrity tests. |
| 158 | +- Run the focused canonical project and path documented in |
| 159 | + `apps/sim/e2e/README.md`. |
| 160 | +- Invoke only `bun run test:e2e`; never invoke raw `playwright test`. |
| 161 | +- Use `--reuse-build` and a single explicit project with `--no-deps` for local |
| 162 | + iteration when the README permits it. |
| 163 | +- Follow the retry policy in the canonical README and Playwright config. It is |
| 164 | + currently zero; do not enable retries or change worker policy in a feature |
| 165 | + test. Do not use `test.only`, unexplained skips, arbitrary browser sleeps, or |
| 166 | + test-local environment bypasses. |
| 167 | +- After focused proof passes, rely on the required complete CI suite. Run a |
| 168 | + repeated stability gate only when the change's scope or acceptance plan calls |
| 169 | + for one. |
| 170 | + |
| 171 | +## Completion checklist |
| 172 | + |
| 173 | +- [ ] Every observable facet has one clear owner, and all applicable owners were |
| 174 | + updated. |
| 175 | +- [ ] Allowed and denied outcomes exist where the feature is gated. |
| 176 | +- [ ] Locators and readiness assertions are semantic and accessible. |
| 177 | +- [ ] Mutations register cleanup first and restore the exact baseline. |
| 178 | +- [ ] No secret or external-egress boundary was weakened. |
| 179 | +- [ ] Related unit and contract-integrity tests are aligned and pass. |
| 180 | +- [ ] Focused execution used the guarded orchestrator and complied with the |
| 181 | + canonical retry policy. |
0 commit comments