Skip to content

ci: split unit tests from static guards, de-&&-chain checks - #2214

Merged
jung-thomas merged 3 commits into
DEVfrom
worktree-ci-split-guards
Sep 9, 2026
Merged

jung-thomas merged 3 commits into
DEVfrom
worktree-ci-split-guards

Conversation

@jung-thomas

Copy link
Copy Markdown
Contributor

Summary

Fixes the chronic CI brittleness where unit-test gates fail daily on policy-linter noise, not real behavioral failures. Developers now ignore the red gate (normalization of deviance), risking actual regressions through.

Data: Last 10 failed CI runs — 8 failed on the guard chain, only 2 on vitest. Both vitest failures were magic-number assertions (expected 15, got 14; expected 50, got 49), not product bugs.

Changes

  1. Split workflows: unit-tests.yml (vitest only) + new static-guards.yml (13 policy linters)
  2. De-&&-chained runners: npm run static-guards runs all checks, reports all failures at once
  3. Pre-push gate: scripts/git-hooks/pre-push runs guards before push (instant local feedback)

Signal restoration

  • Red "unit tests" now means a real behavioral failure (rare, worth reading)
  • Developers see all guard violations at once (no death-by-a-thousand-cuts)
  • Guards run locally first (pre-push), then again in both deploy and scheduled workflows

Testing

  • Verified npm run static-guards passes all 13 checks locally (14.8s)
  • Verified npm run setup:git-hooks installs pre-push hook
  • Both commands work on Windows + Linux (shell detection)

Follow-up (optional, after merge)

Items 4–5 from the original plan (delete exact-count assertions, add auto-fix to guards) are separate PRs; they require curating which specific assertions are worth hardening vs. softening.

…push hook

## Changes

1. **Split CI workflows:**
   - `unit-tests.yml` now runs only vitest (behavioral tests)
   - New `static-guards.yml` runs all 13 guard checks (build collisions, icon imports,
     srv-qa parity, slug-lookup canonicalization, CSRF, GraphQL breaking-changes, etc.)

   This halves alert fatigue: a red "unit tests" now signals a real behavioral failure
   (rare, worth reading), not a policy-linter violation that's noisy by design.

2. **De-`&&`-chained guard runner:**
   - Created `scripts/run-static-guards.ts` that runs all checks to completion
     before reporting, instead of failing on the first violation
   - Reports all failures at once (eliminates "fix one, next one appears" death-by-cuts)
   - Summary at the end shows pass/fail count and which guards failed

3. **Pre-push git hook:**
   - Added `scripts/git-hooks/pre-push` wired via `npm run setup:git-hooks`
   - Runs `npm run static-guards` locally before push (deterministic, instant feedback)
   - Skipped in non-interactive environments (CI, deploy scripts)

## Why

Last 10 failed CI runs: 8 failed on guard chain (`postbuild:apps`), only 2 on real tests.
Guard violations (missing magic comments, new service count off by one, etc.) are high-frequency
noise that drowns out signal. Team has started ignoring CI, risking real regressions.

Splitting unit tests (vitest only) from guards (policy linters) means:
- PR gate is fast + behavioral (genuine failures)
- Guards run everywhere (locally via pre-push, in deploy CI) with instant feedback
- Developer sees all violations at once, not one-by-one
- db-flags.test.js: replace toBe(14) with managedFlagKeys().length-derived
  coverage checks (+ a floor for the registry-membership test) so adding a
  feature flag no longer breaks the suite.
- seed-sapphire-2026-concepts.test.js: exact-count -> floor; shape,
  uniqueness, headline coverage already asserted separately.
- run-static-guards.ts: --fix flag routes to fixable guards + prints hint.
- check-icon-imports.ts / check-kg-meta-formatters-mirror.ts: --fix mode
  applies the mechanical, zero-judgment fix (add side-effect import / copy
  authoritative mirror).
@jung-thomas
jung-thomas merged commit f7791b9 into DEV Sep 9, 2026
6 checks passed
@jung-thomas
jung-thomas deleted the worktree-ci-split-guards branch September 9, 2026 01:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant