Skip to content

Add ratcheting coverage-floor gate to CI - #649

Merged
fpigeonjr merged 3 commits into
masterfrom
gh-627-add-ratcheting-coverage-floor-gate-to-ci
Aug 27, 2026
Merged

Add ratcheting coverage-floor gate to CI#649
fpigeonjr merged 3 commits into
masterfrom
gh-627-add-ratcheting-coverage-floor-gate-to-ci

Conversation

@fpigeonjr

Copy link
Copy Markdown
Contributor

Description

Adds a ratcheting coverage-floor gate to CI, mirroring the mechanism built for ngx-uswds (#237#259).

  • coverage-floor.json (new) — records the four coverage floors (statements, branches, functions, lines), seeded to { statements: 53, branches: 39, functions: 50, lines: 52 }: the coverage measured against test-app's Vitest harness after the Karma→Vitest migration (Migrate test-app runner from Karma/Jasmine to Vitest (Analog) #626) landed, not the pre-migration Karma numbers, since coverage-measurement providers can differ slightly even when both are Istanbul-based.
  • scripts/check-coverage.mjs (new) — reads floors from coverage-floor.json (not hardcoded) and fails with a clear per-metric message (✖ statements 40.00% (floor 53%)) if measured coverage — from test-app/coverage/coverage-summary.json — drops below any floor. --bump rewrites the floor file up to current measured coverage, ratchet-only (never lowers an existing floor); treats a missing/malformed floor as 0 so a corrupt file can never poison the ratchet with NaN.
  • scripts/check-coverage.test.mjs (new) — 7 node --test cases (TDD'd incrementally) covering: pass at/above floor, per-metric failure naming, --bump raising floors, --bump never lowering (ratchet-only), malformed floor values, and a missing coverage summary.
  • test-app/vitest.config.mts — added the json-summary reporter (alongside the existing text-summary/lcovonly) so coverage-summary.json actually exists for the gate to read; this was previously missing.
  • package.json — added coverage:check and coverage:bump scripts.
  • .github/workflows/ci.yml — wired coverage:check in as a required step immediately after the Vitest test run.
  • AGENTS.md (new) — documents the ratchet policy for future contributors/agents: feature/spec-adding PRs must not edit coverage-floor.json — only coverage:bump should, landed as its own deliberate commit, so parallel coverage PRs don't collide on a shared file (the exact problem PR for filter pill changes #259 fixed on ngx-uswds).

Per the QASP "Tested" standard (ADR-0010), this floor is a working gate today; it closes out the epic (#576) once raised to the 80% new-code floor / 90% target. This issue only stands up the mechanism, not the final numbers.

Motivation and Context

Closes #627

Type of Change (Select One and Apply Label)

  • Bug fix (non-breaking change which fixes an issue) → Apply bugfix label
  • New feature (non-breaking change which adds functionality) → Apply enhancement label
  • Breaking change (fix or feature that would cause existing functionality to change) → Apply breaking label
  • Documentation / configuration update → Apply maintenance label

How to Test

  1. npm ci && npm ci --prefix test-app
  2. npm --prefix test-app test — runs the full spec suite via Vitest with coverage; now also emits test-app/coverage/coverage-summary.json.
  3. npm run coverage:check — runs the gate against the seeded floors; should print ✓ Coverage gate passed.
  4. node --test scripts/check-coverage.test.mjs — 7 unit tests for the gate script itself.
  5. Regression check: temporarily lower a value in coverage-floor.json above current measured coverage (e.g. bump statements to 99), re-run npm run coverage:check — it should fail and name the metric with expected vs. actual percentages. Revert the temp change.
  6. CI: confirm the CI workflow runs the new "Enforce coverage-floor gate" step after "Run test-app harness with coverage" and passes on this branch.

Expected result: Test Files 106 passed (106) / Tests 667 passed (667), coverage summary written, and the coverage gate passes at statements 53.56% / branches 39.77% / functions 50.31% / lines 52.84% against the seeded floors.

Screenshots (if appropriate)

N/A — CI/configuration change only.

Checklist

  • Branch name follows convention (e.g. gh-<number>-<slug>)
  • PR title starts with a verb in the imperative mood
  • I have self-reviewed my own code
  • format:check passes (npm run format:check)
  • lint passes (npm run lint)
  • build passes (cd test-app && npm run build)
  • Tests pass and coverage is reported (cd test-app && npm test)
  • If this change requires a documentation update, I have updated it accordingly (AGENTS.md)
  • If there are dependent changes, they have been merged and published in downstream modules

Mirrors the mechanism built for ngx-uswds (#237 -> #259):
- coverage-floor.json holds the four coverage floors (statements,
  branches, functions, lines), seeded to coverage measured after the
  Vitest migration (#626) landed.
- scripts/check-coverage.mjs reads floors from that file and fails
  with a clear per-metric message if measured coverage drops below
  any floor. --bump rewrites the floor file up to current measured
  coverage, ratchet-only (never lowers a floor).
- test-app/vitest.config.mts now emits the json-summary reporter so
  coverage-summary.json exists for the gate to read.
- Wired into CI (.github/workflows/ci.yml) as a required step after
  the Vitest test run.
- AGENTS.md documents the ratchet policy: feature/spec PRs must not
  edit coverage-floor.json; raising it is a separate, deliberate
  coverage:bump commit.

Closes #627
@fpigeonjr fpigeonjr added the maintenance Repo maintenance / tooling label Aug 27, 2026
@fpigeonjr fpigeonjr self-assigned this Aug 27, 2026
@fpigeonjr
fpigeonjr requested a balanced review from Copilot August 27, 2026 21:02

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a CI coverage-floor gate and supporting ratchet tooling.

Changes:

  • Generates JSON coverage summaries and enforces committed coverage floors.
  • Adds ratchet bump tooling and unit tests.
  • Documents and integrates the coverage policy into CI.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
.github/workflows/ci.yml Runs the coverage gate in CI.
AGENTS.md Documents coverage and lint ratchets.
coverage-floor.json Defines initial coverage floors.
package.json Adds coverage check and bump commands.
scripts/check-coverage.mjs Implements coverage enforcement and floor bumping.
scripts/check-coverage.test.mjs Tests the coverage gate CLI.
test-app/vitest.config.mts Enables JSON summary reporting.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/workflows/ci.yml
Comment thread scripts/check-coverage.mjs Outdated
Beyond the coverage-gate policy, document:
- the raw-source dual-workspace repo shape (root library vs. test-app
  tooling workspace) and why specs are colocated under root src/
- the frozen consumer-deep-imports.json publish contract and its
  breaking-change trap when restructuring src/ui-kit
- single-spec run command, passWithNoTests rationale, and that the
  gate scripts' own tests (check-coverage.test.mjs,
  check-lint-baseline.test.mjs, check-baseline-not-increased.test.mjs)
  are not currently wired into any CI workflow
- the check-baseline-not-increased.mjs guard against hand-raising the
  ESLint baseline
- a map of what each CI workflow does
- the dual tsconfig split and test-setup.ts jsdom shims
…verage

- scripts/check-coverage-floor-not-decreased.mjs (new) + tests: CI-only
  guard comparing coverage-floor.json on the PR branch against the base
  branch, failing if any metric's floor decreased. Mirrors the existing
  ESLint baseline guard (check-baseline-not-increased.mjs /
  .github/workflows/lint.yml). Wired into .github/workflows/ci.yml as a
  PR-only step, requiring fetch-depth: 0 on checkout to resolve the base
  SHA's blob.
- scripts/check-coverage.mjs: --bump now preserves istanbul's fractional
  pct value (e.g. 53.56) instead of Math.floor-ing it, so sub-1% gains are
  no longer silently discarded from the ratchet. Re-seeded
  coverage-floor.json to the fractional measured values.
- scripts/check-coverage.test.mjs: added a fractional-coverage bump test.
@fpigeonjr
fpigeonjr marked this pull request as ready for review August 27, 2026 21:22
@fpigeonjr
fpigeonjr requested review from a team and christyhermansen as code owners August 27, 2026 21:22
@fpigeonjr
fpigeonjr merged commit d0ca840 into master Aug 27, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

maintenance Repo maintenance / tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add ratcheting coverage-floor gate to CI

3 participants