Skip to content

Add explicit codecov.yml policy to unblock admin-merge workflow #3476

@MaxGhenis

Description

@MaxGhenis

Summary

codecov/patch enforces a 77.10% patch-coverage threshold derived from the base branch's current coverage. There is no codecov.yml in the repo, so the threshold is implicit and tracks master drift. Recent bug-fix PRs (e.g. #3454) failed this check despite adding regression tests for every change, because fixes that touch non-code paths (exception handlers, whitelist guards, return-None short-circuits) don't proportionally lift coverage.

What goes wrong

  • The check blocks merge via mergeStateStatus: BLOCKED even though mergeable: MERGEABLE — admins have to merge with --admin to proceed.
  • There is no visible threshold-policy document: contributors can't see what target they're shooting at without opening Codecov's UI.
  • The de-facto workflow (admin-merge on coverage failure) undermines the signal — if admin-merge is always fine, the check should not be required; if the check matters, the threshold should be published and tunable.

Suggested fix

Add a codecov.yml at repo root with an explicit policy:

coverage:
  status:
    project:
      default:
        target: auto
        threshold: 1%
    patch:
      default:
        target: 70%    # or whatever realistic floor matches the codebase
        informational: false

Options:

  • Target auto with a threshold of 1–2% — patch can drop coverage by at most 1–2% relative to base (forgiving for bug fixes that touch few lines).
  • Fixed 70% floor — easier to reason about; allows fixes that don't add tests to non-essential defensive paths.
  • Informational = true — keeps the signal visible without blocking merges; admin-merge workaround stops being necessary.

Document the choice in CONTRIBUTING.md so contributors know the target.

Severity

Medium — procedural friction that degrades to "admin-merge everything" in practice, which hides the coverage signal entirely.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions