diff --git a/.github/linters/kernel-findings.rego b/.github/linters/kernel-findings.rego new file mode 100644 index 000000000..962994b59 --- /dev/null +++ b/.github/linters/kernel-findings.rego @@ -0,0 +1,11 @@ +package trivy + +import rego.v1 + +default ignore := false + +# Kernel packages never execute inside a container - only the host's kernel runs - +# so vulnerabilities against them can't be exploited here, regardless of CVE ID. +ignore if { + startswith(input.PkgName, "linux-") +} diff --git a/.github/workflows/vulnerability-scan.yml b/.github/workflows/vulnerability-scan.yml index 58bb586e4..9e51b4909 100644 --- a/.github/workflows/vulnerability-scan.yml +++ b/.github/workflows/vulnerability-scan.yml @@ -14,18 +14,37 @@ jobs: runs-on: ubuntu-latest strategy: matrix: - flavor: [cpp, docs, embedded-cpp, embedded-rust, rust] + include: + - flavor: cpp + dockerfile: .devcontainer/cpp/Dockerfile + - flavor: docs + dockerfile: .devcontainer/docs/Dockerfile + - flavor: embedded-cpp + dockerfile: .devcontainer/cpp/Dockerfile + - flavor: embedded-rust + dockerfile: .devcontainer/rust/Dockerfile + - flavor: rust + dockerfile: .devcontainer/rust/Dockerfile permissions: + contents: read # is needed by actions/checkout security-events: write # is needed by github/codeql-action/upload-sarif to upload sarif files steps: - uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0 with: egress-policy: audit + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + persist-credentials: false - uses: crazy-max/ghaction-container-scan@ffcba8deb5cb3531954cbedbbcd0179d79edf510 # v4.1.0 id: scan + env: + # No direct input for this; trivy binds every CLI flag to a TRIVY_* env var + TRIVY_IGNORE_POLICY: .github/linters/kernel-findings.rego with: + # Unfixed findings (no upstream patch yet) aren't actionable and would just re-appear next scan + ignore_unfixed: true image: ghcr.io/${{ github.repository }}-${{ matrix.flavor }}:latest - dockerfile: .devcontainer/Dockerfile + dockerfile: ${{ matrix.dockerfile }} - uses: github/codeql-action/upload-sarif@db488ddef3bf6cb639b32c2e9a7c0a7ea8271d28 # v4.37.8 if: steps.scan.outputs.sarif != '' with: diff --git a/.mega-linter.yml b/.mega-linter.yml index c404290d6..0c92239af 100644 --- a/.mega-linter.yml +++ b/.mega-linter.yml @@ -28,4 +28,4 @@ FILTER_REGEX_EXCLUDE: (CHANGELOG.md|package-lock.json) # dynamically based upon context (e.g. installed extensions) # hence the exclusion JSON_V8R_FILTER_REGEX_EXCLUDE: (\.vscode) -REPOSITORY_TRIVY_ARGUMENTS: --ignorefile .github/linters/.trivyignore.yml +REPOSITORY_TRIVY_ARGUMENTS: --ignorefile .github/linters/.trivyignore.yml --ignore-unfixed --ignore-policy .github/linters/kernel-findings.rego diff --git a/docs/adr/0001-vulnerability-scanning-noise-reduction.md b/docs/adr/0001-vulnerability-scanning-noise-reduction.md new file mode 100644 index 000000000..1646ec58a --- /dev/null +++ b/docs/adr/0001-vulnerability-scanning-noise-reduction.md @@ -0,0 +1,77 @@ +# Reduce Vulnerability Scanning Noise Instead of Requiring It to Block Merges + +* Status: accepted +* Deciders: Development Team +* Date: 2026-09-05 + +## Context and Problem Statement + +Container images are scanned for vulnerabilities by two independent Trivy invocations: [vulnerability-scan.yml](../../.github/workflows/vulnerability-scan.yml) (`crazy-max/ghaction-container-scan`, scanning each built flavor image daily) and MegaLinter's `REPOSITORY_TRIVY` linter (scanning the repository filesystem on every pull request), alongside Grype, Checkov, Hadolint, secretlint, and betterleaks, all uploading SARIF to GitHub code scanning. + +An audit of the repository's code-scanning alerts found 9,590 open Trivy alerts (169 unique CVEs after de-duplicating across flavors), the oldest dating back to 2024-05-15 — over two years unaddressed. Two structural causes were identified: + +* The overwhelming majority of findings have no upstream fix available yet (unfixed OS-package CVEs), so they are not currently actionable, yet they were reported identically to fixable ones. +* A visible share of the remaining findings were kernel CVEs (e.g. `CVE-2026-74394`, `CVE-2026-72287` against `linux-libc-dev`) that cannot be exploited inside a container, since containers share the host kernel and never execute their own. + +Neither of these two categories can be resolved by changing our own package pins, and manually triaging or dismissing thousands of individual alerts is not a sustainable use of maintainer time. At the same time, supply-chain security is a top priority for this project, so any noise-reduction approach must not risk hiding a real, fixable vulnerability. + +## Decision Drivers + +* **Actionability:** findings should represent something a maintainer can actually act on. +* **Low ongoing effort:** the approach must not require manually re-triaging a large, ever-growing backlog. +* **No loss of real signal:** suppression must be based on documented, auditable criteria — never a blanket "ignore everything" or an undocumented manual dismissal. +* **Reversibility:** anyone should be able to see exactly what is excluded and why, and reconsider it later. + +## Considered Options + +* **Leave scanning output as-is:** keep reporting every finding, unfiltered. +* **Disable or stop running one or more scanners:** remove the source of the noise entirely. +* **Filter to actionable, in-scope findings:** use Trivy's built-in filtering (`--ignore-unfixed`, a Rego `--ignore-policy`) to suppress specific, documented categories of non-actionable findings, while leaving everything else visible. + +## Decision Outcome + +Chosen option: **filter to actionable, in-scope findings**, applied identically to both Trivy invocations ([.mega-linter.yml](../../.mega-linter.yml)'s `REPOSITORY_TRIVY_ARGUMENTS`, and `vulnerability-scan.yml`'s `crazy-max/ghaction-container-scan` step): + +* `--ignore-unfixed` (`ignore_unfixed: true` on the container-scan action) — suppress vulnerabilities with no upstream fix available. Because this is a status-based filter, not an ID-based one, it requires no maintenance list and automatically starts reporting a vulnerability again the moment a fix becomes available. +* A package-class Rego ignore-policy, [.github/linters/kernel-findings.rego](../../.github/linters/kernel-findings.rego), suppressing findings whose `PkgName` starts with `linux-` — kernel/kernel-header packages that cannot execute inside a container regardless of which CVE ID is involved. This was implemented as a Rego policy rather than `.trivyignore`/`.trivyignore.yaml` because those formats only suppress specific CVE IDs one at a time; excluding an entire package class regardless of ID requires Trivy's `--ignore-policy` mechanism. + +This is a deliberate **noise-reduction** decision, not a merge-gating one: the branch-protection ruleset's required `code_scanning` check still only names `CodeQL`, `SonarCloud`, and `zizmor` — Trivy/Grype/Checkov findings remain advisory-only for now. Whether to promote (a filtered, realistic) Trivy signal to a required check is an open follow-up, deferred until the filtered alert volume has been observed for a period. + +### Positive Consequences + +* The bulk of the 9,590-alert backlog is expected to self-clear: GitHub automatically dismisses code-scanning alerts a tool no longer detects on a subsequent analysis, so no manual bulk-dismissal is needed. +* Remaining open alerts should be overwhelmingly ones with an available fix and plausible in-container exploitability — a list small enough to actually triage. +* Both suppression criteria are version-controlled, reviewed via pull request, and self-documenting (unlike ad hoc manual alert dismissals in the GitHub UI, which carry no rationale). + +### Negative Consequences + +* `--ignore-unfixed` means genuinely-real but currently-unfixable vulnerabilities are no longer visible in the default scan output (the full picture remains available on demand via `trivy image --ignore-status ""` / `--show-suppressed`, just not surfaced by default). +* The kernel-package exclusion is a name-prefix heuristic (`PkgName` starting with `linux-`). It is possible, though considered unlikely, that a package matching this prefix does execute meaningful code in some flavor's container — this trade-off is accepted and can be revisited if such a case is found. +* Vulnerability-scan findings still do not block merges even after filtering; a genuinely fixable high-severity CVE introduced by a dependency bump could still be merged. This is an explicitly deferred decision, not a resolved one. + +## Pros and Cons of the Options + +### Leave scanning output as-is + +* ✅ Good, because it requires no change and shows every possible finding. +* ❌ Bad, because it produces the exact alert-fatigue backlog this decision responds to. +* ❌ Bad, because it does not distinguish actionable from non-actionable findings. + +### Disable or stop running one or more scanners + +* ✅ Good, because it removes the noise immediately with minimal configuration. +* ❌ Bad, because it also removes visibility into genuinely fixable, exploitable vulnerabilities. +* ❌ Bad, because it conflicts with supply-chain security being a top priority for this project. + +### Filter to actionable, in-scope findings + +* ✅ Good, because every exclusion is scoped, documented, and reviewable in version control. +* ✅ Good, because it self-corrects as fixes become available (`--ignore-unfixed` is status-based, not a static list). +* ✅ Good, because it does not touch severity or the set of scanners run — only removes findings that are provably not actionable or not exploitable in context. +* ❌ Bad, because it still requires judgment calls (e.g. the kernel-package prefix) that could, in principle, be too broad. + +## Links + +* [Trivy: Filtering](https://trivy.dev/latest/docs/configuration/filtering/) +* [Trivy: Filtering by Rego](https://trivy.dev/latest/docs/configuration/filtering/#by-rego) +* [Open Policy Agent / Rego](https://www.openpolicyagent.org/docs/latest/policy-language/)