Skip to content

ci: SARIF inline static analysis with early-fail lint + isolated spotbugs#1395

Closed
joaodinissf wants to merge 4 commits into
dsldevkit:masterfrom
joaodinissf:ci/sarif-inline-static-analysis
Closed

ci: SARIF inline static analysis with early-fail lint + isolated spotbugs#1395
joaodinissf wants to merge 4 commits into
dsldevkit:masterfrom
joaodinissf:ci/sarif-inline-static-analysis

Conversation

@joaodinissf
Copy link
Copy Markdown
Collaborator

Draft — for measurement. Reshapes static-analysis CI for fast, early PMD/Checkstyle feedback with inline SARIF display on all three tools, dropping the custom Python annotator. Pushed to read real full-reactor job times + confirm inline rendering before finalizing.

Shape (verify.yml, 4 parallel jobs)

  • lintcompile + pmd:pmd + checkstyle:checkstyle (SARIF) + pmd:cpd-check, -T 2C, --fail-never; gates by counting the merged SARIF (+ CPD grep). Goes red in minutes on a violation, on its own check.
  • spotbugscompile + spotbugs:spotbugs (SARIF), -Xmx4g, own parallel lane (the slow analysis).
  • maven-verify — build + tests only; the redundant pmd/checkstyle/spotbugs goals are dropped (now owned by lint/spotbugs).
  • line-endings — unchanged.

All three emit SARIF 2.1.0, merged per tool and uploaded to Code Scanning (security-events: write) for inline annotations + the Security tab.

Why count-gate, not :check

The pmd:check/spotbugs:check goals @Execute-fork a second analysis and can't emit SARIF; run cheaply they lose the classpath and false-positive. We run each report goal once (full-reactor compile → correct + SARIF) and gate by counting. Full rationale, tables, and fidelity caveats: docs/ci-static-analysis-design.md.

Honest framing

Optimizes feedback latency (lint red in ~minutes vs ~14), not total wall-clock (the build/spotbugs floor is ~unchanged), and increases total compute (compile runs per parallel job). Measurement protocol: docs/ci-measurement-protocol.md — real medians TBD.

Note for merge

Required checks change: pmd + checkstylelint + spotbugs (keep line-endings, maven-verify). Commits are intentionally granular; will squash before merge.

🤖 Generated with Claude Code

joaodinissf and others added 3 commits May 30, 2026 22:05
Restructure verify.yml static analysis around early-fail + inline display:

  - lint job: compile + pmd:pmd + checkstyle:checkstyle (SARIF renderers) +
    pmd:cpd-check, -T 2C, --fail-never. Gates by counting the merged SARIF
    (PMD/Checkstyle) + grepping cpd.xml. Goes red in minutes on a violation,
    independent of the build.
  - spotbugs job: compile + spotbugs:spotbugs (SARIF), -Xmx4g, own parallel
    lane (the slow analysis). Same count-gate.
  - maven-verify: drop the redundant checkstyle:check/pmd:*/spotbugs:check
    goals (now owned by lint/spotbugs); build + tests only.
  - SARIF (2.1.0) merged per tool and uploaded to Code Scanning
    (permissions: security-events: write) for inline annotations on all three.

compile runs in-invocation (PMD type-resolving rules need the aux-classpath)
and full-reactor (a -pl subset false-positives). Count-gate, not :check: the
:check goals @Execute-fork a second analysis and can't emit SARIF; see
docs/ci-static-analysis-design.md. Drops the custom Python annotator.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ate)

Code-verified rationale for verify.yml's shape: report-vs-check goal table
(@execute fork behavior per plugin version), Maven reactor failure-flag table,
why report+--fail-never+jq-count is the chosen gate, count-gate==:check
fidelity + config-drift caveats, and the compile-must-be-full-reactor rule.
Referenced from the PR description.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Trustworthy-measurement methodology (N=15-20 dispatch runs in a tight window,
median+IQR, the observed noise floor + signal threshold, failure-latency metric,
pre-flight mirror-health check). Records why the 2026-05 experiment numbers are
void and why the local p2 mirror / tycho.mode=maven are not timing levers.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-advanced-security
Copy link
Copy Markdown

You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool.

What Enabling Code Scanning Means:

  • The 'Security' tab will display more code scanning analysis results (e.g., for the default branch).
  • Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results.
  • You will be able to see the analysis results for the pull request's branch on this overview once the scans have completed and the checks have passed.

For more information about GitHub Code Scanning, check out the documentation.

1 similar comment
@github-advanced-security
Copy link
Copy Markdown

You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool.

What Enabling Code Scanning Means:

  • The 'Security' tab will display more code scanning analysis results (e.g., for the default branch).
  • Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results.
  • You will be able to see the analysis results for the pull request's branch on this overview once the scans have completed and the checks have passed.

For more information about GitHub Code Scanning, check out the documentation.

…GE, revert)

Trips Checkstyle EmptyStatement, PMD SystemPrintln, SpotBugs null-deref to
confirm each renders inline in Code Scanning and each gate goes red. Reverted
immediately after observation.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

/** Trips Checkstyle (EmptyStatement), PMD (SystemPrintln), SpotBugs (null deref). */
public void trigger() {
; // Checkstyle: EmptyStatement
public void trigger() {
; // Checkstyle: EmptyStatement
final String s = null;
System.out.println(s.length()); // PMD: SystemPrintln ; SpotBugs: NP_ALWAYS_NULL

/** Trips Checkstyle (EmptyStatement), PMD (SystemPrintln), SpotBugs (null deref). */
public void trigger() {
; // Checkstyle: EmptyStatement
@joaodinissf
Copy link
Copy Markdown
Collaborator Author

Closing in favor of #1396 — the same shape as a clean single commit (no planted-violation probe, polished description). Validation history lives here.

Left by Claude at João's request.

@joaodinissf joaodinissf deleted the ci/sarif-inline-static-analysis branch May 30, 2026 21:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants