Skip to content

ci(benchmarks): install the modules the JMH job measures - #578

Merged
DemchaAV merged 1 commit into
developfrom
ci/jmh-install-benchmark-deps
Aug 17, 2026
Merged

ci(benchmarks): install the modules the JMH job measures#578
DemchaAV merged 1 commit into
developfrom
ci/jmh-install-benchmark-deps

Conversation

@DemchaAV

Copy link
Copy Markdown
Owner

Why

The weekly JMH run never reaches a benchmark. It dies in Build the JMH runner jar,
resolving a dependency that Maven Central does not have and never will:

Could not find artifact io.github.demchaav:graph-compose-templates:jar:tests:2.2.0 in central

benchmarks/pom.xml depends on the tests-classifier jar of graph-compose-templates
CanonicalBenchmarkSupport builds its fixtures from CvDataFixtures,
InvoiceDataFixtures and ProposalDataFixtures, which live in the templates test
scope because a fixture producing template data cannot sit in the core test scope
without making core depend on templates. That jar is a local build aid: the templates
release profile unbinds attach-test-jar (phase none) so it is never deployed.

The job installed :graph-compose-core and left Maven to resolve the rest, so the only
version of that jar it could ever find is one a source build put in the local
repository. Five of its last six weekly runs failed this way, across four versions —
2.0.0, 2.1.0, 2.1.1 and 2.2.0 — each looking for the jar under the version the release
before it had just published
(run 32004336307).
render-pdf is resolved the same way and survives only because it is published.

Nothing on a pull request could catch it: the workflow runs on a schedule from the
default branch and on manual dispatch, so the job goes red on a Monday, far from the
change that broke it.

What changed

  • benchmarks-jmh.yml installs graph-compose-render-pdf and graph-compose-templates
    from source between the core install and the runner-jar build — the same sequence
    perf-smoke and benchmark-diff in ci.yml already run. The -DskipTests install
    still compiles test sources, which is what produces the tests jar. The step name
    Install root artifact (graph-compose + test-jar) also loses its stale half: the
    artifact is graph-compose-core since the module split, and no core test-jar is
    involved.
  • BenchmarkDependencyInstallGuardTest fails when a job builds benchmarks/pom.xml
    without an earlier step in the same job installing each first-party dependency the
    benchmarks pom declares. The required set is read out of the pom, not a copy of it,
    so a new sibling dependency is guarded the day it is added. Job scoping is the point:
    a file-level check would let one benchmark job borrow another's install steps and
    pass.
  • It is keyed on the presence of an install, never on recognising a command that looks
    wrong, and it fails when it finds no benchmark-building job at all — so a workflow
    shape that moves fails the guard instead of silently emptying it. YAML comments are
    stripped before matching, so an install commented out does not still count as one.
    A job that would rely on -pl … -am to pull a module in transitively reads as
    missing, deliberately: every install in this repository is explicit, and a guard that
    reasons about reachability is one that can be argued with.
  • The guard joins the -Dtest= list in the architecture-and-documentation-guards job,
    which has no path filter and so runs on a workflow-only pull request — the shape that
    introduces this failure.

Verification

./mvnw -B -ntp clean "-Dtest=…" test -pl :graph-compose-coreBUILD SUCCESS,
52 tests, 0 failures (exit code read unpiped). CiGuardListGuardTest resolves the
added name.

The install sequence was run end to end against the failing step: fonts → emoji → core
→ render-pdf → templates, then
./mvnw -B -ntp -f benchmarks/pom.xml clean package -DskipTestsBUILD SUCCESS,
with graph-compose-templates-…-tests.jar on the shade classpath. The runner jar it
produces starts: java -jar benchmarks/target/benchmarks.jar -l lists all 11
benchmarks, so the step after the one that was failing works too.

The guard was confirmed red before it was trusted, three ways: the templates install
deleted from the JMH job (reports job 'jmh' … without first installing graph-compose-templates), the same install downgraded to a YAML comment (same
failure), and -f benchmarks/pom.xml renamed everywhere (reports that no job runs it,
rather than passing over nothing). Dropping both ci.yml templates installs reports
perf-smoke and benchmark-diff by name, so its coverage is all three benchmark jobs,
not one.

Notes

The cron fires from the default branch, so this does not turn the weekly run green on
its own — the job stays red until the commit reaches main with the next release.

No production code, no public API, no @since. The CHANGELOG entry opens
## v2.2.1 — Planned under ### Build, matching the ### Build entries #481, #493 and
#512 shipped for comparable CI fixes.

Lane: build/CI — one workflow job corrected against the pattern its two siblings
already follow, plus the guard that holds all three to it.

The JMH workflow installed the engine from source and let Maven resolve
the rest from Central. One of them is not there to resolve: the benchmarks
read their document fixtures out of the tests-classifier jar of
graph-compose-templates, which the templates release profile unbinds so it
is never published. The job died at dependency resolution before a single
benchmark ran — five of its last six weekly runs, across four versions,
each looking for the jar under the version the release before it had just
published.

It now installs render-pdf and templates from source, the same sequence
the two benchmark jobs in ci.yml already run.
BenchmarkDependencyInstallGuardTest reads every workflow and fails when a
job builds benchmarks/pom.xml without first installing each first-party
dependency the benchmarks pom declares. It takes that list from the pom
rather than a copy of it, so a new sibling dependency is guarded the day
it is added, and it runs in the guard job on every pull request — which is
what the workflow itself cannot do, firing only from the default branch on
a schedule.

./mvnw clean -Dtest=<guard list> test -pl :graph-compose-core — BUILD
SUCCESS, 52 tests. The install sequence verified end to end against the
failing step: the JMH runner jar builds and lists all 11 benchmarks. The
guard was confirmed red three ways first — templates install deleted,
downgraded to a comment, and the benchmark build renamed so no job matches
it.
@DemchaAV
DemchaAV merged commit 368aed7 into develop Aug 17, 2026
12 checks passed
@DemchaAV
DemchaAV deleted the ci/jmh-install-benchmark-deps branch August 17, 2026 08:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant