Skip to content

@W-23599590 Add CLI runner to DataWeave native benchmark harness - #143

Merged
andres-rad merged 16 commits into
masterfrom
W-23599590
Jul 28, 2026
Merged

@W-23599590 Add CLI runner to DataWeave native benchmark harness#143
andres-rad merged 16 commits into
masterfrom
W-23599590

Conversation

@andres-rad

Copy link
Copy Markdown
Contributor

Summary

  • Adds a 4th benchmark runner (benchmarks/runners/cli/) measuring the shipped dw native CLI binary, completing the cross-surface comparison alongside the Node/Python FFI wrappers and JVM engine baseline
  • Introduces a build-gated BenchmarkHarness inside native-cli (enabled only under -Pbenchmark=true; GraalVM tree-shakes it from the production binary when disabled) with a parent/child READY-marker protocol for true cold-start, first-run, and warm metrics
  • Registers a benchmarkCli Gradle task auto-discovered by the root benchmarkCompare aggregator — no changes to the schema, report, or aggregator required

Test plan

  • ./gradlew native-cli:test — 7 new BenchmarkHarnessTest Scala tests pass
  • ./gradlew native-lib:test -PskipNodeTests=false — 2 new JS parity tests (locate.test.mjs, emit.test.mjs) pass without a binary
  • ./gradlew native-cli:benchmarkCli -Pbenchmark=true — runs end-to-end, writes benchmarks/results/cli-<ts>.json conforming to schema/result.schema.json
  • ./gradlew benchmarkCompare -Pbenchmark=true — unified report includes CLI runner alongside existing three
  • ./gradlew build (no -Pbenchmark) — production dw binary contains no reachable benchmark code

@andres-rad
andres-rad requested a review from a team as a code owner July 27, 2026 17:06
Comment thread native-cli/src/main/java/org/mule/weave/cli/DWCLI.java Outdated
Comment thread benchmarks/runners/cli/coldstart.mjs Outdated
Comment thread docs/superpowers/plans/2026-07-27-cli-benchmark-runner.md Outdated
@andres-rad
andres-rad requested a review from mlischetti July 28, 2026 14:41
@andres-rad
andres-rad merged commit 53e245e into master Jul 28, 2026
4 checks passed
@andres-rad
andres-rad deleted the W-23599590 branch July 28, 2026 17:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants