Skip to content

Add Kotlin exploration tests#11921

Open
jpbempel wants to merge 1 commit into
masterfrom
jpbempel/kotlin-exploration-tests
Open

Add Kotlin exploration tests#11921
jpbempel wants to merge 1 commit into
masterfrom
jpbempel/kotlin-exploration-tests

Conversation

@jpbempel

@jpbempel jpbempel commented Jul 12, 2026

Copy link
Copy Markdown
Member

What Does This Do

use okhttp library with branch 5.3 and instrument all methods from
package okhttp3/*

Motivation

Kotlin compiler does tricky thing when generating bytecode that javac does not
Allow to verify and ensure that log probe instrumentation works for kotlin most of the the time

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@jpbempel jpbempel force-pushed the jpbempel/kotlin-exploration-tests branch from 0e750fa to 83d262a Compare July 12, 2026 17:17
@datadog-official

datadog-official Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

🎯 Code Coverage (details)
Patch Coverage: 100.00%
Overall Coverage: 57.03% (-0.01%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: ff19301 | Docs | Datadog PR Page | Give us feedback!

@pr-commenter

pr-commenter Bot commented Jul 12, 2026

Copy link
Copy Markdown

Debugger benchmarks

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
ci_job_date 1783960886 1783961232
end_time 2026-07-13T16:42:51 2026-07-13T16:48:38
git_branch master jpbempel/kotlin-exploration-tests
git_commit_sha 0d23385 ff19301
start_time 2026-07-13T16:41:27 2026-07-13T16:47:13
See matching parameters
Baseline Candidate
ci_job_id 1854912691 1854912691
ci_pipeline_id 124358468 124358468
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
git_commit_date 1783960319 1783960319

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 5 unstable metrics.

See unchanged results
scenario Δ mean agg_http_req_duration_min Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p75 Δ mean agg_http_req_duration_p99 Δ mean throughput
scenario:noprobe unstable
[-12.841µs; +23.403µs] or [-4.435%; +8.084%]
unstable
[-19.728µs; +37.629µs] or [-5.983%; +11.413%]
unstable
[-29.312µs; +50.133µs] or [-8.500%; +14.537%]
unstable
[-116.498µs; +108.624µs] or [-9.136%; +8.518%]
same
scenario:basic unsure
[+1.052µs; +7.280µs] or [+0.400%; +2.768%]
same same unstable
[-379.847µs; -54.445µs] or [-30.059%; -4.308%]
same
scenario:loop unsure
[+1.803µs; +9.253µs] or [+0.020%; +0.104%]
same same same same
Request duration reports for reports
gantt
    title reports - request duration [CI 0.99] : candidate=None, baseline=None
    dateFormat X
    axisFormat %s
section baseline
noprobe (329.721 µs) : 307, 353
.   : milestone, 330,
basic (293.664 µs) : 288, 300
.   : milestone, 294,
loop (8.989 ms) : 8984, 8993
.   : milestone, 8989,
section candidate
noprobe (338.672 µs) : 308, 370
.   : milestone, 339,
basic (298.375 µs) : 292, 305
.   : milestone, 298,
loop (8.988 ms) : 8983, 8994
.   : milestone, 8988,
Loading
  • baseline results
Scenario Request median duration [CI 0.99]
noprobe 329.721 µs [306.668 µs, 352.774 µs]
basic 293.664 µs [287.622 µs, 299.707 µs]
loop 8.989 ms [8.984 ms, 8.993 ms]
  • candidate results
Scenario Request median duration [CI 0.99]
noprobe 338.672 µs [307.764 µs, 369.58 µs]
basic 298.375 µs [291.526 µs, 305.224 µs]
loop 8.988 ms [8.983 ms, 8.994 ms]

@dd-octo-sts

dd-octo-sts Bot commented Jul 12, 2026

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.02 s 13.92 s [+0.0%; +1.5%] (maybe worse)
startup:insecure-bank:tracing:Agent 12.95 s 13.08 s [-1.6%; -0.4%] (maybe better)
startup:petclinic:appsec:Agent 16.82 s 16.70 s [-0.3%; +1.8%] (no difference)
startup:petclinic:iast:Agent 16.76 s 16.92 s [-1.9%; +0.0%] (no difference)
startup:petclinic:profiling:Agent 16.22 s 16.66 s [-7.1%; +1.8%] (no difference)
startup:petclinic:sca:Agent 16.91 s 16.71 s [+0.3%; +2.1%] (maybe worse)
startup:petclinic:tracing:Agent 15.51 s 16.13 s [-7.8%; +0.1%] (no difference)

Commit: ff19301e · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@jpbempel jpbempel force-pushed the jpbempel/kotlin-exploration-tests branch 3 times, most recently from 2b3a187 to 719a066 Compare July 13, 2026 14:00
@jpbempel jpbempel changed the title DRAFT DRAFT Kotlin exploration tests Jul 13, 2026
@datadog-official

datadog-official Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Bits couldn’t fix the CI

🟢 Investigated · 🔴 Fix failed · ⚪ Validate · ⚪ Ready

OkHttp exploration jobs were running against prebuilt repository checkouts that could miss branch-provided *_exploration-tests.patch files, so disabled test fixes like FastFallbackTest were not applied before jvmTest. .gitlab/exploration-tests.yml now copies the patch files into /exploration-tests, and run-exploration-tests.sh applies the matching project patch idempotently before running the project command.


View in Datadog | Reviewed commit b550046

@jpbempel jpbempel force-pushed the jpbempel/kotlin-exploration-tests branch from 719a066 to b550046 Compare July 13, 2026 14:29
@jpbempel jpbempel marked this pull request as ready for review July 13, 2026 16:28
@jpbempel jpbempel requested review from a team as code owners July 13, 2026 16:28
@jpbempel jpbempel requested review from PerfectSlayer, erikayasuda and ojung and removed request for a team July 13, 2026 16:28
@dd-octo-sts

dd-octo-sts Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

  • Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

use okhttp library with branch 5.3 and instrument all methods from
package okhttp3/*
@jpbempel jpbempel force-pushed the jpbempel/kotlin-exploration-tests branch from b550046 to ff19301 Compare July 13, 2026 16:32

@datadog-official datadog-official Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Datadog Autotest: PASS

More details

This PR adds OkHttp Kotlin exploration tests with Docker setup, CI configuration, and test patches. All changes are infrastructure-only (CI config, Docker setup, test disabling via patches). No production code is modified. The happy path should execute successfully; exploration test infrastructure is internal, not user-facing.

Was this helpful? React 👍 or 👎

📊 Validated against 3 scenarios · Open Bits AI session

🤖 Datadog Autotest · Commit b550046 · What is Autotest? · Any feedback? Reach out in #autotest

@jpbempel jpbempel changed the title DRAFT Kotlin exploration tests Add Kotlin exploration tests Jul 13, 2026
@jpbempel jpbempel added comp: debugger Dynamic Instrumentation type: feature Enhancements and improvements tag: no release notes Changes to exclude from release notes labels Jul 13, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b550046f5c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

needs: [ build ]
dependencies:
- build
<<: *common-exploration-tests

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Gate the OkHttp job on the image that contains it

In merge-request pipelines that touch dd-java-agent/agent-debugger, this new job inherits the common changes rule and runs automatically, but the only step that creates /exploration-tests/okhttp is in Dockerfile.exploration-tests and build-exploration-tests-image is still when: manual with no needs edge. The first automatic run of this change will therefore use the already-published $EXPLORATION_TESTS_IMAGE without an OkHttp checkout, causing run-exploration-tests.sh to fail at cd $PROJECT; keep the job manual until the image is published or wire it to an image built in the same pipeline.

Useful? React with 👍 / 👎.

RUN git clone -b okhttp_5.3.x https://github.com/square/okhttp.git
COPY okhttp_exploration-tests.patch .
RUN cd okhttp && git apply /exploration-tests/okhttp_exploration-tests.patch
RUN bash -c "source $HOME/.sdkman/bin/sdkman-init.sh && sdk use java 17.0.10-tem && cd okhttp && ./gradlew dependencies"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Install an Android SDK before configuring OkHttp

The image now configures the cloned OkHttp Gradle build, but this Debian image only installs JDKs/Maven and never provides ANDROID_HOME or local.properties. OkHttp's okhttp module applies the Android library plugin with compileSdk = 35, so project configuration for ./gradlew dependencies/jvmTest requires a valid Android SDK and fails with the usual SDK-location error; install an SDK in the image or disable the Android target before invoking Gradle.

Useful? React with 👍 / 👎.

@datadog-official

datadog-official Bot commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

Bits couldn’t fix the CI

🟢 Investigated · 🔴 Fix failed · ⚪ Validate · ⚪ Ready

org.jsoup.integration.ConnectIT.canInterruptBodyStringRead was failing under line-based jsoup exploration instrumentation because its interrupt-state assertion is too timing-sensitive in that mode. Added a JUnit disabled annotation for that test in dd-java-agent/agent-debugger/exploration-tests/jsoup_exploration-tests.patch, matching the existing exploration-test patch pattern for instrumentation-sensitive jsoup tests.


View in Datadog | Reviewed commit ff19301

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: debugger Dynamic Instrumentation tag: no release notes Changes to exclude from release notes type: feature Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant