Support LAL json{} parsing JSON delivered in text log bodies; fix abortOnFailure wiring and leak-probe timing#13936
Merged
Merged
Conversation
12f6601 to
9d571f6
Compare
There was a problem hiding this comment.
Pull request overview
Note
Copilot couldn't run its full agentic review because no GitHub Actions runner was available. Make sure your repository has a runner available to run Copilot's review, or add a copilot-setup-steps.yml file specifying one with the runs-on attribute. See the docs for more details.
This PR enhances LAL parsing and runtime behavior for OTLP-delivered logs by enabling json {} to parse JSON-shaped text bodies, correctly wiring abortOnFailure into v2-generated code, and improving DSL classloader leak detection timing.
Changes:
- Extend
json {}to fall back to parsing the text body as JSON and normalize the persisted body to JSON for the matching rule. - Fix v2 compiler/codegen to pass each rule’s
abortOnFailureflag into json/yaml/text-regexp parser calls, and make failures rate-limited at WARN when aborting. - Adjust DSL classloader GC/leak-probe arming to avoid delayed leak detection; update docs and extend execution tests/data.
Reviewed changes
Copilot reviewed 18 out of 18 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/classloader/DSLClassLoaderManager.java | Pass settle window to graveyard sweep to support on-demand probe arming. |
| oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/classloader/ClassLoaderGc.java | Rework probe arming/draining to provide timely leak evidence after settle window. |
| oap-server/analyzer/log-analyzer/src/test/resources/scripts/lal/test-lal/feature-cases/execution-basic.yaml | Add feature-case rules for json-text fallback and abortOnFailure=false paths. |
| oap-server/analyzer/log-analyzer/src/test/resources/scripts/lal/test-lal/feature-cases/execution-basic.data.yaml | Add test inputs/expectations for JSON normalization + abortOnFailure behavior. |
| oap-server/analyzer/log-analyzer/src/test/java/org/apache/skywalking/oap/log/analyzer/v2/compiler/LALScriptExecutionTest.java | Assert persisted body “content type” based on rewritten input body oneof. |
| oap-server/analyzer/log-analyzer/src/test/java/org/apache/skywalking/oap/log/analyzer/v2/compiler/LALClassGeneratorBasicTest.java | Update assertions to reflect abortOnFailure being baked into generated calls. |
| oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/dsl/spec/parser/TextParserSpec.java | Thread-safe WARN rate limiting + abortOnFailure parameterization for regexp parser. |
| oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/dsl/spec/parser/ParseFailureWarnLimiter.java | New limiter utility to prevent WARN flooding while tracking suppressed counts. |
| oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/dsl/spec/parser/AbstractParserSpec.java | Remove shared abortOnFailure state; document compile-time flag propagation instead. |
| oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/dsl/spec/filter/FilterSpec.java | Implement json text-body fallback + input normalization; wire abortOnFailure + logging. |
| oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/dsl/ExecutionContext.java | Add input(Object) setter so parsers can swap in rule-local normalized input copies. |
| oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/compiler/rt/LalRuntimeHelper.java | Make regexp group reads null-safe when abortOnFailure=false allows continuation. |
| oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/compiler/LALScriptParser.java | Fix abortOnFailure default to true (documented) during script parsing. |
| oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/v2/compiler/LALClassGenerator.java | Emit parser invocations with per-rule abortOnFailure argument. |
| oap-server/analyzer/log-analyzer/CLAUDE.md | Update internal docs to reflect new generated call signatures. |
| docs/en/setup/backend/log-otlp.md | Document OTLP text-body JSON behavior and json {} fallback/normalization. |
| docs/en/concepts-and-designs/lal.md | Document json {} fallback semantics, normalization scope, and failure logging. |
| docs/en/changes/changes.md | Add CHANGES entry describing the json {} text-body JSON support. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
9d571f6 to
0ce3727
Compare
wankai123
approved these changes
Jul 2, 2026
wankai123
previously approved these changes
Jul 2, 2026
…rtOnFailure wiring and leak-probe timing
- LAL json{} reads the JSON body first and falls back to parsing the text
body as JSON (the OTLP log receiver delivers every string body as text,
even JSON-shaped ones). On a successful text-fallback parse, the matching
rule's context input is swapped to a JSON-bodied copy so that rule persists
the log with content type JSON; the input shared with other rules is never
mutated.
- Fix the abortOnFailure option being silently ignored by the v2 compiler:
the rule's flag is now baked into the generated json/yaml/text-regexp
parser calls (default true, as documented). Aborting parse failures WARN
at most once per minute per parser with the suppressed count reported on
the next emission; abortOnFailure=false failures are DEBUG-only and
continue with a metadata-backed parsed map so parsed.* reads stay
null-safe. Typed-proto inputs (Envoy ALS routing guard) stay quiet.
- Fix delayed classloader leak detection (follow-up to #13935): arm the
unload probe on demand once a pending entry's settle window elapses and
record the collected probe's mint time — sound evidence with single-GC
detection and no drain-time overshoot.
0ce3727 to
076a11f
Compare
wankai123
approved these changes
Jul 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Support LAL
json {}parsing JSON content delivered in a plain-text log bodyThe OTLP log receiver delivers every string body as a text body, even JSON-shaped ones, while
json {}read only the JSON body case — sojson {}rules on OTLP-fed layers silently aborted.json {}now reads the JSON body first and, when it is empty, tries the text body as JSON. On a successful text-fallback parse the matching rule's context input is swapped to a copy with a JSON body, so that rule persists the log with content typeJSON; the shared input is never mutated, and other rules analyzing the same log still see the original text body. Design discussion is summarized in the updatedlal.mdjsonsection andlog-otlp.md.Fix the LAL
abortOnFailureoption being silently ignored in the v2 compilerThe DSL flag was parsed into the rule model but never emitted into the generated code, so
json {}/yaml {}/text { regexp }always used the built-in default andabortOnFailure falsehad no effect (for all three parsers, including the regexp case). The compiler now bakes each rule's flag into the generated parser call (defaulttrue, as documented, fixed from a latentfalsemodel default). A parse failure or regexp non-match that aborts the log is reported at WARN, rate-limited to one report per minute per parser with the suppressed count included in the next report; withabortOnFailure falsethe failure is DEBUG-only, the log continues, andparsed.*reads fall back to metadata fields instead of NPE-dropping the log. Typed-proto inputs (e.g. Envoy ALS entries hitting the MESHjson {}routing guard) stay quiet as before.Fix delayed DSL classloader leak detection (follow-up to #13935)
The unload probe was minted at loader-retire time and recorded its mint time as evidence, so
evidenceUpToMs >= retiredAt + settleMscould never hold on the first collected probe — a real leak needed two class-unloading GC cycles (potentially hours apart on a quiet heap) before the WARN fired. Probes are now armed on demand once a pending entry's settle window has elapsed, and the collected probe's mint time is recorded: sound evidence (no drain-time overshoot from sweeper stalls), single-GC detection, and zero probe cost for loaders that collect normally.CHANGESlog.The CHANGES entry covers the
json {}feature only; theabortOnFailurewiring and the leak-probe timing fix are refinements of unreleased behavior.🤖 Generated with Claude Code