Skip to content

Emit the true body size (and a truncation flag) in instrumentation, not just the preview-capped size #108

@OmarAlJarrah

Description

@OmarAlJarrah

Summary

The request.body.size and response.body.size fields on the http.request / http.response
instrumentation events are derived from the capped preview snapshot, so they report
min(actualSize, bodyPreviewMaxBytes). For any body larger than the preview cap they saturate at
the cap with nothing in that field distinguishing an 8 KiB body from an 8 GB one. Dashboards or
alerts keyed on *.body.size silently flatline at bodyPreviewMaxBytes for large payloads.

This is an observability enhancement, not a correctness bug — the current behavior is intentional and
now documented (see the "Existing docs" note below). The ask is to make the true size observable.

Where the size comes from

Both steps set *.body.size from the preview snapshot length:

sdk-core/src/main/kotlin/org/dexpace/sdk/core/http/pipeline/steps/DefaultInstrumentationStep.kt:211-221

if (requestBody != null) {
    val preview = requestBody.snapshot(options.bodyPreviewMaxBytes)
    ev.field("request.body.size", preview.size.toLong())          // capped at bodyPreviewMaxBytes
        .field("request.body.preview", utf8Preview(preview))
}
val responseBody = response.body
if (responseBody is LoggableResponseBody) {
    val preview = responseBody.snapshot(options.bodyPreviewMaxBytes)
    ev.field("response.body.size", preview.size.toLong())         // capped at bodyPreviewMaxBytes
        .field("response.body.preview", utf8Preview(preview))
    ...
}

The async step is identical:
sdk-core/src/main/kotlin/org/dexpace/sdk/core/http/pipeline/steps/DefaultAsyncInstrumentationStep.kt:307-321.
The failure path likewise reports the preview size for the request body
(DefaultInstrumentationStep.kt:250-254, DefaultAsyncInstrumentationStep.kt:346-350).

snapshot(maxBytes) returns at most maxBytes bytes (and clamps to MAX_BYTE_ARRAY_SIZE), so
preview.size can never exceed bodyPreviewMaxBytes (default 8 KiB, see
HttpInstrumentationOptions.DEFAULT_BODY_PREVIEW_MAX_BYTES).

The true length is already available and already emitted in a separate field —
request.content.length / response.content.length — sourced from contentLength()
(DefaultInstrumentationStep.kt:185, :209). But contentLength() returns -1 for
unknown-length / streaming bodies, and consumers monitoring "body size" naturally reach for the
*.body.size field, which is the capped one.

Existing docs

The current semantics were clarified in the open "clarify bounded body-logging size semantics" docs
PR (#86). It documents that response.body.size is the captured preview size and directs readers
to response.content.length for the true length: "response.body.size is the captured/preview
size, not necessarily the full body size ... Read content.length (not body.size) when you
need the full size." So this issue is purely a follow-up enhancement on top of that documentation —
the field is documented, but a consumer can still only get the true size from a different field
that is -1 for streaming bodies, and there is no explicit signal that the preview was truncated.

Suggested enhancement

One or more of:

  1. Emit the true size when known. Add a *.body.actual_size field (or fold the value into a
    richer event) populated from contentLength() when it is >= 0, so the true size is observable
    alongside the preview.
  2. Add a truncation flag. Emit *.body.preview_truncated (boolean) — true when the body
    exceeded bodyPreviewMaxBytes. For the response body, LoggableResponseBody already distinguishes
    "fit within the cap" from "exceeded the cap" (its contentLength() returns the captured size only
    when the body fit — see docs/http-body-logging-and-concurrency.md), so the truncation state is
    derivable without re-reading the body.
  3. Rename for clarity. Consider *.body.preview_size for the capped field so the name itself
    states that it is the preview length, leaving *.content.length as the true-size field. (This is
    a public field-name change with downstream impact on existing dashboards; gate it behind a
    deliberate decision.)

Options 1 and 2 are additive and the safest first step. Whichever fields are added should be
documented in docs/http-body-logging-and-concurrency.md alongside the existing size-vs-preview
table.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestsdk-coresdk-core toolkit

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions