Skip to content

feat: add four-state JsonField and RawJson tree with additionalProperties pass-through#148

Open
OmarAlJarrah wants to merge 1 commit into
mainfrom
feat/json-field-four-state-model
Open

feat: add four-state JsonField and RawJson tree with additionalProperties pass-through#148
OmarAlJarrah wants to merge 1 commit into
mainfrom
feat/json-field-four-state-model

Conversation

@OmarAlJarrah

Copy link
Copy Markdown
Member

Adds a dependency-free four-state field model for forward-compatible deserialization, plus an immutable additionalProperties holder for lossless unknown-field round-tripping. Both are hand-written sdk-core runtime primitives, fully usable today, that a future generator can target — no codegen is added.

What

sdk-core (no runtime deps beyond Kotlin stdlib)

  • RawJson — an immutable JSON value tree (Obj / Arr / Str / Num / Bool / Null). Numbers are stored as their verbatim literal text, so large long ids and high-precision decimals survive a round-trip without the Double-precision corruption that bites many client SDKs. Typed accessors (toLongOrNull, toBigDecimal, …) parse on demand.
  • JsonField<T> — a sealed four-state container: Missing / Null / Known<T : Any> / Raw(RawJson). Tristate covers the first three; JsonField adds Raw, the "present but does not bind to T" escape hatch a forward-compatible model needs, capturing the original JSON instead of throwing or dropping it. It documents the read-path (four-state) vs. PATCH-path (three-state) boundary and offers explicit, intentionally-lossy toTristate() / fromTristate() interop so the two models are never wrongly merged.
  • AdditionalProperties — an immutable, insertion-ordered snapshot of unknown properties (private ctor + Builder implementing Builder<T> + prefilled newBuilder()), the runtime primitive behind additionalProperties pass-through.

sdk-serde-jackson

  • JsonFieldModule mirrors TristateModule: contextual de/serializers, a bean-property writer that omits Missing fields entirely (so Missing never collapses into null on the wire), and a RawJson ↔ Jackson node converter that streams numbers verbatim. A value that fails to bind to T falls back to Raw rather than failing the parse. Registered in the default ObjectMapper.

The default mapper already tolerates unknown fields, but Jackson then drops them — a read-modify-write loop loses any server-added field. A model mixing in the @JsonAnySetter / @JsonAnyGetter pattern now captures unknowns into AdditionalProperties and re-emits them, making the round-trip lossless (a test demonstrates a large server-added id surviving unchanged).

Tests

  • RawJsonTest, JsonFieldTest, AdditionalPropertiesTest (sdk-core) — value algebra, immutability/defensive copies, number fidelity, Tristate interop, builder semantics.
  • JsonFieldModuleTest, AdditionalPropertiesPassThroughTest (sdk-serde-jackson) — the four states round-trip; a wrong-typed payload becomes Raw and re-emits verbatim; unknown fields are captured and survive read-modify-write.

Gated build (module-scoped, run locally)

./gradlew :sdk-core:test :sdk-core:ktlintCheck :sdk-core:detekt :sdk-core:apiCheck \
          :sdk-serde-jackson:test :sdk-serde-jackson:ktlintCheck :sdk-serde-jackson:detekt :sdk-serde-jackson:apiCheck \
          --no-daemon

Result: BUILD SUCCESSFUL. apiDump was run on both modules and the regenerated api/*.api files are committed.

Closes #50
Closes #52

…ties pass-through

Introduce a dependency-free four-state field model for forward-compatible
deserialization, alongside an immutable additionalProperties holder for
lossless unknown-field round-tripping.

sdk-core (zero non-stdlib deps):
- RawJson: an immutable JSON value tree (Obj/Arr/Str/Num/Bool/Null). Numbers
  keep their verbatim literal text so large ids and high-precision decimals
  round-trip without the Double-precision loss common in client SDKs.
- JsonField<T>: a sealed four-state container — Missing / Null / Known<T:Any> /
  Raw(RawJson). Raw is the "present but unbindable" escape hatch a generated,
  forward-compatible model needs; it preserves the original JSON instead of
  throwing or dropping it. Bidirectional, explicitly-lossy interop with
  Tristate documents the read-path (four-state) vs. PATCH-path (three-state)
  boundary so the two are never wrongly merged.
- AdditionalProperties: an immutable, insertion-ordered snapshot of unknown
  properties (private ctor + Builder + newBuilder), the runtime primitive
  behind additionalProperties pass-through.

sdk-serde-jackson:
- JsonFieldModule mirrors TristateModule: contextual de/serializers, a
  bean-property writer that omits Missing fields, and RawJson<->Jackson node
  conversion that streams numbers verbatim. A value that fails to bind to T
  falls back to Raw rather than failing the parse. Registered in the default
  ObjectMapper.

The default mapper already tolerates unknown fields; a model mixing in the
any-setter/any-getter pattern now captures them into AdditionalProperties so a
read-modify-write loop no longer silently drops server-added fields.

Closes #50
Closes #52
@OmarAlJarrah

Copy link
Copy Markdown
Member Author

This adds RawJson, JsonField<T>, and AdditionalProperties to sdk-core's serde package, plus a Jackson JsonFieldModule that wires deserialization, serialization, and RawJson<->JsonNode conversion. The shape and the mirroring of the existing TristateModule pattern look right, but there's a correctness problem in the number path that defeats the headline lossless-number guarantee, so this needs rework before merging.

Issues

RawJson number conversion is lossy and can produce invalid JSONsdk-serde-jackson/src/main/kotlin/org/dexpace/sdk/serde/jackson/JsonFieldModule.kt:266-275 (RawJsonConversions.fromNode), with the root cause in JacksonObjectMappers.kt:64-87. fromNode builds RawJson.Num(node.asText()), but the default mapper does not enable DeserializationFeature.USE_BIG_DECIMAL_FOR_FLOATS, so Jackson has already parsed every JSON float into a lossy DoubleNode before fromNode ever runs. By the time asText() is called the precision is gone, so Num captures the coerced double text rather than the verbatim wire literal. Against the current default mapper, 1.000000000000000001 round-trips to 1.0, 3.141592653589793238462643383279 collapses to 3.141592653589793, 1.0E-330 becomes 0.0, and 1e400 becomes Infinity — which is not valid JSON and a strict parser will reject it on re-serialization.

This directly contradicts the documented promise in RawJson.kt:35-38 and the "verbatim source" contract on Num.literal (RawJson.kt:24,33-38,139). Large integer ids survive only because Jackson uses LongNode/BigIntegerNode for integral values; every non-integral high-precision number is corrupted. It's not confined to the JsonField.Raw escape hatch either — the AdditionalProperties pass-through funnels through the same fromNode (see AdditionalPropertiesPassThroughTest Model.capture at line 48), so a server-supplied high-precision decimal in the additionalProperties bucket is silently corrupted on a read-modify-write loop, which is exactly the data-loss case this PR is meant to prevent. The existing tests only cover large integer literals (9007199254740993), so the floating-point case slips through entirely.

Fix options: enable USE_BIG_DECIMAL_FOR_FLOATS so floats parse as DecimalNode and asText() yields exact text, or read the number straight from the parser token (JsonParser.getText() at the number token) rather than from an already-coerced JsonNode. Either way, please add round-trip coverage for non-integral high-precision and out-of-double-range literals.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

additionalProperties pass-through on generated models Design a dependency-free four-state JSON field model for generated types

1 participant