Skip to content

feat!: Add per-execution runId, at-most-once tracking, and cross-process tracker resumption#249

Draft
jsonbailey wants to merge 9 commits intomainfrom
jb/aic-2207/update-ai-sdks-billing-spec
Draft

feat!: Add per-execution runId, at-most-once tracking, and cross-process tracker resumption#249
jsonbailey wants to merge 9 commits intomainfrom
jb/aic-2207/update-ai-sdks-billing-spec

Conversation

@jsonbailey
Copy link
Copy Markdown
Contributor

@jsonbailey jsonbailey commented Apr 15, 2026

Summary

  • Per-execution runId: Every tracker gets a unique runId (UUID) included in all track event payloads, enabling per-execution billing isolation
  • At-most-once tracking: Each metric type (duration, tokens, feedback, success/error, time-to-first-token) can only be tracked once per tracker instance — duplicates are dropped with a log warning
  • CreateTracker() factory on ILdAiConfigTracker: Each call returns a new tracker with a fresh runId and independent tracking state. Returns null when the config is disabled
  • ResumptionToken property on ILdAiConfigTracker: URL-safe Base64-encoded JSON containing { runId, configKey, variationKey, version } for cross-process tracker reconstruction
  • CreateTracker(string token, Context context) on ILdAiClient: Reconstructs a tracker from a resumption token with the original runId, enabling deferred feedback from a different process. Model and provider names are set to empty strings on reconstruction

Test plan

  • CreateTrackerReturnsNullWhenDisabled — factory returns null for disabled configs
  • CreateTrackerReturnsNewTrackerWhenEnabled — factory returns distinct instance with same config
  • CreateTrackerReturnsTrackerWithFreshRunId — each tracker gets a unique runId
  • CreateTrackerReturnsTrackerWithIndependentTrackingState — at-most-once state is independent per tracker
  • CreateTrackerCanBeCalledMultipleTimes — factory produces multiple independent trackers
  • ResumptionTokenContainsExpectedFields — token contains exactly runId, configKey, variationKey, version
  • ResumptionTokenIsUrlSafeBase64 — no +, /, or = characters
  • ResumptionTokenIsConsistentAcrossCalls — same token on repeated access
  • CreateTrackerFromResumptionTokenRoundTrips — full round-trip preserves runId
  • CreateTrackerFromResumptionTokenSetsEmptyModelAndProvider — reconstructed tracker has empty model/provider
  • CreateTrackerFromInvalidTokenThrows — malformed input throws ArgumentException
  • CreateTrackerFromNullTokenThrows — null input throws ArgumentNullException
  • CreateTrackerFromTokenMissingRunIdThrows — incomplete payload throws ArgumentException
  • All 72 tests pass (64 existing + 8 new)

🤖 Generated with Claude Code

- Each tracker now carries a runId (UUIDv4) included in all emitted
  events, scoping every metric to a single execution
- At-most-once semantics: duplicate calls to TrackDuration,
  TrackTokens, TrackSuccess/TrackError, TrackFeedback, and
  TrackTimeToFirstToken on the same tracker are dropped with a warning

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@jsonbailey jsonbailey changed the title feat!: Add per-execution runId and at-most-once event tracking feat!: Add per-execution runId, at-most-once tracking, and cross-process tracker resumption Apr 15, 2026
…ess tracker resumption

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@jsonbailey jsonbailey force-pushed the jb/aic-2207/update-ai-sdks-billing-spec branch from fe3cd88 to 4dee48f Compare April 16, 2026 16:53
jsonbailey and others added 7 commits April 16, 2026 16:30
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…Key from resumption token

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…umptionToken static method

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Matches the existing behavior in ResumptionToken where variationKey
is conditionally included. Updates MatchesTrackData test helper to
handle absence of variationKey for configs with empty variation keys.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Per AICONF spec 1.2.7.1, CreateTracker() must always return a new
tracker instance. Removed the null-return guard for disabled configs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…tory

Breaking change: CompletionConfig/Config now return LdAiConfig instead of
ILdAiConfigTracker. The tracker factory is available as a CreateTracker
property on the config, each invocation generates a fresh runId.

- Move CreateTracker from ILdAiConfigTracker to LdAiConfig as a Func
- Remove Config property from ILdAiConfigTracker/LdAiConfigTracker
- Collapse tracker constructors to single public ctor requiring runId
- Replace null-guard exceptions with safe defaults
- Add MetricSummary record and Summary property on tracker
- Fix ResumptionToken key ordering using Utf8JsonWriter

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant