feat!: Add per-execution runId, at-most-once tracking, and cross-process tracker resumption#249
Draft
jsonbailey wants to merge 9 commits intomainfrom
Draft
feat!: Add per-execution runId, at-most-once tracking, and cross-process tracker resumption#249jsonbailey wants to merge 9 commits intomainfrom
jsonbailey wants to merge 9 commits intomainfrom
Conversation
- Each tracker now carries a runId (UUIDv4) included in all emitted events, scoping every metric to a single execution - At-most-once semantics: duplicate calls to TrackDuration, TrackTokens, TrackSuccess/TrackError, TrackFeedback, and TrackTimeToFirstToken on the same tracker are dropped with a warning Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…ess tracker resumption Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fe3cd88 to
4dee48f
Compare
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…Key from resumption token Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…umptionToken static method Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Matches the existing behavior in ResumptionToken where variationKey is conditionally included. Updates MatchesTrackData test helper to handle absence of variationKey for configs with empty variation keys. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Per AICONF spec 1.2.7.1, CreateTracker() must always return a new tracker instance. Removed the null-return guard for disabled configs. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…tory Breaking change: CompletionConfig/Config now return LdAiConfig instead of ILdAiConfigTracker. The tracker factory is available as a CreateTracker property on the config, each invocation generates a fresh runId. - Move CreateTracker from ILdAiConfigTracker to LdAiConfig as a Func - Remove Config property from ILdAiConfigTracker/LdAiConfigTracker - Collapse tracker constructors to single public ctor requiring runId - Replace null-guard exceptions with safe defaults - Add MetricSummary record and Summary property on tracker - Fix ResumptionToken key ordering using Utf8JsonWriter Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
runId(UUID) included in all track event payloads, enabling per-execution billing isolationCreateTracker()factory onILdAiConfigTracker: Each call returns a new tracker with a freshrunIdand independent tracking state. Returnsnullwhen the config is disabledResumptionTokenproperty onILdAiConfigTracker: URL-safe Base64-encoded JSON containing{ runId, configKey, variationKey, version }for cross-process tracker reconstructionCreateTracker(string token, Context context)onILdAiClient: Reconstructs a tracker from a resumption token with the originalrunId, enabling deferred feedback from a different process. Model and provider names are set to empty strings on reconstructionTest plan
CreateTrackerReturnsNullWhenDisabled— factory returns null for disabled configsCreateTrackerReturnsNewTrackerWhenEnabled— factory returns distinct instance with same configCreateTrackerReturnsTrackerWithFreshRunId— each tracker gets a unique runIdCreateTrackerReturnsTrackerWithIndependentTrackingState— at-most-once state is independent per trackerCreateTrackerCanBeCalledMultipleTimes— factory produces multiple independent trackersResumptionTokenContainsExpectedFields— token contains exactly runId, configKey, variationKey, versionResumptionTokenIsUrlSafeBase64— no+,/, or=charactersResumptionTokenIsConsistentAcrossCalls— same token on repeated accessCreateTrackerFromResumptionTokenRoundTrips— full round-trip preserves runIdCreateTrackerFromResumptionTokenSetsEmptyModelAndProvider— reconstructed tracker has empty model/providerCreateTrackerFromInvalidTokenThrows— malformed input throws ArgumentExceptionCreateTrackerFromNullTokenThrows— null input throws ArgumentNullExceptionCreateTrackerFromTokenMissingRunIdThrows— incomplete payload throws ArgumentException🤖 Generated with Claude Code