Feature/cloudflare gateway connector - #13
Open
anassg-lago wants to merge 4 commits into
Open
Conversation
…hit bugs - extract_openai_native/extract_anthropic_native now prefer the response's own model over the requested alias, matching what actually served the call. - Wrapper cache-hit detection: skip billing when a gateway (e.g. Cloudflare) served the response from cache, via .with_raw_response.create(...). - New lago_agent_sdk.gateway.adapters.cloudflare_gateway: extract_cloudflare_log() and resolve_subscription() for the log-extraction half of a standalone connector, verified live across all three of Cloudflare's ingress methods (REST /ai/run, Unified/compat, Native binding) and across native wraps for Anthropic, Gemini, and Mistral through their dedicated passthrough endpoints.
…queue reliability Follow-on to 2bb89a0 (Cloudflare AI Gateway connector). Adds: - Gemini adapter: prefer the response's resolved model over the requested alias, same fix already applied to OpenAI/Anthropic in the prior commit. Extracted the shared resolve_model() helper to adapters/_common.py. - Mistral '-latest' alias resolution for OpenRouter pricing lookups, via union-find over Mistral's mutually-aliasing /v1/models shape. - Selective/lazy pricing warm-up: Cloudflare Workers AI and Mistral pricing are primed reactively on the first wrap()'d call (via _auto_prime_pricing_for), not eagerly at SDK init — OpenRouter is still warmed eagerly. - EventQueue: split permanent (4xx) vs. transient send failures, bounded shutdown drain instead of silently dropping stranded events on exit. - Added verify_ssl config option for local dev against a self-signed Lago. - README: documented the Cloudflare AI Gateway connector (live + backfill paths), removed stale references and roadmap/phase markers.
Workers AI cached tokens were billed twice. `provider="workers-ai"` was missing from `_INPUT_INCLUDES_CACHE_READ`, but Workers AI is only ever reached through Cloudflare's OpenAI-COMPATIBLE endpoint, so its `prompt_tokens` already includes `prompt_tokens_details.cached_tokens`. With the cached portion never subtracted, those tokens were charged at the full input rate AND again at the cache-read rate, which Cloudflare's catalog does publish (verified live for kimi-k2.6, kimi-k2.7-code and glm-5.2). Measured +583% overbill against a real cached call (prompt 23233 / cached 23168) at live catalog rates. Gateway-backfilled Gemini calls could never be priced. The adapter passed Cloudflare's own provider vocabulary through verbatim; a real entry reports provider="google-ai-studio", which matched no vendor in _VENDOR_MAP, so lookup_openrouter missed every time (confirmed against the live 400-model table: miss as google-ai-studio, hit as gemini). The same miss kept it out of _INPUT_INCLUDES_CACHE_READ, so Gemini's cache_read — a subset of its input — was billed on top of input rather than subtracted. A model already carrying its vendor prefix never matched. A real REST-path log reports model="anthropic/claude-opus-4.8" with provider="anthropic", which built "anthropic/anthropic/claude-opus-4.8". Now stripped, but only when the prefix agrees with the resolved vendor, so the lookup stays vendor-gated: a model naming a different vendor is still a miss. Also: `_parse_price` raised InvalidOperation instead of returning None for values >= 1e16, because `.quantize()` sat outside the try. That escaped into emit()'s catch-all and dropped the event as an unknown error rather than taking the normal "no price" path. Returning None also matches what the JS port returns for the same inputs. All 10 distinct (provider, model) pairs across the real captured fixtures now resolve to a live price; three previously missed. The shared golden fixture gains a `precomputed_cases` section carrying verbatim costs from real gateway log entries, plus an optional `provider` on `cases` so per-provider token semantics are pinned cross-repo.
anassg-lago
force-pushed
the
feature/cloudflare-gateway-connector
branch
from
August 7, 2026 09:01
0bb4207 to
7f3d7c0
Compare
`ruff format --check` is a CI gate, and these two files have been failing it since 9cfde77, the branch's first commit — `main` is clean. Both changes are purely cosmetic line-length wraps at the configured 110-char limit; no behavior changes. Unrelated to the billing fixes in the preceding commit, but the PR cannot go green without it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
wrap()auto-detects a client pointed atgateway.ai.cloudflare.com, skips billing oncf-aig-cache-status: HIT, auto-primes Workers AI pricing) and backfill path (gateway.adapters.cloudflare_gatewayextracts a Logs API entry and bills Cloudflare's own metered cost viaemit(usd_cost=..., event_id=...), idempotent across re-runs).-latest, Gemini resolving to a dated snapshot).-latestaliases against Mistral's own/v1/models(union-find over its mutually-aliasing shape) so OpenRouter price lookups hit the right dated model.wrap()'d call, not eagerly at SDK init.verify_sslconfig option for local dev against a self-signed Lago instance.Testing
current_usageevent counts before/after.