Price unreported Cursor events and show real spend coverage - #3129
Price unreported Cursor events and show real spend coverage#3129Yuxin-Qiao wants to merge 10 commits into
Conversation
|
🦞👀 Pull request received. I will update this pull request when review starts. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9fb5d824e6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| inputTokens: usage.inputTokens, | ||
| cachedInputTokens: usage.cacheReadTokens, | ||
| cacheWriteInputTokens: usage.cacheWriteTokens, |
There was a problem hiding this comment.
When an unpriced OpenAI event contains cache tokens, this passes Cursor's disjoint counters directly to codexCostUSD, whose contract treats cached and cache-write tokens as subsets of inputTokens. Cursor instead computes totals by adding all four counters, so an event with 100 input and 900 cache-read tokens is priced as only 100 cached tokens, dropping the uncached input and 800 cached tokens. Normalize the counters to Codex's total-input convention, or use pricing logic that accepts disjoint counters, before publishing the estimate.
Useful? React with 👍 / 👎.
| var modelsForDay = days[dayKey] ?? [:] | ||
| var accumulator = modelsForDay[model] ?? ModelAccumulator() | ||
| accumulator.add(usage) | ||
| accumulator.add(usage, estimatedCents: Self.estimatedListPriceCents(for: usage, model: model)) |
There was a problem hiding this comment.
For historical Cursor events without totalCents, this call omits the event timestamp even though the fallback exposes pricingDate. It therefore always uses current rates; for example, GPT-5.6 Terra/Luna events before the 2026-07-30 cutoff are repriced using the post-cut rates, and the normal 30-day Cursor window can span that boundary. Pass the already-derived event date through so each estimate uses the tariff effective when the usage occurred.
Useful? React with 👍 / 👎.
clawsweeper
Bot
added
merge-risk: 🚨 compatibility
|
Codex review: needs changes before merge. Reviewed August 22, 2026, 6:42 AM ET / 10:42 UTC. ClawSweeper reviewWhat this changesThis PR estimates missing Cursor event costs from bundled model catalogs, improves spend coverage reporting, and tracks priced versus unpriced request coverage. Merge readinessKeep open: malformed non-finite Cursor costs are still treated as missing and replaced with catalog estimates, so one P2 correction is needed before merge. Priority: P2 Review scores
Verification
Live VerificationCommand: Result: FAIL (failed) — execution before step 1 Assertions:
How this fits togetherCursor usage events are decoded into daily cost reports, which feed the cached spend history and the overview dashboard. The new fallback sits between event decoding and daily aggregation, assigning a catalog estimate when Cursor has no vendor cost. flowchart LR
A[Cursor usage event] --> B[Event cost decoder]
B --> C{Vendor cost state}
C -->|Missing| D[Catalog price fallback]
C -->|Valid| E[Reported cost]
C -->|Invalid| F[Unpriced request]
D --> G[Daily cost report]
E --> G
F --> G
G --> H[Spend dashboard and cache]
Before merge
Findings
Agent review detailsSecurityNone. Review metrics
Merge-risk optionsMaintainer options:
Copy recommended automerge instructionTechnical reviewBest possible solution: Represent vendor cost as missing, valid, or invalid so only genuinely absent or null values receive a catalog estimate and malformed values remain unpriced. Do we have a high-confidence way to reproduce the issue? Yes—source inspection gives a high-confidence path: decode a known-model Cursor event with string "NaN" or "Infinity" totalCents, then aggregate it; the decoder yields nil and the new fallback prices it. Is this the best way to solve the issue? No—the fallback is appropriate for genuinely omitted costs, but it must not treat malformed reported costs as omissions. Full review comments:
Overall correctness: patch is incorrect AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 27c7f334e3c4. LabelsLabel justifications:
EvidenceAcceptance criteria:
What I checked:
Likely related people:
Rank-up movesOptional improvements that raise the rating; they are not merge blockers.
Rating scale
Overall follows the weaker of proof and patch quality. Workflow
HistoryReview history (8 earlier review cycles)
|
clawsweeper
Bot
added
rating: 🧂 unranked krab
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7eed14f69f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| inputTokens: usage.inputTokens + usage.cacheReadTokens + usage.cacheWriteTokens, | ||
| cachedInputTokens: usage.cacheReadTokens, | ||
| cacheWriteInputTokens: usage.cacheWriteTokens, |
There was a problem hiding this comment.
When totalCents is absent for a Claude-routed Cursor event containing cache reads or writes, this folded input count reaches the Claude branch of listPriceFallbackCostUSD, where inputTokens, cache reads, and cache creation are treated as disjoint counters. Cached tokens are therefore billed once at the full input rate and again at their cache rate, and the inflated count can also trigger long-context pricing. The fresh evidence relative to the earlier cache-accounting comment is that the updated call now adds both cache counters before invoking a helper that can select Claude; normalize only for the Codex branch or let the helper accept Cursor's disjoint counters.
Useful? React with 👍 / 👎.
| requestCount: requestCount, | ||
| costUSD: costUSD, | ||
| modelsUsed: models.keys.sorted(), | ||
| modelBreakdowns: Self.sortedBreakdowns(breakdowns)) | ||
| modelBreakdowns: Self.sortedBreakdowns(breakdowns), | ||
| estimatedRequestCount: estimatedRequestCount > 0 ? estimatedRequestCount : nil) |
There was a problem hiding this comment.
When one Cursor day contains both a catalog-priced estimate and an unrecognized event whose totalCents is absent, the recognized event makes costUSD non-nil while this entry records only estimatedRequestCount. CostUsageDailyReport.Entry.coverageCounts consequently classifies every remaining request as priced, so the dashboard reports Unpriced 0 even though the unrecognized event has no cost. Track fallback failures in unpricedRequestCount when constructing the entry.
Useful? React with 👍 / 👎.
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 45160c53d3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if usage.totalCents == nil { | ||
| if estimatedCents != nil { | ||
| self.estimatedRequests += 1 | ||
| } else { | ||
| self.unpricedRequests += 1 | ||
| } | ||
| } |
There was a problem hiding this comment.
When Cursor supplies a negative totalCents—a case already covered by reports do not revive a model cost after an invalid cents event—the value is non-nil, so this block records neither an estimate nor an unpriced request even though checkedKnownCostSum rejects the cost. If another model on the same day has a valid cost, the day remains non-nil and coverageCounts consequently classifies the invalid-cost request as priced; if it is the only model, the request disappears from coverage entirely. Validate totalCents before this branch and increment unpricedRequests whenever the reported cost is rejected.
Useful? React with 👍 / 👎.
clawsweeper
Bot
added
rating: 🦪 silver shellfish
45160c5 to
bd3b0be
Compare
|
@clawsweeper re-review |
|
🦞👀 Re-review progress:
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 380c1e5807
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| model: String) -> Double? | ||
| { | ||
| guard usage.totalCents == nil else { return nil } | ||
| if let usd = CostUsagePricing.codexCostUSD( |
There was a problem hiding this comment.
When a user has an exact model entry in custom-pricing.json, this overload implicitly loads that overlay before consulting catalog rates, so an unreported Cursor gpt-* event is priced using the user's native Codex override—or becomes unpriced when the override is partial. This contradicts docs/model-pricing.md, which explicitly says Cursor does not read the overlay, and makes Cursor spend depend on unrelated provider configuration; pass an empty custom-pricing value for this lookup.
Useful? React with 👍 / 👎.
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
clawsweeper
Bot
added
proof: sufficient
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bf9b16bf61
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if let usd = CostUsagePricing.claudeCostUSD( | ||
| model: model, | ||
| inputTokens: usage.inputTokens, |
There was a problem hiding this comment.
When totalCents is missing for Cursor's claude-4.5-sonnet spelling—the spelling already used by Cursor fixtures in CursorUsageEventsFetcherTests.swift—both lookups return nil: the OpenAI lookup does not recognize it, while the bundled Claude table uses claude-sonnet-4-5 and normalizeClaudeModel does not reorder this alias. Without a cached models.dev entry, these known Claude events therefore remain unpriced, defeating the new fallback for a model that the endpoint demonstrably emits; map Cursor's Claude aliases to the catalog identity before this call.
Useful? React with 👍 / 👎.
| if let totalCents = usage.totalCents { | ||
| if totalCents < 0 || !totalCents.isFinite { | ||
| self.unpricedRequests += 1 |
There was a problem hiding this comment.
When one model has both a valid reported cost and a rejected negative/non-finite cost, checkedKnownCostSum invalidates the entire model accumulator, but this branch marks only the rejected request as unpriced. If that is the day's only model, the resulting entry has costUSD == nil, so Entry.coverageCounts forces priced to zero and the valid request disappears from coverage (for two requests it reports only Unpriced 1). The fresh evidence beyond the earlier mixed-model case is that a same-model valid/invalid pair follows the accumulator's fail-closed path and never receives the sibling model cost that made the existing test pass.
Useful? React with 👍 / 👎.
clawsweeper
Bot
added
rating: 🦐 gold shrimp
Cursor emits claude-4.5-sonnet while bundled Claude catalog keys on claude-sonnet-4-5; map alias before fallback pricing so catalog estimates are not silently missed. When the same model has both valid and rejected (negative/non-finite) costs, aggregate costUSD fails closed but per-event priced coverage must still be visible; track per-event pricedRequests and expose via pricedRequestCount to keep coverageCounts honest.
|
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3fa975aecf
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| eventDate: Date, | ||
| model: String) -> Double? | ||
| { | ||
| guard usage.totalCents == nil else { return nil } |
There was a problem hiding this comment.
When Cursor returns a string-encoded non-finite cost such as "totalCents":"NaN" or "Infinity", CursorEventNumber.double collapses the present-but-invalid value to nil, so this guard treats it as an omitted cost, applies fallback pricing, and counts the request as estimated instead of unpriced. The fresh evidence relative to the earlier invalid-cost comment is the decoder's explicit non-finite-to-nil conversion before this branch; preserve whether the field was invalid so only genuinely absent/null costs are estimated.
Useful? React with 👍 / 👎.
1 participant
Summary
CostUsagePricing.listPriceFallbackCostUSD, a shared catalog-only fallback that resolves an unreported cost from the bundled Codex/OpenAI table first and Claude/first-party models.dev routing second. No network access and no provider-name literals.totalCentsfrom that fallback. Estimated events now contribute their list-price value to daily cost and are counted via the existingestimatedRequestCountfield; models absent from both tables stay honestly nil.Coverage: 30 / 30instead of a misleadingCoverage: 0 / 30.unpricedRequestCountso a mixed priced/unpriced day does not report the missing-price request as priced via coverage.totalCents) as unpriced.customPricing: .emptyto the Codex lookup so a user’scustom-pricing.json(exact or partial) cannot change Cursor’s catalog-only estimate, preserving the documented boundary.Real behavior proof (after fix, catalog-only, redacted)
No live Cursor cookie is stored in this repo; proof is via deterministic daily aggregation from the same code path the app uses for
CursorUsageEventsFetcher.makeDailyReport. All values are computed from bundled pricing tables, no network.Focused regressions:
estimatesPriceCachedClaudeTokensWithoutDoubleBillingmixedPricedAndCatalogMissingDayCountsUnpricedRequestsmixedValidAndRejectedCostCountsUnpricedRequestsCommands
swift test --filter CursorUsageEventsFetcherTests- 33/33 passedswift test --filter OverviewSpendSummaryTests- 5/5 passedswift test --filter ProviderArchitectureGatekeeperTests- 38/38 passedswiftformat Sources Testsswiftlint --strict- 0 violations./Scripts/lint.sh lint- passes