Repository navigation
rpc-cost: the three findings the audit left standing - #2821
Merged
Merged
Conversation
Bench 282 has been gated since 2026-09-30 with two blockers and a major that nothing had been done about. All three are the same shape: a value that was never stated read as a value of zero or one, producing a plausible wrong number that eighteen tests passed straight over. 1. block_age multiplied the whole weighted total. Chainstack's archive surcharge is triggered by how old a block is, not by which method asked, and several methods already carry the doubled figure in their own weight: debug_traceTransaction is listed at 2 RU and the empirical pass measured 2 RU even at tip-10. Multiplying again charged trace 3.6 RU against a real 2.0, an 80 % overstatement. The surcharge raises the floor rather than scaling the bill, so the archive weight of a method is now max(its listed weight, chain default x multiplier): a base read goes 1 -> 2, a debug call already at 2 stays 2. The indexer profile is deliberately unchanged at 2.0, because every method in it is 1 RU and the old multiplier reached the same answer there; a fix that moved that number would be over-reaching. 2. The dedicated cohort could not be billed for requests. needed = 0 for all 37 plans, so the overage and max_units branches were unreachable. AWS AMB read 97.82 dollars at every volume against a real 1B bill of 3,097.82, and led every Dedicated cell on it. The root cause is that included_units: null means two opposite things here. GetBlock's dedicated node serves unlimited requests for its monthly fee; AWS AMB includes none and bills every request on top. Both write null. The plan's own overage rate disambiguates, so that decides rather than the cohort: 4 of 37 meter (AMB in three regions, Shyft legacy-scale), 33 stay flat and their allowance is read as unlimited rather than zero. Zeeve's published ceilings now bind too. 3. An unstated capability was read as a granted one. archive: null and trace: null passed both gates, because one tested for the string "false" and the other for a non-nil pointer. 27 of the 106 usage plans are null on each field, and NOWNodes won trace cells on a capability it makes no claim about. Now not ranked, and the reason distinguishes the two cases: "does not serve" is the provider's statement, "not published" is a gap in this catalogue, and only the second is supportable for a null. 13 providers carry an explicit trace: true, so the cells keep a field: 20, 19 and 13 eligible plans at the three volumes. Three published winners move, two of them exactly as the audit predicted by hand before any of this was written: trace @ 10M chainstack 49 was drpc 60 trace @100m chainstack 499 was nownodes 567.75 trace @ 1B drpc 6000 was nownodes 5677.50 (unstated trace) indexer @ 1B onfinality 3624 Six tests, R5 to R7 from the audit's list. Verified by discrimination rather than by passing: with the fixes reverted, four of the five model tests fail and R6 fails, while TestFlatDedicatedPlanStaysFlat passes both ways, which is what it is for. The bench stays gated. The remaining audit items are the circular R1-R4 in the checks plugin, the units and free-allowance columns rendering on 4 of 90 views, and the free tier panel, which is the next piece of work. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Florent asked for a triple check on the values, calling out that Chainstack
bills in RU. Three things came out of it.
The verification standard for this bench is two independent implementations
agreeing on every cell, and I had not re-established it after changing the
model. Rebuilt as a Node scorer reading the same catalogue with js-yaml,
written from the catalogue's field semantics: 2,490 quotes, 0 disagreements
on tier, eligibility, units per request and monthly bill, and all 15 cell
winners identical.
That run also taught something about what independence means here. My first
attempt re-typed the workload profiles into the second implementation and
produced 66 false disagreements, all from two transcription errors in the
dapp and solana-bot mixes. Profiles are INPUT data; retyping them buys no
independence and only adds copying mistakes, so the Go side exports them and
the scorer reads them. The arithmetic is what has to be written twice.
Then the published winners turned out to sit exactly on an allowance.
Chainstack's archive workloads cost 2 RU per request after the block_age
floor, its tiers include 20M / 80M / 200M / 400M RU, and the buckets are 10M
/ 100M / 1B requests. Round times round lands on round: trace and indexer at
10M need exactly 20,000,000 RU, which is exactly Growth, and at 100M exactly
200,000,000, which is exactly Business. Verified by hand, then reproduced by
the scorer.
So 49 dollars is correct and knife-edge. 10.1M requests is 52 dollars, 11M is
79, 15M is 199. Pinned, because a change to the weights, the floor or the
buckets must not move that edge in silence.
And a correction. I reported that the free tier could not be ranked for want
of an allowance metric. Wrong: rpc_free_allowance_requests exists, is
computed by FreeAllowanceRequests, and is live with 53 series. I had queried
{__name__=~"rpc_cost.*"} and the metric is named rpc_free_*.
It matters because converting units to requests REVERSES the raw ranking, and
the raw figures are what I had shown:
onfinality 400,000 RU/day -> 12,000,000 requests/mo
drpc 210,000,000 CU/mo -> 10,500,000
ankr 200,000,000 credits -> 1,000,000
Ankr looks second best in the cohort by allowance and near the bottom by what
it buys, because it charges 200 credits a request against OnFinality's 1 RU.
All three hand-checked and now pinned. The catalogue's own comment says it:
200M credits means nothing until you know a credit buys a two-hundredth of a
request.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bench 282 has been gated since 2026-09-30 with two blockers and a major that nothing had been done about. All three are the same shape: a value that was never stated, read as zero or one, producing a plausible wrong number that eighteen tests passed straight over.
1.
block_agemultiplied the whole weighted totalChainstack's archive surcharge is triggered by how old a block is, not by which method asked, and several methods already carry the doubled figure in their own weight —
debug_traceTransactionis listed at 2 RU and the empirical pass measured 2 RU even at tip-10. Multiplying again charged trace 3.6 RU against a real 2.0, an 80% overstatement.The surcharge raises the floor rather than scaling the bill, so a method's archive weight is now
max(listed weight, chain default x multiplier): a base read goes 1 to 2, a debug call already at 2 stays 2.The indexer profile is deliberately unchanged at 2.0 — every method in it is 1 RU, so the old multiplier reached the same answer there. A fix that moved that number would be over-reaching, and there is a test pinning it.
2. The dedicated cohort could not be billed for requests
needed = 0for all 37 plans, so the overage andmax_unitsbranches were unreachable. AWS AMB read $97.82 at every volume against a real 1B bill of $3,097.82, and led every Dedicated cell on it.Root cause:
included_units: nullmeans two opposite things in this cohort. GetBlock's node serves unlimited requests for its monthly fee; AWS AMB includes none and bills every request on top. Both writenull.The plan's own overage rate disambiguates, so that decides rather than the cohort: 4 of 37 meter (AMB in three regions, Shyft legacy-scale), 33 stay flat with their allowance read as unlimited rather than zero. Zeeve's published ceilings now bind too.
3. An unstated capability was read as a granted one
archive: nullandtrace: nullpassed both gates, because one tested for the string"false"and the other for a non-nil pointer. 27 of the 106 usage plans are null on each field, and NOWNodes won trace cells on a capability it makes no claim about.Now not ranked, and the reason distinguishes the two cases: "does not serve" is the provider's statement, "not published" is a gap in this catalogue, and only the second is supportable for a null. 13 providers carry an explicit
trace: true, so the cells keep a field — 20, 19 and 13 eligible plans at the three volumes.Three published winners move
Two of them exactly as the audit predicted by hand, before any of this was written:
Verified by discrimination, not by passing
Six tests, R5 to R7 from the audit's list. With the fixes reverted, four of the five model tests fail and R6 fails, while
TestFlatDedicatedPlanStaysFlatpasses both ways — which is exactly what it is for, since the 33 flat plans worked before vianeeded = 0and must keep working.The bench stays gated
Remaining audit items: the circular R1-R4 in the checks plugin, the units and free-allowance columns rendering on 4 of 90 views, and the free tier panel — which is the next piece, and the reason this PR exists first.
🤖 Generated with Claude Code