Skip to content

rpc-cost: the three findings the audit left standing - #2821

Merged
Flotapponnier merged 2 commits into
devfrom
fix/rpc-cost-blockers
Oct 5, 2026
Merged

Flotapponnier merged 2 commits into
devfrom
fix/rpc-cost-blockers

Conversation

@Flotapponnier

Copy link
Copy Markdown
Collaborator

Bench 282 has been gated since 2026-09-30 with two blockers and a major that nothing had been done about. All three are the same shape: a value that was never stated, read as zero or one, producing a plausible wrong number that eighteen tests passed straight over.

1. block_age multiplied the whole weighted total

Chainstack's archive surcharge is triggered by how old a block is, not by which method asked, and several methods already carry the doubled figure in their own weight — debug_traceTransaction is listed at 2 RU and the empirical pass measured 2 RU even at tip-10. Multiplying again charged trace 3.6 RU against a real 2.0, an 80% overstatement.

The surcharge raises the floor rather than scaling the bill, so a method's archive weight is now max(listed weight, chain default x multiplier): a base read goes 1 to 2, a debug call already at 2 stays 2.

The indexer profile is deliberately unchanged at 2.0 — every method in it is 1 RU, so the old multiplier reached the same answer there. A fix that moved that number would be over-reaching, and there is a test pinning it.

2. The dedicated cohort could not be billed for requests

needed = 0 for all 37 plans, so the overage and max_units branches were unreachable. AWS AMB read $97.82 at every volume against a real 1B bill of $3,097.82, and led every Dedicated cell on it.

Root cause: included_units: null means two opposite things in this cohort. GetBlock's node serves unlimited requests for its monthly fee; AWS AMB includes none and bills every request on top. Both write null.

The plan's own overage rate disambiguates, so that decides rather than the cohort: 4 of 37 meter (AMB in three regions, Shyft legacy-scale), 33 stay flat with their allowance read as unlimited rather than zero. Zeeve's published ceilings now bind too.

3. An unstated capability was read as a granted one

archive: null and trace: null passed both gates, because one tested for the string "false" and the other for a non-nil pointer. 27 of the 106 usage plans are null on each field, and NOWNodes won trace cells on a capability it makes no claim about.

Now not ranked, and the reason distinguishes the two cases: "does not serve" is the provider's statement, "not published" is a gap in this catalogue, and only the second is supportable for a null. 13 providers carry an explicit trace: true, so the cells keep a field — 20, 19 and 13 eligible plans at the three volumes.

Three published winners move

Two of them exactly as the audit predicted by hand, before any of this was written:

cell now was
trace @ 10M chainstack $49 drpc $60
trace @ 100M chainstack $499 nownodes $567.75
trace @ 1B drpc $6,000 nownodes $5,677.50 (unstated trace)
indexer @ 1B onfinality $3,624

Verified by discrimination, not by passing

Six tests, R5 to R7 from the audit's list. With the fixes reverted, four of the five model tests fail and R6 fails, while TestFlatDedicatedPlanStaysFlat passes both ways — which is exactly what it is for, since the 33 flat plans worked before via needed = 0 and must keep working.

The bench stays gated

Remaining audit items: the circular R1-R4 in the checks plugin, the units and free-allowance columns rendering on 4 of 90 views, and the free tier panel — which is the next piece, and the reason this PR exists first.

🤖 Generated with Claude Code

Flotapponnier and others added 2 commits October 5, 2026 16:23
Bench 282 has been gated since 2026-09-30 with two blockers and a major that
nothing had been done about. All three are the same shape: a value that was
never stated read as a value of zero or one, producing a plausible wrong
number that eighteen tests passed straight over.

1. block_age multiplied the whole weighted total.

Chainstack's archive surcharge is triggered by how old a block is, not by
which method asked, and several methods already carry the doubled figure in
their own weight: debug_traceTransaction is listed at 2 RU and the empirical
pass measured 2 RU even at tip-10. Multiplying again charged trace 3.6 RU
against a real 2.0, an 80 % overstatement.

The surcharge raises the floor rather than scaling the bill, so the archive
weight of a method is now max(its listed weight, chain default x multiplier):
a base read goes 1 -> 2, a debug call already at 2 stays 2. The indexer
profile is deliberately unchanged at 2.0, because every method in it is 1 RU
and the old multiplier reached the same answer there; a fix that moved that
number would be over-reaching.

2. The dedicated cohort could not be billed for requests.

needed = 0 for all 37 plans, so the overage and max_units branches were
unreachable. AWS AMB read 97.82 dollars at every volume against a real 1B
bill of 3,097.82, and led every Dedicated cell on it.

The root cause is that included_units: null means two opposite things here.
GetBlock's dedicated node serves unlimited requests for its monthly fee; AWS
AMB includes none and bills every request on top. Both write null. The plan's
own overage rate disambiguates, so that decides rather than the cohort: 4 of
37 meter (AMB in three regions, Shyft legacy-scale), 33 stay flat and their
allowance is read as unlimited rather than zero. Zeeve's published ceilings
now bind too.

3. An unstated capability was read as a granted one.

archive: null and trace: null passed both gates, because one tested for the
string "false" and the other for a non-nil pointer. 27 of the 106 usage plans
are null on each field, and NOWNodes won trace cells on a capability it makes
no claim about. Now not ranked, and the reason distinguishes the two cases:
"does not serve" is the provider's statement, "not published" is a gap in this
catalogue, and only the second is supportable for a null. 13 providers carry
an explicit trace: true, so the cells keep a field: 20, 19 and 13 eligible
plans at the three volumes.

Three published winners move, two of them exactly as the audit predicted by
hand before any of this was written:

  trace @ 10M    chainstack 49      was drpc 60
  trace @100m    chainstack 499     was nownodes 567.75
  trace @  1B    drpc 6000          was nownodes 5677.50 (unstated trace)
  indexer @ 1B   onfinality 3624

Six tests, R5 to R7 from the audit's list. Verified by discrimination rather
than by passing: with the fixes reverted, four of the five model tests fail
and R6 fails, while TestFlatDedicatedPlanStaysFlat passes both ways, which is
what it is for.

The bench stays gated. The remaining audit items are the circular R1-R4 in
the checks plugin, the units and free-allowance columns rendering on 4 of 90
views, and the free tier panel, which is the next piece of work.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Florent asked for a triple check on the values, calling out that Chainstack
bills in RU. Three things came out of it.

The verification standard for this bench is two independent implementations
agreeing on every cell, and I had not re-established it after changing the
model. Rebuilt as a Node scorer reading the same catalogue with js-yaml,
written from the catalogue's field semantics: 2,490 quotes, 0 disagreements
on tier, eligibility, units per request and monthly bill, and all 15 cell
winners identical.

That run also taught something about what independence means here. My first
attempt re-typed the workload profiles into the second implementation and
produced 66 false disagreements, all from two transcription errors in the
dapp and solana-bot mixes. Profiles are INPUT data; retyping them buys no
independence and only adds copying mistakes, so the Go side exports them and
the scorer reads them. The arithmetic is what has to be written twice.

Then the published winners turned out to sit exactly on an allowance.

Chainstack's archive workloads cost 2 RU per request after the block_age
floor, its tiers include 20M / 80M / 200M / 400M RU, and the buckets are 10M
/ 100M / 1B requests. Round times round lands on round: trace and indexer at
10M need exactly 20,000,000 RU, which is exactly Growth, and at 100M exactly
200,000,000, which is exactly Business. Verified by hand, then reproduced by
the scorer.

So 49 dollars is correct and knife-edge. 10.1M requests is 52 dollars, 11M is
79, 15M is 199. Pinned, because a change to the weights, the floor or the
buckets must not move that edge in silence.

And a correction. I reported that the free tier could not be ranked for want
of an allowance metric. Wrong: rpc_free_allowance_requests exists, is
computed by FreeAllowanceRequests, and is live with 53 series. I had queried
{__name__=~"rpc_cost.*"} and the metric is named rpc_free_*.

It matters because converting units to requests REVERSES the raw ranking, and
the raw figures are what I had shown:

  onfinality     400,000 RU/day   ->  12,000,000 requests/mo
  drpc       210,000,000 CU/mo    ->  10,500,000
  ankr       200,000,000 credits  ->   1,000,000

Ankr looks second best in the cohort by allowance and near the bottom by what
it buys, because it charges 200 credits a request against OnFinality's 1 RU.
All three hand-checked and now pinned. The catalogue's own comment says it:
200M credits means nothing until you know a credit buys a two-hundredth of a
request.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@Flotapponnier
Flotapponnier merged commit 949e0a2 into dev Oct 5, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant