You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Wallet-label benchmark methodology: payment gates and excluded cohorts
#2898
Before proposing a provider adapter, how would you want benchmark №008 to handle per-request payment and intentionally excluded address cohorts?
I operate Obol, an independent x402 blockchain-data service. This is a methodology/intake question, not a request to add a provider or change any published score. I read the harness at commit fd7b756: Provider.Lookup returns HasLabel, Err and Skipped; monitor.go scores label presence separately from the curated-hint accuracy series; anchor_feeder.go repeats the anchors on a nominal 30-minute loop.
Our organization-label lookup returns a free bounded miss, but an existing result requires an x402 payment before its label is returned. An unpaid HTTP 402 is therefore neither a returned entity name nor an accuracy result. It also intentionally excludes individual-person rows, so a public-figure cohort is not a supported output even when a chain is supported.
A possible evaluation contract would:
distinguish free no-match, unpaid challenge, paid returned label, provider error and unsupported cohort;
obtain explicit evaluator payment approval, a hard total budget and bounded retry count before any paid calls; stop rather than silently exceed the cap;
publish payment inclusion/exclusion and cohort selection before sampling, and expose denominators/skips rather than reward selective omissions;
retain both the existing non-generic-name coverage and separate curated-hint agreement measure. Neither is proof of ownership or representative global coverage.
Would that belong as an opt-in paid-provider cohort in the existing harness, or a separate cost-to-obtain-an-organization-label experiment? If neither fits, I will not submit an adapter or follow up. No paid requests, provider comparisons or coverage advantage are being claimed here.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
Before proposing a provider adapter, how would you want benchmark №008 to handle per-request payment and intentionally excluded address cohorts?
I operate Obol, an independent x402 blockchain-data service. This is a methodology/intake question, not a request to add a provider or change any published score. I read the harness at commit fd7b756:
Provider.LookupreturnsHasLabel,ErrandSkipped;monitor.goscores label presence separately from the curated-hint accuracy series;anchor_feeder.gorepeats the anchors on a nominal 30-minute loop.Our organization-label lookup returns a free bounded miss, but an existing result requires an x402 payment before its label is returned. An unpaid HTTP 402 is therefore neither a returned entity name nor an accuracy result. It also intentionally excludes individual-person rows, so a public-figure cohort is not a supported output even when a chain is supported.
A possible evaluation contract would:
Would that belong as an opt-in paid-provider cohort in the existing harness, or a separate cost-to-obtain-an-organization-label experiment? If neither fits, I will not submit an adapter or follow up. No paid requests, provider comparisons or coverage advantage are being claimed here.
All reactions