From 9421422f883b2daf5b2585906b9991a4de54517a Mon Sep 17 00:00:00 2001 From: Alrick Crowe Date: Fri, 14 Aug 2026 14:57:22 -0400 Subject: [PATCH 1/2] =?UTF-8?q?feat:=20add=20blockrun=5Fscrapecheck=20?= =?UTF-8?q?=E2=80=94=20verify=20a=20held=20web=20value=20against=20its=20l?= =?UTF-8?q?ive=20source=20page?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Typed structured tool per the CONTRIBUTING design rule (2 endpoints, fixed input contract). ScrapeCheck independently re-fetches the page and returns an ed25519-signed pass/fail/unverifiable verdict; a claim is never certified unless the re-fetched page contains it. Tiers: verify ($0.01 base) and presence ($0.002 base), both + tx fee via withTxFee. Registered in profiles (full + research), README tools row, count pins bumped 20->21 (profiles tests, brand-numbers.json, badge + alt). Gateway dependency stated in the PR description: routes /v1/scrapecheck/verify(-presence) forward to scrapecheck.fly.dev, which speaks x402 v2 with no API key on either side. --- CONTRIBUTING.md | 2 +- README.md | 15 +++---- brand-numbers.json | 2 +- src/mcp-handler.ts | 2 + src/profiles.ts | 7 ++-- src/tools/scrapecheck.ts | 90 ++++++++++++++++++++++++++++++++++++++++ test/profiles.test.ts | 18 ++++---- 7 files changed, 115 insertions(+), 21 deletions(-) create mode 100644 src/tools/scrapecheck.ts diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ec733a5..81efa1a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -19,7 +19,7 @@ Smoke-test the built server via the MCP stdio handshake: (printf '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2024-11-05","capabilities":{},"clientInfo":{"name":"test","version":"1"}}}\n{"jsonrpc":"2.0","method":"notifications/initialized"}\n{"jsonrpc":"2.0","id":2,"method":"tools/list"}\n'; sleep 2) | node dist/index.js 2>/dev/null ``` -Should return 20 tools including `blockrun_surf` and any new one you add. +Should return 21 tools including `blockrun_surf` and any new one you add. To test locally with Claude Code, point it at your dev build: diff --git a/README.md b/README.md index 78b2a75..00b3aec 100644 --- a/README.md +++ b/README.md @@ -6,12 +6,12 @@

Agents can't sign up for accounts. Agents can't enter credit cards.
Agents can only sign transactions.

-BlockRun MCP gives your agent 20 tools — markets, research, web search, images, video, on-chain data, and live Polymarket trading — paid per call in USDC. No accounts. No API keys. No dashboards.

+BlockRun MCP gives your agent 21 tools — markets, research, web search, images, video, on-chain data, and live Polymarket trading — paid per call in USDC. No accounts. No API keys. No dashboards.

Read the odds and place the bet, from one self-custody wallet.


-20 tools  +21 tools  Agent native  No API keys  Read and trade Polymarket  @@ -42,7 +42,7 @@ claude mcp add blockrun -s user -- npx -y @blockrun/mcp@latest --- -> **BlockRun MCP** is an open-source [Model Context Protocol](https://modelcontextprotocol.io) server that gives Claude — and any MCP-compatible agent — 20 tools for real-time data and real actions: 70 LLMs, image & video generation, prediction-market data, live web/X search, on-chain queries across 40 chains, and **the ability to place real, USDC-settled bets on Polymarket**. Authentication is a wallet signature (no API keys); you pay per call in USDC via the [x402](https://x402.org) protocol (no credit cards, no subscriptions). One self-custody wallet on Base or Solana. MIT licensed. +> **BlockRun MCP** is an open-source [Model Context Protocol](https://modelcontextprotocol.io) server that gives Claude — and any MCP-compatible agent — 21 tools for real-time data and real actions: 70 LLMs, image & video generation, prediction-market data, live web/X search, on-chain queries across 40 chains, and **the ability to place real, USDC-settled bets on Polymarket**. Authentication is a wallet signature (no API keys); you pay per call in USDC via the [x402](https://x402.org) protocol (no credit cards, no subscriptions). One self-custody wallet on Base or Solana. MIT licensed. ## 🏆 First of its kind — the signal → trade loop in Claude Code @@ -56,7 +56,7 @@ Every other data integration was built for **human developers** — create an ac **Agents can't do any of that.** BlockRun MCP is built for the agent-first world: -- **One wallet, every source** — 20 tools behind a single self-custody wallet. No per-vendor signups. +- **One wallet, every source** — 21 tools behind a single self-custody wallet. No per-vendor signups. - **No API keys** — your wallet signature *is* authentication. - **No credit cards** — pay per request in USDC via [x402](https://x402.org), fractions of a cent each. - **Starts free** — the free tier (`blockrun_chat mode:"free"`, `blockrun_dex`, crypto `blockrun_price`, `blockrun_models`) costs $0. @@ -71,7 +71,7 @@ Every other data integration was built for **human developers** — create an ac | ------------------- | -------------------------------- | ------------------------- | ----------------------------------------- | | **Setup** | Account + API key *per vendor* | Account/key for 1 vendor | **Wallet auto-created, no signup** | | **Payment** | Credit card, monthly minimums | Credit card / vendor plan | **USDC per-call via x402** | -| **Data sources** | One per integration | One vendor | **20 tools — LLMs, media, markets, chain**| +| **Data sources** | One per integration | One vendor | **21 tools — LLMs, media, markets, chain**| | **Place real bets** | Build it yourself | Rare | **Yes — Polymarket CLOB, confirm-gated** | | **Pay-chain** | — | — | **Base + Solana** | | **Agent budgets** | Manual | — | **Built-in per-agent delegation** | @@ -147,7 +147,7 @@ Expose a trimmed tool set so the client loads fewer schemas into context. Pass ` | Profile | Tools | |---------|-------| -| `full` *(default)* | everything (20 tools) | +| `full` *(default)* | everything (21 tools) | | `media` | `wallet` `models` `image` `video` `realface` `music` `speech` | | `trading` | `wallet` `price` `dex` `markets` `surf` `defi` `rpc` `polymarket_read` `polymarket` | | `research` | `wallet` `models` `chat` `search` `exa` `surf` | @@ -215,6 +215,7 @@ Claude reads the odds with `blockrun_markets` and — with your confirmation — | `blockrun_polymarket` | **Trade on Polymarket** (CLOB V2): place/cancel real bets, positions, redeem winnings — signed locally, settled in pUSD from a gasless deposit wallet. Confirm-gated, $25/order default cap. [Details ↓](#-polymarket-trading) | free tool; bets are your funds | | `blockrun_surf` | Surf (asksurf.ai) — 83 endpoints: CEX data, on-chain SQL (13 chains, 80+ tables), 100M+ labeled wallets, Polymarket + Kalshi, social mindshare, news, Surf-1.5 chat with citations | $0.0095/call | | `blockrun_exa` | Neural web search (Exa) — research, competitors, papers, URL content | $0.01/query | +| `blockrun_scrapecheck` | ScrapeCheck — verify a held web value (price, title, availability) against its live source page; signed pass/fail/unverifiable verdict, offline-verifiable | $0.012 full / $0.0040 presence | | `blockrun_search` | Grok Live Search — web + X/Twitter + news with citations | $0.025 × max_results | | `blockrun_dex` | Live DEX prices via DexScreener | free | | `blockrun_rpc` | Raw JSON-RPC on 40 chains (Ethereum, Base, Solana, Bitcoin, Sui, NEAR, …) via Tatum | $0.002/call | @@ -391,7 +392,7 @@ The server runs a non-blocking npm registry check at startup and prints an `Upda ## FAQ **What is BlockRun MCP?** -An open-source MCP server that gives Claude and other agents 20 tools for real-time data and real actions (trading, media, on-chain), paid per call in USDC. No accounts, no API keys. +An open-source MCP server that gives Claude and other agents 21 tools for real-time data and real actions (trading, media, on-chain), paid per call in USDC. No accounts, no API keys. **Do I need API keys or accounts?** No. A wallet is auto-created locally on first run; you fund it with USDC. No signups, no dashboards, no key rotation. diff --git a/brand-numbers.json b/brand-numbers.json index c74dbdd..13cf537 100644 --- a/brand-numbers.json +++ b/brand-numbers.json @@ -21,7 +21,7 @@ "aliases": 229 }, "mcp": { - "tools": 20 + "tools": 21 }, "chains": { "rpc": 40 diff --git a/src/mcp-handler.ts b/src/mcp-handler.ts index a9d890d..96a9c2f 100644 --- a/src/mcp-handler.ts +++ b/src/mcp-handler.ts @@ -24,6 +24,7 @@ import { registerSurfTool } from "./tools/surf.js"; import { registerRpcTool } from "./tools/rpc.js"; import { registerDefiTool } from "./tools/defi.js"; import { registerPolymarketReadTool, registerPolymarketTool } from "./tools/polymarket.js"; +import { registerScrapecheckTool } from "./tools/scrapecheck.js"; import { resolveTools, type ToolName } from "./profiles.js"; /** @@ -75,6 +76,7 @@ export function initializeMcpServer( defi: () => registerDefiTool(server, budget), polymarket_read: () => registerPolymarketReadTool(server), polymarket: () => registerPolymarketTool(server), + scrapecheck: () => registerScrapecheckTool(server, budget), }; for (const [name, register] of Object.entries(registrars) as [ToolName, () => void][]) { diff --git a/src/profiles.ts b/src/profiles.ts index 34ec67c..94a4f21 100644 --- a/src/profiles.ts +++ b/src/profiles.ts @@ -28,7 +28,8 @@ export type ToolName = | "rpc" | "defi" | "polymarket_read" - | "polymarket"; + | "polymarket" + | "scrapecheck"; // `as const satisfies` keeps the literal tuple type (so the exhaustiveness // guard below can see the actual entries) AND rejects any entry that isn't a @@ -36,7 +37,7 @@ export type ToolName = export const ALL_TOOLS = [ "wallet", "chat", "models", "image", "music", "speech", "video", "realface", "search", "exa", "markets", "price", "dex", "modal", "phone", "surf", "rpc", "defi", - "polymarket_read", "polymarket", + "polymarket_read", "polymarket", "scrapecheck", ] as const satisfies readonly ToolName[]; // Compile-time guard: if a new ToolName is added to the union but not to @@ -60,7 +61,7 @@ export const PROFILES: Record = { trading: ["wallet", "price", "dex", "markets", "surf", "defi", "rpc", "polymarket_read", "polymarket"], // Web research & analysis: live search, neural search, Surf's news/SQL, // and chat for synthesis, plus wallet and the model catalogue. - research: ["wallet", "models", "chat", "search", "exa", "surf"], + research: ["wallet", "models", "chat", "search", "exa", "surf", "scrapecheck"], // Minimal LLM gateway: just chat + model discovery + wallet. chat: ["wallet", "models", "chat"], }; diff --git a/src/tools/scrapecheck.ts b/src/tools/scrapecheck.ts new file mode 100644 index 0000000..aab30f8 --- /dev/null +++ b/src/tools/scrapecheck.ts @@ -0,0 +1,90 @@ +// src/tools/scrapecheck.ts +// +// ScrapeCheck (scrapecheck.fly.dev) — independent verification of a held web +// value against its live source page. Typed structured tool (not a skill): +// two endpoints, a fixed input contract the LLM should respect, one price +// each. The agent holds a value it got from somewhere else (a search result, +// a scraper, another tool) and needs to know it is on the source page right +// now before acting on it. +// +// Settlement: per call to ScrapeCheck's Base treasury via the gateway +// (route proposed in this PR — see PR description). No API key exists on +// either side: x402 payment is the credential. + +import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js"; +import { TOOL_ANNOTATIONS } from "../tool-annotations.js"; +import { z } from "zod"; +import { reserveBudget, recordSpending } from "../utils/budget.js"; +import { withTxFee } from "../utils/tx-fee.js"; +import { asStructuredContent } from "../utils/body.js"; +import { getClient } from "../utils/wallet.js"; +import { formatError, extractErrorMessage } from "../utils/errors.js"; +import type { BudgetState } from "../types.js"; + +type RawClient = { + requestWithPaymentRaw: (endpoint: string, body: unknown) => Promise; +}; + +// Base prices published in ScrapeCheck's OpenAPI and charged by its 402 +// challenge (verified against the live challenge header, x402 v2): +// /verify $0.01, /verify-presence $0.002. The gateway adds its flat tx fee. +export function estimateScrapecheckCost(tier: string): number { + return withTxFee(tier === "presence" ? 0.002 : 0.01); +} + +export function registerScrapecheckTool(server: McpServer, budget: BudgetState): void { + server.registerTool( + "blockrun_scrapecheck", + { + description: `Verify a web value you already hold against its live source page (ScrapeCheck). Not retrieval: it checks a value, it does not find data. + +Call when a price, title, availability, or any page value came from a search result, a scraper, or another tool, and you are about to act on it. ScrapeCheck independently re-fetches the page and returns a signed pass/fail/unverifiable verdict — a claim is never certified unless the re-fetched page contains it, and anything unconfirmed is unverifiable, never pass. Verdicts are ed25519-signed and verifiable offline. + +Tiers: +- verify — full check: is the value on the page AND does it answer what was asked ($${withTxFee(0.01).toFixed(3)}/check charged) +- presence — cheap screen: does the value appear on the page at all; never returns pass ($${withTxFee(0.002).toFixed(4)}/check charged) + +Scope: server-rendered pages; JS-only content returns unverifiable rather than a guess.`, + annotations: TOOL_ANNOTATIONS.readOnlyOpenWorld, + inputSchema: { + url: z.string().describe("Source page the value came from (public http/https)"), + claim: z + .record(z.string(), z.union([z.string(), z.number(), z.boolean()])) + .describe('The value(s) to check, as {field: value}, e.g. {"price": "$45"}'), + asked: z.string().describe("The question the value is supposed to answer, plain text"), + tier: z + .enum(["verify", "presence"]) + .optional() + .describe('"verify" (default, full check) or "presence" (cheap screen, never returns pass)'), + agent_id: z.string().optional().describe("Agent identifier for budget tracking and enforcement."), + }, + }, + async ({ url, claim, asked, tier, agent_id }) => { + try { + const chosenTier = tier === "presence" ? "presence" : "verify"; + const estimatedCost = estimateScrapecheckCost(chosenTier); + const gate = reserveBudget(budget, agent_id, estimatedCost); + if (!gate.allowed) { + return { + content: [{ type: "text", text: `${gate.reason}. Use blockrun_wallet action:"report" to see usage or action:"delegate" to increase agent budget.` }], + isError: true, + }; + } + try { + const client = getClient() as unknown as RawClient; + const endpoint = chosenTier === "presence" ? "/v1/scrapecheck/verify-presence" : "/v1/scrapecheck/verify"; + const result = await client.requestWithPaymentRaw(endpoint, { url, claim, asked }); + recordSpending(budget, estimatedCost, agent_id); + return { + content: [{ type: "text", text: JSON.stringify(result, null, 2) }], + structuredContent: asStructuredContent(result), + }; + } finally { + gate.release(); + } + } catch (err) { + return { content: [{ type: "text", text: formatError(extractErrorMessage(err)) }], isError: true }; + } + } + ); +} diff --git a/test/profiles.test.ts b/test/profiles.test.ts index 9a06de8..6409d41 100644 --- a/test/profiles.test.ts +++ b/test/profiles.test.ts @@ -4,16 +4,16 @@ import assert from "node:assert/strict"; import { ALL_TOOLS, PROFILES, resolveProfileName, resolveTools } from "../src/profiles.js"; const EXPECTED_COUNTS: Record = { - full: 20, + full: 21, media: 7, trading: 9, - research: 6, + research: 7, chat: 3, }; -test("ALL_TOOLS has the full 20-tool set", () => { - assert.equal(ALL_TOOLS.length, 20); - assert.equal(new Set(ALL_TOOLS).size, 20, "no duplicates"); +test("ALL_TOOLS has the full 21-tool set", () => { + assert.equal(ALL_TOOLS.length, 21); + assert.equal(new Set(ALL_TOOLS).size, 21, "no duplicates"); }); test("resolveProfileName precedence: --profile flag > env > default", () => { @@ -46,16 +46,16 @@ test("every profile includes wallet (needed to pay)", () => { } }); -test("unknown profile name falls back to full (20 tools)", () => { +test("unknown profile name falls back to full (21 tools)", () => { const { profile, tools } = resolveTools(["--profile", "nonsense"], {}); assert.equal(profile, "full"); - assert.equal(tools.size, 20); + assert.equal(tools.size, 21); }); test("no args → full", () => { const { profile, tools } = resolveTools([], {}); assert.equal(profile, "full"); - assert.equal(tools.size, 20); + assert.equal(tools.size, 21); }); test("Object.prototype key names fall back to full instead of crashing", () => { @@ -64,7 +64,7 @@ test("Object.prototype key names fall back to full instead of crashing", () => { for (const name of ["constructor", "__proto__", "toString", "hasOwnProperty"]) { const { profile, tools } = resolveTools(["--profile", name], {}); assert.equal(profile, "full", `${name} should fall back to full`); - assert.equal(tools.size, 20, `${name} should expose all 20 tools`); + assert.equal(tools.size, 21, `${name} should expose all 21 tools`); } }); From ff6f0265db0b11f55a05854089477edf1bca63ab Mon Sep 17 00:00:00 2001 From: VickyXAI <115643921+VickyXAI@users.noreply.github.com> Date: Tue, 18 Aug 2026 14:07:05 -0700 Subject: [PATCH 2/2] fix(scrapecheck): guard the URL and the chain before the ledger, and put the route under verify:prices MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Maintainer follow-up on #104. The tool itself was a faithful copy of exa.ts; these are the three conventions a pass-through partner route has to carry, plus the test that pins them. SSRF. `url` was a bare z.string() forwarded raw. This process never fetches it — the gateway does, then ScrapeCheck does — which is exactly the case blockrun_video and blockrun_image already guard, and the comment there says why: a URL pointing at localhost / the metadata endpoint / the private network was "forwarded, quoted, and PAID for before failing (or worse, succeeding) server-side". file://, localhost, 169.254.169.254 and wildcard-DNS forms like 127.0.0.1.nip.io all went through. Now: scheme check plus isBlockedFetchHostResolved(), BEFORE reserveBudget, so a rejected URL never touches the ledger. Chain. Settlement goes to ScrapeCheck's Base treasury and sol.blockrun.ai does not carry these routes, but getClient() returns the Solana client whenever the session chain is Solana. Now fails closed with the same actionable message price.ts and realface.ts use, ahead of the SSRF check so the answer a Solana user gets is "switch chains", not "that host is private". verify:prices. The base price here is set by ScrapeCheck rather than by our own catalogue, so it can move with no BlockRun deploy — the exact drift that script exists to catch, and it had no scrapecheck rows. Both routes are registered; they report `? no 402 (HTTP 404)` until the gateway routes exist, which is the honest signal (unreachable is counted separately and never reads as passing). Also: the description told the model verdicts were "verifiable offline" while the handler returned the vendor payload verbatim — no schema, no pinned key, no signature check. Softened to say what the tool actually does, since a tool whose whole value is "don't trust the last hop" should not ask the agent to trust this one on its word. README gains the Base-only note alongside the other Base-only tools. test/scrapecheck.test.ts: 8 cases — both prices, unknown-tier fails closed to the expensive tier, five SSRF forms rejected with zero network and zero spend, the happy path booking exactly once, presence hitting the cheap route, the Solana guard, and the budget gate stopping the call rather than merely recording it. Verified: tsc clean, sync-brand-numbers --check clean, 349/349 tests, stdio handshake lists 21 tools, verify:prices exits 0 with 0 under-reserving. --- README.md | 4 +- scripts/verify-prices.ts | 7 ++ src/tools/scrapecheck.ts | 61 ++++++++++++++-- test/scrapecheck.test.ts | 150 +++++++++++++++++++++++++++++++++++++++ 4 files changed, 213 insertions(+), 9 deletions(-) create mode 100644 test/scrapecheck.test.ts diff --git a/README.md b/README.md index 00b3aec..e2ee2b4 100644 --- a/README.md +++ b/README.md @@ -215,7 +215,7 @@ Claude reads the odds with `blockrun_markets` and — with your confirmation — | `blockrun_polymarket` | **Trade on Polymarket** (CLOB V2): place/cancel real bets, positions, redeem winnings — signed locally, settled in pUSD from a gasless deposit wallet. Confirm-gated, $25/order default cap. [Details ↓](#-polymarket-trading) | free tool; bets are your funds | | `blockrun_surf` | Surf (asksurf.ai) — 83 endpoints: CEX data, on-chain SQL (13 chains, 80+ tables), 100M+ labeled wallets, Polymarket + Kalshi, social mindshare, news, Surf-1.5 chat with citations | $0.0095/call | | `blockrun_exa` | Neural web search (Exa) — research, competitors, papers, URL content | $0.01/query | -| `blockrun_scrapecheck` | ScrapeCheck — verify a held web value (price, title, availability) against its live source page; signed pass/fail/unverifiable verdict, offline-verifiable | $0.012 full / $0.0040 presence | +| `blockrun_scrapecheck` | ScrapeCheck — verify a held web value (price, title, availability) against its live source page; pass/fail/unverifiable verdict carrying an ed25519 signature you can check against ScrapeCheck's published key | $0.012 full / $0.0040 presence | | `blockrun_search` | Grok Live Search — web + X/Twitter + news with citations | $0.025 × max_results | | `blockrun_dex` | Live DEX prices via DexScreener | free | | `blockrun_rpc` | Raw JSON-RPC on 40 chains (Ethereum, Base, Solana, Bitcoin, Sui, NEAR, …) via Tatum | $0.002/call | @@ -296,7 +296,7 @@ blockrun_wallet action:"setup" # shows the Solana address + fun Then send USDC (SPL) on the **Solana** network — from Coinbase (pick "Solana"), Phantom, Solflare, or Backpack. Switch back with `blockrun_wallet action:"chain" chain:"base"`. The server keeps both wallets; switching just changes which one pays. -**Base-only** — these fall back to Base regardless of active chain: `blockrun_music`, `blockrun_speech`, `blockrun_video`, paid `blockrun_realface`, paid stock `blockrun_price`, and native Anthropic (`claude-*`) passthrough. In Solana mode they return a "switch to Base" message instead of charging. `blockrun_image` pays on either chain. +**Base-only** — these fall back to Base regardless of active chain: `blockrun_music`, `blockrun_speech`, `blockrun_video`, `blockrun_scrapecheck`, paid `blockrun_realface`, paid stock `blockrun_price`, and native Anthropic (`claude-*`) passthrough. In Solana mode they return a "switch to Base" message instead of charging. `blockrun_image` pays on either chain. --- diff --git a/scripts/verify-prices.ts b/scripts/verify-prices.ts index add76a0..5c34039 100644 --- a/scripts/verify-prices.ts +++ b/scripts/verify-prices.ts @@ -27,6 +27,7 @@ import { estimateSurfCost, SURF_PRICE_USD } from "../src/tools/surf.js"; import { estimateSearchCost } from "../src/tools/search.js"; import { estimateCost as estimateImageCost } from "../src/tools/image.js"; import { estimateExaCost } from "../src/tools/exa.js"; +import { estimateScrapecheckCost } from "../src/tools/scrapecheck.js"; import { estimateChatCost, promptCharSize } from "../src/tools/chat.js"; import { estimateVideoCost } from "../src/tools/video.js"; import { MARKETS_PRICE_USD } from "../src/tools/markets.js"; @@ -111,6 +112,12 @@ const PROBES: Probe[] = [ // Routes with no exported estimator: pin the documented figure instead, so a // gateway reprice still trips this gate rather than only the skill docs. { label: "exa/search", path: "exa/search?query=t", expected: withTxFee(0.01) }, + // ScrapeCheck is a pass-through partner: the BASE price is set by ScrapeCheck, + // not by our own catalogue, so it can move without a BlockRun deploy — the + // exact drift this sweep exists to catch. Base only (settles to ScrapeCheck's + // Base treasury), so `[sol: not served]` on these two rows is correct, not a gap. + { label: "scrapecheck/verify", path: "scrapecheck/verify", body: { url: "https://example.com", claim: { price: "$1" }, asked: "what is the price?" }, expected: estimateScrapecheckCost("verify") }, + { label: "scrapecheck/presence", path: "scrapecheck/verify-presence", body: { url: "https://example.com", claim: { price: "$1" }, asked: "what is the price?" }, expected: estimateScrapecheckCost("presence") }, { label: "defillama/protocols", path: "defillama/protocols", expected: withTxFee(0.005) }, { label: "rpc/ethereum (single)", path: "rpc/ethereum", body: { jsonrpc: "2.0", id: 1, method: "eth_blockNumber", params: [] }, expected: withTxFee(0.002) }, diff --git a/src/tools/scrapecheck.ts b/src/tools/scrapecheck.ts index aab30f8..520b1d7 100644 --- a/src/tools/scrapecheck.ts +++ b/src/tools/scrapecheck.ts @@ -7,9 +7,9 @@ // a scraper, another tool) and needs to know it is on the source page right // now before acting on it. // -// Settlement: per call to ScrapeCheck's Base treasury via the gateway -// (route proposed in this PR — see PR description). No API key exists on -// either side: x402 payment is the credential. +// Settlement: per call to ScrapeCheck's Base treasury via the gateway, the +// same pass-through shape as Surf (see the gateway's x402-partner.ts). Base +// only — see the getChain() guard below. import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js"; import { TOOL_ANNOTATIONS } from "../tool-annotations.js"; @@ -17,7 +17,8 @@ import { z } from "zod"; import { reserveBudget, recordSpending } from "../utils/budget.js"; import { withTxFee } from "../utils/tx-fee.js"; import { asStructuredContent } from "../utils/body.js"; -import { getClient } from "../utils/wallet.js"; +import { getClient, getChain } from "../utils/wallet.js"; +import { isBlockedFetchHostResolved } from "../utils/ssrf.js"; import { formatError, extractErrorMessage } from "../utils/errors.js"; import type { BudgetState } from "../types.js"; @@ -28,6 +29,13 @@ type RawClient = { // Base prices published in ScrapeCheck's OpenAPI and charged by its 402 // challenge (verified against the live challenge header, x402 v2): // /verify $0.01, /verify-presence $0.002. The gateway adds its flat tx fee. +// +// PROVISIONAL until the gateway routes exist: what the caller is actually +// charged is whatever /api/v1/scrapecheck/* quotes in its `payment-required` +// header, which is the gateway's published price, not ScrapeCheck's base. +// `npm run verify:prices` probes both routes — if this estimator ever reserves +// LESS than the live quote, the budget gate under-reserves on every call and +// the ledger under-counts permanently. Fix this constant, not the script. export function estimateScrapecheckCost(tier: string): number { return withTxFee(tier === "presence" ? 0.002 : 0.01); } @@ -38,16 +46,16 @@ export function registerScrapecheckTool(server: McpServer, budget: BudgetState): { description: `Verify a web value you already hold against its live source page (ScrapeCheck). Not retrieval: it checks a value, it does not find data. -Call when a price, title, availability, or any page value came from a search result, a scraper, or another tool, and you are about to act on it. ScrapeCheck independently re-fetches the page and returns a signed pass/fail/unverifiable verdict — a claim is never certified unless the re-fetched page contains it, and anything unconfirmed is unverifiable, never pass. Verdicts are ed25519-signed and verifiable offline. +Call when a price, title, availability, or any page value came from a search result, a scraper, or another tool, and you are about to act on it. ScrapeCheck independently re-fetches the page and returns a pass/fail/unverifiable verdict — a claim is never certified unless the re-fetched page contains it, and anything unconfirmed is unverifiable, never pass. Verdicts carry an ed25519 signature you can check against ScrapeCheck's published key; this tool returns the verdict as-is and does NOT verify that signature for you. Tiers: - verify — full check: is the value on the page AND does it answer what was asked ($${withTxFee(0.01).toFixed(3)}/check charged) - presence — cheap screen: does the value appear on the page at all; never returns pass ($${withTxFee(0.002).toFixed(4)}/check charged) -Scope: server-rendered pages; JS-only content returns unverifiable rather than a guess.`, +Scope: server-rendered pages; JS-only content returns unverifiable rather than a guess. Settles on Base only. Public http(s) URLs only.`, annotations: TOOL_ANNOTATIONS.readOnlyOpenWorld, inputSchema: { - url: z.string().describe("Source page the value came from (public http/https)"), + url: z.string().url().describe("Source page the value came from (public http/https)"), claim: z .record(z.string(), z.union([z.string(), z.number(), z.boolean()])) .describe('The value(s) to check, as {field: value}, e.g. {"price": "$45"}'), @@ -61,6 +69,45 @@ Scope: server-rendered pages; JS-only content returns unverifiable rather than a }, async ({ url, claim, asked, tier, agent_id }) => { try { + // Settlement goes to ScrapeCheck's Base treasury; sol.blockrun.ai does + // not carry these routes. Fail closed with the same actionable message + // the other Base-only paid tools use (price.ts, realface.ts) rather + // than letting the call 404 against the wrong gateway. + if (getChain() !== "base") { + return { + content: [{ type: "text", text: formatError("blockrun_scrapecheck settles on Base only. Switch BlockRun to Base (run blockrun_wallet with action:chain chain:base) and fund the Base wallet with USDC.") }], + isError: true, + }; + } + + // SSRF guard on the caller-supplied URL, mirroring blockrun_video and + // blockrun_image. This process never fetches the URL — the GATEWAY and + // then ScrapeCheck do — so this is defense-in-depth plus a saved round + // trip: a URL pointing at localhost / the metadata endpoint / the + // private network would otherwise be forwarded, quoted, and PAID for + // before failing (or worse, succeeding) server-side. Resolved, not + // literal: wildcard-DNS names like 127.0.0.1.nip.io are public strings + // that map to private addresses. Runs BEFORE reserveBudget so a + // rejected URL never touches the ledger. + let parsed: URL; + try { + parsed = new URL(url); + } catch { + return { content: [{ type: "text", text: formatError(`url is not a valid URL: ${url}`) }], isError: true }; + } + if (parsed.protocol !== "http:" && parsed.protocol !== "https:") { + return { + content: [{ type: "text", text: formatError(`url must be an http(s) URL — got scheme "${parsed.protocol}"`) }], + isError: true, + }; + } + if (await isBlockedFetchHostResolved(parsed.hostname)) { + return { + content: [{ type: "text", text: formatError(`url resolves to a private/loopback/link-local address (${parsed.hostname}) — refusing to forward it to the gateway.`) }], + isError: true, + }; + } + const chosenTier = tier === "presence" ? "presence" : "verify"; const estimatedCost = estimateScrapecheckCost(chosenTier); const gate = reserveBudget(budget, agent_id, estimatedCost); diff --git a/test/scrapecheck.test.ts b/test/scrapecheck.test.ts new file mode 100644 index 0000000..46aae98 --- /dev/null +++ b/test/scrapecheck.test.ts @@ -0,0 +1,150 @@ +// Run with: npm test (tsx --experimental-test-module-mocks --test) +// +// blockrun_scrapecheck is a pass-through partner route: the caller hands us a +// URL and we pay a THIRD PARTY to fetch it. That makes two guards load-bearing, +// and both have to fire before a single micro-dollar is reserved: +// +// 1. SSRF. This process never fetches the URL — the gateway does, then +// ScrapeCheck does. Without the guard a URL pointing at localhost / the +// metadata endpoint / the private network is forwarded, quoted and PAID +// for before it fails (or worse, succeeds) server-side. Same hole +// blockrun_video closed; same fix. +// 2. Chain. Settlement goes to ScrapeCheck's BASE treasury and sol.blockrun.ai +// does not carry the routes, so a Solana session must fail closed with an +// actionable message rather than 404 against the wrong gateway. +// +// Plus the cost table, because the base price here is set by ScrapeCheck rather +// than by our own catalogue and can move without a BlockRun deploy. +import { test, mock } from "node:test"; +import assert from "node:assert/strict"; +import type { BudgetState } from "../src/types.js"; + +let activeChain: "base" | "solana" = "base"; +let rawCalls = 0; +let lastEndpoint = ""; +let lastBody: unknown = null; + +mock.module("../src/utils/wallet.js", { + namedExports: { + getChain: () => activeChain, + getClient: () => ({ + requestWithPaymentRaw: async (endpoint: string, body: unknown) => { + rawCalls++; + lastEndpoint = endpoint; + lastBody = body; + return { verdict: "pass", signature: "ed25519:deadbeef" }; + }, + }), + }, +}); +// Hostname-keyed, no DNS: the real resolver is covered by ssrf.test.ts; here we +// only need "this hostname is private" to be decidable offline. +mock.module("../src/utils/ssrf.js", { + namedExports: { + isBlockedFetchHostResolved: async (hostname: string) => + hostname === "169.254.169.254" || hostname === "127.0.0.1.nip.io" || hostname === "localhost", + isBlockedFetchHost: () => false, + }, +}); + +const { registerScrapecheckTool, estimateScrapecheckCost } = await import("../src/tools/scrapecheck.js"); + +function makeHarness(limit: number | null = null) { + let handler: ((args: Record) => Promise) | undefined; + const server = { + registerTool: (_n: string, _c: unknown, h: any) => { handler = h; }, + server: { getClientCapabilities: () => ({}) }, + } as any; + const budget: BudgetState = { limit, spent: 0, calls: 0, agents: new Map() }; + registerScrapecheckTool(server, budget); + return { call: (args: Record) => handler!(args), budget }; +} + +const OK = { claim: { price: "$45" }, asked: "what is the price?" }; +const text = (res: any) => res.content.map((c: any) => c.text).join("\n"); + +test("reserves the CHARGED price, not ScrapeCheck's base", () => { + // base + the gateway's flat tx fee (src/utils/tx-fee.ts). Reserving the base + // would leave the gate short on every call. + assert.equal(estimateScrapecheckCost("verify"), 0.012); + assert.equal(estimateScrapecheckCost("presence"), 0.004); +}); + +test("an unknown tier reserves the EXPENSIVE tier, never the cheap one", () => { + // Fail closed: the handler normalises to "verify", so the estimator must + // agree. Reserving $0.004 for a $0.012 call is how a cap becomes a lie. + for (const t of ["", "VERIFY", "full", "nonsense", "Presence"]) { + assert.equal(estimateScrapecheckCost(t), 0.012, `tier "${t}" must not reserve the presence price`); + } +}); + +test("SSRF: private, loopback, metadata and non-http(s) URLs are refused before ANY call or reservation", async () => { + for (const url of [ + "file:///etc/passwd", + "ftp://example.com/x", + "http://169.254.169.254/latest/meta-data/", + "https://127.0.0.1.nip.io/page", + "http://localhost:8080/admin", + ]) { + rawCalls = 0; + const { call, budget } = makeHarness(); + const res = await call({ url, ...OK }); + assert.equal(res.isError, true, text(res)); + assert.match(text(res), /http\(s\) URL|private\/loopback\/link-local|not a valid URL/); + assert.equal(rawCalls, 0, `paid call was made for ${url}`); + assert.equal(budget.spent, 0, `budget was touched for ${url}`); + } +}); + +test("a public https page still goes through and books the spend once", async () => { + rawCalls = 0; + const { call, budget } = makeHarness(); + const res = await call({ url: "https://shop.example.com/item/1", ...OK }); + assert.notEqual(res.isError, true, text(res)); + assert.equal(rawCalls, 1); + assert.equal(lastEndpoint, "/v1/scrapecheck/verify"); + assert.deepEqual(lastBody, { url: "https://shop.example.com/item/1", ...OK }); + assert.equal(budget.spent, 0.012); +}); + +test("tier:presence hits the cheap route and books the cheap price", async () => { + rawCalls = 0; + const { call, budget } = makeHarness(); + const res = await call({ url: "https://shop.example.com/item/1", ...OK, tier: "presence" }); + assert.notEqual(res.isError, true, text(res)); + assert.equal(lastEndpoint, "/v1/scrapecheck/verify-presence"); + assert.equal(budget.spent, 0.004); +}); + +test("Solana sessions fail closed with the switch-chain message — no call, no spend", async () => { + activeChain = "solana"; + rawCalls = 0; + const { call, budget } = makeHarness(); + const res = await call({ url: "https://shop.example.com/item/1", ...OK }); + activeChain = "base"; + assert.equal(res.isError, true, text(res)); + assert.match(text(res), /Base only/); + assert.match(text(res), /action:chain chain:base/); + assert.equal(rawCalls, 0, "a Solana session must not reach the Base-only route"); + assert.equal(budget.spent, 0); +}); + +test("the chain guard runs before the SSRF guard's DNS work and before the ledger", async () => { + // Ordering matters for the error the user sees: on Solana with a bad URL the + // actionable answer is "switch chains", not "that host is private". + activeChain = "solana"; + const { call } = makeHarness(); + const res = await call({ url: "http://169.254.169.254/", ...OK }); + activeChain = "base"; + assert.match(text(res), /Base only/); +}); + +test("the budget gate rejects before paying, and a rejected call books nothing", async () => { + rawCalls = 0; + const { call, budget } = makeHarness(0.005); // under the $0.012 verify reserve + const res = await call({ url: "https://shop.example.com/item/1", ...OK }); + assert.equal(res.isError, true, text(res)); + assert.match(text(res), /blockrun_wallet/); + assert.equal(rawCalls, 0, "the gate must stop the call, not just record it"); + assert.equal(budget.spent, 0); +});