From 98d59b65171f0307ceeb1bbe4df3b60c3180f4fb Mon Sep 17 00:00:00 2001
From: Gil Desmarais <git@desmarais.de>
Date: Mon, 17 Aug 2026 15:10:14 +0200
Subject: [PATCH 1/8] docs(ruby-gem): align MCP and capture guides with
 AutoFallback

Document YAML capture, apply's zero-item ship gate, inspect recon, and mcp.json Botasaurus env so agents stop teaching Faraday-only auto.
---
 .../ruby-gem/guides/ai-agent-workflows.mdx    | 33 ++++++----
 .../guides/capturing-feed-configs.mdx         | 41 +++++-------
 .../docs/ruby-gem/reference/mcp-server.mdx    | 63 ++++++++++---------
 3 files changed, 72 insertions(+), 65 deletions(-)

diff --git a/src/content/docs/ruby-gem/guides/ai-agent-workflows.mdx b/src/content/docs/ruby-gem/guides/ai-agent-workflows.mdx
index df090b5b..a13e92af 100644
--- a/src/content/docs/ruby-gem/guides/ai-agent-workflows.mdx
+++ b/src/content/docs/ruby-gem/guides/ai-agent-workflows.mdx
@@ -12,8 +12,9 @@ import { Code } from "@astrojs/starlight/components";
 Without MCP, agents must invoke shell commands, manage output buffers, and parse unstructured text. With MCP:
 
 - The agent automatically discovers available tools, arguments, and return types.
-- Responses are structured (JSON Feed objects, configuration schemas, and RSS XML).
+- Responses are structured (JSON Feed items, YAML configs, RSS XML).
 - Agents can inspect, capture, validate, and verify feeds iteratively in a closed feedback loop.
+- `html2rss://runtime` reports whether Botasaurus is configured in this MCP process (boolean only).
 
 ## Client Setup
 
@@ -41,7 +42,10 @@ Add `html2rss` to your Cursor MCP settings (`~/.cursor/mcp.json` or `.cursor/mcp
     "mcpServers": {
       "html2rss": {
         "command": "mise",
-        "args": ["exec", "--", "html2rss", "mcp"]
+        "args": ["exec", "--", "html2rss", "mcp"],
+        "env": {
+          "BOTASAURUS_SCRAPER_URL": "http://127.0.0.1:4010"
+        }
       }
     }
   }
@@ -61,7 +65,10 @@ Add `html2rss` to your Claude Desktop configuration (`~/Library/Application Supp
     "mcpServers": {
       "html2rss": {
         "command": "mise",
-        "args": ["exec", "--", "html2rss", "mcp"]
+        "args": ["exec", "--", "html2rss", "mcp"],
+        "env": {
+          "BOTASAURUS_SCRAPER_URL": "http://127.0.0.1:4010"
+        }
       }
     }
   }
@@ -71,23 +78,25 @@ Add `html2rss` to your Claude Desktop configuration (`~/Library/Application Supp
 
 ## Autonomous Workflow Patterns
 
+**Budget:** `scrape_url` is one call. Durable config is `capture_config` → `validate_config` → `apply_config`. Call `inspect_url` only when the result is weak or you need recon (final URL, status, https→http, native RSS/Atom).
+
 ### Pattern A: One-Shot Content Scraping
 
 When an agent needs articles immediately without saving a feed configuration:
 
-1. The agent calls `scrape_url` with the target URL.
-2. `html2rss` runs auto-source extraction (Schema.org, JSON state, semantic HTML) and returns a JSON Feed items array.
-3. If the page is protected or rendered with JavaScript, the agent calls `inspect_url` to diagnose the structure, then retries `scrape_url` with `strategy: "botasaurus"`.
+1. Call `scrape_url` with `strategy: "auto"`. Auto already runs Faraday then Botasaurus.
+2. An empty item list is still success for articles-now. Read `html2rss://runtime` if `botasaurus_configured` is false.
+3. Call `inspect_url` only if the result is weak or you need recon. Do not retry `scrape_url` with explicit `faraday` after auto.
 
 ### Pattern B: Iterative Feed Config Authoring
 
 When an agent is tasked with creating a durable YAML feed configuration:
 
-1. **Inspect:** The agent calls `inspect_url` to check content type, SST node counts, and eligible scrapers.
-2. **Capture:** The agent runs `capture_config` to derive CSS selectors for items, title, link, and description.
-3. **Refine:** The agent reviews the derived selectors or asks the human user for domain-specific adjustments.
-4. **Validate:** The agent passes the configuration to `validate_config` to verify schema conformance.
-5. **Apply:** The agent tests the final configuration with `apply_config` to produce and inspect live RSS XML.
+1. **Capture:** Call `capture_config`. It returns YAML (`items` + `enhance: true`). Check `_meta.articles_count` and `has_selectors`.
+2. **Recon (optional):** Call `inspect_url` for `final_url`, `status`, `scheme_downgrade`, and `alternate_feeds` if the draft is weak.
+3. **Rewrite:** If the destination is html2rss-configs, add `directory.topics` and explicit channel `title`/`url`. Strive to keep `enhance: true` (set `false` only when chrome leaks into items).
+4. **Validate:** Pass `yaml` (or `config`) to `validate_config`. Exactly one of those arguments.
+5. **Apply:** Call `apply_config`. `isError` plus `_meta.item_count: 0` means the config is not shippable — channel title in the XML is not success.
 
 ## JavaScript-Rendered Sites (Botasaurus)
 
@@ -100,4 +109,4 @@ For dynamic JavaScript single-page applications or sites protected by anti-bot m
   lang="bash"
 />
 
-Ensure `BOTASAURUS_SCRAPER_URL` is accessible (typically `http://127.0.0.1:4010`) in the environment where the MCP server runs. Agents can then pass `strategy: "botasaurus"` to `scrape_url`, `inspect_url`, and `capture_config`.
+Put `BOTASAURUS_SCRAPER_URL` (typically `http://127.0.0.1:4010`) in the MCP server `env` block above — a shell export does not reach the Cursor/Claude MCP process. Confirm with `html2rss://runtime` (`botasaurus_configured: true`). AutoFallback then hops without a second tool call.
diff --git a/src/content/docs/ruby-gem/guides/capturing-feed-configs.mdx b/src/content/docs/ruby-gem/guides/capturing-feed-configs.mdx
index 9b8bb7ce..aa153525 100644
--- a/src/content/docs/ruby-gem/guides/capturing-feed-configs.mdx
+++ b/src/content/docs/ruby-gem/guides/capturing-feed-configs.mdx
@@ -5,11 +5,11 @@ description: Derive a reusable YAML feed config from a URL with html2rss capture
 
 import { Aside, Code } from "@astrojs/starlight/components";
 
-`html2rss capture` analyzes a page through the auto-source pipeline and prints a reusable feed config with derived CSS selectors. Use it when you want a first draft faster than hand-writing selectors from scratch.
+`html2rss capture` analyzes a page through the auto-source pipeline and prints a reusable YAML draft with an **items selector plus `enhance: true`** — not title/url/description selector soup. `enhance: true` fills missing article fields at feed-build time.
 
 <Aside type="tip" title="Treat capture as a draft">
-  Selector quality depends on page structure. Complex layouts often need a manual `items_selector` hint or
-  small edits before you ship the config.
+  Selector quality depends on page structure. Catalog feeds still need `directory.topics` and explicit channel
+  `title`/`url`. Strive to keep `enhance: true`; set it `false` only when page chrome leaks into items.
 </Aside>
 
 ## When to Capture
@@ -57,8 +57,7 @@ See the [CLI reference](/ruby-gem/reference/cli-reference/#capture) for the full
 <Code
   code={`
   require 'html2rss'
-  require 'yaml'
-  # Derive a config hash (:channel and :selectors)
+  # Derive a config hash (channel + items selector with enhance: true)
   config = Html2rss.capture('https://example.com/articles')
   # Pin strategy or provide an items hint
   config = Html2rss.capture(
@@ -66,12 +65,7 @@ See the [CLI reference](/ruby-gem/reference/cli-reference/#capture) for the full
     strategy: :botasaurus,
     items_selector: '.article-card'
   )
-  # Serialize with string keys (same wire form as hand-written YAML)
-  File.write(
-    'my-feed.yml',
-    YAML.dump(Html2rss::HashUtil.deep_stringify_keys(config))
-  )
-  # Use immediately
+  File.write('my-feed.yml', Html2rss::Config.to_yaml(config))
   feed = Html2rss.feed(config)
 `}
   lang="ruby"
@@ -79,34 +73,31 @@ See the [CLI reference](/ruby-gem/reference/cli-reference/#capture) for the full
 
 ## How It Works
 
-1. **Request** — fetches the page with the chosen strategy
-2. **Discover** — runs AutoSource to extract articles
-3. **Analyze** — normalizes the page into an SST document and maps segment positions back to articles
-4. **Derive** — builds CSS selectors from SST tag paths for items, title, link, and description
-5. **Assemble** — returns a config hash ready for YAML or `Html2rss.feed`
+1. **Request** — `FeedPipeline` (AutoFallback when `:auto`)
+2. **Discover** — AutoSource extracts admitted articles
+3. **Segment** — SST Segmenter strategies `:list` → `:cluster` → `:semantic`
+4. **Gate** — emit an items selector only when enough articles match
+5. **Assemble** — `{ items: { selector:, enhance: true } }` plus channel
 
-Capture segment discovery currently uses the list Segmenter strategy only (not AutoSource cluster/semantic heuristics). When the draft is weak, pass `--items_selector` or refine selectors by hand.
+When the quality gate fails, selectors are omitted (`has_selectors: false`) rather than inventing attribute selectors. Hint with `--items_selector` or refine by hand.
 
 ## What Capture Derives
 
 Capture focuses on:
 
 - `channel.url` (and related channel defaults)
-- `selectors.items`
-- `selectors.title`
-- `selectors.url` (derived href selector)
-- `selectors.description` when a distinct description root exists
+- `selectors.items` with `enhance: true`
 
-It does not invent author, `published_at`, categories, or enclosure selectors. Add those manually when the page exposes them reliably.
+It does not invent per-field title/url/description selectors, author, `published_at`, categories, or enclosure selectors. `enhance: true` fills missing article fields via the HTML article extractor at feed-build time.
 
-Description is omitted when it would resolve to the invalid CSS selector `.` (item root equals description root).
+MCP `capture_config` returns the same YAML. `validate_config` / `apply_config` accept that YAML string (XOR a config hash). `apply_config` is `isError` when the feed has zero items.
 
 ## Recommended Follow-Up
 
 1. Validate: `html2rss validate my-feed.yml`
 2. Render: `html2rss feed my-feed.yml`
-3. Tighten selectors, strategy, or `request.botasaurus` options if needed
-4. For Feed Directory contributions, add `directory.topics` (see [Creating Custom Feeds](/creating-custom-feeds/#sharing-your-config))
+3. Tighten the items selector, strategy, or `request.botasaurus` options if needed
+4. For Feed Directory contributions, add `directory.topics` and keep `enhance: true` unless chrome leaks (see [Creating Custom Feeds](/creating-custom-feeds/#sharing-your-config))
 
 ## Related
 
diff --git a/src/content/docs/ruby-gem/reference/mcp-server.mdx b/src/content/docs/ruby-gem/reference/mcp-server.mdx
index 0da546c3..624aa6bf 100644
--- a/src/content/docs/ruby-gem/reference/mcp-server.mdx
+++ b/src/content/docs/ruby-gem/reference/mcp-server.mdx
@@ -37,13 +37,13 @@ The MCP server supports two transport mechanisms:
 
 ## Strategy Resolution in MCP
 
-Inside the MCP server, passing `strategy: "auto"` (or omitting strategy) resolves directly to `faraday` without running the multitenant Botasaurus fallback chain.
+`scrape_url` and `capture_config` with `strategy: "auto"` (the default) run Faraday → Botasaurus AutoFallback. One call is enough; do not retry with explicit `faraday` after `auto`.
 
-If a scraped target is JavaScript-rendered, Cloudflare-protected, or returns an empty article list under Faraday:
+`inspect_url` is the exception: `auto` stays on Faraday for a cheap diagnostic. Pin `strategy: "botasaurus"` on inspect when you need browser rendering.
 
-1. Use `inspect_url` to confirm page structure and scraper eligibility.
-2. Retry `scrape_url` or `capture_config` with `strategy: "botasaurus"`.
-3. Ensure the Botasaurus service is running and `BOTASAURUS_SCRAPER_URL` is set in the environment (e.g. `http://127.0.0.1:4010`).
+Read `html2rss://runtime` for `botasaurus_configured` (boolean only). Set `BOTASAURUS_SCRAPER_URL` on the **MCP process** env (see [AI agent workflows](/ruby-gem/guides/ai-agent-workflows/)).
+
+**Tool-call budget:** `scrape_url` = 1 call. Durable config = `capture_config` → `validate_config` → `apply_config`. Call `inspect_url` only when scrape/capture is weak or you need recon.
 
 ## Tools Reference
 
@@ -59,51 +59,55 @@ One-shot article extraction as JSON Feed items. Use when you need articles immed
   - `limit` _(integer, optional)_: Maximum articles to keep. Default: `25`.
   - `items_selector` _(string, optional)_: CSS selector hint for items.
 - **Return value:** JSON string containing an array of article objects (`title`, `url`, `description`, `published_at`, `author`, `image`, `categories`).
-- **Metadata (`_meta`):** Includes `total`, `strategy`, and `channel_title`.
-- **Error handling:** Unhandled exceptions return `isError: true` with error message text.
+- **Metadata (`_meta`):** Includes `total`, `requested_strategy`, `channel_title`, and Status tallies.
+- **Error handling:** Unhandled exceptions return `isError: true`. An empty item list is still success (articles-now is not a ship gate).
 
 ### `inspect_url`
 
-Diagnostic page analysis. Inspects HTTP response headers, content type, scraper eligibility, Semantic Structural Tree (SST) node count, and discovered article segments.
+Diagnostic page analysis plus recon from the same fetch. `auto` uses Faraday; pin `botasaurus` when you need browser rendering.
 
 - **Parameters:**
   - `url` _(string, required)_: Source page URL.
   - `strategy` _(string, optional)_: Request strategy (`auto`, `faraday`, `botasaurus`). Default: `auto`.
 - **Return value:** Pretty-printed JSON object with diagnostic details:
-  - `content_type`: Detected MIME type.
-  - `html_response`: Boolean indicating whether the payload was parsed as HTML.
-  - `scraper_eligibility`: Eligible scrapers (e.g. `wordpress_api`, `schema`, `microdata`, `semantic_html`).
-  - `sst_stats`: Node count and degradation status.
-  - `sst.segment_stats`: Number of segments discovered, strategies matched, and sample CSS tag paths.
+  - `requested_url` / `final_url`: requested URL vs `Response#url` after redirects.
+  - `status`: HTTP status when the strategy reports one.
+  - `scheme_downgrade`: `true` when the fetch went https → http.
+  - `alternate_feeds`: `rel=alternate` RSS/Atom links from the document head (no `/feed` path guessing).
+  - `content_type` / `html_response` / `scraper_eligibility` / `sst_stats` / `sst.segment_stats`.
 
 ### `capture_config`
 
-Analyzes a target URL and derives a reusable `html2rss` feed configuration hash with calculated CSS selectors. Full schema options live in resource `html2rss://schema`.
+Analyzes a target URL and derives a reusable YAML draft (`items` selector + `enhance: true`). Same serializer as CLI `html2rss capture`. Full schema options live in resource `html2rss://schema`.
 
 - **Parameters:**
   - `url` _(string, required)_: Source page URL.
   - `strategy` _(string, optional)_: Request strategy (`auto`, `faraday`, `botasaurus`). Default: `auto`.
   - `items_selector` _(string, optional)_: Optional CSS selector hint for items.
-- **Return value:** Pretty-printed JSON configuration hash containing `:channel` and `:selectors`.
-- **Metadata (`_meta`):** Includes `articles_count`, `channel_title`, `has_selectors` (boolean), and `strategy`.
+- **Return value:** YAML document (string keys). Treat as a draft: catalog feeds still need `directory.topics` and explicit channel `title`/`url`. Strive to keep `enhance: true`.
+- **Metadata (`_meta`):** Includes `articles_count`, `channel_title`, `has_selectors`, and `requested_strategy`.
 
 ### `validate_config`
 
-Validates a feed configuration hash against the official `html2rss` JSON schema (`html2rss://schema`) and runtime constraints.
+Validates a feed configuration against the official `html2rss` JSON schema (`html2rss://schema`) and runtime constraints.
 
-- **Parameters:**
-  - `config` _(object, required)_: Feed configuration object with `channel` and `selectors`.
+- **Parameters** (exactly one of `config` or `yaml`):
+  - `config` _(object, optional)_: Feed configuration object.
+  - `yaml` _(string, optional)_: Feed configuration YAML (same form as CLI capture / catalog files).
 - **Return value:** Text response `Config is valid.` on success.
-- **Error handling:** Returns `isError: true` with a serialized JSON error dictionary if schema validation fails.
+- **Error handling:** Returns `isError: true` with a serialized JSON error dictionary if schema validation fails, or if both/neither argument is provided.
 
 ### `apply_config`
 
-Executes a validated feed configuration against a page and returns valid RSS 2.0 XML.
+Executes a validated feed configuration against a page and returns RSS 2.0 XML. This is the ship gate: `isError` when the feed has zero items.
 
 - **Parameters:**
   - `url` _(string, required)_: Source page URL (populates `channel.url` if omitted from config).
-  - `config` _(object, required)_: Feed configuration object.
+  - `config` _(object, optional)_: Feed configuration object (XOR `yaml`).
+  - `yaml` _(string, optional)_: Feed configuration YAML (XOR `config`).
 - **Return value:** RSS 2.0 XML string.
+- **Metadata (`_meta`):** `item_count` from the rendered RSS `items` (not a regex over the XML).
+- **Error handling:** `isError: true` when the scrape produced no items, or on unhandled exceptions.
 
 ## Resources Reference
 
@@ -114,6 +118,7 @@ The MCP server exposes reference data under the `html2rss://` URI scheme:
 | **`html2rss://schema`**     | `application/json` | The complete JSON Schema for `html2rss` feed configurations, including selector rules, extractors, and request controls. |
 | **`html2rss://extractors`** | `application/json` | Alphabetical list of all registered extractor names (`attribute`, `html`, `href`, `text`, `static`, etc.).               |
 | **`html2rss://strategies`** | `application/json` | List of all registered request strategies (`auto`, `faraday`, `botasaurus`, `local_file`).                               |
+| **`html2rss://runtime`**    | `application/json` | Boolean `botasaurus_configured` from process env. Never includes the scraper URL.                                        |
 
 ## Prompts Reference
 
@@ -121,19 +126,21 @@ The server publishes guided prompt workflows for AI assistants:
 
 ### `scrape-webpage`
 
-Guided one-shot extraction prompt that instructs the agent to run `scrape_url`, inspect the response if empty or JS-gated, and retry with `botasaurus` when needed.
+One `scrape_url` call with `strategy: auto` (AutoFallback already hops). Call `inspect_url` only if the result is weak or you need recon.
 
 - **Arguments:**
   - `url` _(string, required)_: Target URL to scrape.
 
 ### `capture-feed-config`
 
-Four-step workflow prompt for generating durable feed configurations:
+Four-step workflow for a durable feed config:
+
+1. Call `capture_config` — YAML draft. Check `_meta.articles_count` and `has_selectors`. Strive `enhance: true`.
+2. If weak or you need recon, call `inspect_url`. Auto already hops to Botasaurus.
+3. Call `validate_config` with `yaml` (or `config`) — must not be `isError`.
+4. Call `apply_config` — `isError` if zero items. Confirm `_meta.item_count` before shipping.
 
-1. Call `capture_config` to derive initial selectors and check `articles_count`.
-2. If selectors are weak, call `inspect_url` and retry with `botasaurus`.
-3. Call `validate_config` to ensure schema conformance.
-4. Call `apply_config` to verify the resulting RSS feed XML.
+If the destination is html2rss-configs, rewrite the draft for `directory.topics` and explicit channel `title`/`url`.
 
 - **Arguments:**
   - `url` _(string, required)_: Target URL to analyze.

From cb68e560820811744f24a31ef475073f12539032 Mon Sep 17 00:00:00 2001
From: Gil Desmarais <git@desmarais.de>
Date: Mon, 17 Aug 2026 15:34:57 +0200
Subject: [PATCH 2/8] docs(ruby-gem): document MCP envelope results

Point MCP guides at ok/next_step/guidance/payload so agents stop reading _meta or a raw items array.
---
 .../ruby-gem/guides/ai-agent-workflows.mdx    |  8 ++--
 .../guides/capturing-feed-configs.mdx         |  2 +-
 .../docs/ruby-gem/reference/mcp-server.mdx    | 39 ++++++++++++-------
 3 files changed, 29 insertions(+), 20 deletions(-)

diff --git a/src/content/docs/ruby-gem/guides/ai-agent-workflows.mdx b/src/content/docs/ruby-gem/guides/ai-agent-workflows.mdx
index a13e92af..91c9cd0c 100644
--- a/src/content/docs/ruby-gem/guides/ai-agent-workflows.mdx
+++ b/src/content/docs/ruby-gem/guides/ai-agent-workflows.mdx
@@ -12,7 +12,7 @@ import { Code } from "@astrojs/starlight/components";
 Without MCP, agents must invoke shell commands, manage output buffers, and parse unstructured text. With MCP:
 
 - The agent automatically discovers available tools, arguments, and return types.
-- Responses are structured (JSON Feed items, YAML configs, RSS XML).
+- Responses are structured envelopes (`ok`, `next_step`, `guidance`, `payload`) — articles in `payload.items`, YAML in `payload.yaml`, RSS in `payload.rss`.
 - Agents can inspect, capture, validate, and verify feeds iteratively in a closed feedback loop.
 - `html2rss://runtime` reports whether Botasaurus is configured in this MCP process (boolean only).
 
@@ -85,18 +85,18 @@ Add `html2rss` to your Claude Desktop configuration (`~/Library/Application Supp
 When an agent needs articles immediately without saving a feed configuration:
 
 1. Call `scrape_url` with `strategy: "auto"`. Auto already runs Faraday then Botasaurus.
-2. An empty item list is still success for articles-now. Read `html2rss://runtime` if `botasaurus_configured` is false.
+2. An empty item list is still success for articles-now. Read `payload.items`. Follow `next_step` (including `read_runtime` / `html2rss://runtime` if Botasaurus is unset).
 3. Call `inspect_url` only if the result is weak or you need recon. Do not retry `scrape_url` with explicit `faraday` after auto.
 
 ### Pattern B: Iterative Feed Config Authoring
 
 When an agent is tasked with creating a durable YAML feed configuration:
 
-1. **Capture:** Call `capture_config`. It returns YAML (`items` + `enhance: true`). Check `_meta.articles_count` and `has_selectors`.
+1. **Capture:** Call `capture_config`. YAML is `payload.yaml` (`items` + `enhance: true`). Check `payload.articles_count` and `has_selectors`.
 2. **Recon (optional):** Call `inspect_url` for `final_url`, `status`, `scheme_downgrade`, and `alternate_feeds` if the draft is weak.
 3. **Rewrite:** If the destination is html2rss-configs, add `directory.topics` and explicit channel `title`/`url`. Strive to keep `enhance: true` (set `false` only when chrome leaks into items).
 4. **Validate:** Pass `yaml` (or `config`) to `validate_config`. Exactly one of those arguments.
-5. **Apply:** Call `apply_config`. `isError` plus `_meta.item_count: 0` means the config is not shippable — channel title in the XML is not success.
+5. **Apply:** Call `apply_config`. `isError` plus `payload.item_count: 0` means the config is not shippable — channel title in the XML is not success.
 
 ## JavaScript-Rendered Sites (Botasaurus)
 
diff --git a/src/content/docs/ruby-gem/guides/capturing-feed-configs.mdx b/src/content/docs/ruby-gem/guides/capturing-feed-configs.mdx
index aa153525..180f46d6 100644
--- a/src/content/docs/ruby-gem/guides/capturing-feed-configs.mdx
+++ b/src/content/docs/ruby-gem/guides/capturing-feed-configs.mdx
@@ -90,7 +90,7 @@ Capture focuses on:
 
 It does not invent per-field title/url/description selectors, author, `published_at`, categories, or enclosure selectors. `enhance: true` fills missing article fields via the HTML article extractor at feed-build time.
 
-MCP `capture_config` returns the same YAML. `validate_config` / `apply_config` accept that YAML string (XOR a config hash). `apply_config` is `isError` when the feed has zero items.
+MCP `capture_config` returns that YAML in `payload.yaml`. `validate_config` / `apply_config` accept the YAML string (XOR a config hash). `apply_config` is `isError` when the feed has zero items (`payload.item_count`).
 
 ## Recommended Follow-Up
 
diff --git a/src/content/docs/ruby-gem/reference/mcp-server.mdx b/src/content/docs/ruby-gem/reference/mcp-server.mdx
index 624aa6bf..5c7b2ed5 100644
--- a/src/content/docs/ruby-gem/reference/mcp-server.mdx
+++ b/src/content/docs/ruby-gem/reference/mcp-server.mdx
@@ -43,7 +43,17 @@ The MCP server supports two transport mechanisms:
 
 Read `html2rss://runtime` for `botasaurus_configured` (boolean only). Set `BOTASAURUS_SCRAPER_URL` on the **MCP process** env (see [AI agent workflows](/ruby-gem/guides/ai-agent-workflows/)).
 
-**Tool-call budget:** `scrape_url` = 1 call. Durable config = `capture_config` → `validate_config` → `apply_config`. Call `inspect_url` only when scrape/capture is weak or you need recon.
+**Tool-call budget:** `scrape_url` = 1 call. Durable config = `capture_config` → `validate_config` → `apply_config`. Call `inspect_url` only when scrape/capture is weak or you need recon. Follow envelope `next_step` / `guidance` rather than guessing the next tool.
+
+## Result envelope
+
+Every tool returns the same JSON object in both the text content and `structuredContent` (there is no `_meta` quality channel):
+
+```json
+{ "ok": true, "next_step": "validate_config", "guidance": "...", "payload": {} }
+```
+
+`isError` matches `ok: false`. Hosts that parsed scrape text as a raw JSON array must read `payload.items` instead.
 
 ## Tools Reference
 
@@ -58,9 +68,8 @@ One-shot article extraction as JSON Feed items. Use when you need articles immed
   - `strategy` _(string, optional)_: Request strategy (`auto`, `faraday`, `botasaurus`). Default: `auto`.
   - `limit` _(integer, optional)_: Maximum articles to keep. Default: `25`.
   - `items_selector` _(string, optional)_: CSS selector hint for items.
-- **Return value:** JSON string containing an array of article objects (`title`, `url`, `description`, `published_at`, `author`, `image`, `categories`).
-- **Metadata (`_meta`):** Includes `total`, `requested_strategy`, `channel_title`, and Status tallies.
-- **Error handling:** Unhandled exceptions return `isError: true`. An empty item list is still success (articles-now is not a ship gate).
+- **Return value:** Envelope with `payload.items` (JSON Feed article objects: `title`, `url`, `description`, `published_at`, `author`, `image`, `categories`). Also `payload.total`, `requested_strategy`, `channel_title`, and `admission_drops` when present.
+- **Error handling:** Unhandled exceptions return `isError: true` with `payload.class` / `payload.message`. An empty item list is still success (`ok: true`; follow `next_step`, which may be `inspect_url` or `read_runtime`).
 
 ### `inspect_url`
 
@@ -69,12 +78,13 @@ Diagnostic page analysis plus recon from the same fetch. `auto` uses Faraday; pi
 - **Parameters:**
   - `url` _(string, required)_: Source page URL.
   - `strategy` _(string, optional)_: Request strategy (`auto`, `faraday`, `botasaurus`). Default: `auto`.
-- **Return value:** Pretty-printed JSON object with diagnostic details:
+- **Return value:** Envelope whose `payload` is the diagnostic object:
   - `requested_url` / `final_url`: requested URL vs `Response#url` after redirects.
   - `status`: HTTP status when the strategy reports one.
   - `scheme_downgrade`: `true` when the fetch went https → http.
   - `alternate_feeds`: `rel=alternate` RSS/Atom links from the document head (no `/feed` path guessing).
   - `content_type` / `html_response` / `scraper_eligibility` / `sst_stats` / `sst.segment_stats`.
+- **next_step:** `done` when native alternate feeds exist, `capture_config` when articles were found, otherwise `scrape_url`.
 
 ### `capture_config`
 
@@ -84,8 +94,8 @@ Analyzes a target URL and derives a reusable YAML draft (`items` selector + `enh
   - `url` _(string, required)_: Source page URL.
   - `strategy` _(string, optional)_: Request strategy (`auto`, `faraday`, `botasaurus`). Default: `auto`.
   - `items_selector` _(string, optional)_: Optional CSS selector hint for items.
-- **Return value:** YAML document (string keys). Treat as a draft: catalog feeds still need `directory.topics` and explicit channel `title`/`url`. Strive to keep `enhance: true`.
-- **Metadata (`_meta`):** Includes `articles_count`, `channel_title`, `has_selectors`, and `requested_strategy`.
+- **Return value:** Envelope with `payload.yaml` (string keys; same serializer as CLI `html2rss capture`). Treat as a draft: catalog feeds still need `directory.topics` and explicit channel `title`/`url`. Strive to keep `enhance: true`.
+- **payload:** Also `articles_count`, `channel_title`, `has_selectors`, and `requested_strategy`. Follow `next_step` (`validate_config` or `inspect_url`).
 
 ### `validate_config`
 
@@ -94,8 +104,8 @@ Validates a feed configuration against the official `html2rss` JSON schema (`htm
 - **Parameters** (exactly one of `config` or `yaml`):
   - `config` _(object, optional)_: Feed configuration object.
   - `yaml` _(string, optional)_: Feed configuration YAML (same form as CLI capture / catalog files).
-- **Return value:** Text response `Config is valid.` on success.
-- **Error handling:** Returns `isError: true` with a serialized JSON error dictionary if schema validation fails, or if both/neither argument is provided.
+- **Return value:** Envelope with empty `payload` on success (`next_step`: `apply_config`).
+- **Error handling:** Returns `isError: true` with `payload.errors` if schema validation fails, or `next_step: validate_config` if both/neither argument is provided.
 
 ### `apply_config`
 
@@ -105,9 +115,8 @@ Executes a validated feed configuration against a page and returns RSS 2.0 XML.
   - `url` _(string, required)_: Source page URL (populates `channel.url` if omitted from config).
   - `config` _(object, optional)_: Feed configuration object (XOR `yaml`).
   - `yaml` _(string, optional)_: Feed configuration YAML (XOR `config`).
-- **Return value:** RSS 2.0 XML string.
-- **Metadata (`_meta`):** `item_count` from the rendered RSS `items` (not a regex over the XML).
-- **Error handling:** `isError: true` when the scrape produced no items, or on unhandled exceptions.
+- **Return value:** Envelope with `payload.rss` (RSS 2.0 XML) and `payload.item_count` from the rendered RSS `items` (not a regex over the XML).
+- **Error handling:** `isError: true` when the scrape produced no items (`payload.item_count: 0`), or on unhandled exceptions.
 
 ## Resources Reference
 
@@ -117,7 +126,7 @@ The MCP server exposes reference data under the `html2rss://` URI scheme:
 | :-------------------------- | :----------------- | :----------------------------------------------------------------------------------------------------------------------- |
 | **`html2rss://schema`**     | `application/json` | The complete JSON Schema for `html2rss` feed configurations, including selector rules, extractors, and request controls. |
 | **`html2rss://extractors`** | `application/json` | Alphabetical list of all registered extractor names (`attribute`, `html`, `href`, `text`, `static`, etc.).               |
-| **`html2rss://strategies`** | `application/json` | List of all registered request strategies (`auto`, `faraday`, `botasaurus`, `local_file`).                               |
+| **`html2rss://strategies`** | `application/json` | Published MCP request strategies (`auto`, `faraday`, `botasaurus`). Does not list `local_file`.                          |
 | **`html2rss://runtime`**    | `application/json` | Boolean `botasaurus_configured` from process env. Never includes the scraper URL.                                        |
 
 ## Prompts Reference
@@ -135,10 +144,10 @@ One `scrape_url` call with `strategy: auto` (AutoFallback already hops). Call `i
 
 Four-step workflow for a durable feed config:
 
-1. Call `capture_config` — YAML draft. Check `_meta.articles_count` and `has_selectors`. Strive `enhance: true`.
+1. Call `capture_config` — YAML is `payload.yaml`. Check `payload.articles_count` and `has_selectors`. Strive `enhance: true`.
 2. If weak or you need recon, call `inspect_url`. Auto already hops to Botasaurus.
 3. Call `validate_config` with `yaml` (or `config`) — must not be `isError`.
-4. Call `apply_config` — `isError` if zero items. Confirm `_meta.item_count` before shipping.
+4. Call `apply_config` — `isError` if zero items. Confirm `payload.item_count` before shipping.
 
 If the destination is html2rss-configs, rewrite the draft for `directory.topics` and explicit channel `title`/`url`.
 

From 7fc7d0f9bb1fa388f45a978d14b85ca2dabe4ee6 Mon Sep 17 00:00:00 2001
From: Gil Desmarais <git@desmarais.de>
Date: Wed, 19 Aug 2026 01:49:39 +0200
Subject: [PATCH 3/8] docs(domain): document admission, enhance leftover, and
 Status drops

Cleanup owns feed-item admission; enhance fills missing card fields and drops listing chrome. Status.admission_drops and auto_feed_result/--explain expose the same telemetry.
---
 .../ruby-gem/guides/managing-feed-configs.mdx |  4 +-
 .../docs/ruby-gem/reference/auto-source.mdx   | 46 +++++++++++--------
 .../docs/ruby-gem/reference/selectors.mdx     |  6 ++-
 3 files changed, 35 insertions(+), 21 deletions(-)

diff --git a/src/content/docs/ruby-gem/guides/managing-feed-configs.mdx b/src/content/docs/ruby-gem/guides/managing-feed-configs.mdx
index d346afa0..e1a635f2 100644
--- a/src/content/docs/ruby-gem/guides/managing-feed-configs.mdx
+++ b/src/content/docs/ruby-gem/guides/managing-feed-configs.mdx
@@ -76,7 +76,9 @@ Prefer `Html2rss.feed_result` when one scrape must render as both RSS and JSON F
 - `to_rss` / `to_json_feed(feed_url:)` — render formats
 - `status` — scrape telemetry (`Html2rss::Status`)
 
-`status.to_h` is the stable observability payload. Always includes `version` and `dedup_dropped`. When present, it may also include `scraper_tallies`, `selected_strategy`, `attempt_count`, and `strategy_attempts` (auto-fallback attempts; empty outside `strategy: auto`).
+`status.to_h` is the stable observability payload. Always includes `version` and `dedup_dropped`. When present, it may also include `scraper_tallies`, `selected_strategy`, `attempt_count`, `strategy_attempts` (auto-fallback attempts; empty outside `strategy: auto`), and `admission_drops` (Cleanup reason → count).
+
+For URL-only auto discovery, `Html2rss.auto_feed_result(url)` returns the same `FeedResult` (CLI `html2rss auto --explain` prints `status.to_h` on stderr).
 
 <Code
   code={`
diff --git a/src/content/docs/ruby-gem/reference/auto-source.mdx b/src/content/docs/ruby-gem/reference/auto-source.mdx
index ce9c9bc6..2007795c 100644
--- a/src/content/docs/ruby-gem/reference/auto-source.mdx
+++ b/src/content/docs/ruby-gem/reference/auto-source.mdx
@@ -20,25 +20,33 @@ To enable it, add `auto_source: {}` to your configuration:
 
 ## How It Works
 
-`auto_source` uses the following strategies to find content:
-
-1.  **`wordpress_api`:** Detects the `<link rel="https://api.w.org/">` tag used by WordPress and pulls posts from the REST API without parsing article HTML. See [WordPress API](/ruby-gem/reference/wordpress-api/).
-2.  **`sitemap`:** Automatically locates XML sitemap documents (`<link rel="sitemap">`, `/sitemap.xml`, or `/robots.txt`), filtering entries by priority and recency, with support for Google News tags (`<news:news>`).
-3.  **`meta_oembed`:** Extracts single-item articles, video pages, and media updates from OpenGraph/Twitter meta tags and resolves JSON oEmbed endpoints (`<link rel="alternate" type="application/json+oembed">`).
-4.  **`schema`:** Parses `<script type="json/ld">` tags containing structured data (e.g., [Schema.org](https://schema.org/)), extracting authors and publishers.
-5.  **`microdata`:** Extracts HTML Microdata annotations (`itemscope itemtype`).
-6.  **`microformats2`:** Parses Microformats2 `h-entry` markup, extracting `p-name`, `e-content`, `u-url`, `dt-published`, `p-author`, `p-category`, and `u-photo` / `u-featured` media.
-7.  **`json_state`:** Single-page applications often stash pre-rendered article data in `<script type="application/json">` tags or global variables
-    such as `window.__NEXT_DATA__`, `window.__NUXT__`, or `window.STATE`. The JSON-state scraper walks those blobs, finds arrays with
-    `title`/`url` pairs, and converts them into feed items.
-8.  **`xhr_articles`:** When Botasaurus runs the browser tier, it always captures JSON XHR/fetch response bodies. This scraper reuses those
-    captured bodies (no extra HTTP requests) and extracts article-like arrays the same way `json_state` does. Empty for Faraday or Botasaurus HTTP-request tiers.
-9.  **`semantic_html`:** Searches for semantic HTML5 tags like `<article>`, `<main>`, and `<section>`, with fallback clustering for anchorless lists.
-10. **`html`:** Analyzes HTML structure to find frequently occurring item selectors, with tag-structure and class-clustering fallbacks.
-
-`auto_source` also automatically extracts and prepends kicker/teaser labels to article titles when present, and discovers channel-level `author` and `image` (favicon, touch-icon, or OpenGraph images) from the host page.
-
-You can also restrict the maximum number of discovered articles via the `limit:` keyword argument in Ruby (`Html2rss.auto_source(url, limit: 10)`) or the `--limit` CLI flag.
+One fetch, then scrapers run in **tiers**. Within a tier, results merge. The pipeline stops when Cleanup has admitted enough articles (`limit`, default `25`).
+
+1. **In-page structured:** `schema`, `microdata`, `microformats2`, `json_state`, `xhr_articles`
+2. **Follow-up IO:** `wordpress_api`, `sitemap`, `meta_oembed`
+3. **Heuristic:** `semantic_html`
+4. **Heuristic:** `html` — skipped when an earlier tier already admitted at least one clean article
+
+What each scraper does:
+
+- **`schema`:** Parses `<script type="json/ld">` tags containing structured data (e.g., [Schema.org](https://schema.org/)), extracting authors and publishers.
+- **`microdata`:** Extracts HTML Microdata annotations (`itemscope itemtype`).
+- **`microformats2`:** Parses Microformats2 `h-entry` markup, extracting `p-name`, `e-content`, `u-url`, `dt-published`, `p-author`, `p-category`, and `u-photo` / `u-featured` media.
+- **`json_state`:** Walks in-page JSON (`<script type="application/json">`, `window.__NEXT_DATA__`, `window.__NUXT__`, `window.STATE`) for arrays with `title`/`url` pairs.
+- **`xhr_articles`:** Reuses JSON XHR/fetch bodies captured during a Botasaurus **browser** scrape (no extra HTTP). Empty for Faraday and Botasaurus HTTP-request tiers.
+- **`wordpress_api`:** Detects `<link rel="https://api.w.org/">` and pulls posts from the REST API. See [WordPress API](/ruby-gem/reference/wordpress-api/).
+- **`sitemap`:** Locates XML sitemaps (`<link rel="sitemap">`, `/sitemap.xml`, or `/robots.txt`), filtering by priority and recency, with Google News tags (`<news:news>`).
+- **`meta_oembed`:** OpenGraph/Twitter meta tags plus JSON oEmbed (`<link rel="alternate" type="application/json+oembed">`).
+- **`semantic_html`:** Semantic HTML5 (`<article>`, `<main>`, `<section>`), with fallback clustering for anchorless lists.
+- **`html`:** Repeated item structure (list / cluster / semantic segmenter). Last resort only.
+
+After scrapers, **Cleanup** is the admission gate: merge, dedupe, hard-exclude non-article destinations, drop junk titles, trim to `limit`. Scoring ranks and demotes; it does not refill `limit` after Cleanup.
+
+Drop reasons (counts) land on `Html2rss::Status#admission_drops`. Print them with `html2rss auto URL --explain` (JSON on stderr; the feed stays on stdout). Prefer `Html2rss.auto_feed_result` in Ruby when you need that status object.
+
+`auto_source` also prepends kicker/teaser labels to titles when present, and discovers channel-level `author` and `image` (favicon, touch-icon, or OpenGraph images) from the host page.
+
+Ruby `limit:` and CLI `--limit` cap kept articles (default `25`).
 
 **`json_state` / `xhr_articles` Limitations:** both scrapers require discoverable arrays of hashes containing clear `title` and `url` fields. Minified or
 obfuscated payloads, heavily encoded values, or blobs that require executing embedded functions are ignored. `xhr_articles` only sees JSON bodies captured during the current Botasaurus browser scrape (capped by the scrape API).
diff --git a/src/content/docs/ruby-gem/reference/selectors.mdx b/src/content/docs/ruby-gem/reference/selectors.mdx
index 616d33c1..0eabe9b9 100644
--- a/src/content/docs/ruby-gem/reference/selectors.mdx
+++ b/src/content/docs/ruby-gem/reference/selectors.mdx
@@ -28,7 +28,11 @@ At a minimum, you need an `items` selector to define the list of articles and a
 
 ## Automatic Item Enhancement
 
-To simplify configuration, `html2rss` can automatically extract the `title`, `url`, and `image` from each item. This feature is enabled by default.
+With `enhance: true` (the default), `html2rss` fills **missing** fields on each matched item from the card HTML: `title`, `url`, `description`, `published_at`, `image`, `categories`, and enclosures. Explicit selectors still win when they produce a value.
+
+Description leftover text drops listing chrome (CTA lines such as “read more”, date-shaped chips, type chips, title echoes, section names). When the `items` selector is a heading or wrapping link with no date or description, extraction can climb to the parent card — it aborts if that parent looks like a listing (multiple headings or article links).
+
+Capture emits `items` + `enhance: true` only. Keep `enhance: true` unless page chrome leaks into items.
 
 <Code
   code={`

From f28c492a617be02268faa12eef74f8d884386e13 Mon Sep 17 00:00:00 2001
From: Gil Desmarais <git@desmarais.de>
Date: Wed, 19 Aug 2026 01:49:44 +0200
Subject: [PATCH 4/8] docs(use-cases): sync capture, MCP logs, and Botasaurus
 request schema

Capture stamps strategy and supports --explain; MCP daemons log to stderr at info. Replace scroll_to_bottom with scroll and document the OpenAPI 2.0 window_size object.
---
 .../docs/ruby-gem/guides/ai-agent-workflows.mdx    |  4 ++++
 .../ruby-gem/guides/backward-compatibility.mdx     | 14 +++++++++++++-
 .../ruby-gem/guides/capturing-feed-configs.mdx     |  7 +++++--
 .../docs/ruby-gem/guides/custom-http-requests.mdx  |  2 +-
 .../ruby-gem/guides/handling-dynamic-content.mdx   |  6 +++---
 5 files changed, 26 insertions(+), 7 deletions(-)

diff --git a/src/content/docs/ruby-gem/guides/ai-agent-workflows.mdx b/src/content/docs/ruby-gem/guides/ai-agent-workflows.mdx
index 91c9cd0c..30613020 100644
--- a/src/content/docs/ruby-gem/guides/ai-agent-workflows.mdx
+++ b/src/content/docs/ruby-gem/guides/ai-agent-workflows.mdx
@@ -110,3 +110,7 @@ For dynamic JavaScript single-page applications or sites protected by anti-bot m
 />
 
 Put `BOTASAURUS_SCRAPER_URL` (typically `http://127.0.0.1:4010`) in the MCP server `env` block above — a shell export does not reach the Cursor/Claude MCP process. Confirm with `html2rss://runtime` (`botasaurus_configured: true`). AutoFallback then hops without a second tool call.
+
+## Daemon logs (stderr)
+
+`html2rss mcp` uses stdout for JSON-RPC, so the daemon logs to **stderr**. Unless you set `LOG_LEVEL`, the MCP process uses `info` (the gem library default stays `warn`). A foreground watcher sees the start banner, each tool call, and pipeline fallbacks. Use `LOG_LEVEL=debug` for more detail or `LOG_LEVEL=warn` to quiet it.
diff --git a/src/content/docs/ruby-gem/guides/backward-compatibility.mdx b/src/content/docs/ruby-gem/guides/backward-compatibility.mdx
index 39ebd2b4..a4901c4e 100644
--- a/src/content/docs/ruby-gem/guides/backward-compatibility.mdx
+++ b/src/content/docs/ruby-gem/guides/backward-compatibility.mdx
@@ -7,6 +7,17 @@ import { Code } from "@astrojs/starlight/components";
 
 This page outlines recent breaking changes, purged legacy aliases, and migration steps for older feed configurations.
 
+## Current Botasaurus request schema
+
+`request.botasaurus` matches the scrape-API OpenAPI 2.0 `ScrapeRequest` (unknown keys rejected):
+
+1. **`scroll`** — boolean. There is no `scroll_to_bottom` key; use `scroll: true` for infinite lists.
+2. **`window_size`** — object `{ width: 1920, height: 1080 }`, not a two-item array.
+3. **`wait_timeout_seconds`** — integer `1..20` (omit to use the scrape-API default of `15`).
+4. **`max_retries`** — integer `0..3` (omit to use the scrape-API default of `2`).
+
+Run `html2rss validate config.yml` after editing Botasaurus options.
+
 ## 0.26.0
 
 Upgrade notes for configs and integrators targeting gem **0.26.0**:
@@ -97,4 +108,5 @@ When upgrading to modern `html2rss` releases:
 4. **Move channel-level transport keys**: Ensure `strategy` and `headers` are defined at the top level of the YAML file.
 5. **Use `enclosure` for RSS media**: Do not rely on `image` becoming an RSS `<enclosure>`; select podcast/media URLs with `enclosure`.
 6. **Prefer `feed_result` for dual-format / cache**: Integrators that need RSS + JSON Feed from one scrape (or Marshal caching) should use `Html2rss.feed_result` and `status.to_h`.
-7. **Validate configurations**: Run `html2rss validate config.yml` to ensure your YAML conforms to the current schema.
+7. **Align `request.botasaurus` with OpenAPI 2.0**: Replace `scroll_to_bottom` with `scroll`, use `{ width, height }` for `window_size`, and keep `wait_timeout_seconds` in `1..20`.
+8. **Validate configurations**: Run `html2rss validate config.yml` to ensure your YAML conforms to the current schema.
diff --git a/src/content/docs/ruby-gem/guides/capturing-feed-configs.mdx b/src/content/docs/ruby-gem/guides/capturing-feed-configs.mdx
index 180f46d6..86c56af4 100644
--- a/src/content/docs/ruby-gem/guides/capturing-feed-configs.mdx
+++ b/src/content/docs/ruby-gem/guides/capturing-feed-configs.mdx
@@ -36,6 +36,8 @@ Print YAML to stdout:
   html2rss capture https://example.com/articles --items_selector ".post-card"
   # Analyze a local HTML fixture
   html2rss capture https://example.com/articles --input ./page.html
+  # Quality JSON on stderr; YAML stays on stdout
+  html2rss capture https://example.com/articles --explain
   # Save the draft
   html2rss capture https://example.com/articles > my-feed.yml
 `}
@@ -46,9 +48,10 @@ Common options:
 
 - `--strategy` — `auto`, `faraday`, `botasaurus`, or `local_file` (default `auto`)
 - `--items_selector` — CSS selector hint for item cards
-- `--limit` — maximum articles kept while deriving selectors
+- `--limit` — maximum articles kept while deriving selectors (default `25`)
 - `--max-redirects` / `--max-requests` — request budget overrides
 - `--input` — local HTML file (sets `local_file` strategy)
+- `--explain` — print capture quality JSON on stderr (`articles_count`, `channel_title`, `has_selectors`, `segment_strategy`, `selected_strategy`, `admission_drops`); YAML stays on stdout
 
 See the [CLI reference](/ruby-gem/reference/cli-reference/#capture) for the full flag list.
 
@@ -77,7 +80,7 @@ See the [CLI reference](/ruby-gem/reference/cli-reference/#capture) for the full
 2. **Discover** — AutoSource extracts admitted articles
 3. **Segment** — SST Segmenter strategies `:list` → `:cluster` → `:semantic`
 4. **Gate** — emit an items selector only when enough articles match
-5. **Assemble** — `{ items: { selector:, enhance: true } }` plus channel
+5. **Assemble** — `{ items: { selector:, enhance: true } }` plus channel. When AutoFallback selects a concrete transport (or you pin one), Capture **stamps** `strategy:` into the YAML so later `html2rss feed` / `Html2rss.feed` replay the same transport.
 
 When the quality gate fails, selectors are omitted (`has_selectors: false`) rather than inventing attribute selectors. Hint with `--items_selector` or refine by hand.
 
diff --git a/src/content/docs/ruby-gem/guides/custom-http-requests.mdx b/src/content/docs/ruby-gem/guides/custom-http-requests.mdx
index e9302d8b..91110c28 100644
--- a/src/content/docs/ruby-gem/guides/custom-http-requests.mdx
+++ b/src/content/docs/ruby-gem/guides/custom-http-requests.mdx
@@ -73,7 +73,7 @@ Request budgets are configured under `request`, not as top-level keys:
 
 - `request.max_redirects` limits redirect hops
 - `request.max_requests` limits the total request budget for the feed build
-- `request.botasaurus.*` is reserved for Botasaurus-only behavior such as navigation mode, retries, header/cookie forwarding, and scroll options
+- `request.botasaurus.*` is reserved for Botasaurus-only behavior such as navigation mode, retries, header/cookie forwarding, and `scroll`
 
 ## Common Use Cases
 
diff --git a/src/content/docs/ruby-gem/guides/handling-dynamic-content.mdx b/src/content/docs/ruby-gem/guides/handling-dynamic-content.mdx
index dd206831..89e930b8 100644
--- a/src/content/docs/ruby-gem/guides/handling-dynamic-content.mdx
+++ b/src/content/docs/ruby-gem/guides/handling-dynamic-content.mdx
@@ -22,7 +22,7 @@ When a site requires browser rendering or anti-bot bypass by default, you can ex
     botasaurus:
       navigation_mode: google_get_bypass
       wait_for_selector: ".article-card"
-      scroll_to_bottom: true
+      scroll: true
   channel:
     url: https://example.com/app
   selectors:
@@ -44,7 +44,7 @@ Botasaurus extraction is ideal when:
 - **Content loads after page load** - JavaScript fetches data dynamically
 - **Single Page Applications (SPAs)** - React, Vue, Angular apps
 - **Anti-bot interstitials** - Cloudflare, Google search referrers, or challenge pages
-- **Scroll-triggered content** - Infinite scroll lists (`scroll_to_bottom: true`)
+- **Scroll-triggered content** - Infinite scroll lists (`scroll: true`)
 
 ## Botasaurus Request Controls
 
@@ -70,7 +70,7 @@ Configure browser actions under `request.botasaurus`:
   strategy: botasaurus
   request:
     botasaurus:
-      scroll_to_bottom: true
+      scroll: true
   `}
   lang="yaml"
 />

From 9677b3a0d6f9850d2d053104d6c53a28baacfb5f Mon Sep 17 00:00:00 2001
From: Gil Desmarais <git@desmarais.de>
Date: Wed, 19 Aug 2026 01:49:52 +0200
Subject: [PATCH 5/8] docs(adapters): document CLI --explain and Botasaurus
 OpenAPI 2.0 options

auto/capture --explain print telemetry on stderr. MCP inspect payload includes admission and XHR recon. request.botasaurus matches the scrape-API contract (no extra keys, window_size object).
---
 .../docs/ruby-gem/reference/cli-reference.mdx | 12 ++++++---
 .../docs/ruby-gem/reference/mcp-server.mdx    | 24 ++++++++++++++---
 .../docs/ruby-gem/reference/strategy.mdx      | 26 ++++++++++---------
 3 files changed, 43 insertions(+), 19 deletions(-)

diff --git a/src/content/docs/ruby-gem/reference/cli-reference.mdx b/src/content/docs/ruby-gem/reference/cli-reference.mdx
index d4e8355d..679bdc5f 100644
--- a/src/content/docs/ruby-gem/reference/cli-reference.mdx
+++ b/src/content/docs/ruby-gem/reference/cli-reference.mdx
@@ -26,6 +26,7 @@ Automatically discovers items from a page and prints the generated RSS or JSONFe
   html2rss auto https://example.com/articles --limit 10 ; \\
   BOTASAURUS_SCRAPER_URL="http://localhost:4010" html2rss auto https://example.com/protected --strategy botasaurus ; \\
   html2rss auto https://example.com/articles --items_selector ".post-card" ; \\
+  html2rss auto https://example.com/articles --explain ; \\
   html2rss auto --input sample.html ; \\
   html2rss auto https://example.com/articles --input sample.html
 `}
@@ -38,11 +39,12 @@ Available options:
 
 - `--strategy`: Optional request strategy (`auto`, `faraday`, `botasaurus`, `local_file`). Defaults to `auto`, which tries `faraday` -> `botasaurus`.
 - `--format`: Output format for the auto-sourced feed (`rss` or `jsonfeed`). Defaults to `rss`.
-- `--limit`: Maximum number of articles to extract during discovery (defaults to `20`).
+- `--limit`: Maximum number of articles to extract during discovery (defaults to `25`).
 - `--items_selector`: Optional CSS selector hint for item extraction.
 - `--max-redirects`: Maximum redirects to follow per request.
 - `--max-requests`: Maximum requests to allow for this feed build (defaults to `4` for auto discovery).
 - `--input`: Local HTML file path to read input from without making network requests. When provided without a `URL`, `html2rss` attempts to detect the base URL from `<link rel="canonical">` or OpenGraph metadata in the HTML.
+- `--explain`: Print `Html2rss::Status` JSON to stderr (`version`, `dedup_dropped`, and when present `scraper_tallies`, `selected_strategy`, `attempt_count`, `strategy_attempts`, `admission_drops`). Stdout stays the RSS or JSON Feed.
 
 #### URL Surface Guidance For `auto`
 
@@ -131,13 +133,14 @@ The CLI keeps `strategy` as a top-level override and writes runtime request limi
 
 ### Capture
 
-Analyzes a URL using auto-source structural analysis and prints a reusable YAML feed configuration with derived CSS selectors to stdout.
+Analyzes a URL using auto-source structural analysis and prints a reusable YAML feed configuration (`items` selector + `enhance: true`) to stdout.
 
 <Code
   code={`
   html2rss capture https://example.com/articles ; \\
   html2rss capture https://example.com/articles --strategy botasaurus ; \\
   html2rss capture https://example.com/articles --items_selector ".post-card" ; \\
+  html2rss capture https://example.com/articles --explain ; \\
   html2rss capture --input sample.html
 `}
   lang="bash"
@@ -153,6 +156,9 @@ Available options:
 - `--max-redirects`: Maximum redirects to follow per request.
 - `--max-requests`: Maximum requests to allow for this feed build.
 - `--input`: Local HTML file path to read input from without making network requests.
+- `--explain`: Print capture quality JSON to stderr (`articles_count`, `channel_title`, `has_selectors`, `segment_strategy`, `selected_strategy`, `admission_drops`). Stdout stays YAML.
+
+When AutoFallback (or a pinned strategy) selects a concrete transport, the printed YAML includes a top-level `strategy:` so later `html2rss feed` uses the same hop.
 
 ### MCP
 
@@ -174,7 +180,7 @@ Available options:
 - `--transport`: MCP transport protocol (`stdio` or `http`). Defaults to `stdio`.
 - `--port`: Port for HTTP transport (default: `8080`). Bound strictly to `127.0.0.1` (loopback only).
 
-HTTP transport requires the `rack`, `rackup`, and `webrick` gems. See the [MCP Server Reference](/ruby-gem/reference/mcp-server/) and [AI Agent Workflows Guide](/ruby-gem/guides/ai-agent-workflows/) for full configuration and tool schemas.
+HTTP transport requires the `rack`, `rackup`, and `webrick` gems. stdio keeps JSON-RPC on stdout; the daemon logs to stderr and defaults to `LOG_LEVEL=info` unless you set `LOG_LEVEL`. See the [MCP Server Reference](/ruby-gem/reference/mcp-server/) and [AI Agent Workflows Guide](/ruby-gem/guides/ai-agent-workflows/) for full configuration and tool schemas.
 
 ### Schema
 
diff --git a/src/content/docs/ruby-gem/reference/mcp-server.mdx b/src/content/docs/ruby-gem/reference/mcp-server.mdx
index 5c7b2ed5..a95a2ecf 100644
--- a/src/content/docs/ruby-gem/reference/mcp-server.mdx
+++ b/src/content/docs/ruby-gem/reference/mcp-server.mdx
@@ -34,6 +34,7 @@ The MCP server supports two transport mechanisms:
 
 - **Loopback binding:** The HTTP transport only binds to `127.0.0.1`. Never expose it to public interfaces without authentication and reverse proxy origin controls.
 - **Dependencies:** The HTTP transport requires the `rack`, `rackup`, and `webrick` gems. If they are not present, starting the server raises a descriptive `LoadError`.
+- **Logging:** stdio JSON-RPC uses stdout. The daemon logs to stderr and defaults to `LOG_LEVEL=info` unless `LOG_LEVEL` is set (`debug` for more, `warn` to quiet).
 
 ## Strategy Resolution in MCP
 
@@ -47,14 +48,24 @@ Read `html2rss://runtime` for `botasaurus_configured` (boolean only). Set `BOTAS
 
 ## Result envelope
 
-Every tool returns the same JSON object in both the text content and `structuredContent` (there is no `_meta` quality channel):
+Every tool returns the same JSON object in both the text content and `structuredContent`:
 
-```json
-{ "ok": true, "next_step": "validate_config", "guidance": "...", "payload": {} }
-```
+<Code
+  code={`
+  {
+    "ok": true,
+    "next_step": "validate_config",
+    "guidance": "...",
+    "payload": {}
+  }
+`}
+  lang="json"
+/>
 
 `isError` matches `ok: false`. Hosts that parsed scrape text as a raw JSON array must read `payload.items` instead.
 
+`next_step` is one of: `done`, `inspect_url`, `validate_config`, `apply_config`, `scrape_url`, `capture_config`, `read_runtime`.
+
 ## Tools Reference
 
 The server registers 5 primary tools for AI agent orchestration.
@@ -84,6 +95,9 @@ Diagnostic page analysis plus recon from the same fetch. `auto` uses Faraday; pi
   - `scheme_downgrade`: `true` when the fetch went https → http.
   - `alternate_feeds`: `rel=alternate` RSS/Atom links from the document head (no `/feed` path guessing).
   - `content_type` / `html_response` / `scraper_eligibility` / `sst_stats` / `sst.segment_stats`.
+  - `articles_count` and `admission_drops` when the response is HTML (Cleanup drop-reason counts).
+  - `blocked_surface` when an interstitial signature is detected.
+  - `xhr_capture` when the inspect strategy is `botasaurus` (`count`, redacted `sample_endpoints`, `candidate_articles`).
 - **next_step:** `done` when native alternate feeds exist, `capture_config` when articles were found, otherwise `scrape_url`.
 
 ### `capture_config`
@@ -129,6 +143,8 @@ The MCP server exposes reference data under the `html2rss://` URI scheme:
 | **`html2rss://strategies`** | `application/json` | Published MCP request strategies (`auto`, `faraday`, `botasaurus`). Does not list `local_file`.                          |
 | **`html2rss://runtime`**    | `application/json` | Boolean `botasaurus_configured` from process env. Never includes the scraper URL.                                        |
 
+`validate_config` / `apply_config` reject `strategy: local_file` and `request.local_file_path`. Use CLI `--input` for fixtures.
+
 ## Prompts Reference
 
 The server publishes guided prompt workflows for AI assistants:
diff --git a/src/content/docs/ruby-gem/reference/strategy.mdx b/src/content/docs/ruby-gem/reference/strategy.mdx
index d0710e68..d67999cb 100644
--- a/src/content/docs/ruby-gem/reference/strategy.mdx
+++ b/src/content/docs/ruby-gem/reference/strategy.mdx
@@ -30,6 +30,8 @@ Under `auto`, `Html2rss.feed_result(...).status` exposes scrape telemetry: `sele
 
 `faraday` is the standard static HTTP transport. When no custom `User-Agent` header is supplied in `headers`, `html2rss` sends `html2rss/<VERSION>` by default.
 
+HTML is accepted when `Content-Type` includes `text/html`, or when the body sniffs as HTML (`<!DOCTYPE html` / `<html`) and the response is not JSON. Gzip, deflate, and brotli bodies are decoded even when `Content-Encoding` is missing; unlabeled brotli is tried for `octet-stream`.
+
 ## `botasaurus`
 
 `botasaurus` delegates page fetching to a Botasaurus scrape API endpoint. It runs as the second tier of `auto`, or when you pin `strategy: botasaurus`.
@@ -43,6 +45,8 @@ html2rss enforces local request policy preflight and timeout budget. When a tota
 
 During the **browser** execution tier, the Botasaurus scrape API always captures JSON XHR/fetch response bodies (`xhr_responses`). html2rss forwards them as `Response#captured_responses` so AutoSource `xhr_articles` can extract articles without extra requests. The HTTP-request tier returns an empty capture list. Caps and filtering live in the scrape API (see its README); article-likeness filtering is a client concern.
 
+The client speaks OpenAPI 2.0 `ScrapeSuccess` / `ScrapeError` envelopes. Success requires an `html` string. Extra `request.botasaurus` keys fail YAML validation.
+
 ### Configuration
 
 <Code
@@ -54,21 +58,20 @@ During the **browser** execution tier, the Botasaurus scrape API always captures
   request:
     botasaurus:
       navigation_mode: auto
-      max_retries: 1
+      max_retries: 2
       headless: false
   `}
   lang="yml"
 />
 
-Supported `request.botasaurus` options:
+Supported `request.botasaurus` options (unknown keys are rejected):
 
-- `execution_mode` (`auto`, `request`, `browser`; default `auto`)
+- `execution_mode` (`auto`, `request`, `browser`; scrape-API default `auto` when omitted)
 - `navigation_mode` (`auto`, `get`, `google_get`, `google_get_bypass`, `organic_get`; default `auto`)
-- `max_retries` (`0..3`; default `1`)
+- `max_retries` (`0..3`; omit to use the scrape-API default of `2`)
 - `wait_for_selector` (string)
-- `wait_timeout_seconds` (integer)
+- `wait_timeout_seconds` (integer `1..20`; omit to use the scrape-API default of `15`)
 - `scroll` (boolean)
-- `scroll_to_bottom` (boolean)
 - `block_images` (boolean)
 - `block_images_and_css` (boolean)
 - `block_trackers` (boolean)
@@ -76,7 +79,7 @@ Supported `request.botasaurus` options:
 - `headless` (boolean, default `false`)
 - `proxy` (string)
 - `user_agent` (string)
-- `window_size` (two-item integer array, for example `[1920, 1080]`)
+- `window_size` (`{ width: 1920, height: 1080 }` — both positive integers, required together)
 - `lang` (string, for example `en-US`)
 - `headers` (hash, forwarded to the Botasaurus scrape API alongside config-level `headers`)
 - `cookies` (hash)
@@ -90,7 +93,7 @@ Example scrape-API payload shape:
   {
     "url": "https://example.com",
     "navigation_mode": "auto",
-    "max_retries": 1,
+    "max_retries": 2,
     "headless": false
   }
   `}
@@ -103,11 +106,10 @@ Example scrape-API payload shape:
   code={`
   BOTASAURUS_SCRAPER_URL="http://localhost:4010" \\
   html2rss auto https://example.com/updates --strategy botasaurus
-
-BOTASAURUS_SCRAPER_URL="http://localhost:4010" \\
-html2rss feed my_config.yml --strategy botasaurus
+  BOTASAURUS_SCRAPER_URL="http://localhost:4010" \\
+  html2rss feed my_config.yml --strategy botasaurus
 `}
-lang="bash"
+  lang="bash"
 />
 
 ## `local_file`

From 486e0b784018f03cfe3a6e932257f9a5e6a7cab1 Mon Sep 17 00:00:00 2001
From: Gil Desmarais <git@desmarais.de>
Date: Wed, 19 Aug 2026 01:49:52 +0200
Subject: [PATCH 6/8] docs(infrastructure): document Faraday HTML sniffing and
 Botasaurus validation

Faraday decodes unlabeled gzip/brotli and sniffs HTML bodies. Invalid Botasaurus keys (scroll_to_bottom, array window_size) fail validation; --explain surfaces admission drops.
---
 src/content/docs/troubleshooting/troubleshooting.mdx | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/src/content/docs/troubleshooting/troubleshooting.mdx b/src/content/docs/troubleshooting/troubleshooting.mdx
index 5719a615..f5879be1 100644
--- a/src/content/docs/troubleshooting/troubleshooting.mdx
+++ b/src/content/docs/troubleshooting/troubleshooting.mdx
@@ -30,7 +30,7 @@ The `auto` flow is URL-surface sensitive.
   - search result pages
   - client-rendered app-shell entrypoints
 
-If extraction quality is poor, switch to a more specific listing/update URL before tuning selectors.
+If extraction quality is poor, switch to a more specific listing/update URL before tuning selectors. `html2rss auto URL --explain` prints admission drop counts on stderr so you can see why items were dropped (junk titles, non-article destinations) without mixing JSON into the feed.
 
 ### Empty Feeds
 
@@ -95,12 +95,13 @@ Then run with:
 
 Common configuration-related errors:
 
-- **`UnsupportedResponseContentType`:** The website returned content that html2rss can't parse (not HTML or JSON).
+- **`UnsupportedResponseContentType`:** The website returned content that html2rss can't parse (not HTML or JSON). Faraday still treats bodies that sniff as HTML (`<!DOCTYPE html` / `<html`) as HTML, and decodes gzip/brotli even when `Content-Encoding` is missing.
 - **`UnsupportedStrategy`:** The specified strategy is not available. Use `auto`, `faraday`, `botasaurus`, or `local_file`.
 - **`BOTASAURUS_SCRAPER_URL is required for strategy=botasaurus.`:** Set `BOTASAURUS_SCRAPER_URL` to your Botasaurus scrape API base URL when using `--strategy botasaurus`.
 - **`BOTASAURUS_SCRAPER_URL is invalid`:** Fix the URL format and retry.
 - **`Configuration must include at least 'selectors' or 'auto_source'`:** You need to specify either manual selectors or enable auto-source.
 - **`stylesheet.type invalid`:** Only `text/css` and `text/xsl` are supported for stylesheets.
+- **`request.botasaurus` validation:** Unknown keys fail (`scroll_to_bottom` is not valid — use `scroll: true`). `window_size` must be `{ width:, height: }`. `wait_timeout_seconds` must be `1..20`.
 
 ### Missing Item Parts
 

From c38e7d8b7fa72d456366fe03b3243c2305e7e2a5 Mon Sep 17 00:00:00 2001
From: Gil Desmarais <git@desmarais.de>
Date: Wed, 19 Aug 2026 01:50:39 +0200
Subject: [PATCH 7/8] docs(adapters): fill MCP capture payload and apply
 next_step

Capture envelopes include optional segment/strategy/admission fields. Zero-item apply points agents at inspect_url.
---
 src/content/docs/ruby-gem/reference/mcp-server.mdx | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/content/docs/ruby-gem/reference/mcp-server.mdx b/src/content/docs/ruby-gem/reference/mcp-server.mdx
index a95a2ecf..194fc30d 100644
--- a/src/content/docs/ruby-gem/reference/mcp-server.mdx
+++ b/src/content/docs/ruby-gem/reference/mcp-server.mdx
@@ -109,7 +109,7 @@ Analyzes a target URL and derives a reusable YAML draft (`items` selector + `enh
   - `strategy` _(string, optional)_: Request strategy (`auto`, `faraday`, `botasaurus`). Default: `auto`.
   - `items_selector` _(string, optional)_: Optional CSS selector hint for items.
 - **Return value:** Envelope with `payload.yaml` (string keys; same serializer as CLI `html2rss capture`). Treat as a draft: catalog feeds still need `directory.topics` and explicit channel `title`/`url`. Strive to keep `enhance: true`.
-- **payload:** Also `articles_count`, `channel_title`, `has_selectors`, and `requested_strategy`. Follow `next_step` (`validate_config` or `inspect_url`).
+- **payload:** Also `articles_count`, `channel_title`, `has_selectors`, `requested_strategy`, and when present `segment_strategy`, `selected_strategy`, `admission_drops`. Follow `next_step` (`validate_config` or `inspect_url`).
 
 ### `validate_config`
 
@@ -130,7 +130,7 @@ Executes a validated feed configuration against a page and returns RSS 2.0 XML.
   - `config` _(object, optional)_: Feed configuration object (XOR `yaml`).
   - `yaml` _(string, optional)_: Feed configuration YAML (XOR `config`).
 - **Return value:** Envelope with `payload.rss` (RSS 2.0 XML) and `payload.item_count` from the rendered RSS `items` (not a regex over the XML).
-- **Error handling:** `isError: true` when the scrape produced no items (`payload.item_count: 0`), or on unhandled exceptions.
+- **Error handling:** `isError: true` when the scrape produced no items (`payload.item_count: 0`, `next_step: inspect_url`), or on unhandled exceptions.
 
 ## Resources Reference
 

From 11aa7a9b595a20923813f29df8adee812c77dce2 Mon Sep 17 00:00:00 2001
From: Gil Desmarais <git@desmarais.de>
Date: Wed, 19 Aug 2026 16:34:57 +0200
Subject: [PATCH 8/8] docs(ruby-gem): anchor 0.27.0 migrations and write
 forward
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

Forward-facing guides teach present MCP and capture behavior; breaking contract changes live under backward-compatibility §0.27.0. Fixes inspect_url auto staying on Faraday in MCP prompts.
---
 .../ruby-gem/guides/ai-agent-workflows.mdx    |  6 +--
 .../guides/backward-compatibility.mdx         | 42 +++++++++++--------
 .../guides/capturing-feed-configs.mdx         |  2 +-
 .../docs/ruby-gem/reference/cli-reference.mdx |  2 +-
 .../docs/ruby-gem/reference/mcp-server.mdx    |  8 ++--
 .../docs/troubleshooting/troubleshooting.mdx  |  2 +-
 6 files changed, 35 insertions(+), 27 deletions(-)

diff --git a/src/content/docs/ruby-gem/guides/ai-agent-workflows.mdx b/src/content/docs/ruby-gem/guides/ai-agent-workflows.mdx
index 30613020..c3064ce6 100644
--- a/src/content/docs/ruby-gem/guides/ai-agent-workflows.mdx
+++ b/src/content/docs/ruby-gem/guides/ai-agent-workflows.mdx
@@ -84,9 +84,9 @@ Add `html2rss` to your Claude Desktop configuration (`~/Library/Application Supp
 
 When an agent needs articles immediately without saving a feed configuration:
 
-1. Call `scrape_url` with `strategy: "auto"`. Auto already runs Faraday then Botasaurus.
-2. An empty item list is still success for articles-now. Read `payload.items`. Follow `next_step` (including `read_runtime` / `html2rss://runtime` if Botasaurus is unset).
-3. Call `inspect_url` only if the result is weak or you need recon. Do not retry `scrape_url` with explicit `faraday` after auto.
+1. Call `scrape_url` with `strategy: "auto"`. Auto runs Faraday then Botasaurus when `BOTASAURUS_SCRAPER_URL` is set on the MCP process.
+2. An empty item list is still success for articles-now. Read `payload.items`. Follow `next_step` (including `read_runtime` / `html2rss://runtime` when Botasaurus is unset).
+3. Call `inspect_url` when the result is weak or you need recon (`final_url`, `status`, `scheme_downgrade`, native RSS/Atom).
 
 ### Pattern B: Iterative Feed Config Authoring
 
diff --git a/src/content/docs/ruby-gem/guides/backward-compatibility.mdx b/src/content/docs/ruby-gem/guides/backward-compatibility.mdx
index a4901c4e..4e79df5b 100644
--- a/src/content/docs/ruby-gem/guides/backward-compatibility.mdx
+++ b/src/content/docs/ruby-gem/guides/backward-compatibility.mdx
@@ -5,16 +5,20 @@ description: "Migration guide and backward-compatibility reference for html2rss
 
 import { Code } from "@astrojs/starlight/components";
 
-This page outlines recent breaking changes, purged legacy aliases, and migration steps for older feed configurations.
+This page outlines breaking changes, purged legacy aliases, and migration steps when upgrading the gem.
 
-## Current Botasaurus request schema
+## 0.27.0
 
-`request.botasaurus` matches the scrape-API OpenAPI 2.0 `ScrapeRequest` (unknown keys rejected):
+Upgrade notes for configs and integrators targeting gem **0.27.0** (current `master` after `0.26.0`):
 
-1. **`scroll`** — boolean. There is no `scroll_to_bottom` key; use `scroll: true` for infinite lists.
-2. **`window_size`** — object `{ width: 1920, height: 1080 }`, not a two-item array.
-3. **`wait_timeout_seconds`** — integer `1..20` (omit to use the scrape-API default of `15`).
-4. **`max_retries`** — integer `0..3` (omit to use the scrape-API default of `2`).
+1. **MCP result envelope** — Every MCP tool returns `{ ok, next_step, guidance, payload }` in text and `structuredContent`. Read `payload.items`, `payload.yaml`, and `payload.rss`; scrape text is not a raw item array.
+2. **MCP AutoFallback** — `scrape_url` and `capture_config` with `strategy: "auto"` run Faraday → Botasaurus in one call. `inspect_url` with `auto` stays on Faraday (pin `botasaurus` when you need browser rendering for inspect).
+3. **Capture output** — `html2rss capture` and MCP `capture_config` emit an items selector plus `enhance: true` only (no per-field title/url/description selector soup). Capture stamps `strategy:` when AutoFallback selects a concrete transport.
+4. **CLI `--explain`** — `html2rss auto --explain` and `html2rss capture --explain` print telemetry JSON on stderr; stdout stays the feed or YAML. Status includes `admission_drops` when Cleanup dropped items.
+5. **`Html2rss.auto_feed_result`** — URL-only auto discovery returns `FeedResult` with the same `status` telemetry as config builds.
+6. **`request.botasaurus` OpenAPI 2.0** — Keys must match the scrape-API `ScrapeRequest` (unknown keys rejected). Use `scroll: true` (not `scroll_to_bottom`), `window_size: { width:, height: }` (not a two-item array), and `wait_timeout_seconds` in `1..20`. See [Strategy](/ruby-gem/reference/strategy/#botasaurus).
+7. **Botasaurus wire format** — Client expects OpenAPI 2.0 `ScrapeSuccess` / `ScrapeError` envelopes from the scrape API.
+8. **`--limit` default** — Auto discovery default is `25` (`AutoSource::DEFAULT_LIMIT`).
 
 Run `html2rss validate config.yml` after editing Botasaurus options.
 
@@ -30,7 +34,7 @@ Upgrade notes for configs and integrators targeting gem **0.26.0**:
 Upgrade notes for configs and integrators targeting gem **0.25.0**:
 
 1. **Removed `link` selector alias** — Custom selector configurations must use the canonical `url` key. The legacy `link` selector alias and exclusivity checks have been removed.
-2. **`auto` article limit (`--limit N`)** — Discovery now supports a configurable article limit (`--limit N` CLI flag, default `20`) and `Html2rss.auto_source(url, limit: 20)` in Ruby.
+2. **`auto` article limit (`--limit N`)** — Discovery supports a configurable limit (`--limit N` CLI flag). Default was `20` in 0.25.0; **0.27.0** default is `25` (`Html2rss.auto_source(url, limit: 25)`).
 3. **Channel metadata in `auto`** — `auto_source` automatically extracts channel-level `author` and `image` (favicon, touch-icon, or OpenGraph images) from the host page.
 4. **Actionable `NoFeedItemsExtracted` error** — `auto` mode fails loud with diagnostic surface classifications (blocked surface, app-shell, unsupported extraction surface) when zero items are found.
 
@@ -100,13 +104,17 @@ Update any occurrences of `link`, `pubDate`, or `updated` in your `selectors` bl
 
 ## Migration Checklist
 
-When upgrading to modern `html2rss` releases:
+When upgrading to **0.27.0** or later:
 
-1. **Migrate Browserless to Botasaurus**: Update any feeds using `strategy: browserless` or `request.browserless` to `strategy: botasaurus` or `strategy: auto` with `BOTASAURUS_SCRAPER_URL`.
-2. **Rename URL selectors**: Ensure article URL selectors use `url` rather than `link`.
-3. **Rename date selectors**: Ensure date selectors use `published_at` rather than `updated` or `pubDate`.
-4. **Move channel-level transport keys**: Ensure `strategy` and `headers` are defined at the top level of the YAML file.
-5. **Use `enclosure` for RSS media**: Do not rely on `image` becoming an RSS `<enclosure>`; select podcast/media URLs with `enclosure`.
-6. **Prefer `feed_result` for dual-format / cache**: Integrators that need RSS + JSON Feed from one scrape (or Marshal caching) should use `Html2rss.feed_result` and `status.to_h`.
-7. **Align `request.botasaurus` with OpenAPI 2.0**: Replace `scroll_to_bottom` with `scroll`, use `{ width, height }` for `window_size`, and keep `wait_timeout_seconds` in `1..20`.
-8. **Validate configurations**: Run `html2rss validate config.yml` to ensure your YAML conforms to the current schema.
+1. **MCP integrators**: Read tool results from the envelope `payload`; configure `BOTASAURUS_SCRAPER_URL` on the MCP process env.
+2. **Align `request.botasaurus` with OpenAPI 2.0**: Use `scroll`, `{ width, height }` for `window_size`, and `wait_timeout_seconds` in `1..20` (see [Strategy](/ruby-gem/reference/strategy/#botasaurus)).
+3. **Validate configurations**: Run `html2rss validate config.yml`.
+
+When upgrading from releases before **0.26.0**:
+
+4. **Migrate Browserless to Botasaurus**: Use `strategy: botasaurus` or `strategy: auto` with `BOTASAURUS_SCRAPER_URL`.
+5. **Rename URL selectors**: Use `url` rather than `link`.
+6. **Rename date selectors**: Use `published_at` rather than `updated` or `pubDate`.
+7. **Move channel-level transport keys**: Put `strategy` and `headers` at the top level of the YAML file.
+8. **Use `enclosure` for RSS media**: Select podcast/media URLs with `enclosure`; `image` does not become RSS `<enclosure>`.
+9. **Prefer `feed_result` for dual-format / cache**: Use `Html2rss.feed_result` / `Html2rss.auto_feed_result` and `status.to_h`.
diff --git a/src/content/docs/ruby-gem/guides/capturing-feed-configs.mdx b/src/content/docs/ruby-gem/guides/capturing-feed-configs.mdx
index 86c56af4..8f11d655 100644
--- a/src/content/docs/ruby-gem/guides/capturing-feed-configs.mdx
+++ b/src/content/docs/ruby-gem/guides/capturing-feed-configs.mdx
@@ -5,7 +5,7 @@ description: Derive a reusable YAML feed config from a URL with html2rss capture
 
 import { Aside, Code } from "@astrojs/starlight/components";
 
-`html2rss capture` analyzes a page through the auto-source pipeline and prints a reusable YAML draft with an **items selector plus `enhance: true`** — not title/url/description selector soup. `enhance: true` fills missing article fields at feed-build time.
+`html2rss capture` analyzes a page through the auto-source pipeline and prints a reusable YAML draft: an **items selector plus `enhance: true`**. At feed-build time, `enhance: true` fills missing article fields from each matched card.
 
 <Aside type="tip" title="Treat capture as a draft">
   Selector quality depends on page structure. Catalog feeds still need `directory.topics` and explicit channel
diff --git a/src/content/docs/ruby-gem/reference/cli-reference.mdx b/src/content/docs/ruby-gem/reference/cli-reference.mdx
index 679bdc5f..c9f1675b 100644
--- a/src/content/docs/ruby-gem/reference/cli-reference.mdx
+++ b/src/content/docs/ruby-gem/reference/cli-reference.mdx
@@ -180,7 +180,7 @@ Available options:
 - `--transport`: MCP transport protocol (`stdio` or `http`). Defaults to `stdio`.
 - `--port`: Port for HTTP transport (default: `8080`). Bound strictly to `127.0.0.1` (loopback only).
 
-HTTP transport requires the `rack`, `rackup`, and `webrick` gems. stdio keeps JSON-RPC on stdout; the daemon logs to stderr and defaults to `LOG_LEVEL=info` unless you set `LOG_LEVEL`. See the [MCP Server Reference](/ruby-gem/reference/mcp-server/) and [AI Agent Workflows Guide](/ruby-gem/guides/ai-agent-workflows/) for full configuration and tool schemas.
+HTTP transport requires the `rack`, `rackup`, and `webrick` gems. MCP daemon logging (stderr, `LOG_LEVEL`) is documented in the [MCP Server Reference](/ruby-gem/reference/mcp-server/). See also the [AI Agent Workflows Guide](/ruby-gem/guides/ai-agent-workflows/) for client setup and tool schemas.
 
 ### Schema
 
diff --git a/src/content/docs/ruby-gem/reference/mcp-server.mdx b/src/content/docs/ruby-gem/reference/mcp-server.mdx
index 194fc30d..193ec010 100644
--- a/src/content/docs/ruby-gem/reference/mcp-server.mdx
+++ b/src/content/docs/ruby-gem/reference/mcp-server.mdx
@@ -38,9 +38,9 @@ The MCP server supports two transport mechanisms:
 
 ## Strategy Resolution in MCP
 
-`scrape_url` and `capture_config` with `strategy: "auto"` (the default) run Faraday → Botasaurus AutoFallback. One call is enough; do not retry with explicit `faraday` after `auto`.
+`scrape_url` and `capture_config` with `strategy: "auto"` (the default) run Faraday → Botasaurus AutoFallback in one call.
 
-`inspect_url` is the exception: `auto` stays on Faraday for a cheap diagnostic. Pin `strategy: "botasaurus"` on inspect when you need browser rendering.
+`inspect_url` with `auto` uses Faraday for a cheap diagnostic. Pin `strategy: "botasaurus"` when you need browser rendering for inspect.
 
 Read `html2rss://runtime` for `botasaurus_configured` (boolean only). Set `BOTASAURUS_SCRAPER_URL` on the **MCP process** env (see [AI agent workflows](/ruby-gem/guides/ai-agent-workflows/)).
 
@@ -62,7 +62,7 @@ Every tool returns the same JSON object in both the text content and `structured
   lang="json"
 />
 
-`isError` matches `ok: false`. Hosts that parsed scrape text as a raw JSON array must read `payload.items` instead.
+`isError` matches `ok: false`. Scrape results expose articles in `payload.items`.
 
 `next_step` is one of: `done`, `inspect_url`, `validate_config`, `apply_config`, `scrape_url`, `capture_config`, `read_runtime`.
 
@@ -161,7 +161,7 @@ One `scrape_url` call with `strategy: auto` (AutoFallback already hops). Call `i
 Four-step workflow for a durable feed config:
 
 1. Call `capture_config` — YAML is `payload.yaml`. Check `payload.articles_count` and `has_selectors`. Strive `enhance: true`.
-2. If weak or you need recon, call `inspect_url`. Auto already hops to Botasaurus.
+2. If weak or you need recon, call `inspect_url` (`auto` uses Faraday; pin `botasaurus` for browser rendering).
 3. Call `validate_config` with `yaml` (or `config`) — must not be `isError`.
 4. Call `apply_config` — `isError` if zero items. Confirm `payload.item_count` before shipping.
 
diff --git a/src/content/docs/troubleshooting/troubleshooting.mdx b/src/content/docs/troubleshooting/troubleshooting.mdx
index f5879be1..6da865bb 100644
--- a/src/content/docs/troubleshooting/troubleshooting.mdx
+++ b/src/content/docs/troubleshooting/troubleshooting.mdx
@@ -101,7 +101,7 @@ Common configuration-related errors:
 - **`BOTASAURUS_SCRAPER_URL is invalid`:** Fix the URL format and retry.
 - **`Configuration must include at least 'selectors' or 'auto_source'`:** You need to specify either manual selectors or enable auto-source.
 - **`stylesheet.type invalid`:** Only `text/css` and `text/xsl` are supported for stylesheets.
-- **`request.botasaurus` validation:** Unknown keys fail (`scroll_to_bottom` is not valid — use `scroll: true`). `window_size` must be `{ width:, height: }`. `wait_timeout_seconds` must be `1..20`.
+- **`request.botasaurus` validation:** Unknown keys are rejected. Valid scroll, window size, and wait-timeout shapes are documented in [Strategy — Botasaurus](/ruby-gem/reference/strategy/#botasaurus).
 
 ### Missing Item Parts