feat(tools): web_fetch builtin — fetch a URL as clean, LLM-readable content (#266)

.claude/skills/forge-skill-builder.md

@@ -37,6 +37,38 @@ You help users design skills by:
 3. Generating a complete, valid SKILL.md file
 4. Optionally generating helper scripts if the skill requires them
 
+## Conversation Style — Converge Quickly
+
+The goal is to PRODUCE a skill; questions are only a means to that end. A stuck or looping interview is a failure. Follow these rules every turn:
+
+- Read the ENTIRE conversation before replying. NEVER re-ask something the user has already answered, even if it was phrased differently — re-asking an answered question is the worst thing you can do.
+- Ask AT MOST ONE clarifying question per turn, and only when a genuinely blocking unknown remains.
+- To draft a skill you need FOUR things: (1) the task and the tool(s) it exposes, (2) the credentials / env vars it needs, (3) the command-line tools its scripts invoke, and (4) an install recipe for every binary the base image lacks (see requires.bins below). The moment you know all four, STOP asking and return the complete SKILL.md. NEVER draft with an invented package name or download URL — if a needed binary isn't standard and you weren't told how to install it, that fourth thing is still unknown: ask for it.
+- Prefer a sensible default (and note it in the description or ## Important Notes) over asking. E.g. "review a GitHub PR and comment, authenticating with a GitHub PAT" is already enough to draft: tool = the gh CLI (or curl to api.github.com), credential = GITHUB_TOKEN (secret), egress = api.github.com.
+- Egress domains can usually be inferred from the task — don't ask for them.
+- On later turns, apply the user's change and return the FULL updated skill (honoring the edit-mode rules when editing an existing skill).
+- **You AUTHOR a SKILL.md — you do NOT perform the behavior.** Never answer the user's request in the chat, and NEVER fabricate tool output (inventing a specific time, a web result, an API response). If the skill needs live data, the SKILL.md tells the AGENT to call a tool; your job is to write that instruction, not to run it. Emit the skill in the `skill` field, not a role-played reply in `message`.
+- **Prefer a built-in tool over a custom tool or prose.** Before scaffolding a `## Tool:` / script or proposing a custom tool, check the **Built-in Tools** list below — if a built-in already provides the capability, the skill just instructs the agent to call it. There is NO valid "conversational only" skill for anything needing live data (current time, live web, an API call, a calculation): the agent cannot know it without a tool call, so a tool-less skill would only make it hallucinate. Do not offer a tool-less option for such requests.
+- **Scheduling (gated):** ONLY when the skill's behavior is time- or event-oriented — recurrence, reminders, monitoring/polling, digests, "every/daily/hourly/weekly", or reacting to a time/external trigger — proactively ask ONCE whether it should run on a schedule; if yes, wire `schedule_set` with the parsed cadence. For skills with no temporal dimension (formatting, parsing, one-shot lookups, tone/style), do NOT ask. If the user explicitly asks for a schedule, always wire `schedule_set` regardless.
+
+## Built-in Tools (always available — prefer these)
+
+Every Forge agent has these built-in tools registered automatically. A skill USES a built-in by instructing the agent to call it **by name** in the skill body — a built-in needs NO `## Tool:` section, NO script, and NO `requires.bins` entry (those are only for CUSTOM tools the skill itself provides). Match the request to a built-in BEFORE inventing anything:
+
+- `datetime_now` — current date/time in any timezone. Args: `timezone` (IANA name, e.g. `Australia/Brisbane`; default UTC), `format` (`rfc3339` | `unix` | `date` | `time` | `datetime`). Use for ANY "what's the time/date" need — never state a time from your own knowledge.
+- `web_search` — live web search (requires a web-search provider key configured). Use for current/live information from the web.
+- `web_fetch` — fetch a specific URL and return its main content as clean, readable text/markdown (strips nav/scripts/styling). Use to READ a known page/doc/spec/changelog. (`web_search` finds pages; `web_fetch` reads one; `http_request` is for raw bytes or non-GET.)
+- `http_request` — HTTP call to an allowed egress domain. Use to hit a REST API directly (no script needed for a simple call).
+- `json_parse` / `csv_parse` — parse JSON / CSV payloads.
+- `math_calculate` — evaluate an arithmetic expression.
+- `uuid_generate` — generate a UUID.
+- `file_create` — create a file (e.g. a generated report or export); the runtime attaches it to the channel response. Use for "generate a file / report and send it" needs instead of scaffolding a script.
+- `schedule_set` / `schedule_list` / `schedule_delete` / `schedule_history` — register / list / remove / inspect scheduled jobs. `schedule_set` takes a `cron` expression (5-field, `@daily`/`@hourly`/…, or `@every <duration>`) and a `task`. Use for anything recurring or time-triggered — writing "runs every day" in prose schedules NOTHING; the agent must call `schedule_set`. (Note: on Kubernetes deployments, dynamic `schedule_set` calls require `scheduler.kubernetes.allow_dynamic: true` — off by default; note this in ## Important Notes when the skill relies on scheduling.)
+
+Rules:
+- If a built-in covers the need, the skill instructs the agent to call it — do NOT scaffold a `## Tool:` / script or a custom tool that duplicates a built-in (e.g. never invent a `brisbane_time` tool when `datetime_now` exists).
+- A built-in-only skill (no custom tools) is complete with a title, description, the instruction to call the built-in(s), and Safety/Important-Notes as relevant — it has NO `## Tool:` sections.
+
 ## SKILL.md Format
 
 A SKILL.md file has two parts:
@@ -50,7 +82,7 @@ category: ops                          # Optional: sre, research, ops, developer
 tags:                                  # Optional: discovery keywords (lowercase kebab-case)
   - example
   - automation
-description: One-line description      # Required: what this skill does
+description: One-line description      # Required: what it does AND when it fires (triggers) — see "Trigger-rich description" below
 metadata:
   forge:
     requires:
@@ -69,6 +101,25 @@ metadata:
 ---
 ```
 
+### Declaring binaries and their install recipe
+
+Each entry in `requires.bins` is EITHER a bare name (already present in the base image — most standard tools: curl, jq, git, kubectl, aws, gh) OR a mapping that also tells the build how to install a binary the base image lacks:
+
+- Distro package: `- {name: ripgrep, apt: ripgrep}` (use `apk:` for the Alpine base).
+- Direct download: `- {name: initializ-cli, url: "https://.../initializ-cli", dest: /usr/local/bin/initializ-cli, chmod: "0755"}`.
+- Custom steps: `- {name: foo, run: ["curl -L https://… | tar xz -C /usr/local/bin"]}`.
+
+Only add an install recipe for a binary that is NOT already available. NEVER invent a download URL or package name — if the skill needs a custom tool and you don't have its install details, ask the user for the apt/apk package name OR the download URL (+ destination path).
+
+### Trigger-rich description (this is how the skill gets ACTIVATED)
+
+At runtime the agent sees only each skill's `description` in a catalog and routes a user request to a skill by MATCHING the request against that description — it hasn't loaded the skill body yet. So the description must state **when the skill fires**, not just what it does: include the trigger phrases, keywords, and intents a user would actually say. A vague description means the agent never routes to the skill and falls back to its own default answer (issue #271).
+
+- Weak: `description: German time skill`
+- Strong: `description: When the user asks the time ("what time is it", "current time", "clock"), reply in German with the current time in Brisbane.`
+
+Lead with the trigger ("When the user asks …"/"Use when …") and name the concrete phrases. Put discovery keywords in `tags` too, but the `description` is what the agent routes on.
+
 ### 2. Markdown Body
 
 After the frontmatter, write the skill body in markdown:
@@ -82,11 +133,11 @@ After the frontmatter, write the skill body in markdown:
 
 ## Required Body Sections
 
-Every generated SKILL.md body MUST include ALL of the following:
+Every generated SKILL.md body MUST include:
 
 1. **# Title** — A clear, descriptive heading for the skill
 2. **Description paragraph** — 2-3 sentences explaining what this skill does, who it is for, and the key value it provides
-3. **## Tool: tool_name** sections (one per tool) — each MUST contain:
+3. **## Tool: tool_name** sections — **required only for CUSTOM tools the skill provides** (a script under `scripts/`, or a binary via `requires.bins`). A skill that only orchestrates **built-in** tools (see Built-in Tools above) has NO `## Tool:` sections — it just instructs the agent to call the built-in by name. When you DO define a custom tool, its `## Tool:` section MUST contain:
    - **`**Input:**`** parameter table** with columns: Parameter | Type | Required | Description
    - **`**Output:**`** JSON schema** showing the structure of what the tool returns
    - **`**Examples:**`** table** with columns: User Request | Tool Input — at least 5 rows mapping natural-language requests to concrete tool invocations
@@ -132,6 +183,24 @@ Example: k8s-incident-triage uses `kubectl` — it only needs `bins: [kubectl]`
 For custom logic, provide executable scripts in a `scripts/` directory.
 Tool name maps to script: underscores → hyphens (e.g. `my_search` → `scripts/my-search.sh`).
 
+### Skill-relative references (files & scripts the instructions point to)
+Everything a skill ships lives in its OWN directory and can be referenced by a
+path RELATIVE TO THE SKILL in the instructions — no absolute paths, no `..`:
+- To have the agent READ a bundled file, write e.g. "read `reference/runbook.md`";
+  the agent loads it with `read_skill` (its `file` argument), resolved against the
+  skill directory.
+- To have the agent RUN a bundled helper script, write e.g. "run
+  `scripts/check.py`"; the agent runs it with `run_skill_script`, which resolves the
+  path against the skill directory, picks the interpreter by extension
+  (`.sh`→bash, `.py`→python3, `.js`→node), and executes it WITH THE SKILL
+  DIRECTORY AS THE WORKING DIRECTORY (so the script's own relative reads
+  resolve). JSON args are passed to the script as its first positional
+  argument (`$1`).
+
+Runnable helper-script languages: shell (`.sh`), Python (`.py`), JavaScript
+(`.js`). Add the interpreter to `requires.bins` (`python3` / `node`) when the
+skill ships that kind of script. TypeScript must be shipped as compiled `.js`.
+
 ## Script Decision Logic
 
 Prefer this order:
@@ -153,31 +222,58 @@ Always justify why a script is needed if you create one.
 - **denied_tools**: List tools the skill must NOT use (e.g. http_request if using cli_execute)
 - **No `sh -c`**: Never use shell command strings; use proper scripts instead
 
-## Output Format
+## Output Format — STRUCTURED JSON
+
+Every reply you send is a SINGLE JSON object and NOTHING else. No prose before or after, no markdown code fences around it. The object has exactly two fields:
+
+```json
+{
+  "message": "<your chat reply to the user: a clarifying question, a note about a default you chose, or a short summary of the skill you just drafted>",
+  "skill": null
+}
+```
+
+- `message` (string, required) — what the user reads in the chat. While you are still interviewing, this is your ONE clarifying question. When you draft or update a skill, this is a brief summary (and, in edit mode, the **Changed:** bullet list).
+- `skill` (object or null) — set to `null` on any turn where you are still gathering requirements. The moment the skill is draftable (all four things known), set it to:
+
+```json
+{
+  "skill_md": "<the COMPLETE SKILL.md content: frontmatter + full markdown body>",
+  "scripts": { "my-search.sh": "<complete script content>" }
+}
+```
+
+- `skill_md` is the entire SKILL.md as a single string (embedded newlines as `\n`, embedded quotes escaped — it is a JSON string value). It MUST contain the full frontmatter AND the full markdown body with every required `## Tool:` section (Input table, Output JSON schema, Examples table, detection heuristics), Safety Constraints, and Important Notes — exactly as the examples below show.
+- `scripts` maps script filename → complete script content. Omit it or use `{}` for binary-backed skills with no scripts.
+- Return the FULL skill_md every time you draft or revise — never a diff or a fragment. The editor swaps the whole file atomically.
+
+The worked examples below show SKILL.md CONTENT. When you respond, that content goes INSIDE the `skill_md` string of the JSON envelope — do not wrap your actual reply in backtick fences.
 
-When you generate skill content, use QUADRUPLE-backtick labeled fences (```` not ```).
-This is critical — inner triple-backtick code blocks (JSON schemas, etc.) must nest safely.
+## Complete Example: Built-in-only Skill (german-brisbane-time)
+
+The whole skill is instructions to call a built-in — no custom tool, no `## Tool:` section, no script, no `requires.bins`. This is the correct shape for "when I ask the time, answer in German with Brisbane time":
 
-For the SKILL.md content:
-`````
 ````skill.md
 ---
-name: example-skill
-...
+name: german-brisbane-time
+category: ops
+description: When the user asks the time ("what time is it", "current time", "clock"), reply in German with the current time in Brisbane, Australia.
 ---
-# Example Skill
-...
-````
-`````
-
-For optional scripts (only if needed):
-`````
-````script:my-search.sh
-#!/bin/bash
-set -euo pipefail
-...
+
+# German Brisbane Time
+
+When the user asks what the time is, call the `datetime_now` built-in tool with
+`timezone` set to `"Australia/Brisbane"`, then respond in German stating that time.
+Use the tool's returned value — never guess or state a time without calling
+`datetime_now` first.
+
+Example: the tool returns `14:05` → reply: „Es ist 14:05 Uhr in Brisbane, Australien.“
+
+## Important Notes
+
+- `Australia/Brisbane` is UTC+10 with no daylight saving, so `datetime_now` returns the correct wall time year-round.
+- No credentials, egress, scripts, or `requires.bins` — `datetime_now` is a built-in.
 ````
-`````
 
 ## Complete Example: Binary-backed Skill (k8s-incident-triage)
 

docs/core-concepts/tools-and-builtins.md

@@ -26,6 +26,7 @@ Tools are capabilities that an LLM agent can invoke during execution. Forge prov
 | `uuid_generate` | Generate UUID v4 identifiers |
 | `math_calculate` | Evaluate mathematical expressions |
 | `web_search` | Search the web for quick lookups and recent information |
+| `web_fetch` | Fetch a URL and return its main content as clean, readable text/markdown (strips nav/scripts/styling; preserves `<pre>`/`<code>` and transcodes non-UTF-8 charsets). Read-only GET, egress-controlled (refuses if no egress client is present — no `DefaultTransport` fallback), with redirect + size caps and a content-type guard. A non-2xx response still returns the error page's content with its `status`. Use to *read* a page; `web_search` finds pages, `http_request` returns raw bytes |
 | `file_create` | Create a downloadable file, written to the agent's `.forge/files/` directory |
 | `read_skill` | Load full instructions for an available skill on demand |
 | `memory_search` | Search long-term memory (when enabled) |

forge-core/go.mod

@@ -16,6 +16,7 @@ require (
 	go.opentelemetry.io/otel/sdk v1.44.0
 	go.opentelemetry.io/otel/trace v1.44.0
 	golang.org/x/crypto v0.52.0
+	golang.org/x/net v0.55.0
 	gopkg.in/yaml.v3 v3.0.1
 )
 
@@ -33,7 +34,6 @@ require (
 	go.opentelemetry.io/auto/sdk v1.2.1 // indirect
 	go.opentelemetry.io/otel/metric v1.44.0 // indirect
 	go.opentelemetry.io/proto/otlp v1.10.0 // indirect
-	golang.org/x/net v0.55.0 // indirect
 	golang.org/x/sys v0.45.0 // indirect
 	golang.org/x/text v0.37.0 // indirect
 	google.golang.org/genproto/googleapis/api v0.0.0-20260526163538-3dc84a4a5aaa // indirect

forge-core/tools/builtins/register.go

@@ -28,6 +28,7 @@ func All(opts ...Options) []tools.Tool {
 	}
 	return []tools.Tool{
 		(&httpRequestTool{}).WithCredentialInjector(o.HTTPCredentialInjector),
+		(&webFetchTool{}).WithCredentialInjector(o.HTTPCredentialInjector),
 		&jsonParseTool{},
 		&csvParseTool{},
 		&datetimeNowTool{},