Skip to content

forge.yaml tools: entries don't register builtins — banner claims tools the agent doesn't have #281

Description

@initializ-mk

Observed (live, k8s-triage agent)

forge.yaml declared:

tools:
  - name: file_read
    type: builtin

The startup banner printed Tools: 1 (file_read), but the registered tools log line (and the LLM's tool definitions) did not include file_read. The model, whose skill told it to read a file with a tool it didn't have, fabricated the file's contents with file_create and analyzed its own invention — twice, in separate sessions — producing confidently wrong answers.

Why

  • builtins.RegisterAll(reg) registers a fixed default set (register.go All()); file_read is only in CodeAgentReadTools, registered when the code-agent skill is active.
  • cfg.Config.Tools (the tools: list) is never consulted for builtin registration. Its only uses: the banner (forge-cli/runtime/runner.go:2653-2660 prints the names verbatim) and BuiltinTools deny-list protection.

So the tools: block is cosmetic for builtins — the banner reports configuration, not reality.

Impact

Misleading operator UX plus a real hallucination hazard: a skill referencing a configured-but-unregistered tool steers the model into inventing data instead of failing loudly.

Suggested fix (either)

  1. Make tools: [{name: file_read, type: builtin}] actually register the named builtin (lookup across All() + the code-agent bundles), erroring at startup on unknown names; or
  2. At minimum, derive the banner from reg.List() (the truth) and warn on configured-but-unregistered entries.

Found while live-testing context compression (#241/#279); unrelated to compression itself.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions