Observed (live, k8s-triage agent)
forge.yaml declared:
tools:
- name: file_read
type: builtin
The startup banner printed Tools: 1 (file_read), but the registered tools log line (and the LLM's tool definitions) did not include file_read. The model, whose skill told it to read a file with a tool it didn't have, fabricated the file's contents with file_create and analyzed its own invention — twice, in separate sessions — producing confidently wrong answers.
Why
builtins.RegisterAll(reg) registers a fixed default set (register.go All()); file_read is only in CodeAgentReadTools, registered when the code-agent skill is active.
cfg.Config.Tools (the tools: list) is never consulted for builtin registration. Its only uses: the banner (forge-cli/runtime/runner.go:2653-2660 prints the names verbatim) and BuiltinTools deny-list protection.
So the tools: block is cosmetic for builtins — the banner reports configuration, not reality.
Impact
Misleading operator UX plus a real hallucination hazard: a skill referencing a configured-but-unregistered tool steers the model into inventing data instead of failing loudly.
Suggested fix (either)
- Make
tools: [{name: file_read, type: builtin}] actually register the named builtin (lookup across All() + the code-agent bundles), erroring at startup on unknown names; or
- At minimum, derive the banner from
reg.List() (the truth) and warn on configured-but-unregistered entries.
Found while live-testing context compression (#241/#279); unrelated to compression itself.
Observed (live, k8s-triage agent)
forge.yaml declared:
The startup banner printed
Tools: 1 (file_read), but theregistered toolslog line (and the LLM's tool definitions) did not includefile_read. The model, whose skill told it to read a file with a tool it didn't have, fabricated the file's contents withfile_createand analyzed its own invention — twice, in separate sessions — producing confidently wrong answers.Why
builtins.RegisterAll(reg)registers a fixed default set (register.goAll());file_readis only inCodeAgentReadTools, registered when the code-agent skill is active.cfg.Config.Tools(thetools:list) is never consulted for builtin registration. Its only uses: the banner (forge-cli/runtime/runner.go:2653-2660prints the names verbatim) andBuiltinToolsdeny-list protection.So the
tools:block is cosmetic for builtins — the banner reports configuration, not reality.Impact
Misleading operator UX plus a real hallucination hazard: a skill referencing a configured-but-unregistered tool steers the model into inventing data instead of failing loudly.
Suggested fix (either)
tools: [{name: file_read, type: builtin}]actually register the named builtin (lookup acrossAll()+ the code-agent bundles), erroring at startup on unknown names; orreg.List()(the truth) and warn on configured-but-unregistered entries.Found while live-testing context compression (#241/#279); unrelated to compression itself.