Skip to content

Skill activation: model answers from its own defaults instead of routing to a matching installed skill #271

Description

@initializ-mk

Symptom

With a correct, installed skill (German-Brisbane-time), the agent still answers from its own default behavior instead of using the skill:

User: What time is it?
Agent: I can tell you the current time, but I need a timezone or location. For example: "UTC", "New York", or "Brisbane".

User: use the skill you have to answer
Agent: In Brisbane ist es gerade 14:48 Uhr.

The forced call proves the skill is installed and functional — activation/dispatch is the failure, not the skill content.

Root cause

Skill activation is 100% model-judgment via lazy read_skill, with no trigger matching and no routing directive. Skills are listed in the system prompt under ## Available Skills as - name: description, and the preamble (forge-cli/runtime/runner.go:3381) only says:

"To use a skill, call read_skill with the skill name … to load its full instructions, then follow them."

That is purely mechanical — it explains HOW to load a skill but never instructs the model to prefer a matching skill over its own default answer. For a terse query like "what time is it?", the model's built-in reflex (ask for a timezone) wins because nothing told it to consult the catalog first.

Fix (two-sided)

1. Runtime preamble — add a routing directive (primary)

Strengthen the ## Available Skills preamble so the model routes before answering, e.g.:

"Before answering a request from your own knowledge or default behavior, check whether it matches an available skill's description below. If it does, call read_skill to load that skill and follow it instead of answering directly. Skills exist precisely to override your defaults for these cases."

This is the biggest lever — the catalog descriptions can be perfectly good (they were here) and the model still won't route without being told to.

2. Skill Builder — emit trigger-rich descriptions (ties to #270)

Generated skill description frontmatter should include the trigger phrases that fire it, not just what it does — e.g. "Triggers when the user asks the time ('what time is it', current time, clock). Replies in German with Brisbane time." A matchable catalog line makes the routing directive in (1) reliable.

Acceptance

  • After installing a skill whose description matches a query, the agent uses it without the user having to say "use the skill."
  • Preamble instructs prefer-matching-skill-over-default-answer.
  • Skill Builder generates trigger-oriented descriptions.
  • Regression check: a query that matches no skill still answers normally (no over-routing).

Open question

Whether to add a deterministic trigger-match layer (keyword/intent → auto-read_skill) in addition to the prompt directive, for cases where prompt-only routing proves unreliable. Prompt directive first; measure; escalate to deterministic matching if needed.

Related: #270 (Skill Builder authoring correctness), #252 (interview convergence)

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingforge-cliAffects the forge-cli command-line tool (init, run, build, mcp commands)toolsAffects builtin or adapter tools (cli_execute, http_request, browser, etc.)

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions