Make search_issues semantic by default

[kelsey-myers]

Summary

Make search_issues use semantic search by default on hosts that support it, falling back to lexical on GHES.

Why

Closes github/plan-track-agentic-toolkit#729

An A/B experiment on the remote server compared lexical, semantic, hybrid and agent-chosen variants of search_issues over a week. Semantic reduced the number of searches an agent needed per session to 1.7, against lexical's 2.4, with query length roughly flat. Fewer searches means fewer round-trips of results into the agent's context.

Semantic search is not available on GitHub Enterprise Server, where the API returns 403 for search_type=semantic, so those hosts keep lexical behaviour.

What changed

• search_issues sends search_type=semantic on dotcom and GHEC.
• Free-text quotes are stripped from the query, since quoting asks the backend for an exact phrase match and defeats semantic matching. Quotes around qualifier values (label:"needs triage") are preserved.
• The tool and query descriptions now describe the engine the host will actually use. Steering a caller toward natural language is wrong advice on a lexical-only instance.
• New WithHost tool option, so tools with host-specific capabilities can adapt. utils.ParseHostType exposes the classification parseAPIHost was already doing internally.
• search_pull_requests is unaffected.

advanced_search is deliberately left alone. The semantic query class already extends the advanced one, so the parameter changes nothing when semantic runs, and on the lexical fallback it makes things worse: under advanced search an unparenthesised boolean binds repo: to only its own clause, so the search escapes the repository the caller asked about. Prod-verified over four query shapes; the existing field. conditional in prepareSearchArgs is unchanged, since field genuinely does require it.

Eval details — how the query description was chosen

The query parameter description was picked by A/B testing four phrasings against a set of benchmark cases that prompt with synonym lists ("auth, authentication, or login problems") to see whether the model reaches for boolean OR. OR isn't supported by semantic search, so the query silently degrades to lexical.

Measured as: of the queries emitted for those cases, how many contained OR.

Wording	OR usage	sonnet-4.5	gemini-2.5-pro	haiku-4.5
A — shipped "When the user gives alternative wordings, include them as plain words rather than joining them with OR."	6/15 (40%)	1/5	5/5	0/5
B "Semantic matching already finds related wording, so synonyms don't need listing or joining with OR."	8/19 (42%)	3/7	5/5	0/7
"Avoid boolean OR operators, which fall back to lexical search."	9/17 (53%)	4/5	5/5	0/7
lexical description (control)	7/12 (58%)	3/5	4/4	0/3
C "Semantic matching already covers related wording and synonyms." (no mention of OR)	10/17 (59%)	3/5	5/5	2/7

Findings:

• Naming the alternative behaviour is what works. A and B both do, and both beat the negative phrasing.
• Negation alone does nothing. "Avoid boolean OR" is indistinguishable from a description that never mentions OR.
• Silence is worst. C was the only condition where haiku emitted OR at all.
• gemini-2.5-pro is unmoved by any wording, 5/5 throughout.

MCP impact

• Tool schema or behavior changed

search_issues descriptions changed, and the underlying search engine changes on non-GHES hosts. Parameters are unchanged.

Prompts tested (tool changes only)

• "Find issues about login failing after a password reset in github/github-mcp-server"
• "Search for issues labelled needs triage in this repo"
• "Any open issues about slow rendering performance?"

Security / limits

• Data exposure, filtering, or token/size limits considered

Semantic issue search sits in a separate API rate limit family: 10 requests/minute versus 30/minute for standard search. This applies per request based on search_type, so making semantic the default lowers the effective rate limit for issue search on supported hosts. The experiment recorded no rate limit errors across ~76k semantic calls.

Tool renaming

• I am not renaming tools as part of this PR

Lint & tests

• Linted locally with ./script/lint
• Tested locally with ./script/test

Docs

• Updated (README / docs / examples)

[kelsey-myers]

Reasoning for why OSS as opposed to the remote server is detailed in https://github.com/github/plan-track-agentic-toolkit/issues/729 (tradeoffs dropdown) - I'm open to discussion on this though!

[Copilot]

Pull request overview

Makes issue search semantic by default on supported hosts, with lexical fallback for GHES.

Changes:

• Adds host-aware tool configuration and semantic search mode.
• Preserves qualifier quotes while removing free-text quotes.
• Updates tests, snapshots, and generated documentation.

Show a summary per file

File	Description
README.md	Updates query guidance.
pkg/utils/api.go	Exposes host classification.
pkg/github/tools.go	Adds host-aware tool options.
pkg/github/search_utils.go	Implements semantic search preparation.
pkg/github/search_semantic_test.go	Tests quote handling and descriptions.
pkg/github/issues.go	Makes issue search host-aware.
pkg/github/issues_test.go	Updates semantic request expectations.
pkg/github/inventory.go	Propagates tool options.
pkg/github/__toolsnaps__/search_issues.snap	Updates default schema snapshot.
pkg/github/__toolsnaps__/search_issues_ff_fields_param.snap	Updates flagged schema snapshot.
internal/ghmcp/server.go	Configures host type for stdio.
docs/insiders-features.md	Updates generated query guidance.
docs/feature-flags.md	Updates generated query guidance.

Review details

• Files reviewed: 13/13 changed files
• Comments generated: 2
• Review effort level: Medium