docs: confirm robots.txt exists for AI bot crawling on docs site

[Copilot]

The GEO audit flagged that the docs site at github.github.com/gh-aw scored 0/18 on the robots.txt check, projecting a +18 point lift per page if fixed.

docs/public/robots.txt already exists (landed in #47597) with comprehensive Allow: / rules for all required AI crawlers:

User-agent: GPTBot / OAI-SearchBot / ChatGPT-User
User-agent: anthropic-ai / ClaudeBot / Claude-SearchBot
User-agent: PerplexityBot / Perplexity-User
User-agent: Google-Extended / Googlebot / Google-CloudVertexBot
User-agent: Bingbot / cohere-ai / DuckAssistBot / xAI-Bot / Amazonbot / AI2Bot / YouBot / CCBot

Sitemap: https://github.github.com/gh-aw/sitemap-index.xml

This PR confirms the file is in place and closes the audit issue.

• Fixes [geo-optimizer] Add robots.txt to docs site to allow AI bot crawling #47641

[github-actions]

Hey @Copilot 👋 — thanks for working on closing the GEO audit issue (#47641). This PR is still in draft with zero file changes, which needs clarification:

• Confirm the implementation — The PR body describes confirming that docs/public/robots.txt exists (from chore: bump AWF to v0.27.39 and MCP gateway to v0.4.4 #47597), but there are no changes in this diff. Is the audit resolved as-is, or are additional changes needed to the robots.txt file or site configuration?
• Mark as ready — Once you have confirmed the audit is actually closed, remove draft status and ensure any necessary changes are included.
• Add verification — If the robots.txt file landing in chore: bump AWF to v0.27.39 and MCP gateway to v0.4.4 #47597 is sufficient, consider adding a brief note in the PR body confirming the GEO check now passes.

The fix came in #47597, so this PR role is just to formally close the audit — make sure the diff reflects that work!

Generated by ✅ Contribution Check · sonnet46 · 52.3 AIC · ⌖ 10.3 AIC · ⊞ 6.2K · ◷

[github-actions]

🤖 PR Triage

Field	Value
Category	docs
Risk	🟢 Low
Priority Score	38/100
Recommended Action	defer

Score breakdown: Impact 15 + Urgency 10 + Quality 13

Rationale: Draft PR, no files changed (empty diff), addresses a GEO/robots.txt documentation gap. Low urgency, low impact. Defer until the author completes the draft with actual changes.

Generated by 🔧 PR Triage Agent · sonnet46 · 38.1 AIC · ⌖ 6.75 AIC · ⊞ 5.7K · ◷