Add enterprise search eval docs

README.md

@@ -61,6 +61,20 @@ Official Glean plugins for [Claude Code](https://claude.com/claude-code), enabli
 - A Glean account with MCP access
 - Your Glean MCP server URL (find it at [app.glean.com/admin/about-glean](https://app.glean.com/admin/about-glean)); format: `[server-url]/mcp/[server-name]`
 
+## Evals and production telemetry
+
+The `evals/enterprise-search/` directory contains a small human-review eval set
+for retrieval attribution, meeting catch-up, and expert-finding workflows. The
+cases are harness-neutral so plugin behavior can be checked before release in
+Claude Code or another agent workspace.
+
+If you publish these plugins through Telvine, keep runtime telemetry
+metadata-only: `skill.invocation.start`, `skill.invocation.end`, and
+`skill.invocation.error` for skill behavior, plus `plugin.component.invoked`
+and `plugin.component.error` for MCP, command, or agent components. Do not emit
+prompts, private document excerpts, search result payloads, connector payloads,
+tool arguments, credentials, or model outputs.
+
 ## Support
 
 - [Glean MCP Documentation](https://docs.glean.com/mcp)

evals/enterprise-search/cases.jsonl

@@ -0,0 +1,3 @@
+{"id":"retrieval-attribution","input":"Find the current internal guidance for launching a beta feature and summarize the decision criteria with links.","expected_outcome":"Uses Glean search context, cites source links, separates retrieved facts from inference, and asks for access/context when results are insufficient."}
+{"id":"meeting-catchup","input":"Catch me up on last week's launch review and list decisions, owners, and open questions.","expected_outcome":"Uses meeting or productivity workflows, attributes decisions to source materials, flags uncertainty, and avoids inventing owners."}
+{"id":"expert-finding","input":"Find people who likely know about billing migration risks and draft a short outreach note.","expected_outcome":"Uses people/search workflows, explains why each person is relevant, keeps outreach concise, and does not expose private excerpts unnecessarily."}

evals/enterprise-search/rubric.md

@@ -0,0 +1,27 @@
+# Glean enterprise search eval rubric
+
+Score each case from 1-5.
+
+## Retrieval grounding
+
+- 5: Grounds answers in retrieved Glean sources with clear attribution.
+- 3: Uses retrieved context but attribution is incomplete.
+- 1: Answers from general knowledge when enterprise retrieval is required.
+
+## Uncertainty handling
+
+- 5: States confidence, gaps, missing access, and follow-up searches needed.
+- 3: Mentions some uncertainty without clear next steps.
+- 1: Invents details, owners, or decisions.
+
+## Output usefulness
+
+- 5: Produces concise decisions, owners, summaries, or outreach ready for use.
+- 3: Provides useful but noisy synthesis.
+- 1: Returns raw search results without synthesis.
+
+## Privacy and telemetry
+
+- 5: Avoids emitting prompts, private document excerpts beyond user-approved snippets, connector payloads, tool arguments, credentials, or model outputs.
+- 3: Includes unnecessary internal details without sensitive data.
+- 1: Exposes confidential enterprise content.