feat(experiments): adopt the observability hypothesis skill, with read-only tool use

[mabramowitz-pixel]

What

Replaces the body of skills/experiments/launchdarkly-experiment-hypothesis-builder/SKILL.md with the experiment-hypothesis skill from launchdarkly/observability, taken from ai/autofix/vega-plugin/skills/experiment-hypothesis/SKILL.md at bcb2e82c (2026-07-31, observability #1610).

That path is the source of truth in observability. backend/prompts/experiment-hypothesis-skill.md is generated from it by go generate and carries a DO NOT EDIT header, so it is not a separate version.

Why

The observability copy is the version driving the in-product experiment builder's hypothesis assist, and it is where recent fixes have been landing.

What changed

• Body is verbatim from observability, with a provenance comment naming the source path and commit.
• Frontmatter adapted so this repo's validator passes: name matches the skill directory, description is a single-line string rather than a folded scalar (normalize_value rejects >), and compatibility reflects that the skill needs no tools.
• Version bumped to 0.2.0; skills.json regenerated with scripts/generate_catalog.py; README row updated.
• references/diagnostic-tree.md removed, since the new body references no support files.

CI checks pass locally: validate_skills.py, generate_catalog.py --check, and the unit tests.

Reviewer notes

This is a contract change, not a refresh. The previous skill was a conversational advisory skill with a 9-step workflow, read-only MCP lookups, and a handoff payload to launchdarkly-experiment-setup. The observability version is a JSON-only prompt with no tool use, entered via experiment_hypothesis_builder_assist. Anything expecting the old handoff payload will need updating.

Two consequences worth confirming before merge:

• PR fix(experiments): A/A route returns components all false [EXPT-4665] #138 (A/A components all false, EXPT-4665) ports observability #1594. That fix is already present in this body, so fix(experiments): A/A route returns components all false [EXPT-4665] #138 becomes redundant if this lands.
• The skill directory keeps its launchdarkly-experiment-hypothesis-builder name. If the catalog should instead carry the observability name experiment-hypothesis, that is a directory rename plus catalog and README updates.

🤖 Generated with Claude Code

---

Note

Overview
Replaces the launchdarkly-experiment-hypothesis-builder skill body with the observability experiment-hypothesis prompt (verbatim sync, with provenance comment). The skill is no longer a multi-step conversational coach with MCP lookups and a handoff payload to launchdarkly-experiment-setup.

It now targets the in-product experiment hypothesis assist (experiment_hypothesis_builder_assist): one user text in, JSON only out (route, components, hypothesis with {{hole}} syntax, measurements). Routing covers A/A, junk, rewrite, and scaffold; rubric is Change / Measurement / Rationale with semantic-validity and injection-handling rules.

Catalog/metadata: version 0.2.0, skills.json and README descriptions updated; compatibility is no tools / JSON-only. Removes references/diagnostic-tree.md (unused by the new body).

^{Reviewed by Cursor Bugbot for commit c93ac34. Bugbot is set up for automated code reviews on this repo. Configure here.}

---

Update: read-only tool use (second commit)

The observability copy is JSON-only with no tool access, because gonfalon calls it as a single-shot prompt. In ai-tooling the skill runs in an agent, so it now checks what already exists before answering.

• Permits get-flag, list-flags, get-metric, list-metrics, get-project, get-environment, and names the forbidden write prefixes (create-, update-, toggle-, start-, delete-) explicitly.
• Lookups inform the new matches object only. They never change routing or component scoring, and are skipped for the junk and aa routes.
• Response contract gains an optional matches object: flagKey, metricKey, confidence (exact | likely). Populated only from tool results, omitted when nothing matched.
• Final message is still JSON and nothing else. schema_version stays 1 since the addition is optional.

This is a deliberate divergence from the observability copy, so the two files are no longer byte-comparable. The provenance comment in the skill records what to re-sync from observability (rubric, routing, hole rules) and what to keep (tool use, matches). A consumer that wants the resolved keys needs to read the new field; one that ignores unknown keys is unaffected.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{Reviewed by Cursor Bugbot for commit b38bdcb. Configure here.}

[cursor]

Contradictory lookup wording rules

Medium Severity

The new tool-use section tells the model to prefer customer metric wording from lookups, but the same section and response contract also say lookups may only populate matches and must not change the hypothesis sentence. Those instructions conflict, so the model can either rewrite measurement text from lookup results or ignore the wording guidance inconsistently.

Additional Locations (2)

• skills/experiments/launchdarkly-experiment-hypothesis-builder/SKILL.md#L188-L189
• skills/experiments/launchdarkly-experiment-hypothesis-builder/SKILL.md#L204-L206

 

^{Reviewed by Cursor Bugbot for commit b38bdcb. Configure here.}