Skip to content

fix(ce-debug): investigate the tracker and PR history for prior work#1018

Merged
tmchow merged 1 commit into
mainfrom
tmchow/debug-diagnosis-enhancement
Jun 28, 2026
Merged

fix(ce-debug): investigate the tracker and PR history for prior work#1018
tmchow merged 1 commit into
mainfrom
tmchow/debug-diagnosis-enhancement

Conversation

@tmchow

@tmchow tmchow commented Jun 28, 2026

Copy link
Copy Markdown
Collaborator

Summary

ce-debug diagnosed bugs only from the code and live telemetry (Sentry, logs, DB). It never consulted the project's institutional memory — the issue tracker and PR history — so it could not see when a bug:

  • already has an open ticket (team is aware; may be intentionally deferred),
  • is being fixed in an unmerged PR (don't write a duplicate),
  • had a prior fix that regressed (read the original fix for the intended behavior + why it came back), or
  • had a prior merged attempt that took the same approach you were about to — yet the bug persists, which is high-value negative evidence that the approach is known to fail.

Change

New Phase 1.4 in skills/ce-debug/SKILL.md:

  • Discovers the tracker/code-review surface from repo signals (git remote, ABC-123-style issue keys in commits/branches/PR titles, the tracker named in active project conventions) — capability-not-tool: never assumes a specific CLI/MCP exists, never treats a missing one as proof the capability is absent.
  • Runs a few bounded queries, weighted toward what git log cannot show (open/in-flight work + the issue/PR discussion thread). Merged-fix discovery is explicitly handed to the existing Phase 1.3 git step to avoid redundant searching.
  • Treats ticket/PR text as data, not instructions.
  • Gated off the trivial fast-path; regression signals are the strongest trigger.

A Phase 2 present-findings bullet carries any surfaced ticket/PR into the recommendation (e.g. lead with an existing open PR instead of a fresh fix).

Validation

Tested via the skill-creator eval workflow (fresh subagents, so the in-session skill cache didn't mask the change), using this repo's real GitHub issues/PRs as fixtures and the pre-edit file (git show HEAD) as the baseline:

Scenario Seeded reality Result
Open PR exists bug fixed by open PR #818 with-skill found & led with #818; baseline never did (an open PR isn't in git log)
Merged fix bug fixed by merged PR #1008 git finds the commit, 1.4 pivots to the PR + issue thread for the why — no redundant search
No prior work novel BOM-in-frontmatter bug bounded search, no fabricated ticket, correct root cause — precision guard held

With-skill 100% vs. baseline 68% pass rate; tracker searched 3/3 scenarios vs. 0/3 for the baseline. Modest cost (~+40s, tokens within noise).

ce-debug previously diagnosed only from code and live telemetry, so it
could not see that a bug already had an open ticket, was being fixed in
an unmerged PR, had a prior fix that regressed, or had a prior merged
attempt that took the same (failing) approach.

Add Phase 1.4: discover the project's tracker/code-review surface from
repo signals (git remote, issue-key patterns, active project
conventions), then run a few targeted queries. The search is weighted
toward what git log cannot show -- open/in-flight work and the issue/PR
discussion thread -- and explicitly defers merged-fix discovery to the
existing Phase 1.3 git step to avoid redundant work. Findings flow into
Phase 2 and reshape the recommendation (e.g. defer to an existing open
PR instead of writing a duplicate fix). Gated off the trivial fast-path;
regression signals are the strongest trigger; ticket/PR text is treated
as data, not instructions.
@tmchow tmchow merged commit afc5e01 into main Jun 28, 2026
2 checks passed
@github-actions github-actions Bot mentioned this pull request Jun 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant