PDX-490: feat(mcp) — error_category + retryable on test-run failures (Thread D)

[mrdailey99]

Summary

Adds two optional, additive fields to both provar_automation_testrun.steps[] (JUnitStepResult) and provar_testrun_rca.failures[] (FailureReport):

• error_category — one of INFRASTRUCTURE | ASSERTION | LOCATOR | TIMEOUT | OTHER. Absent when no known pattern matches.
• retryable — true only for INFRASTRUCTURE (Connection reset / ECONNRESET / socket hang up / Failed to read client socket message) and TIMEOUT (TimeoutException). false for LOCATOR (NoSuchElementException), ASSERTION (AssertionException), and OTHER (SessionNotCreatedException / WebDriverException / ClassNotFoundException / LicenseException / InvalidPasswordException). Absent when error_category is absent.

Identical classifier logic lives in both files (classifyErrorCategory in rcaTools.ts, classifyStepErrorCategory in antTools.ts) so a consumer sees the same label regardless of whether they consume the testrun output or the RCA report.

The existing root_cause_category (17 fine-grained buckets, drives recommendation text) is unchanged — error_category is an additional coarse-grained classifier that exists to drive automated retry policy.

Why

Today an agent calling provar_automation_testrun or provar_testrun_rca has no machine-friendly retry hint — to decide whether a failure is transient or deterministic, it has to re-parse human-readable strings. That leads to two failure modes:

• agents retry locator failures forever (waste of time, the DOM has not changed),
• agents give up on a single ECONNRESET blip that would succeed on the next attempt.

error_category + retryable give the agent one machine-checkable boolean to gate retries on.

Scope

• src/mcp/tools/rcaTools.ts — adds ErrorCategory type, classifyErrorCategory(), isRetryable(), fields on FailureReport, population inside buildFailureReports, tool description update.
• src/mcp/tools/antTools.ts — adds JUnitErrorCategory type, classifyStepErrorCategory(), isStepRetryable(), fields on JUnitStepResult, population inside extractStepsFromJUnit.
• src/mcp/tools/automationTools.ts — tool-description update for provar_automation_testrun.
• docs/mcp.md — documents the new fields on both tools.
• test/unit/mcp/rcaTools.test.ts — 8 new tests covering all five categories, the "no match" case, the OTHER case, and the additive-no-breaking-change case.
• test/unit/mcp/antTools.test.ts — 6 new tests on parseJUnitResults covering all categories plus the no-match and pass-step cases.

Cross-thread coordination

[touches antTools.ts] — Thread B (PDX-491 H1) will also touch antTools.ts in a sibling PR — specifically the body of parseJUnitResults / extractStepsFromJUnit (the parser body). The edits in this PR are isolated to:

• the JUnitStepResult interface declaration (one new union type + two optional fields), and
• the field-population block inside the per-testcase loop in extractStepsFromJUnit (an additive if (errorMessage) { ... step.error_category = ...; step.retryable = ...; }).

Both edits are additive and confined to small contiguous spans, so a clean rebase with Thread B's PR should be straightforward.

Test plan

• node ../../../node_modules/typescript/bin/tsc -p . --pretty --noEmit — clean (no errors)
• node ../../../node_modules/eslint/bin/eslint.js on every changed file — clean (no warnings)
• node_modules/.bin/nyc node_modules/.bin/mocha "test/**/*.test.ts" — 1169 passing, 1 pending
• node scripts/lint-script-names.cjs — OK
• No external/customer-facing docs touched (docs/provar-mcp-public-docs.md, docs/university-of-provar-mcp-course.md) — the additive field rollout will be flagged for the Provar team to mirror in customer docs.

🤖 Generated with Claude Code

[github-actions]

Quality Orchestrator

🟢 LOW · 6 / 100 · All changed files have mapped tests.

---

🧪 Tests to Run · Running 3 of 51 tests

• unit/mcp/antTools.test.ts
• unit/mcp/automationTools.test.ts
• unit/mcp/rcaTools.test.ts

▶ Run command

npx vitest run \
  unit/mcp/antTools.test.ts \
  unit/mcp/automationTools.test.ts \
  unit/mcp/rcaTools.test.ts

---

_{⚡ quality-orchestrator  ·  /qo stub <file>  ·  qo analyze-local}

[Copilot]

Pull request overview

This PR adds two additive, optional fields (error_category, retryable) to MCP tool outputs for test-run failures so downstream consumers can make machine-friendly retry decisions without re-parsing human-readable error text.

Changes:

• Add coarse-grained error classification + retryability flags to provar_testrun_rca failure reports and to provar_automation_testrun step results.
• Update MCP tool descriptions and docs to describe the new fields.
• Add unit test coverage for all categories + “no match” behavior in both RCA and JUnit parsing.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/mcp/tools/rcaTools.ts	Adds error_category/retryable to FailureReport and populates them from failure text; updates tool description.
src/mcp/tools/antTools.ts	Adds error_category/retryable to JUnitStepResult and populates them during JUnit step extraction.
src/mcp/tools/automationTools.ts	Updates provar_automation_testrun tool description to mention the new step fields.
docs/mcp.md	Documents the new fields for both tools and updates example output.
test/unit/mcp/rcaTools.test.ts	Adds tests validating RCA failure error_category/retryable behavior.
test/unit/mcp/antTools.test.ts	Adds tests validating JUnit step error_category/retryable behavior.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[mrdailey99]

Addressed in 669b517: rca tool description now correctly scopes error_category/retryable to mode='rca'; root_cause_category count corrected to 17.