v1.6.4 Release

What's Changed

• PDX-515: fix(mcp): additive org_describe workspace discovery (no regression) by @mrdailey99 in #233

• .metadata requirement— a candidate now only qualifies if it contains a metadata directory. Without this, candidate 1 (the parent, which almost always exists) would always win and make candidates 2 & 3 unreachable. Now parent falls through to the fallbacks when it isn't a real workspace. *

• Basename verbatim— candidates 2 & 3 use the project folder name as-is (MyProject) instead of the old dash-lowercased form (my-project). The projectNameDashes helper that produced the dashed form was removed as dead code. *

• The per-candidate assertPathAllowed skip (silently skipping out-of-policy candidates before any fs access) is unchanged. *

• Develop by @mrdailey99 in #234

Full Changelog: v1.6.3...v1.6.4

v1.6.3 Release

Highlights

• org_describe reads Provar's cached metadata. provar_org_describe now resolves the Provar IDE .metadata SfObject cache layout and returns the object's real fields instead of field_count: 0. Adds an optional environment input (defaults to default).
• New screen-context validation. A new rule (UI-SCREEN-CONTEXT-001, major) flags a UiAssert or post-Save step left under the create (action=New) screen, where at runtime it would verify a saved record on a page Salesforce has already navigated away from. Scoped to action=New, so valid Edit/View patterns don't fire.
• Screen-target check now runs on real test cases. UI-SCREEN-TARGET-001 now reads the uri attribute Provar writes (not just hand-edited XML) and recognizes Visualforce/Lightning navigation params (page, pageObject, flexiPage, recordType, …), so it doesn't false-flag valid Salesforce CPQ/Billing screens.

What's Changed

• PDX-515: fix(mcp): read Provar IDE SfObject org-describe cache layout by @mrdailey99 in #227
• PDX-516: feat(mcp): validate UiAssert screen context (action=New vs View) by @mrdailey99 in #228
• PDX-517: refactor(mcp): dedup screen-context rule on structured test_item_id by @mrdailey99 in #231
• PDX-518: fix(mcp): read UiWithScreen target from uri attribute by @mrdailey99 in #230
• Develop by @mrdailey99 in #229

Full Changelog: v1.6.2...v1.6.3

v1.6.2 Release

What's Changed

• PDX-514: fix(mcp): base estimatedTokens on content text only by @mrdailey99 in #225
• Develop by @mrdailey99 in #226

Full Changelog: v1.6.1...v1.6.2

v1.6.1 Release

v1.6.1

A focused patch release: verbose local test runs no longer fail on large output, and AI-generated control-flow steps now use Provar's real step identifiers.

Fixed

• Verbose local test runs no longer fail with ENOBUFS. provar_automation_testrun (and the other sf-backed tools) used to abort with an opaque ENOBUFS error when a run — especially a DETAILED Salesforce UI run — produced
  more output than the in-memory capture buffer, even though Provar had actually run and written results to disk. Child output is now streamed to disk instead of buffered in memory, so large and verbose runs complete and return their
  results. A residual buffer error now returns an actionable message pointing to the on-disk results and the --json fallback. Validated end-to-end on Windows, macOS, and Linux.
• Control-flow and list-comparison steps now use Provar's real step identifiers. The test-step reference, validator allow-list, schema, and loop prompts referenced control.TryCatchFinally (which does not exist in Provar — the
  canonical step is control.Finally) and a malformed list.ListCompareListCompare. These are corrected everywhere, so AI-generated try/catch/finally and list-compare steps are valid Provar steps that load and run, and the validator
  no longer emits or flags the wrong ids. A new test pins every documented step id to the validator's allow-list to prevent drift.

Upgrade notes

• Non-breaking. No new inputs, outputs, or configuration — both changes are internal correctness and robustness fixes.

v1.6.0 Release

v1.6.0

A major step-up for the local validator: it now mirrors what actually loads and runs in Provar, surfaces severity through validity, and ships a higher default quality bar — plus two new MCP resources that expose the validator's contract to AI clients.

Highlights

• The local validator now mirrors Provar's own load/runtime behaviour. Dozens of structural and best-practice checks that previously lived only in the Quality Hub backend now run locally, so provar_testcase_validate catches load-blocking and runtime defects with no API key required.
• Validity now reflects severity. A critical best-practice violation (e.g. a hallucinated apiId, a non-integer testItemId) now gates is_valid instead of quietly passing — an AI agent can trust is_valid: true again.
• A single, tri-state verdict. Every validate tool returns status: valid | needs_improvement | invalid, alongside the effective quality_threshold and meets_quality_threshold, so agents have one unambiguous signal to gate on.
• A higher default quality bar. The default quality threshold is raised 80 → 90, tunable per call (quality_threshold) or globally (PROVAR_MCP_QUALITY_THRESHOLD).
• Two new MCP resources expose the validator's contract: the structured Provar test-step schema and a canonical Validation Rule Registry.

New

• MCP resource provar://schema/test-step — the structured JSON contract for the Provar test-case XML (root, generic apiCall, every step type with required/optional args and value classes).
• MCP resource provar://docs/validation-rules — the canonical registry of every validation rule across both layers (id, severity, weight, what it checks, and whether it gates is_valid).
• status, quality_threshold, meets_quality_threshold output fields on provar_testcase_validate and the suite/plan/project validate tools.
• quality_threshold input plus the PROVAR_MCP_QUALITY_THRESHOLD env var (precedence: per-call arg → env → 90).
• Context-aware comparisonType validation — the valid comparison set is scoped by step type (AssertValues, UI Assert, …) instead of one flat list.
• Project-aware test_case_id allocation — generated test cases take the next id in the surrounding Provar project rather than a hard-coded id="1"; the chosen id is surfaced on the response.
• PROVAR_PLUGIN_NOT_FOUND error code when the Provar Automation plugin is missing.

Changed

• Validity bridge: critical best-practice violations are surfaced as is_valid-gating issues, deduplicated against the Layer-1 rule that already owns the same concept.
• Severity alignment with Quality Hub: UI-BINDING-ORDER-001 and VAR-NAMING-001 reclassified critical → major (they hard-fail at runtime but do not block loading).
• Test-case generator fidelity: UiDoAction is serialised as uiInteraction, and UiAssert field assertions are nested for correct Provar IDE rendering.

Fixed

• Best-practices engine no longer crashes on numeric tag values.
• RENDER-CASE-001 scoped to the six real valueClass values, removing false positives.
• TC_010 accepts any integer test-case id and treats id as optional (the guid is the real identifier).
• Windows: the sf executable and its arguments are quoted so project paths containing spaces work.

Upgrade notes

• Mostly non-breaking. New inputs, env vars, and output fields are additive.
• Behaviour change to note: with the validity bridge and the higher default threshold (90), a test case that previously returned is_valid: true may now report status: "needs_improvement" (score below 90) or "invalid" (a critical violation now gates validity). Set quality_threshold per call or PROVAR_MCP_QUALITY_THRESHOLD globally to restore the previous 80 bar if needed.

v1.5.3 Release

v1.5.3

Stable release promoting develop → main. This release sharpens Salesforce UI
test-step authoring correctness and brings Microsoft Dynamics 365 + Power
Platform to first-class parity in the Provar MCP, plus a consolidated MCP
configuration reference.

✨ New: Microsoft Dynamics 365 & Power Platform support (PDX-498)

Provar 3.0.7 ships four new NitroXConnect:ms-* UI connection variants. The MCP
now recognizes them end-to-end — no more API-UNKNOWN-001 on Dynamics/Power
Platform test cases.

• Four new shorthands for provar_testcase_generate:
  MSDynamics365Connect, MSDataverseConnect, MSPowerAppConnect,
  MSPowerPageConnect
• Two new validation rules:
  • UI-NITROX-CONNECT-ARGS-001 (critical) — rejects Apex-only args and
    cross-variant args (e.g. powerAppName on a Dynamics 365 step)
  • UI-NITROX-VARIANT-ARG-001 (minor) — flags missing variant args, while
    allowing runtime-bound <apiParam> data-driven patterns
• Docs: new "Microsoft Dynamics & Power Platform Steps (Provar 3.0.7+)"
  section in the step reference, plus Pilot Guide Scenario 14

🐛 Salesforce UI step nesting — generate + validate now agree (PDX-495 / 496 / 497)

Previously the generator could emit flat UI action steps that Provar IDE renders
incorrectly, while the validator scored them 100/clean — generator and validator
disagreed about valid output.

• Generator (PDX-495): provar_testcase_generate now auto-nests UI action
  steps (UiDoAction, UiAssert, UiRead, …) inside their parent
  UiWithScreen <clause name="substeps">. New grouping_mode flag
  (auto default / flat / single-screen) as an escape hatch. Depth-first
  testItemId numbering is preserved.
• Validator (PDX-496): local fallback validator now mirrors the Quality Hub
  UI-NEST-STRUCT-001 rule (severity major, weight 7), walking the full
  ancestor chain across control-flow wrappers (IfThen/ForEach/etc.). Applies to
  7 APIs: UiDoAction, UiAssert, UiRead, UiFill, UiNavigate,
  UiWithRow, UiHandleAlert.
• Alignment (PDX-497): introduced a single shared canonical-set module
  (uiActionApiIds.ts) so generator and validator draw from the same 7-API
  source of truth. Replaced fragile substring matching with exact Set.has()
  lookups (no false positives). UiWithRow's dual role (container + action) is
  now handled via a recursive walker.
• Follow-ups: root-level UiWithRow now also requires a wrapping
  UiWithScreen, and tool outputs now include explicit "required container"
  messaging to guide authors.

📚 MCP Configuration reference (PDX-499)

• New single ## Configuration reference section in docs/mcp.md covering
  every CLI flag and environment variable in one place
• Documents eight previously-buried knobs: --auto-defects, PROVAR_HOME,
  PROVAR_MCP_MAX_TOOL_DEPTH, PROVAR_MCP_EMIT_TOKEN_META,
  PROVAR_MCP_VALIDATION_DIR, PROVAR_NO_UPDATE_CHECK, PROVAR_AUTO_DEFECTS,
  and the CLI-only --allowed-paths callout
• npx-first restructure of Quick start / Client configuration with per-OS and
  per-client examples

✅ Verification

• yarn test:only — 1315 passing, 1 pending
• node scripts/mcp-smoke.cjs — 57/57 pass
• yarn lint / yarn compile — clean

Full changelog: v1.5.2...v1.5.3

v1.5.2 Release

ProvarDX v1.5.2 Release

Highlights

• Quieter validators, louder failure modes — strict properties validation catches typos like testCases vs testCase before the run happens, and the testrun surfaces a RUN-001 warning when sf reports success but zero tests actually ran.
• Smarter retry signal — failures now include error_category + retryable so AI agents and CI can distinguish a flaky Connection reset from a real assertion failure without regexing the message.
• Org-aware authoring (read-only) — new provar_org_describe tool reads Provar's workspace .metadata cache to surface required fields, types, and existence — no live SF connection needed at generation time.
• Canonical valueClass for dates and numbers — date/datetime values now emit valueClass="date" / "datetime", and numbers use valueClass="decimal" per PROVAR_TEST_STEP_REFERENCE.md. Stops the silent-discard failure mode where dates serialised as strings vanished without error.

New

• provar_org_describe MCP tool: reads <workspace>/.metadata/<connection>/* (JSON, XML, and legacy .object formats) and returns object/field describe data. Gracefully handles cache-miss with details.suggestion. All paths run through assertPathAllowed.
• Strict properties validator (provar_properties_validate) detects unknown keys at top-level, metadata.*, and environment.* with Levenshtein-based "did you mean" suggestions. Emits warnings (not errors) so future Provar additions don't break older MCP clients.
• Zero-tests guard on provar_automation_testrun — gated on parsedAny && stepCount === 0 so it only fires when the tool genuinely has JUnit data showing zero tests ran (not when JUnit data is missing/unparseable).
• DATA-001 warning when a test case containing <dataTable> is referenced via the top-level testCase/testCases array rather than a .testinstance (data-driven iteration silently fails in direct mode).
• error_category + retryable optional fields on JUnitStepResult and FailureReport. Categories: INFRASTRUCTURE, ASSERTION, LOCATOR, TIMEOUT, OTHER. Retryable only for INFRASTRUCTURE and TIMEOUT.

Fixed

• valueClass="integer" was emitted for numeric values; canonical reference is valueClass="decimal". All numeric strings (integer and decimal forms) now emit as decimal.
• Datetime regex was anchored only at the start, so 2026-05-19T10:30:00not-a-zone was misclassified as datetime. Now end-anchored with optional fractional seconds + timezone.
• Date fields silently serialised as valueClass="string" and discarded by Provar's runtime, surfacing only when a downstream validation rule failed. Now correctly typed.
• testCasePlanMode resolver bypassed assertPathAllowed on the properties-file override path (security regression). Now enforces policy + canonicalises via fs.realpathSync to prevent symlink escape from the allowed-paths boundary.
• XML required-flag parsing in legacy .object cache files compared against the string "true", but fast-xml-parser returns boolean true by default — required fields were misclassified as nillable. Now handles both forms.
• provar_properties_validate emitted "Warnings emitted programmatically follow the shape WARNING [<CODE>]: <message>" too broadly — the validator also emits placeholder warnings without a code. Wording now scoped to formatWarning() output only.
• provar_testrun_rca tool description implied mode="failures" returns error_category / retryable, but those fields only appear on failures[] in mode="rca". Description corrected.

Internal / infrastructure

• New src/mcp/utils/warningCodes.ts shared enum + formatWarning() helper. All warning emission sites now reference the same canonical codes (SCHEMA-001, RUN-001, DATA-001, PROVARHOME-001, PARALLEL-001, JUNIT-001).
• discoverWorkspace in provar_org_describe policy-checks every candidate dir before any fs call, so the ~/Provar/... home fallback never touches the filesystem when home sits outside --allowed-paths.

Upgrade notes

• Non-breaking. All new warnings are additive in the response (warnings[] array). Existing callers see no behavior change unless they consume the new fields.
• valueClass="decimal" for numeric arguments is a generator-side change — newly generated test cases will produce the canonical XML; existing test cases on disk are unaffected.

v1.5.1 Release

ProvarDX v1.5.1 Release

Highlights

• Smaller, faster MCP handshake — opt into compact tool schemas and load only the tool groups you need. ~36% fewer handshake tokens with compact mode alone, up to ~57% when combined with group filtering.
• Smarter validation loops — agents get tunable response detail, run-over-run diffs, and a single completeness signal that's safe to gate on.
• Single-call test authoring — test-case generation is now a true one-shot construction, with a runtime guard so agents stop iterating in the wrong direction.
• Reliable connection + environment resolution in .testproject files.

Tool-catalog footprint

Tokens sent to the LLM on tools/list (≈4 chars/token):

Configuration	Tools	~Tokens	Savings vs default
Standard (all groups, full descriptions)	41	18,355	—
Compact (all groups, compact descriptions)	41	11,758	−36%
Authoring profile (compact + 4 groups)	21	7,906	−57%

Per-tool savings are largest where they matter most — testcase_generate alone drops from ~2,070 tokens of description to a fraction of that in compact mode.

New

• Compact schema mode and tool-group filtering for trimmed startup payloads.
• detail, baseline_run_id, run_id, and completeness_score on validation tools.
• fields parameter on inspect / list tools to scope responses to only what's needed.
• Depth guard and token-attribution middleware across all tools.
• Construct-vs-amend contract carried into test-case tool titles, descriptions, and a runtime check on empty steps.

Guidance & prompt improvements

• Test-case authoring rewritten as a single-call construction contract — agents now produce a complete test case in one call instead of looping through construct → amend → re-amend cycles. End-to-end authoring of a multi-step Salesforce flow drops from typically 3–5 tool calls to 1.
• Construct-vs-amend semantics surfaced at three layers (tool title, description, runtime guard) so agents that skim only the title still get the contract.
• Validation tools now return a single completeness_score (0–100) so agents have one number to gate on, instead of inferring stop/continue from violation arrays.
• Compact tool descriptions are tuned to keep the contract (when to call, prerequisites, common failure modes) while dropping prose — the signal agents actually use stays intact.

Fixed

• Validation stop decisions now account for all violation levels (plan metadata, suite, best-practices) instead of stopping while issues remain.
• Read-only validation diffs work without writing new results.
• Validation baselines are now scoped to their original project context, so a baseline from one project can't silently diff against another.
• Unknown tool-group names now warn instead of silently disabling everything.
• Release builds now reliably fetch the latest NitroX schemas instead of falling back to a bundled copy.
• Connection + environment resolution in .testproject files.
• Various agent-loop and review-pass hardening for the test-case authoring path.

Upgrade notes

• Non-breaking. All new parameters and env vars are opt-in.
• Existing callers see no behavior change.

v.1.5.0

Provar MCP v1.5.0

The first stable release of the Provar MCP server — a Model Context Protocol integration that lets AI agents drive Provar automation end-to-end.

What's New

• Zero-install startup via npx @provartesting/provardx-cli mcp start — no Salesforce CLI required
• Auto-update check at startup notifies when a newer version is available
• Full path allowlist enforcement, shell injection prevention, and input validation across all tools
• Tool names in underscore format (provar_automation_testrun, etc.) for broad MCP client compatibility

NitroX Component Authoring

• provar_nitrox_discover, read, generate, patch, and validate tools for full FactComponent lifecycle
• AJV JSON Schema validation (NX_SCHEMA_* issues) alongside semantic rules NX001–NX010
• Component catalog and FactComponent/FactPackage schemas fetched from source at release time

Quality Hub Integrations

• provar_qualityhub_connect, testrun, defect_create, testcase_retrieve, and examples_retrieve tools
• Secure API key storage and validation
• provar_qualityhub_display and provar_qualityhub_testrun_report for in-session results

Test Authoring & Execution Loop

• provar_testcase_step_edit for targeted step-level edits
• provar_testrun_rca for AI-driven root cause analysis on failures
• provar_connection_list for discovering available Salesforce connections
• provar_nitrox_validate with full XML correctness checking and local semantic rules

Guided AI Workflows

• Built-in MCP prompts for onboarding, troubleshooting, orchestration, and test migration workflows
• provar://docs/tool-guide resource for in-context tool selection guidance

Platform

• Windows support: .cmd/.bat executable resolution and path case-insensitivity handling
• macOS and Linux fully supported via npx

v1.1.0

Added the following commands:

• Automation Setup
• Automation Project Compile
• Automation Metadata Download
• Automation Test Run