Skip to content

[v5] Remove VSTest support from the test adapter and make MTP the default#9726

Open
Evangelink wants to merge 2 commits into
dev/v5from
dev/amauryleve/dev-v5
Open

[v5] Remove VSTest support from the test adapter and make MTP the default#9726
Evangelink wants to merge 2 commits into
dev/v5from
dev/amauryleve/dev-v5

Conversation

@Evangelink

@Evangelink Evangelink commented Jul 8, 2026

Copy link
Copy Markdown
Member

Summary

First slice of the v5 work to run MSTest exclusively on Microsoft.Testing.Platform (MTP). Targets the long-lived dev/v5 branch (created from main).

This PR removes the VSTest reflection surface (ITestExecutor/ITestDiscoverer and friends) from MSTest.TestAdapter and makes MTP the only engine by removing the EnableMSTestRunner opt-in. MSTest now runs solely in MTP mode.

Crucially, this does not mean MSTest stops working under vstest.console. Starting with VSTest 18.10, VSTest knows how to host an MTP test host under VSTest, so an MTP-only test app (like MSTest v5) is still runnable through vstest.console — it just runs as an MTP app (MTP extensions are not surfaced in that hosting mode). This PR bumps VSTest to 18.10.0-preview-26357-05 so that path works, which is what keeps the VSTest-based tests green.

Changes

  • buildTransitive/common/MSTest.TestAdapter.targets — remove the EnableMSTestRunner opt-in / VSTest coexistence branch; the adapter is now always a testing platform application (IsTestingPlatformApplication=true, GenerateProgramFile=false). Adds a transitional MSTest_WarnOnObsoleteEnableMSTestRunner warning when a leftover EnableMSTestRunner value is detected (suppressible via SuppressMSTestRunnerObsoleteWarning).
  • Properties/AssemblyInfo.cs — drop [assembly: TestExtensionTypes(typeof(MSTestDiscoverer), typeof(MSTestExecutor))] (the VSTest extension registration).
  • EqtTraceLogger.cs — deleted. PlatformServiceProvider.AdapterTraceLogger defaults to NopTraceLogger, and MTP installs MTPTraceLogger, so no bootstrap is needed.
  • MSTestExecutor.cs — remove ITestExecutor, [ExtensionUri], the parameterless ctor, the EqtTrace bootstrap, the public RunTests(...) overloads, and the now-dead TestCase-based RunTestsAsync overload + ToUnitTestElement (MTP always runs from sources). The internal source-based execution engine used by MSTestBridgedTestFramework is untouched.
  • MSTestDiscoverer.cs — remove ITestDiscoverer, [FileExtension]/[DefaultExecutorUri], the parameterless ctor, and the public DiscoverTests(...) entry point. The internal discovery engine used by the MTP bridge is untouched.
  • Folder consolidation — delete the VSTestAdapter/ and TestingPlatformAdapter/ folders and move all their files (the shared MSTestExecutor/MSTestDiscoverer engine + the MTP extension files) to the MSTest.TestAdapter project root. Namespaces unchanged.
  • Directory.Packages.props — bump VSTest (MicrosoftNETTestSdkVersion and Microsoft.TestPlatform.Filter.Source) to 18.10.0-preview-26357-05 so VSTest can host the MTP test host.
  • MSTestAdapter.UnitTests — the removed VSTest reflection surface broke these tests: deleted the tests asserting on removed VSTest attributes and rewrote the behavioral tests to drive the internal MTP engine (DiscoverTestsAsync / RunTestsAsync(sources, ...)).
  • FrameworkHandlerAdapter.cs (VSTestBridge) — IFrameworkHandle.EnableShutdownAfterTestRun became [Obsolete] ("has no effect") in the new object model; suppress CS0618 while keeping the interface implementation and historical pass-through.

Why this is only a first slice

The MTP path still consumes the VSTest object model through Microsoft.Testing.Extensions.VSTestBridge (VSTestRunTestExecutionRequest, IFrameworkHandle, TestCase, …). Fully decoupling from the object model is a separate, larger effort the codebase already flags as a long-term goal ("the eventual goal should be to stop using the VSTestBridge altogether").

Verification

Full repo builds cleanly (0 errors), including all MSTest.TestAdapter TFMs (net462, net8.0, net9.0, UWP, WinUI). The rewritten MSTestAdapter.UnitTests classes pass (12 tests).

Follow-ups (subsequent PRs on dev/v5)

  • Remove EnableMSTestRunner / VSTest coexistence from Microsoft.Testing.Platform.MSBuild.targets and every other reference.
  • MSTest.Sdk: remove UseVSTest and Sdk/VSTest/; make the Runner the only engine.
  • Remove the VSTestBridge object-model coupling from the adapter (neutral request model).
  • Drop the isMTP flag that threads through MSTestExecutorTestExecutionManagerUnitTestDiscovererAssemblyEnumeratorWrapper (always true now); requires updating the kept MSTestAdapter.PlatformServices.UnitTests call sites.
  • Update samples and docs; remove EnableMSTestRunner from all .csproj.
  • Update --help/--info acceptance expectations if any option surface changes.

Copilot AI review requested due to automatic review settings July 8, 2026 09:05
@Evangelink Evangelink changed the base branch from main to dev/v5 July 8, 2026 09:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This is the first slice of the v5 effort to drop full VSTest support from MSTest.TestAdapter and run MSTest exclusively on Microsoft.Testing.Platform (MTP). It removes the VSTest reflection surface (ITestExecutor/ITestDiscoverer implementations, extension-registration attributes, parameterless constructors, and public entry points) and the EnableMSTestRunner opt-in, making the adapter always a testing-platform application. The internal execution/discovery engine consumed by MSTestBridgedTestFramework is left intact, and the change verified that the src project still builds across all TFMs. It targets the long-lived dev/v5 branch and is deliberately a draft: VSTest-specific test projects are knowingly left broken for follow-up PRs.

Changes:

  • Strip VSTest coupling from MSTestExecutor/MSTestDiscoverer (interfaces, [ExtensionUri]/[FileExtension]/[DefaultExecutorUri], parameterless ctors, public RunTests/DiscoverTests) and drop the EqtTrace bootstrap plus the EqtTraceLogger file.
  • Remove the [assembly: TestExtensionTypes(...)] VSTest extension registration.
  • Simplify MSTest.TestAdapter.targets to always set IsTestingPlatformApplication=true and GenerateProgramFile=false, removing the EnableMSTestRunner coexistence branch.
Show a summary per file
File Description
src/Adapter/MSTest.TestAdapter/VSTestAdapter/MSTestExecutor.cs Drops ITestExecutor, [ExtensionUri], parameterless ctor, EqtTrace bootstrap, and public RunTests overloads; keeps internal engine used by the MTP bridge.
src/Adapter/MSTest.TestAdapter/VSTestAdapter/MSTestDiscoverer.cs Drops ITestDiscoverer, file-extension/executor-uri attributes, parameterless ctor, and public DiscoverTests; keeps internal discovery path.
src/Adapter/MSTest.TestAdapter/VSTestAdapter/EqtTraceLogger.cs Deleted; default NopTraceLogger + MTP's MTPTraceLogger cover logging.
src/Adapter/MSTest.TestAdapter/Properties/AssemblyInfo.cs Removes VSTest extension-type registration (also inadvertently drops the file's UTF-8 BOM — flagged).
src/Adapter/MSTest.TestAdapter/buildTransitive/common/MSTest.TestAdapter.targets Removes EnableMSTestRunner opt-in; adapter is now unconditionally an MTP application.

Review details

  • Files reviewed: 5/5 changed files
  • Comments generated: 1
  • Review effort level: Medium

Comment thread src/Adapter/MSTest.TestAdapter/Properties/AssemblyInfo.cs Outdated
@Evangelink Evangelink marked this pull request as ready for review July 8, 2026 10:48

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

🤖 Automated review by GitHub Copilot. Generated by the Expert Code Review workflow. To request a follow-up action, reply by tagging @copilot directly.

# Dimension Verdict
20 Build Infrastructure & Dependencies 🟡 1 MODERATE (suggestion for follow-up)

✅ 21/22 dimensions clean.

  • Build Infrastructure — users with EnableMSTestRunner=false silently get MTP on v5 upgrade (tracked for follow-up)

Dimension notes:

  • 1–3 (Correctness, Threading, Security): Clean. EqtTraceLogger removal is safe — PlatformServiceProvider.AdapterTraceLogger defaults to NopTraceLogger.Instance and MSTestBridgedTestFramework ctor sets MTPTraceLogger before any execution/discovery call. No new shared state or threading concerns.
  • 4 (Public API): No breaking change — MSTestDiscoverer and MSTestExecutor were already internal sealed. The public members existed only to satisfy the VSTest interface contracts.
  • 5–9 (Performance, Cross-TFM, Resources, Defensive Coding, Localization): N/A or clean. Module initializer does less work now (perf improvement).
  • 10–14 (Test dimensions): N/A — no test files changed. Acknowledged that CI is expected to be red until follow-up test project updates land.
  • 15–17 (Structure, Naming, Docs): Clean. Minor note: // TODO: Verify why VSTest annotates the IEnumerable as nullable. at MSTestExecutor.cs:77 is now stale (VSTest no longer calls this path) and lacks an issue reference per repo policy — can be cleaned up when the isMTP flag removal follow-up lands.
  • 18–19 (Analyzer, IPC): N/A.
  • 20 (Build Infrastructure): See inline comment on .targets file — suggestion for migration warning in follow-up.
  • 21 (Scope & PR Discipline): Well-scoped first slice. Follow-ups are enumerated clearly. Consider tracking them as GitHub issues for discoverability.
  • 22 (PowerShell): N/A.

MSBuild Authoring (supplemental): The .targets change follows correct MSBuild patterns — Condition=" '$(Prop)' == '' " for safe defaults in a transitive targets file. No anti-patterns detected.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@Evangelink Evangelink force-pushed the dev/amauryleve/dev-v5 branch from 64edcfd to f3f35d0 Compare July 10, 2026 15:27
@github-actions

This comment has been minimized.

2 similar comments
@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

…ault

Starting with v5, MSTest runs solely on Microsoft.Testing.Platform (MTP). The
VSTest reflection surface is removed from MSTest.TestAdapter and the adapter is
always a testing platform application (IsTestingPlatformApplication defaults to
true). EnableMSTestRunner is no longer used; setting it emits an info message.

Starting with VSTest 18.10, vstest.console can host an MTP test application
(executable), so MSTest keeps working under 'dotnet test'/vstest in MTP mode
(MTP extensions are not surfaced through VSTest).

Adapter:
- Remove the VSTest surface from MSTestExecutor/MSTestDiscoverer: [ExtensionUri],
  [DefaultExecutorUri], [FileExtension], ITestExecutor/ITestDiscoverer, the
  reflection parameterless ctors, and the public RunTests/DiscoverTests entry
  points; the internal MTP-facing async methods remain. Cancel() becomes internal.
- Delete the VSTest EqtTraceLogger (and its module-initializer hook) and the
  [assembly: TestExtensionTypes] registration; update InternalAPI accordingly.
- Default IsTestingPlatformApplication to true and key the localized-satellite
  copy logic off it (v5 is always MTP), replacing the removed EnableMSTestRunner.
- Emit an info message (not a warning) when the obsolete EnableMSTestRunner
  property is set; suppress with SuppressMSTestRunnerObsoleteMessage.

Packages/test infrastructure:
- Bump Microsoft.NET.Test.Sdk / Microsoft.TestPlatform / Filter.Source to
  18.10.0-preview-26357-05 (VSTest that can host an MTP test host).
- Always include the test-tools feed in generated NuGet.config; make the shared
  MSTest acceptance asset use the MTP dotnet test runner; drop the addPublicFeeds
  parameter and Microsoft.NET.Test.Sdk from the MTP asset template (avoids a
  System.Memory 4.0.2.0 vs 4.0.5.0 binding-redirect crash on net462 publish).
- Convert AppDomainTests to validate .NET Framework AppDomain isolation under MTP
  (net462 MTP exe, runsettings toggling DisableAppDomain).
- Remove MSTest.VstestConsoleWrapper.IntegrationTests (drove net462 MSTest library
  assets through the classic vstest.console TranslationLayer, unsupported in v5;
  coverage is duplicated in-process by MSTest.IntegrationTests).
- Skip classic-VSTest-runner acceptance tests on .NET 10+ SDK and drop the
  EnableMSTestRunner=false opt-out coverage.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@Evangelink Evangelink force-pushed the dev/amauryleve/dev-v5 branch from f3f35d0 to 629efef Compare July 13, 2026 12:05
@github-actions

This comment has been minimized.

The addPublicFeeds parameter was removed from TestAsset.GenerateAssetAsync, but
this call site was missed during the rebase, breaking the build with CS1739.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown
Contributor

🧪 Test quality grade — PR #9726

GradeTestNotes
B (80–89) mod UseCooperativeCancellationForTimeoutAnalyzerTests.
WhenTimeoutAttributeWithoutCooperativeCancellation_
CodeFixOffersTaskRunTestMethod
Strong assertion: verifies both diagnostic location and transformed code; body is mildly long due to raw string literals — consider extracting expected snippets to named local variables for readability.
A (90–100) mod UseCooperativeCancellationForTimeoutAnalyzerTests.
WhenTaskRunTestMethodAttributeWithTimeout_
NoDiagnostic
No issues found. Note: orphaned stray lines after the closing brace in the raw diff may indicate a copy-paste artifact that could break compilation.

This advisory comment was generated automatically. Grades are heuristic
and informational — they do not block merging. Re-run with
/grade-tests.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

🤖 Automated content by GitHub Copilot. Generated by the Grade Tests on PR (on open / sync) workflow. · 80.5 AIC · ⌖ 3.99 AIC · ⊞ 8.9K · [◷]( · )

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants