Description
The test MCP tool has no argument for running a subset of a test suite, so every invocation runs the whole thing. There is no workaround through the existing arguments: directory is explicitly not forwarded as a positional target, per the note in _parseTest:
// NOTE: 'directory' is intentionally not added here. It is applied as the
// working directory in [_runToolCommand], not as a positional test target.
and tags / exclude_tags only filter by annotations that already exist in the test source.
The CLI itself has no such gap. very_good test test/foo_test.dart works today: FlutterTestOptions.parse keeps argResults.rest and the command forwards it to the runner. So this is a hole in the MCP surface only, and an agent driving the CLI through MCP is strictly less capable than one shelling out.
That matters for agent workflows in two ways:
- Latency. On a mid-sized app suite, running everything to check one directory turns a ~10s feedback loop into a ~50s one.
- Context cost. The tool returns the runner's full output. On a suite with a failure that produces a large widget-tree or stack dump, a single call can return well over 100k characters, nearly all of it irrelevant to the tests the caller cared about. Being able to narrow the run is the cheapest available mitigation.
Requirements
Additional Context
One behaviour worth documenting in the argument description rather than working around: targeting specific files already disables the test optimization step, via TestCLIRunner.isTargettingTestFiles(options.rest) in both test.dart and dart_test_command.dart. That is existing CLI behaviour and should stay consistent — callers just need to know it.
I have a patch ready and will open a PR referencing this issue. Happy to adjust the argument name or shape (array vs. comma-separated string) to whatever you prefer.
Description
The
testMCP tool has no argument for running a subset of a test suite, so every invocation runs the whole thing. There is no workaround through the existing arguments:directoryis explicitly not forwarded as a positional target, per the note in_parseTest:and
tags/exclude_tagsonly filter by annotations that already exist in the test source.The CLI itself has no such gap.
very_good test test/foo_test.dartworks today:FlutterTestOptions.parsekeepsargResults.restand the command forwards it to the runner. So this is a hole in the MCP surface only, and an agent driving the CLI through MCP is strictly less capable than one shelling out.That matters for agent workflows in two ways:
Requirements
testtool accepts optional test paths and forwards them as positional targets.restrather than consumed as an option value.very_good testandvery_good dart test.Additional Context
One behaviour worth documenting in the argument description rather than working around: targeting specific files already disables the test optimization step, via
TestCLIRunner.isTargettingTestFiles(options.rest)in bothtest.dartanddart_test_command.dart. That is existing CLI behaviour and should stay consistent — callers just need to know it.I have a patch ready and will open a PR referencing this issue. Happy to adjust the argument name or shape (array vs. comma-separated string) to whatever you prefer.