Skip to content

MCP Tools in Drills : Drill Get and Drill Resource Get - #3232

Open
dynamicdhx wants to merge 30 commits into
mainfrom
users/dhruvbharuka/drillgetanddrillresourceget
Open

MCP Tools in Drills : Drill Get and Drill Resource Get #3232
dynamicdhx wants to merge 30 commits into
mainfrom
users/dhruvbharuka/drillgetanddrillresourceget

Conversation

@dynamicdhx

@dynamicdhx dynamicdhx commented Aug 10, 2026

Copy link
Copy Markdown

What does this PR do?

`Adds new Resilience Management drill tooling to the Azure MCP “resilience” area, enabling agents/users to list/get drills and drill targets (resources) for a service group, and wires these tools into server-level documentation and consolidation.

Changes:

Introduces azmcp resilience drill get and azmcp resilience drill resource get commands (options, models, JSON context, service APIs, and setup registration).
Adds unit tests for the new commands and extends recorded/live integration tests to cover drill listing.
Updates consolidated tool mapping plus azmcp-commands.md and e2eTestPrompts.md to include the new tools.`

[Add additional context, screenshots, or information that helps reviewers]

GitHub issue number?

[Link to the GitHub issue this PR addresses]

Pre-merge Checklist

  • Required for All PRs
    • Read contribution guidelines
    • PR title clearly describes the change
    • Commit history is clean with descriptive messages (cleanup guide)
    • Added comprehensive tests for new/modified functionality
    • Created a changelog entry if the change falls among the following: new feature, bug fix, UI/UX update, breaking change, or updated dependencies. Follow the changelog entry guide
  • For MCP tool changes:
    • One tool per PR: This PR adds or modifies only one MCP tool for faster review cycles
    • Updated servers/Azure.Mcp.Server/README.md and/or servers/Fabric.Mcp.Server/README.md documentation
    • Validate README.md changes running the script ./eng/scripts/Process-PackageReadMe.ps1. See Package README
    • For new or modified tool descriptions, ran ToolDescriptionEvaluator and obtained a score of 0.4 or more and a top 3 ranking for all related test prompts
    • For tools with new names, including new tools or renamed tools, update consolidated-tools.json
    • For renamed tools, follow the Tool Rename Checklist and tag the PR with the breaking-change label
    • For new tools associated with Azure services or publicly available tools/APIs/products, add URL to documentation in the PR description
  • Extra steps for Azure MCP Server tool changes:
    • Updated command list in servers/Azure.Mcp.Server/docs/azmcp-commands.md
    • Ran ./eng/scripts/Update-AzCommandsMetadata.ps1 to update tool metadata in azmcp-commands.md (required for CI)
    • Updated test prompts in servers/Azure.Mcp.Server/docs/e2eTestPrompts.md
    • 👉 For Community (non-Microsoft team member) PRs:
      • Security review: Reviewed code for security vulnerabilities, malicious code, or suspicious activities before running tests (crypto mining, spam, data exfiltration, etc.)
      • Manual tests run: added comment /azp run mcp - pullrequest - live to run Live Test Pipeline

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds new Resilience Management drill tooling to the Azure MCP “resilience” area, enabling agents/users to list/get drills and drill targets (resources) for a service group, and wires these tools into server-level documentation and consolidation.

Changes:

  • Introduces azmcp resilience drill get and azmcp resilience drill resource get commands (options, models, JSON context, service APIs, and setup registration).
  • Adds unit tests for the new commands and extends recorded/live integration tests to cover drill listing.
  • Updates consolidated tool mapping plus azmcp-commands.md and e2eTestPrompts.md to include the new tools.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
tools/Azure.Mcp.Tools.ResilienceManagement/tests/Azure.Mcp.Tools.ResilienceManagement.Tests/ResilienceManagementCommandTests.cs Adds recorded/live integration coverage for listing drills.
tools/Azure.Mcp.Tools.ResilienceManagement/tests/Azure.Mcp.Tools.ResilienceManagement.Tests/Drills/DrillGetCommandTests.cs Adds unit tests for drill list/get behavior.
tools/Azure.Mcp.Tools.ResilienceManagement/tests/Azure.Mcp.Tools.ResilienceManagement.Tests/Drills/Resources/DrillResourceGetCommandTests.cs Adds unit tests for drill target list/get behavior.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Services/IResilienceManagementService.cs Extends service contract with drill + drill-target APIs.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Services/ResilienceManagementService.cs Implements ARM calls for listing/getting drills and drill targets.
tools/Azure.Mcp.Tools.ResilienceManagement/src/ResilienceManagementSetup.cs Registers the new drill command group + command DI registrations.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Options/ResilienceManagementOptionDescriptions.cs Adds option description text for --drill.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Options/Drills/DrillGetOption.cs Defines CLI/tool options for drill list/get.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Options/Drills/Resources/DrillResourceGetOption.cs Defines CLI/tool options for drill target list/get.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Models/DrillInfo.cs Adds response model for drill details.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Commands/ResilienceManagementJsonContext.cs Registers new drill types for source-gen JSON serialization.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Commands/Drills/DrillGetCommand.cs Implements drill list/get command and error mapping.
tools/Azure.Mcp.Tools.ResilienceManagement/src/Commands/Drills/Resources/DrillResourceGetCommand.cs Implements drill target list/get command and error mapping.
servers/Azure.Mcp.Server/src/Resources/consolidated-tools.json Adds drill tools into the consolidated resilience management tool.
servers/Azure.Mcp.Server/docs/e2eTestPrompts.md Adds e2e prompts for the two new drill tools.
servers/Azure.Mcp.Server/docs/azmcp-commands.md Documents new CLI commands for drills and drill targets.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread tools/Azure.Mcp.Tools.ResilienceManagement/src/Models/DrillInfo.cs Outdated
Comment thread servers/Azure.Mcp.Server/src/Resources/consolidated-tools.json
@dynamicdhx
dynamicdhx requested a review from a team as a code owner August 11, 2026 06:56
@dynamicdhx

dynamicdhx commented Aug 11, 2026 via email

Copy link
Copy Markdown
Author

- Add missing final newlines in DrillGetCommand.cs, DrillInfo.cs, DrillGetOption.cs, and DrillGetCommandTests.cs
- Fix import ordering in ResilienceManagementSetup.cs (alphabetical: Drills, Goals, Recovery, UsagePlans, then Services, then Microsoft)

@g2vinay Vinay Gera (g2vinay) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding the drill tooling, Dhruv — the command/service/options code is clean and follows the existing resilience patterns closely, so this is mostly there. The blocker is that CI is red, and I was able to reproduce three separate failures locally (3 failed / 90 passed running the toolset tests at the head commit). None of them are design problems — they're all test/CI hygiene and quick to clear.

1. The new integration tests have no recordings (this is what's failing the test legs)

Should_list_drills, Should_list_drill_resources, and Should_return_400_when_drill_missing_for_drill_resource_get all fail in playback with ClientResultException: Service request failed / "exception during start playback" — the classic "there's no recording for this test" signature. The tell is that assets.json still points at the old tag (..._5ec122e793) and isn't part of this PR, so the new tests were never recorded and pushed. Good news is the provisioning is already there (step 8 in test-resources-post.ps1 creates a drill and polls for a drill resource), so you should just need to run the drill tests in Record mode, test-proxy push, and commit the updated assets.json. (Ref: docs/recorded-tests.md.)

While you're in there: Should_return_400_when_drill_missing_for_drill_resource_get never actually calls Azure — it just omits the required --drill and expects a 400 — but because it lives in the recorded suite it still needs a recording. That exact case is already covered by the DrillResourceGetCommandTests unit test, so I'd drop it from the recorded suite rather than record an empty interaction.

2. dotnet format hasn't been run (this is failing the Analyze leg)

dotnet format --verify-no-changes flags four things:

  • Missing final newline in DrillInfo.cs, DrillGetCommand.cs, and DrillGetOption.cs.
  • Import ordering in ResilienceManagementSetup.cs — the new Commands.Drills usings landed after the Goals.* ones, but should be alphabetical (Drills before Goals).

A quick dotnet format + commit clears all of it.

3. Spelling check is red

The all-caps deployment-output key DRILLNAME (used in the post script and the integration tests) gets spell-checked as drillname, which isn't in the dictionary. The sibling keys are already allow-listed (recoveryplanname, servicegroupname, …), so just add drillname to tools/Azure.Mcp.Tools.ResilienceManagement/cspell.yaml in alphabetical order.

Checklist items still outstanding

  • Changelog — these are new user-facing tools, so they need an entry under servers/Azure.Mcp.Server/changelog-entries/.
  • ToolDescriptionEvaluator — please paste the results into the PR description (≥ 0.4 and top-3 for the drill prompts). The descriptions read well; we just need the numbers on record.

Minor (non-blocking)

DrillInfo.Tags is a Dictionary<string,string>? — prefer IReadOnlyDictionary<string,string>? to match UsagePlanInfo and avoid exposing a mutable collection on the response model (same thing Copilot flagged).

Once the recordings are in, format is clean, and the spelling word is added, CI should go green and I'm happy to re-review. Thanks again!

@g2vinay Vinay Gera (g2vinay) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Following up on my earlier review with in-line comments.

Comment thread tools/Azure.Mcp.Tools.ResilienceManagement/src/Models/DrillInfo.cs Outdated
@dynamicdhx

Copy link
Copy Markdown
Author

Copilot resolve the merge conflicts in this pull request

Co-authored-by: dynamicdhx <310516170+dynamicdhx@users.noreply.github.com>

Copilot AI commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Copilot resolve the merge conflicts in this pull request

Resolved by merging the current main branch and reconciling the release-consumed changelog entry in ee7dc01.

Co-authored-by: dynamicdhx <310516170+dynamicdhx@users.noreply.github.com>

@g2vinay Vinay Gera (g2vinay) left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two follow-ups from a skill/consistency pass over the new drill tools

Co-authored-by: dynamicdhx <310516170+dynamicdhx@users.noreply.github.com>
Comment thread eng/scripts/Deploy-TestResources.ps1 Outdated
@dynamicdhx

Copy link
Copy Markdown
Author

Tool Description Evaluator Results

Test 1

Expected Tool: resilience_drill_get
Prompt: List all resilience drills in service group <service_group>

Results

Rank Score Tool Status
1 0.751840 resilience_drill_get EXPECTED
2 0.615782 resilience_drill_resource_get
3 0.563313 resilience_recovery_plan_get
4 0.513548 resilience_goal_assignment_get
5 0.487196 resilience_goal_template_get

Test 2

Expected Tool: resilience_drill_get
Prompt: Get the details of resilience drill <drill_name> in service group <service_group>

Results

Rank Score Tool Status
1 0.730860 resilience_drill_get EXPECTED
2 0.636920 resilience_drill_resource_get
3 0.558141 resilience_recovery_plan_get
4 0.525329 resilience_goal_assignment_get
5 0.484862 resilience_goal_template_get

Test 3

Expected Tool: resilience_drill_resource_get
Prompt: List all drill resources for resilience drill <drill_name> in service group <service_group>

Results

Rank Score Tool Status
1 0.755046 resilience_drill_resource_get EXPECTED
2 0.754533 resilience_drill_get
3 0.531550 resilience_recovery_plan_get
4 0.510866 resilience_recovery_plan_resource_get
5 0.494711 group_resource_list

Test 4

Expected Tool: resilience_drill_resource_get
Prompt: List all drill targets for resilience drill <drill_name> in service group <service_group>

Results

Rank Score Tool Status
1 0.741283 resilience_drill_resource_get EXPECTED
2 0.721324 resilience_drill_get
3 0.504911 resilience_recovery_plan_get
4 0.504342 resilience_goal_assignment_get
5 0.491443 resilience_goal_template_get

Test 5

Expected Tool: resilience_drill_resource_get
Prompt: Show the resources targeted by resilience drill <drill_name> in service group <service_group>

Results

Rank Score Tool Status
1 0.735897 resilience_drill_resource_get EXPECTED
2 0.690784 resilience_drill_get
3 0.509586 resilience_recovery_plan_get
4 0.486840 resilience_goal_assignment_get
5 0.482897 resilience_recovery_plan_resource_get

Test 6

Expected Tool: resilience_drill_resource_get
Prompt: Get the complete details of drill resource <resource_name> for resilience drill <drill_name> in service group <service_group>

Results

Rank Score Tool Status
1 0.709874 resilience_drill_resource_get EXPECTED
2 0.677170 resilience_drill_get
3 0.523816 resilience_recovery_plan_get
4 0.494171 resilience_goal_assignment_get
5 0.490590 resilience_recovery_plan_resource_get

Test 7

Expected Tool: resilience_drill_resource_get
Prompt: Get drill target <resource_name> for resilience drill <drill_name> in service group <service_group>

Results

Rank Score Tool Status
1 0.733168 resilience_drill_resource_get EXPECTED
2 0.660466 resilience_drill_get
3 0.477953 resilience_goal_assignment_get
4 0.477938 resilience_recovery_plan_get
5 0.460337 resilience_goal_template_get

Test 8

Expected Tool: resilience_drill_resource_get
Prompt: Retrieve the ARM properties of drill resource <resource_name> for resilience drill <drill_name> in service group <service_group>

Results

Rank Score Tool Status
1 0.697110 resilience_drill_resource_get EXPECTED
2 0.628869 resilience_drill_get
3 0.483541 resilience_recovery_plan_get
4 0.462251 resilience_goal_resource_get
5 0.452077 resilience_recovery_plan_resource_get

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants