[azure-ai-ml] Fix internal pipeline Command dropping interactive services during serialization

[ayushhgarg-work]

Description

Fixes an issue where the internal pipeline Command node (azure.ai.ml._internal.entities.command.Command) silently dropped node-level interactive services (SSH, JupyterLab, TensorBoard, VS Code, etc.) during serialization. Because the services field was never emitted into the pipeline REST request, the backend received no service definitions, Designer did not create the Interactive services run setting, and Singularity/Aether jobs produced no ssh/jupyterlab/vscode endpoints.

Root cause

Command._to_rest_object in the internal entities module serialized only limits and resources, unlike the public Command serializer (azure.ai.ml.entities._builders.command.Command) which already serializes services. The internal node also had no services property, so the field could not survive serialization even when set on the node instance.

Changes

• Added a services property/setter (using _resolve_job_services) and __init__ handling to the internal Command, mirroring the public Command.
• Command._to_rest_object now serializes services via get_rest_dict_for_node_attrs (only when present, so behavior is unchanged when no services are set).
• Command._from_rest_object_to_init_params now rehydrates services back into typed JobService objects for round-trip parity.
• Distributed (subclass of the internal Command) inherits the fix automatically.
• Added regression test test_interactive_services_in_internal_command_node.
• Updated CHANGELOG.

Related work items

• ICM 822596251 — AMLSingularity | ssh/jupyterlab link doesn't appear
• Bug 5466757 — [azure-ai-ml][Pipeline][Interactive services] Internal Command drops services during serialization

Testing

• Verified services now serialize into the REST request: {"my_ssh": {"job_service_type": "SSH"}, "my_jupyter": {"job_service_type": "JupyterLab"}}.
• Verified to-rest/from-rest round-trip preserves typed JobService objects.
• Added and ran a regression unit test (passes).
• Ran the internal pipeline + component unit suites; all pass except one unrelated pre-existing failure (test_artifacts_in_additional_includes, which fails due to a missing azure-devops az CLI extension in the local environment).

All SDK Contribution checklist:

• The pull request does not introduce [breaking changes]
• CHANGELOG is updated for new features, bug fixes or other significant changes.
• I have read the contribution guidelines.

General Guidelines and Best Practices

• Title of the pull request is clear and informative.
• There are a small number of commits, each of which have an informative message.

Testing Guidelines

• Pull request includes test coverage for the included changes.

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).
9 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Pull request overview

Fixes internal pipeline Command nodes dropping interactive services during REST serialization and deserialization.

Changes:

• Adds service resolution and REST round-trip support.
• Adds regression coverage for SSH and JupyterLab services.
• Documents the fix in the changelog.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
azure/ai/ml/_internal/entities/command.py	Preserves interactive services during REST conversion.
tests/internal/unittests/test_pipeline_job.py	Tests serialization and typed deserialization.
CHANGELOG.md	Records the bug fix.

[Copilot]

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

[github-actions]

[Pilot] PR Pipeline Failure Analysis

A CI pipeline failed on this pull request. Here is an automated analysis of what went wrong and how to get the build green.

What failed

The macOS Python 3.11 CI job (macos311) failed for azure-ai-ml — both the sdist and whl test runs. Five unit tests failed:

• TestDataOperations.test_mount_persistent and test_mount_non_persistent
• TestDatastoreOperations.test_mount_persistent and test_mount_non_persistent
• TestPackageInstallation.test_package_installed_below_cpython_3_13

The test_package_installed_below_cpython_3_13 test name strongly suggests it expects a dependency not to be installed above CPython 3.12, but the macOS 3.11 runner environment may have a version mismatch or a package that was unexpectedly installed/updated. The mount-related failures (test_mount_persistent, test_mount_non_persistent) in both TestDataOperations and TestDatastoreOperations are likely pre-existing test failures unrelated to this PR's changes (which focus on interactive services serialization), but they should be confirmed.

These appear to be test failures, not build or infrastructure failures.

Recommended next steps

• Check whether test_mount_persistent, test_mount_non_persistent, and test_package_installed_below_cpython_3_13 were already failing on main before this PR (i.e., pre-existing failures not introduced by this change).
• If these tests are pre-existing failures, document them and consider adding skip markers or opening a tracking issue.
• If any failure was introduced by this PR, review the mount-related datastore/data operation code paths to ensure nothing was inadvertently changed.
• Re-run the pipeline to rule out transient issues.
• See the CI troubleshooting guide: https://aka.ms/ci-fix
• Push new commits to address any failures introduced by this PR; this comment updates automatically on the next failing run.

Raw pipeline analysis (azsdk ci analyze)

Analyzing pipeline https://github.com/Azure/azure-sdk-for-python/pull/48238...
--------------------------------------------------------------------------------
Failed Tests
--------------------------------------------------------------------------------
{
  "/tmp/6613139/LLM Artifacts - macos311 - 1/ml-azure-ai-ml-test-junit-sdist.xml": [
    "sdk.ml.azure-ai-ml.tests.dataset.unittests.test_data_operations.TestDataOperations.test_mount_persistent",
    "sdk.ml.azure-ai-ml.tests.dataset.unittests.test_data_operations.TestDataOperations.test_mount_non_persistent",
    "sdk.ml.azure-ai-ml.tests.datastore.unittests.test_datastore_operations.TestDatastoreOperations.test_mount_persistent",
    "sdk.ml.azure-ai-ml.tests.datastore.unittests.test_datastore_operations.TestDatastoreOperations.test_mount_non_persistent",
    "sdk.ml.azure-ai-ml.tests.internal_utils.unittests.test_dev_requirements.TestPackageInstallation.test_package_installed_below_cpython_3_13"
  ],
  "/tmp/6613139/LLM Artifacts - macos311 - 1/ml-azure-ai-ml-test-junit-whl.xml": [
    "sdk.ml.azure-ai-ml.tests.dataset.unittests.test_data_operations.TestDataOperations.test_mount_persistent",
    "sdk.ml.azure-ai-ml.tests.dataset.unittests.test_data_operations.TestDataOperations.test_mount_non_persistent",
    "sdk.ml.azure-ai-ml.tests.datastore.unittests.test_datastore_operations.TestDatastoreOperations.test_mount_persistent",
    "sdk.ml.azure-ai-ml.tests.datastore.unittests.test_datastore_operations.TestDatastoreOperations.test_mount_non_persistent",
    "sdk.ml.azure-ai-ml.tests.internal_utils.unittests.test_dev_requirements.TestPackageInstallation.test_package_installed_below_cpython_3_13"
  ]
}

Copilot detected the failing pipeline and generated the analysis above. To have it attempt a fix automatically, reply with @copilot please fix the failing pipeline on this PR.

Generated by Pipeline Analysis - Next Steps · 33.1 AIC · ⌖ 6.33 AIC · ⊞ 6.7K · ◷