HDDS-15690. Support Node-ID for DiskBalancer Commands by Gargi-jais11 · Pull Request #10755 · apache/ozone

Gargi-jais11 · 2026-07-14T08:27:55Z

What changes were proposed in this pull request?

ozone admin datanode diskbalancer status command fails when passing DataNode ID instead of address

Minor (Usability / CLI Consistency)

The ozone admin datanode diskbalancer status command currently only accepts a DataNode's network address/host string. If a user attempts to pass a valid datanodeUuid (DataNode ID), the command fails or throws an unsupported/invalid argument error.

For better usability and alignment with other ozone admin commands (which typically support both UUIDs and addresses to identify cluster nodes), this command should be updated to resolve and accept the DataNode ID.

root@Nodename ~]# ozone admin datanode list -id scm1
Datanode: a3b63511-bdf8-4fa1-8ab6-d19c0e806f84 (/default/10.140.95.199/Nodename/3 pipelines)
Operational State: IN_SERVICE
Health State: HEALTHY
Total volume count: 1
Healthy volume count: 1
Related pipelines:
18acb50f-830d-426d-a8c4-a2d0577c12ab/RATIS/THREE/RATIS/OPEN/Follower
1f6743b3-3eb4-4c9b-994c-3655217dc9b4/RATIS/THREE/RATIS/OPEN/Leader
f5ef1508-579a-4bee-9933-a04c525c36dd/RATIS/ONE/RATIS/OPEN/Leader


Datanode: bc55ef13-ba30-4086-9f80-a5c9abc263ed (/default/10.140.181.1/Nodename-2/3 pipelines)
Operational State: IN_SERVICE
Health State: HEALTHY
Total volume count: 1
Healthy volume count: 1
Related pipelines:
a5bf9a58-1eb2-4036-9e3a-c8e508942482/RATIS/THREE/RATIS/OPEN/Leader
85ef6b85-f3c1-4ce6-bd9d-8ada883a88fe/RATIS/ONE/RATIS/OPEN/Leader
a2075c08-4167-462c-acb1-1bdf03cfc47a/RATIS/THREE/RATIS/OPEN/Follower


Datanode: c557e715-f1ef-458b-8df0-04dc63b54e0e (/default/10.140.163.5/Nodename-3/1 pipelines)
Operational State: IN_SERVICE
Health State: HEALTHY
Total volume count: 1
Healthy volume count: 1
Related pipelines:
2e4b6954-f2be-47b4-bd68-0fa5c8bc4eca/RATIS/ONE/RATIS/OPEN/Leader 
Now check the status of  disk balancer

DiskBalancer output:-

[root@Nodename ~]# ozone admin datanode diskbalancer status a3b63511-bdf8-4fa1-8ab6-d19c0e806f84
Error on node [a3b63511-bdf8-4fa1-8ab6-d19c0e806f84]: Invalid host name: local host is: "Nodename/10.140.213.128"; destination host is: "a3b63511-bdf8-4fa1-8ab6-d19c0e806f84":19864; java.net.UnknownHostException: Invalid host name: local host is: "Nodename/10.140.213.128"; destination host is: "a3b63511-bdf8-4fa1-8ab6-d19c0e806f84":19864; java.net.UnknownHostException; For more details see:  http://wiki.apache.org/hadoop/UnknownHost; For more details see:  http://wiki.apache.org/hadoop/UnknownHost
Failed to get DiskBalancer status from nodes: [a3b63511-bdf8-4fa1-8ab6-d19c0e806f84]

PR Changes:

Allows uuid also to be passed for the diskbaalncer command using --node-id flag but when scm is down giving uuid will fail as it needs scm to be resolved for sending the command. In that case use IP or hsotname.
When hostname/ipAddress or --in-service-datanode is given then datanode is displayed in hostname(ipAddpress:port) format
else when uuid is given it simply shows uuid at datanode place.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-15690

How was this patch tested?

Added unit tests.

bash-5.1$ ozone admin datanode diskbalancer report --node-id 59c14bfa-1ccd-45e4-83e6-8c2c3a5de873,0d4a065f-db6c-4649-9906-1a4df09ffbdf
Report result:
Datanode: 59c14bfa-1ccd-45e4-83e6-8c2c3a5de873
Aggregate VolumeDataDensity: 0.00%
IdealUsage: 8.00% | Threshold: 10.00% | ThresholdRange: (0.00%, 18.00%)

Volume Details:

StorageID                                     StoragePath                                OzoneCapacity  OzoneAvailable       OzoneUsed     ContainerPreAllocatedSpace   EffectiveUsedSpace     Utilization   VolumeDensity
DS-57df847c-49a9-4b3e-84a5-b851bc58e2be       /data/hdds/hdds                               1006.75 GB       926.22 GB         4.18 MB                            0 B             80.53 GB           8.00%           0.00%

-------

Datanode: 0d4a065f-db6c-4649-9906-1a4df09ffbdf
Aggregate VolumeDataDensity: 0.00%
IdealUsage: 8.00% | Threshold: 10.00% | ThresholdRange: (0.00%, 18.00%)

Volume Details:

StorageID                                     StoragePath                                OzoneCapacity  OzoneAvailable       OzoneUsed     ContainerPreAllocatedSpace   EffectiveUsedSpace     Utilization   VolumeDensity
DS-298606de-78ea-47d1-abe0-e2f80f06b6bc       /data/hdds/hdds                               1006.75 GB       926.22 GB         4.18 MB                            0 B             80.53 GB           8.00%           0.00%


Note:
  - Aggregate VolumeDataDensity: Sum of per-volume density (deviation from ideal); higher means more imbalance.
  - IdealUsage: Target utilization (0-100%) when volumes are evenly balanced.
  - ThresholdRange: Acceptable deviation (percent); volumes within IdealUsage +/- Threshold are considered balanced.
  - VolumeDensity: Deviation of a particular volume's utilization from IdealUsage.
  - Utilization: how much a particular volume is utilized (effectiveUsedSpace / ozoneCapacity) in %.
  - OzoneCapacity: Ozone data volume capacity.
  - OzoneAvailable: Ozone data volume available space.
  - OzoneUsed: Ozone data volume used space.
  - ContainerPreAllocatedSpace: Space reserved for containers not yet written to disk.
  - EffectiveUsedSpace: This is the actual used space of volume which is visible to the diskBalancer : (ozoneCapacity minus ozoneAvailable) + containerPreAllocatedSpace + move delta.
  - move delta: source volume space to be reclaimed after move completion; this value is reflected only when diskBalancer is running else it is 0.

bash-5.1$ ozone admin datanode diskbalancer status --node-id 59c14bfa-1ccd-45e4-83e6-8c2c3a5de873,0d4a065f-db6c-4649-9906-1a4df09ffbdf
Status result:
Datanode                                                     Status     Threshold(%)    BandwidthInMB   Threads    StopAfterDiskEven  ContainerStates                SuccessMove  FailureMove  BytesMoved(MB)  EstBytesToMove(MB) EstTimeLeft(min)    
59c14bfa-1ccd-45e4-83e6-8c2c3a5de873                         STOPPED    10.0000         10              5          true               CLOSED,QUASI_CLOSED            0            0            0               0                  0                   
0d4a065f-db6c-4649-9906-1a4df09ffbdf                         STOPPED    10.0000         10              5          true               CLOSED,QUASI_CLOSED            0            0            0               0                  0                   

Note:
 - EstBytesToMove is calculated based on the target disk even state with the configured threshold.
 - EstTimeLeft is calculated based on EstimatedBytesToMove and configured disk bandwidth.
 - Both EstimatedBytes and EstTimeLeft could be non-zero while no containers can be moved, especially when the configured threshold or disk capacity is too small.
 - ContainerStates indicates which container lifecycle states are eligible for balancing; by default, CLOSED and QUASI_CLOSED are allowed.
bash-5.1$ ozone admin datanode diskbalancer update -b 100 -t 4 -p 10 --node-id 59c14bfa-1ccd-45e4-83e6-8c2c3a5de873,0d4a065f-db6c-4649-9906-1a4df09ffbdf
Updated DiskBalancer configuration on nodes: [59c14bfa-1ccd-45e4-83e6-8c2c3a5de873, 0d4a065f-db6c-4649-9906-1a4df09ffbdf]
bash-5.1$ ozone admin datanode diskbalancer start --node-id 59c14bfa-1ccd-45e4-83e6-8c2c3a5de873,0d4a065f-db6c-4649-9906-1a4df09ffbdf
Started DiskBalancer on nodes: [59c14bfa-1ccd-45e4-83e6-8c2c3a5de873, 0d4a065f-db6c-4649-9906-1a4df09ffbdf]
bash-5.1$ ozone admin datanode diskbalancer stop --node-id 59c14bfa-1ccd-45e4-83e6-8c2c3a5de873,0d4a065f-db6c-4649-9906-1a4df09ffbdf
Stopped DiskBalancer on nodes: [59c14bfa-1ccd-45e4-83e6-8c2c3a5de873, 0d4a065f-db6c-4649-9906-1a4df09ffbdf]
bash-5.1$ ozone admin datanode diskbalancer report --node-id 59c14bfa-1ccd-45e4-83e6-8c2c3a5de873,0d4a065f-db6c-4649-9906-1a4df09ffbdf --json
[ {
  "datanode" : "59c14bfa-1ccd-45e4-83e6-8c2c3a5de873",
  "action" : "report",
  "status" : "success",
  "volumeDensity" : "0.00%",
  "idealUsage" : "8.00%",
  "threshold %" : "4.00%",
  "thresholdRange" : "(4.00%, 12.00%)",
  "volumes" : [ {
    "storageId" : "DS-57df847c-49a9-4b3e-84a5-b851bc58e2be",
    "storagePath" : "/data/hdds/hdds",
    "ozoneCapacity" : "1006.75 GB",
    "ozoneAvailable" : "926.22 GB",
    "ozoneUsed" : "4.18 MB",
    "containerPreAllocatedSpace" : "0 B",
    "effectiveUsedSpace" : "80.53 GB",
    "utilization" : "8.00%",
    "volumeDensity" : "0.00%"
  } ]
}, {
  "datanode" : "0d4a065f-db6c-4649-9906-1a4df09ffbdf",
  "action" : "report",
  "status" : "success",
  "volumeDensity" : "0.00%",
  "idealUsage" : "8.00%",
  "threshold %" : "4.00%",
  "thresholdRange" : "(4.00%, 12.00%)",
  "volumes" : [ {
    "storageId" : "DS-298606de-78ea-47d1-abe0-e2f80f06b6bc",
    "storagePath" : "/data/hdds/hdds",
    "ozoneCapacity" : "1006.75 GB",
    "ozoneAvailable" : "926.22 GB",
    "ozoneUsed" : "4.18 MB",
    "containerPreAllocatedSpace" : "0 B",
    "effectiveUsedSpace" : "80.53 GB",
    "utilization" : "8.00%",
    "volumeDensity" : "0.00%"
  } ]
} ]
bash-5.1$ ozone admin datanode diskbalancer status --node-id 59c14bfa-1ccd-45e4-83e6-8c2c3a5de873,0d4a065f-db6c-4649-9906-1a4df09ffbdf --json
[ {
  "datanode" : "59c14bfa-1ccd-45e4-83e6-8c2c3a5de873",
  "action" : "status",
  "status" : "success",
  "serviceStatus" : "STOPPED",
  "threshold" : 4.0,
  "bandwidthInMB" : 100,
  "threads" : 10,
  "stopAfterDiskEven" : true,
  "containerStates" : "CLOSED,QUASI_CLOSED",
  "successMove" : 0,
  "failureMove" : 0,
  "bytesMovedMB" : 0,
  "estBytesToMoveMB" : 0,
  "estTimeLeftMin" : 0
}, {
  "datanode" : "0d4a065f-db6c-4649-9906-1a4df09ffbdf",
  "action" : "status",
  "status" : "success",
  "serviceStatus" : "STOPPED",
  "threshold" : 4.0,
  "bandwidthInMB" : 100,
  "threads" : 10,
  "stopAfterDiskEven" : true,
  "containerStates" : "CLOSED,QUASI_CLOSED",
  "successMove" : 0,
  "failureMove" : 0,
  "bytesMovedMB" : 0,
  "estBytesToMoveMB" : 0,
  "estTimeLeftMin" : 0
} ]
bash-5.1$ ozone admin datanode diskbalancer update -b 100 -t 4 -p 10 --node-id 59c14bfa-1ccd-45e4-83e6-8c2c3a5de873,0d4a065f-db6c-4649-9906-1a4df09ffbdf --json
[ {
  "datanode" : "59c14bfa-1ccd-45e4-83e6-8c2c3a5de873",
  "action" : "update",
  "status" : "success",
  "configuration" : {
    "threshold" : 4.0,
    "bandwidthInMB" : 100,
    "parallelThread" : 10
  }
}, {
  "datanode" : "0d4a065f-db6c-4649-9906-1a4df09ffbdf",
  "action" : "update",
  "status" : "success",
  "configuration" : {
    "threshold" : 4.0,
    "bandwidthInMB" : 100,
    "parallelThread" : 10
  }
} ]
bash-5.1$ ozone admin datanode diskbalancer start --node-id 59c14bfa-1ccd-45e4-83e6-8c2c3a5de873,0d4a065f-db6c-4649-9906-1a4df09ffbdf --json
[ {
  "datanode" : "59c14bfa-1ccd-45e4-83e6-8c2c3a5de873",
  "action" : "start",
  "status" : "success"
}, {
  "datanode" : "0d4a065f-db6c-4649-9906-1a4df09ffbdf",
  "action" : "start",
  "status" : "success"
} ]
bash-5.1$ ozone admin datanode diskbalancer stop --node-id 59c14bfa-1ccd-45e4-83e6-8c2c3a5de873,0d4a065f-db6c-4649-9906-1a4df09ffbdf --json
[ {
  "datanode" : "59c14bfa-1ccd-45e4-83e6-8c2c3a5de873",
  "action" : "stop",
  "status" : "success"
}, {
  "datanode" : "0d4a065f-db6c-4649-9906-1a4df09ffbdf",
  "action" : "stop",
  "status" : "success"
} ]

rich7420 · 2026-07-15T05:44:50Z

@Gargi-jais11 thanks for the patch!

Copilot

Pull request overview

Updates the ozone admin datanode diskbalancer CLI to accept datanode UUIDs (Node IDs) in addition to host/IP forms, resolving UUIDs via SCM before issuing DiskBalancer RPCs and showing UUIDs in the display output. This improves CLI usability/consistency with other admin commands and reduces user confusion when supplying node identifiers.

Changes:

Add UUID detection + UUID→CLIENT_RPC address resolution via SCM for DiskBalancer subcommands.
Include UUID in human/JSON output display strings (hostname (ip:port / uuid)), and document the new accepted identifier forms.
Add/extend unit tests covering UUID resolution, mixed valid/invalid inputs, JSON output, and state-reset behavior across invocations.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
hadoop-ozone/cli-admin/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/AbstractDiskBalancerSubCommand.java	Resolves UUID args via SCM, tracks resolution failures, resets per-invocation state, and formats display names via SCM metadata.
hadoop-ozone/cli-admin/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DiskBalancerSubCommandUtil.java	Adds UUID detection/resolution helpers, CLIENT_RPC address helper, and appends UUID to display formatting.
hadoop-ozone/cli-admin/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DiskBalancerStatusSubcommand.java	Clears cached per-run status state and widens output formatting to accommodate UUID in display.
hadoop-ozone/cli-admin/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DiskBalancerReportSubcommand.java	Clears cached per-run report state to prevent cross-invocation leakage.
hadoop-ozone/cli-admin/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DiskBalancerCommands.java	Updates CLI usage docs to describe UUID-capable positional args and output format changes.
hadoop-ozone/cli-admin/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DatanodeParameters.java	Updates positional parameter label and help text to reflect UUID support.
hadoop-ozone/cli-admin/src/test/java/org/apache/hadoop/hdds/scm/cli/datanode/TestDiskBalancerSubCommandUtil.java	New unit tests for UUID detection, UUID resolution, and UUID-inclusive display formatting.
hadoop-ozone/cli-admin/src/test/java/org/apache/hadoop/hdds/scm/cli/datanode/TestDiskBalancerSubCommands.java	Adds tests for UUID arguments across status/start/stop/update, mixed validity, JSON output, and state reset.
hadoop-hdds/docs/content/feature/DiskBalancer.md	Documents UUID support in the DiskBalancer CLI user docs.
hadoop-hdds/docs/content/feature/DiskBalancer.zh.md	Same as above for the Chinese user docs.
hadoop-hdds/docs/content/design/diskbalancer.md	Updates design docs to reflect SCM usage for UUID resolution in addition to batch discovery.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

slfan1989

Thanks for working on this! The overall approach looks good to me, and the added test coverage is helpful. I left one minor comment about preserving non-UUID targets when SCM resolution is unavailable, but it does not look like a blocker to me.

Gargi-jais11 · 2026-07-17T06:28:36Z

@ChenSammi Please take a look again. I have updated the code to support --node-id for uuid.

rich7420

LGTM! thanks for the update!

support node-id for diskbalancer commands

a0f07e4

Gargi-jais11 marked this pull request as ready for review July 14, 2026 08:28

Gargi-jais11 requested review from ChenSammi and slfan1989 July 14, 2026 08:28

rich7420 reviewed Jul 15, 2026

View reviewed changes

Comment thread ...zone/cli-admin/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DatanodeParameters.java Outdated

rich7420 reviewed Jul 15, 2026

View reviewed changes

Comment thread ...-admin/src/main/java/org/apache/hadoop/hdds/scm/cli/datanode/DiskBalancerSubCommandUtil.java Outdated

ChenSammi requested a review from Copilot July 16, 2026 12:23

Copilot started reviewing on behalf of ChenSammi July 16, 2026 12:24 View session

Copilot AI reviewed Jul 16, 2026

View reviewed changes

slfan1989 approved these changes Jul 17, 2026

View reviewed changes

Gargi-jais11 marked this pull request as draft July 17, 2026 06:11

create explicit --node-id for uuid

b084347

Gargi-jais11 marked this pull request as ready for review July 17, 2026 06:27

rich7420 approved these changes Jul 18, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HDDS-15690. Support Node-ID for DiskBalancer Commands#10755

HDDS-15690. Support Node-ID for DiskBalancer Commands#10755
Gargi-jais11 wants to merge 2 commits into
apache:masterfrom
Gargi-jais11:HDDS-15690

Gargi-jais11 commented Jul 14, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

rich7420 commented Jul 15, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

slfan1989 left a comment

Uh oh!

Gargi-jais11 commented Jul 17, 2026

Uh oh!

rich7420 left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

Gargi-jais11 commented Jul 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

What is the link to the Apache JIRA

How was this patch tested?

Uh oh!

Uh oh!

Uh oh!

rich7420 commented Jul 15, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

slfan1989 left a comment

Choose a reason for hiding this comment

Uh oh!

Gargi-jais11 commented Jul 17, 2026

Uh oh!

rich7420 left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Gargi-jais11 commented Jul 14, 2026 •

edited

Loading