feat(services/hdfs-native): support rename with if_not_exists

[PDGGK]

Which issue does this PR close?

Part of #7828 — RFC-7818. Said on #8089 that hdfs_native would follow webdav; this is that.

Rationale for this change

hdfs_native took _args: OpRename and ignored it, so if_not_exists was rejected by the correctness-check layer. The sibling hdfs service already implements this; the two now behave the same way.

Two places needed the flag.

1. The destination is stat'd before the rename, and an existing file is deleted.

Ok(status) => {
    if status.isdir { return Err(IsADirectory) }
    else { self.client.delete(&to_path, true).await? }   // <- destroys the target
}

test_rename_with_if_not_exists_returns_condition_not_match asserts the target still reads back with its original content, so returning ConditionNotMatch has to happen before the delete. The check is placed after the isdir arm, keeping the existing IsADirectory answer for a directory destination — same ordering as hdfs.

2. The not-found arm pre-creates the destination, and that create can lose a race.

HdfsError::FileNotFound(_) => {
    self.client.create(&to_path, WriteOptions::default().create_parent(true)).await?
}

WriteOptions::default() is overwrite: false (hdfs-native-0.14.3 client.rs:53), so if another writer takes the destination between the stat and this call, the NameNode raises FileAlreadyExistsException, which the crate maps to HdfsError::AlreadyExists (hdfs/proxy.rs:347) and this service maps to ErrorKind::AlreadyExists. Under if_not_exists the contract calls for ConditionNotMatch, so a small mapper translates it — the same shape as map_hdfs_rename_error in services/hdfs.

The final rename(..., overwrite = true) stays as it is: by that point the destination is a file this call just created, so overwrite = false would reject our own placeholder.

Verification

Against a real HDFS cluster, not by reading. The fixture in fixtures/hdfs/docker-compose-hdfs-cluster.yml uses network_mode: host, which does not reach the host on Docker Desktop for macOS, so I ran the same two images on a bridge network with dfs.datanode.hostname=localhost and dfs.client.use.datanode.hostname=true. Same Hadoop 3.2.1 images, same OPENDAL_HDFS_NATIVE_* variables as the CI action.

Rename suite:

running 10 tests
test behavior::test_rename_source_dir                                     ... ok
test behavior::test_rename_non_existing_source                            ... ok
test behavior::test_rename_self                                           ... ok
test behavior::test_rename_target_dir                                     ... ok
test behavior::test_rename_with_if_not_exists_nested                      ... ok
test behavior::test_rename_file                                           ... ok
test behavior::test_rename_with_if_not_exists                             ... ok
test behavior::test_rename_nested                                         ... ok
test behavior::test_rename_with_if_not_exists_returns_condition_not_match ... ok
test behavior::test_rename_overwrite                                      ... ok

test result: ok. 10 passed; 0 failed

test_rename_overwrite is in that set, so the default path still replaces the destination.

And the control that makes those numbers mean something — dropping the ConditionNotMatch return while keeping the capability bit:

---- behavior::test_rename_with_if_not_exists_returns_condition_not_match ----
test panicked: rename must fail: ()

test result: FAILED. 9 passed; 1 failed

Exactly one test. The check, not the capability bit, is doing the work.

Whole-suite counts, same cluster, --test-threads 1, unmodified tree vs this branch:

upstream/main	113 passed; 0 failed
this branch	116 passed; 0 failed

The three added tests are exactly the delta in the total, and nothing else moved.

Worth flagging for anyone reproducing this locally: run it single-threaded. With the default thread count a single-DataNode cluster on this machine fails a handful of read-stream tests — a different handful on each run, on the unmodified tree as well — so a parallel run is not a usable control.

cargo clippy -p opendal-service-hdfs-native --all-features --all-targets -- -D warnings and cargo fmt are clean.

On not adding a unit test

Setting the capability is what enrolls the service in the three behaviour tests above, and those run against a real NameNode. A unit test over the error mapper would only restate the if_not_exists && guard that is visible in the diff. Happy to add one if you would rather have it.

Separable

If you would rather land the minimum, the mapper in the not-found arm can be dropped and the stat-path check alone still passes all three behaviour tests — the mapper only changes which error a concurrent writer sees. I have kept it because AlreadyExists leaking out of an if_not_exists call is the one error kind that operation is defined not to produce.

Are there any user-facing changes?

Yes — op.rename_with(from, to).if_not_exists(true) now works on hdfs_native, returning ErrorKind::ConditionNotMatch when the destination exists. Ordinary rename is unchanged.

[Xuanwo]

Why we need this?

[PDGGK]

It handles a race, and it is separable from the rest — dropped in the latest push, so this PR is now only the capability plus the check.

For the record, what it did: in the not-found arm the destination is pre-created with WriteOptions::default(), which does not overwrite. If another writer takes the destination between the get_file_info above and that create, the NameNode answers FileAlreadyExistsException, hdfs-native maps it to HdfsError::AlreadyExists (hdfs/proxy.rs:347) and parse_hdfs_error maps that to ErrorKind::AlreadyExists. Under if_not_exists the contract says that case is ConditionNotMatch, so the mapper translated it.

It is not needed for the RFC behaviour: the three rename_with_if_not_exists behaviour tests are single-threaded and pass without it — just re-ran the full rename suite against a real cluster on this push, 10 passed / 0 failed. Without it a caller that loses that race sees AlreadyExists instead of ConditionNotMatch, which is the only difference.

Happy to send it as its own PR if you want that error kind corrected; otherwise it can stay as it is.