Skip to content

feat(fs): copy with metadata#7535

Open
dentiny wants to merge 1 commit into
apache:mainfrom
dentiny:hjiang/copy-with-metadata
Open

feat(fs): copy with metadata#7535
dentiny wants to merge 1 commit into
apache:mainfrom
dentiny:hjiang/copy-with-metadata

Conversation

@dentiny
Copy link
Copy Markdown
Contributor

@dentiny dentiny commented May 17, 2026

Rationale for this change

Currently, copy behavior is inconsistent across multiple platforms: on macos user metadata is copied over, but left on linux.

Verification script

#[tokio::test]
    async fn fs_copy_does_not_preserve_user_metadata_on_linux() {
        let temp_dir = tempfile::tempdir().expect("create temp dir");
        let root = temp_dir.path().to_string_lossy().to_string();

        let backend = FsBuilder::default()
            .root(&root)
            .build()
            .expect("build fs backend");

        let src = "src.txt";
        let dst = "dst.txt";

        let src_abs = temp_dir.path().join(src);
        std::fs::write(&src_abs, b"hello world").expect("write src");

        // Set a `user.foo` xattr directly on the source file. Some
        // filesystems (e.g. tmpfs mounted with `nouser_xattr`) reject this;
        // in that case skip the test instead of failing, since we cannot
        // observe the behavior we want to verify.
        if let Err(e) = xattr::set(&src_abs, "user.foo", b"bar") {
            // ENOTSUP == EOPNOTSUPP == 95 on Linux, mirroring the check in
            // `FsCore::get_user_metadata`.
            if e.kind() == std::io::ErrorKind::Unsupported || e.raw_os_error() == Some(95) {
                eprintln!("skipping fs_copy xattr test: xattr unsupported on {root}");
                return;
            }
            panic!("failed to set xattr on source: {e}");
        }

        // Sanity check: the source file now reports the user metadata via
        // the public stat API, which reads xattrs under the hood.
        let src_meta = backend.stat(src, OpStat::default()).await.expect("stat src");
        let src_user_md: HashMap<String, String> = src_meta
            .into_metadata()
            .user_metadata()
            .cloned()
            .unwrap_or_default();
        assert_eq!(
            src_user_md.get("foo").map(String::as_str),
            Some("bar"),
            "source file should expose its user metadata via stat",
        );

        backend
            .copy(src, dst, OpCopy::default(), OpCopier::default())
            .await
            .expect("copy");

        let dst_abs = temp_dir.path().join(dst);
        assert!(dst_abs.exists(), "destination file should exist after copy");
        assert_eq!(
            std::fs::read(&dst_abs).expect("read dst"),
            b"hello world".as_slice(),
            "file content should be copied verbatim",
        );

        // The actual behavior under test: user metadata is lost on copy.
        let dst_meta = backend.stat(dst, OpStat::default()).await.expect("stat dst");
        let dst_user_md = dst_meta
            .into_metadata()
            .user_metadata()
            .cloned()
            .unwrap_or_default();
        assert!(
            dst_user_md.is_empty(),
            "fs_copy must NOT preserve user metadata on Linux, but got {dst_user_md:?}",
        );

        // Cross-check at the raw xattr layer to rule out a stat-layer bug.
        let raw_dst_attrs: Vec<_> = xattr::list(&dst_abs)
            .expect("list dst xattrs")
            .filter(|name| name.to_string_lossy().starts_with("user."))
            .collect();
        assert!(
            raw_dst_attrs.is_empty(),
            "destination file should have no `user.*` xattrs after fs_copy, got {raw_dst_attrs:?}",
        );
    }

What changes are included in this PR?

This PR explicitly copies user metadata on unix platform.

Are there any user-facing changes?

No.

AI Usage Statement

Gpt-5.5 helped me made the code change, I did the review.

@dentiny dentiny requested a review from Xuanwo as a code owner May 17, 2026 05:37
@dosubot dosubot Bot added size:M This PR changes 30-99 lines, ignoring generated files. releases-note/feat The PR implements a new feature or has a title that begins with "feat" labels May 17, 2026
.map_err(new_std_io_error)?;

// `write_with_user_metadata` is only supported in unix platform.
#[cfg(unix)]
Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

write with metadata is only supported on unix for now

#[cfg(unix)]
write_with_user_metadata: true,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

releases-note/feat The PR implements a new feature or has a title that begins with "feat" size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant