Skip to content

Add checksum and clone template to S3 Anchor #22819

Open
JasMehta08 wants to merge 2 commits into
root-project:masterfrom
JasMehta08:s3-anchor-checksum-clone-template
Open

Add checksum and clone template to S3 Anchor #22819
JasMehta08 wants to merge 2 commits into
root-project:masterfrom
JasMehta08:s3-anchor-checksum-clone-template

Conversation

@JasMehta08

@JasMehta08 JasMehta08 commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

This Pull request:

(Is a part of the GSoC 2026 project S3 Backend for RNTuple.)

Adds checksum and a configurable clone path to the S3 Anchor.

Changes or fixes:

  • Add an XXH3 64-bit checksum to the S3 anchor, computed over a canonical compact JSON representation of all data fields. Verified on read.
  • Add fCloneTemplate field to RNTupleAnchorS3 for resolving attribute-set (hidden) ntuple base URLs. Serialized into the anchor JSON and extracted on read (optional, for forward compatibility with older anchors).

Checklist:

  • tested changes locally

Comment thread tree/ntuple/inc/ROOT/RPageStorageS3.hxx Outdated
Comment thread tree/ntuple/inc/ROOT/RPageStorageS3.hxx Outdated
@JasMehta08
JasMehta08 force-pushed the s3-anchor-checksum-clone-template branch from 1373142 to 5f539d7 Compare July 15, 2026 12:56
Comment thread tree/ntuple/inc/ROOT/RPageStorageS3.hxx
Comment thread tree/ntuple/test/ntuple_storage_s3.cxx Outdated
Comment thread tree/ntuple/test/ntuple_storage_s3.cxx Outdated
@JasMehta08
JasMehta08 requested a review from silverweed July 15, 2026 13:33
@JasMehta08
JasMehta08 force-pushed the s3-anchor-checksum-clone-template branch from 5f539d7 to bfe8630 Compare July 18, 2026 05:46
@JasMehta08 JasMehta08 changed the title Add checksum and clone template to S3 Anchor Add checksum to S3 Anchor and Clone template as file-constant Jul 18, 2026
@JasMehta08 JasMehta08 changed the title Add checksum to S3 Anchor and Clone template as file-constant Add checksum to S3 Anchor and change clone template as file-constant Jul 18, 2026
@JasMehta08
JasMehta08 force-pushed the s3-anchor-checksum-clone-template branch from bfe8630 to 5945e72 Compare July 18, 2026 06:14
@JasMehta08

Copy link
Copy Markdown
Contributor Author

@silverweed and @jblomer

I have removed the clone template from the anchor, and made it a file-constant, please have a new review of the commits

Comment thread tree/ntuple/src/RPageStorageS3.cxx Outdated
@github-actions

github-actions Bot commented Jul 20, 2026

Copy link
Copy Markdown

Test Results

    23 files      23 suites   3d 17h 30m 12s ⏱️
 3 874 tests  3 874 ✅ 0 💤 0 ❌
78 880 runs  78 880 ✅ 0 💤 0 ❌

Results for commit e38221d.

♻️ This comment has been updated with latest results.

@JasMehta08
JasMehta08 force-pushed the s3-anchor-checksum-clone-template branch 2 times, most recently from 1d7e1db to 5b189a8 Compare July 20, 2026 07:22
@JasMehta08
JasMehta08 force-pushed the s3-anchor-checksum-clone-template branch from 5b189a8 to 06b9980 Compare July 20, 2026 08:47
@JasMehta08

JasMehta08 commented Jul 20, 2026

Copy link
Copy Markdown
Contributor Author

@silverweed ,

I have updated the implementation, and removed CanonicalPayload() from both ToJSoN() as well as CreateFromJSoN(). I removed the duplicate work that was being done by it. ToJSoN() using the anchor itself to create and checksum, and attaches it at the end. In CreateFromJSoN() it strips the checksum of the anchor, then using the remaining anchor to create a new hash and check against the checksum.

@silverweed

Copy link
Copy Markdown
Contributor

Thanks! When @jblomer approves we can merge

@JasMehta08
JasMehta08 force-pushed the s3-anchor-checksum-clone-template branch from 06b9980 to 396482c Compare July 22, 2026 17:54
@JasMehta08

Copy link
Copy Markdown
Contributor Author

@jblomer and @silverweed

I have added the cloneTemplate into the in the anchor for now, for backwards compatibility. I have not made changes to CreateFromJSON() as it would require adding an additional field to the anchor struct, and will add it with the reader changes. Or would it be better to just add it now?

@JasMehta08
JasMehta08 force-pushed the s3-anchor-checksum-clone-template branch from 396482c to 0b8e053 Compare July 23, 2026 19:34
@JasMehta08 JasMehta08 changed the title Add checksum to S3 Anchor and change clone template as file-constant Add checksum and clone template to S3 Anchor Jul 23, 2026
@JasMehta08
JasMehta08 force-pushed the s3-anchor-checksum-clone-template branch from 0b8e053 to 723b666 Compare July 24, 2026 09:22
@JasMehta08

Copy link
Copy Markdown
Contributor Author

@silverweed and @jblomer

there was a small issue with the clang formatting that I have fixed, the other failures dont seem to be related to changes due to this PR.

@jblomer jblomer left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, one tiny comment.

Comment thread tree/ntuple/src/RPageStorageS3.cxx Outdated
@JasMehta08
JasMehta08 force-pushed the s3-anchor-checksum-clone-template branch from 723b666 to e38221d Compare July 24, 2026 11:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants