fix(services/b2): percent-encode the object path in presigned requests - #8076
Merged
Conversation
The three presign arms interpolate the path raw, while the ordinary
read and write paths in the same crate encode it:
// backend.rs, presign Stat and Read
let url = format!(
"{}/file/{}/{}?Authorization={}",
auth_info.download_url, self.core.bucket, path, resp.authorization_token
);
// backend.rs, presign Write
req = req.header("X-Bz-File-Name", build_abs_path(&self.core.root, path));
// core.rs:144-149, download_file_by_name -- the reference
let url = format!("{}/file/{}/{}",
auth_info.download_url, self.bucket, percent_encode_path(&path));
// core.rs:262, upload_file -- the reference
req = req.header(X_BZ_FILE_NAME, percent_encode_path(&p));
Measured by handing the formatted strings to http::Request::get:
dir/file.txt path=/file/bkt/dir/file.txt query=Authorization=TOK
a#b.txt path=/file/bkt/a query=None
a?b.txt path=/file/bkt/a query=b.txt?Authorization=TOK
a b.txt BUILD ERROR: invalid uri character
a%20b.txt sends a%20b.txt, which B2 decodes to "a b.txt"
The `#` case is the worst: the URL is truncated to a different object
*and* the whole ?Authorization= is swallowed as a fragment, so the
presigned URL carries no token at all. `a b.txt` cannot be presigned even
though op.read("a b.txt") on the same object works, because that goes
through the encoding path. And the write header sends a%20b.txt where
op.write("a%20b.txt") sends a%2520b.txt, so a presigned upload lands on a
different key than an ordinary write for the same OpenDAL path.
The first line above is the one that matters for blast radius: a path
with no reserved characters encodes to itself, because
percent_encode_path leaves `/` alone. Presigned URLs for ordinary object
names are byte-identical to what they are today.
No new tests: these are format! arguments inside an async method whose
only seam is a live B2 authorization call, and asserting on them would
mean restructuring the presign arms. The three existing unit tests pass,
fmt and clippy are clean.
Note for rebasing: apache#7801 renames build_abs_path at these same three
sites. If it lands first the X-Bz-File-Name line needs a one-word
rebase.
Xuanwo
approved these changes
Aug 14, 2026
Member
|
Next time, please avoid submitting similar PRs. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
None — found while reading the service.
Rationale for this change
The three presign arms interpolate the object path raw, while the ordinary read and write paths in the same crate encode it.
Presign (
backend.rs), Stat and Read:Presign Write:
The reference,
core.rs:Handing the formatted strings to
http::Request::getshows what a presigned URL actually becomes:dir/file.txtpath=/file/bkt/dir/file.txt,query=Authorization=TOK✅a#b.txtpath=/file/bkt/a,query=Nonea?b.txtpath=/file/bkt/a,query=b.txt?Authorization=TOKa b.txtBUILD ERROR: invalid uri charactera%20b.txta%20b.txt, which B2 decodes toa b.txtThe
#row is the worst: the URL is truncated to a different object and the entire?Authorization=is swallowed as a fragment, so the presigned URL carries no token at all.a b.txtcannot be presigned even thoughop.read("a b.txt")on the same object works — that call goes through the encoding path. And the write header sendsa%20b.txtwhereop.write("a%20b.txt")sendsa%2520b.txt, so a presigned upload lands on a different key than an ordinary write for the same OpenDAL path.Blast radius — the first row is the one that matters: a path with no reserved characters encodes to itself, because
percent_encode_pathleaves/alone. Presigned URLs for ordinary object names are byte-identical to what they are today; only names that are currently broken change.What changes are included in this PR?
One call to
percent_encode_pathat each of the three sites, making presign agree withdownload_file_by_nameandupload_file.Tests
No new tests, deliberately: these are
format!arguments and a header value inside an async method whose only seam is a live B2 authorization call, and asserting on them would mean restructuring the presign arms — which I would rather not fold into a fix. The 3 existing unit tests pass;cargo fmt --all -- --checkandcargo clippy -p opendal-service-b2 --all-targetsare clean with zero warnings.Note for rebasing
#7801 renames
build_abs_pathat these same three sites. If it lands first, theX-Bz-File-Nameline needs a one-word rebase.Are there any user-facing changes?
Yes, for
services-b2:presign_read/presign_stat/presign_writeon an object whose name contains a character that is reserved in a URL now address that object, keep their authorization token, and agree with what a non-presigned read or write would do. Presigned URLs for names without reserved characters are unchanged.