fix(services/lakefs): follow the pagination cursor instead of stopping after one page - #8088
Merged
Merged
Conversation
…g after one page The lister decodes LakefsListResponse -- whose Pagination carries has_more and next_offset -- and then sets ctx.done = true unconditionally. Both fields appear in the crate only in their own declarations at core.rs:319 and :321; nothing reads them. So a listing returns the first server page and reports success. Nothing errors, nothing warns; the caller simply receives fewer entries than exist. lakefs does not declare list_with_limit, so no &amount is sent and the server's own default page size applies, and recursive listing is emulated by driving this lister per directory, so every level truncates independently. The cursor is now fed back as after on later pages. The parameter already exists and the lister already sends the caller's start_after on the first page, so this only fills in the pages after it. The encoding belongs in the same change rather than a follow-up: core.rs:147 concatenated after raw, and next_offset is an object path chosen by the server. Driving the cursor is what makes a non-empty after reachable at all, so shipping one without the other would introduce exactly the defect apache#8073 fixed across five services -- a key with a space aborting the request, one with # silently rewinding the page.
erickguan
approved these changes
Aug 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
None — found while reading the service.
Rationale for this change
The lister decodes
LakefsListResponse, whosePaginationcarries the cursor:and then sets
ctx.done = trueunconditionally.has_moreandnext_offsetappear in the crate only in their own declarations (core.rs:319,:321) — nothing reads them.So a listing returns the first server page and reports success. Nothing errors, nothing warns; the caller simply receives fewer entries than exist. lakefs does not declare
list_with_limit, so no&amountis sent and the server's own default page size applies. And since it does not declarelist_with_recursiveeither, a recursive listing is emulated by driving this lister per directory — so every level truncates independently.What changes are included in this PR?
The cursor is fed back as
afteron later pages. The parameter already exists (core.rs:146), and the lister already sends the caller'sstart_afteron the first page with the comment "start after should only be set for the first page" — this fills in the pages after it.Why the encoding is in the same PR
core.rs:147concatenatedafterraw:next_offsetis an object path chosen by the server. Driving the cursor is what makes a non-emptyafterreachable at all — before this change the parameter was only ever set from the caller'sstart_after. Shipping the cursor without the encoding would therefore introduce exactly the defect #8073 fixed across five services: a key with a space aborts the request withinvalid uri character, one with#truncates the parameter and silently rewinds the page, one with&grafts a stray parameter. Splitting them would mean knowingly landing a regression and fixing it afterwards.percent_encode_pathis the right helper here —next_offsetis a path, and it is whatprefixon line 135 already uses.Tests
None. The behaviour is the server's pagination contract, and there is no directory under
.github/servicesfor lakefs, so nothing exercises the lister either way.What I can and cannot claim, stated plainly: that the fields are parsed and never read is verified by grep over the crate; that
afteris the parameter to feednext_offsetback into rests on the two names and onafteralready being this endpoint's resume parameter. I have no lakeFS instance to confirm it against. If you would rather see it exercised first, say so and I will leave it.cargo build,cargo clippy -p opendal-service-lakefs --all-targets(zero warnings) andcargo fmt --all -- --checkare clean.Are there any user-facing changes?
Yes: listing a directory with more entries than one server page returns all of them instead of silently stopping at the first page.