feat: add async pagination support#156
Conversation
Pagination was sync-only: Paginator walks pages over an HttpClient and a consumer must block a thread per page. This adds AsyncPaginator, the non-blocking counterpart that drives the same page-to-page walk over an AsyncHttpClient and returns a CompletableFuture, so no thread parks waiting on a page. AsyncPaginator reuses the existing PaginationStrategy/Page wire-convention seam unchanged, so any strategy written for Paginator works here. It exposes forEachAsync(Consumer) and collectAllAsync(), preserves the maxPages safety cap, and closes each Response after the strategy parses it (including on the exceptional path). The driver is trampolined so a long run of synchronously-completed page futures stays stack-safe. The surface stays transport-agnostic and in sdk-core with no new dependencies (java.util.concurrent only); Reactor/coroutines bridges belong in the adapter modules.
|
This adds a non-blocking IssuesCancelling the returned future is a no-op —
Pagination package docs not updated — |
Pagination was sync-only.
Paginatorwalks pages over anHttpClient, which means a consumer must block a thread per page. This addsAsyncPaginator— the non-blocking counterpart that drives the same page-to-page walk over anAsyncHttpClientand returns aCompletableFuture, so no thread parks waiting on a page.What this adds
org.dexpace.sdk.core.pagination.AsyncPaginator<T>, mirroringPaginator<T>:forEachAsync(Consumer<in T>): CompletableFuture<Void>— walks every item across every page, invoking the consumer per item; the future completes when the walk finishes or completes exceptionally on transport/parse/consumer failure.collectAllAsync(): CompletableFuture<MutableList<T>>— convenience that buffers all items.@JvmOverloadsconstructor shape asPaginator(client, initialRequest, strategy, optionalmaxPages).Design notes
PaginationStrategy/Pagewire-convention types, so any strategy written for the syncPaginator(cursor, page-number,Link-header) works here with no changes.maxPagesstill bounds the number of HTTP exchanges, and eachResponseis closed after the strategy parses it — including on the exceptional path, before the result future fails.thenCompose, so a long run of already-complete pages does not overflow the stack. A 5,000-page synchronous-completion test covers this.sdk-coreand uses onlyjava.util.concurrent(CompletableFuture). Reactor/coroutinesFlow/Fluxbridges remain the responsibility of the adapter modules, per the layering inCLAUDE.md.Tests
A new
AsyncPaginatorTestexercises multi-page walks (both synchronously-completed and completed on a background executor), themaxPagescap, transport failures, strategy-parse failures (asserting the response is still closed), consumer-thrown aborts, response-close counting, empty terminal pages, and deep stack-safety. Backed by a newStubAsyncHttpClientfake that can complete inline or via an executor.Gated build (run locally; root build skipped per repo guidance to avoid R8 + kover aggregate)
All green.
:sdk-core:apiDumpwas run and the regeneratedsdk-core/api/sdk-core.apiis committed.Dependency note
Issue #34 lists #30 (pagination-stack unification) as a blocker. That unification (PR #145) is still open and not on
main, so this targets the currentpaginationsurface (Paginator+PaginationStrategy+Page). If #145 lands,AsyncPaginatorshould be re-pointed at the unified surface; because it only depends onPaginationStrategy/Page, that follow-up is mechanical.Closes #34