feat(java): let the storage API reach the local filesystem - #960
Open
keksmd wants to merge 2 commits into
Open
Conversation
The storage API merged in apache#958 has no implementation, so nothing can open a GraphAr file yet. This adds the local filesystem adapter that the same issue asked for, keeping the module free of any GraphAr layout, format, or query concern: it resolves a URI to a path, opens a seekable input over it, and writes through a position-reporting output. Directories are created on demand for an output file, an existing target is refused unless the caller asked to replace it, and a read past the end of a file reports the end of the stream rather than a partial buffer. Closes the adapter half of apache#953. Not-tested: only the file scheme is exercised; other schemes are the concern of later adapters.
Review feedback on apache#960: a write from a buffer without a backing array allocated a fresh staging array on every call. The output is single threaded already, because it tracks its own position, so one instance buffer can serve every such write. It is allocated on first use, so an output that only ever sees heap buffers never pays for it. Adds the missing test for a direct buffer larger than one transfer chunk, which the previous test did not reach: with the copy loop reduced to a single pass the new test fails with "expected:<20000> but was:<8192>".
SemyonSinchenko
approved these changes
Aug 21, 2026
Member
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reason for this PR
Closes the remaining half of #953. The storage API merged in #958 defines
Storage,InputFile,OutputFile,SeekableInput, andPositionOutput,but no implementation ships with it, so nothing in the Java tree can open a
GraphAr file yet. Every later layer of #947 (physical IO, Parquet backend,
reader, writer) needs at least one working adapter to be testable.
What changes are included in this PR?
A local filesystem adapter in a new
graphar-storage-localmodule:LocalStorageresolves afile:URI (and a bare path) to ajava.nio.Pathand hands back input and output files.
LocalSeekableInputreads at an absolute position and reports the end of thestream instead of a short buffer.
LocalOutputFile/LocalPositionOutputcreate parent directories on demand,refuse an existing target unless the caller asked to replace it, and report
the number of bytes written so far.
The module depends only on
graphar-storage-api. It carries no GraphAr layout,format, projection, or query concern, per the boundary agreed in #953.
Are these changes tested?
Yes.
LocalStorageTestcovers the round trip, positional reads, end-of-streambehaviour, directory creation, and the refusal to overwrite an existing target.
Are there any user-facing changes?
A new module and package
org.apache.graphar.storage.local. Nothing existingchanges.
Checklist
make cpplintbefore submitting when changed files are in thecppdirectory.pre-commit runbefore commit the changed files.