Add PedraDB SnapshotStore adapter and compare in backend benches - #19
Draft
jaredLunde wants to merge 3 commits into
Draft
Add PedraDB SnapshotStore adapter and compare in backend benches#19jaredLunde wants to merge 3 commits into
jaredLunde wants to merge 3 commits into
Conversation
Wire a pedradb feature (rocksdb-compat git dep) that implements the same on-disk fold contract as fjall/rocksdb, extend the conformance suite, and compare all three engines in snapshot_backends. Co-authored-by: Jared Lunde <jared.lunde@gmail.com>
Depend on rocksdb-compat directly (import as rocksdb_compat) instead of renaming the package to pedradb. Co-authored-by: Jared Lunde <jared.lunde@gmail.com>
Pin rocksdb-compat to latest Pedra (5b6789c), run backends one-at-a-time when entries ≥ 50M so a ~250 GiB disk can hold a 1B fold, and add progress logging / settle headroom checks / backend selection env knobs. Co-authored-by: Jared Lunde <jared.lunde@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an opt-in
pedradbbackend (rocksdb-compat@5b6789c) and extendssnapshot_backendsso fjall / RocksDB / Pedra compare on the same route-fold workload.500M results (unsettled)
Caveats: settle skipped for all three (disk headroom). 1B attempted, aborted (~420+ GiB needed on a 254 GiB disk).
Bottom line: Pedra wins point-get latency; Rocks wins hydrate; fjall wins space / miss reject / hit tails.
Changes
rocksdb-compatpinned to5b6789cTest plan