|
|
Rust bindings for RayforceDB, a high-performance columnar database designed for analytics and data operations. The core is written in pure C with minimal overhead — combining columnar storage with SIMD vectorization for lightning-fast analytics on time-series and big-data workloads.
The bindings call the core's C API directly (no marshalling shim), so reads are
zero-copy where it counts: a numeric column is exposed as a &[T] slice rather than
copied element by element.
Full Documentation: https://rs.rayforcedb.com/
- Fluent API — chainable, intuitive query builders that read like the operation; real operator overloads for arithmetic, methods for comparisons.
- Zero-Copy & High Performance — build a column in a single
memcpy, read it back as a borrow; minimal overhead between Rust and the RayforceDB runtime via the C API. - Type-Safe — a full value model (atoms, vectors, lists, dicts, tables) with
ToValue/FromValueconversions and optionalchronotemporals. - Lightweight — the core is less than a 1 MB footprint.
- Batteries included — CSV & splayed I/O, binary serialization, and a TCP/IPC client.
use rayforce::{col, Runtime, Table, Value};
// One live runtime per process; the scope brackets its whole life.
Runtime::scope(|_rt| {
let quotes = Table::new(
&["symbol", "bid", "ask"],
&[
Value::sym_vec(&["AAPL", "AAPL", "AAPL", "GOOG", "GOOG", "GOOG"]),
Value::vec(&[100.0f64, 101.0, 102.0, 200.0, 201.0, 202.0]),
Value::vec(&[110.0f64, 111.0, 112.0, 210.0, 211.0, 212.0]),
],
)?;
let result = quotes
.select()
.agg("max_bid", col("bid").max())
.agg("min_bid", col("bid").min())
.agg("avg_ask", col("ask").avg())
.agg("count", col("bid").count())
.filter(col("bid").ge(110.0).and(col("ask").gt(100.0)))
.by("symbol")
.execute()?;
println!("{result}");
Ok(())
})?;┌────────┬─────────┬─────────┬─────────┬───────┐
│ symbol │ max_bid │ min_bid │ avg_ask │ count │
│ SYM │ F64 │ F64 │ F64 │ I64 │
├────────┼─────────┼─────────┼─────────┼───────┤
│ GOOG │ 202.0 │ 200.0 │ 211.0 │ 3 │
├────────┴─────────┴─────────┴─────────┴───────┤
│ 1 rows (1 shown) 5 columns (5 shown) │
└──────────────────────────────────────────────┘
The RayforceDB core and the rayforce-q IPC client are C. Both ship inside the
crate, so nothing is fetched at build time — the build script compiles them and statically
links librayforce.a.
# Cargo.toml
[dependencies]
rayforce = { git = "https://github.com/RayforceDB/rayforce-rs" }Requirements: a C toolchain (make, clang) and libclang for bindgen.
The C sources are git submodules addressed over SSH (git@github.com:), and Cargo
fetches a git dependency's submodules itself. Without a GitHub SSH key, rewrite the
URLs to https and make Cargo fetch through git, which honours the rewrite:
git config --global url."https://github.com/".insteadOf "git@github.com:"# ~/.cargo/config.toml
[net]
git-fetch-with-cli = trueA crates.io dependency needs none of this — the sources ship inside the crate.
The C sources live in git submodules under rayforce-sys/vendor/, addressed over SSH,
so a checkout needs them initialized:
git clone --recurse-submodules git@github.com:RayforceDB/rayforce-rs.git
# in an existing clone:
git submodule sync --recursive # picks up a URL change in .gitmodules
git submodule update --init --recursive
cargo build
cargo testWithout a GitHub SSH key, rewrite the submodule URLs to https once with
git config --global url."https://github.com/".insteadOf "git@github.com:".
Each release links one pinned core version. It lives in two places that must agree — the
rayforce-sys/vendor/rayforce submodule, and the CORE_VERSION / CORE_COMMIT constants
in rayforce-sys/build.rs that get stamped into librayforce.a (a crate unpacked from
crates.io has no git history for the core's Makefile to read a version from).
To move the pin, move both:
git -C rayforce-sys/vendor/rayforce fetch --tags
git -C rayforce-sys/vendor/rayforce checkout v2.6.0
git add rayforce-sys/vendor/rayforce
git -C rayforce-sys/vendor/rayforce rev-parse --short=7 HEAD # CORE_COMMIT
$EDITOR rayforce-sys/build.rs # CORE_VERSION, CORE_COMMIT
./scripts/check-vendored-pin.sh # names the mismatch if they disagree
cargo test --workspacerayforce-sys/vendor/rayforce-q works the same way, minus the constants — nothing is
stamped from it.
To build against a core you are changing instead, point the build script at your own checkout. These take precedence over the vendored copies:
export RAYFORCE_SRC=/path/to/rayforce
export RAYFORCE_Q_SRC=/path/to/rayforce-qSuch a checkout is built in place, so incremental state is preserved — except across a
core-flavour switch. Release and debug objects share every filename, so the first build
after RAYFORCE_CORE_DEBUG changes drops every object under src/ and the
librayforce.a beside them, and records the flags in an untracked .stamp. Nothing
tracked by git is touched.
bindgen locates libclang via LIBCLANG_PATH. This is deliberately not set in the
repo's .cargo/config.toml. If bindgen can't auto-detect libclang, set it yourself:
# macOS (CTL):
export LIBCLANG_PATH="/Library/Developer/CommandLineTools/usr/lib"
# Linux:
export LIBCLANG_PATH="$(dirname "$(find /usr/lib -name 'libclang*.so*' | head -1)")"The chrono is on by default for date/time/timestamp conversions.
Built with ❤️ for high-performance data processing | MIT Licensed | RayforceDB