Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,8 @@ crate-type = ["cdylib"]
# with (ADR 0002), so a revision is the honest way to say which one.
# A local checkout is used instead with a `paths` override in
# `.cargo/config.toml`, which is untracked on purpose.
zudb = { package = "zu", git = "https://github.com/tamnd/zu", rev = "95c7c9909f3a2624515d27eb436da52936016960" }
zu-common = { git = "https://github.com/tamnd/zu", rev = "95c7c9909f3a2624515d27eb436da52936016960" }
zudb = { package = "zu", git = "https://github.com/tamnd/zu", rev = "130f67db924bcd0f766ee814b0da2edae32150d4" }
zu-common = { git = "https://github.com/tamnd/zu", rev = "130f67db924bcd0f766ee814b0da2edae32150d4" }
# N-API by way of napi-rs (ADR 0002). `napi9` is the version of N-API
# this addon declares it needs, which is what makes one binary work
# across Node 24, Node 26, Electron and Bun without a rebuild: the
Expand Down
22 changes: 21 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,9 +53,29 @@ for (const { name } of await conn.query(`MATCH (p:Person) RETURN p.name AS name`

It is the whole engine and not a reduced one: writes, transactions, the appender, registered frames and streams, all of it, on bytes that are not a file. `conn.memory` says which kind you have, since `path` cannot quite answer it on a filesystem that allows a colon in a name. Nothing survives the last connection, which is the point: a test, a script, or five minutes with the language costs no cleanup and leaves no `social.zu1` in a directory somebody has to notice later.

## A second connection, made from the first

`duplicate()` is another connection to the same database, made from a connection rather than from a path. It is how a pool is written.

```ts
import { connect } from "zudb";

await using conn = await connect("social.zu1");
await using other = await conn.duplicate();

const rows = await other.query(`MATCH (p:Person) RETURN p.name AS name`);
console.log(rows.length);
```

It forks off the database the connection already holds rather than opening the file again, so it costs a schema load and no path lookup, and it works on a database in memory, where there is no path to open a second time. That was the gap worth closing: a pool that seeds itself and lets the first connection go had no way to a second one at all.

The two are connections in every sense rather than two names for one. Each has its own prepared statements, its own caches and its own transaction, so a task taking one from a pool is not in whatever transaction the last borrower left open, and closing one does not close the other. What they share is the write side: they queue behind each other to write and each sees what the other has committed. Two of them also read at once, where two statements on one connection queue, which is the other reason to reach for this.

The switches come across, including `bigIntMode` and `temporal`, because a pool handing out connections that answered differently from the one it was seeded with would be a trap. Other clients spell this call `cursor()`, after the way every embedded database has spelled it for thirty years. That name is taken here by `conn.cursor()`, which is a cursor over the rows of one statement and a different thing entirely, so this one says what it does.

## What works today

`connect`, `query`, `exec`, `stream`, `close`, `dispose` and `await using`. Named parameters both ways, including lists, records and nesting. Every scalar the engine has, plus nodes, edges and paths with their tables named rather than numbered, and `ZuDate`, `ZuTime`, `ZuTimestamp` and `ZuDuration`, with `{ temporal: true }` and `toTemporal()` for the runtimes that have `Temporal`. Read-only connections, databases in memory, memory and thread limits. `bigIntMode`, per statement or per connection. An `AbortSignal` on any statement. The full error surface above, and `isZuError` to recognize it. Streaming, as an async iterable, as batches and as a Web Stream. Transactions, with `inTransaction` on the connection. An appender, for loading rows a batch at a time, and `load` for building a whole database out of columns and an edge list. Registered frames, so an Arrow table or an object of typed arrays is something a statement can match on without the rows being copied. `columnar`, for a result read down its columns as the buffers themselves rather than across its rows as objects. Prepared statements, compiled at the line that asked and run as often as wanted, and `explain` and `profile`, as a tree a program walks and as the listing a person reads. Both module formats, typed separately.
`connect`, `query`, `exec`, `stream`, `close`, `dispose` and `await using`. `duplicate`, for a second connection made from the first. Named parameters both ways, including lists, records and nesting. Every scalar the engine has, plus nodes, edges and paths with their tables named rather than numbered, and `ZuDate`, `ZuTime`, `ZuTimestamp` and `ZuDuration`, with `{ temporal: true }` and `toTemporal()` for the runtimes that have `Temporal`. Read-only connections, databases in memory, memory and thread limits. `bigIntMode`, per statement or per connection. An `AbortSignal` on any statement. The full error surface above, and `isZuError` to recognize it. Streaming, as an async iterable, as batches and as a Web Stream. Transactions, with `inTransaction` on the connection. An appender, for loading rows a batch at a time, and `load` for building a whole database out of columns and an edge list. Registered frames, so an Arrow table or an object of typed arrays is something a statement can match on without the rows being copied. `columnar`, for a result read down its columns as the buffers themselves rather than across its rows as objects. Prepared statements, compiled at the line that asked and run as often as wanted, and `explain` and `profile`, as a tree a program walks and as the listing a person reads. Both module formats, typed separately.

Build it with `npm run build`, and run the suite with `npm test`. Nothing is published yet, so `npm i zudb` is not a thing you can type at anybody's terminal, but everything it will do is built and installed on every run of the release workflow.

Expand Down
35 changes: 35 additions & 0 deletions binding.d.cts
Original file line number Diff line number Diff line change
Expand Up @@ -1017,6 +1017,41 @@ export declare class Connection {
* because profiling it would apply the write.
*/
profile(statement: string, params?: Record<string, ZuParam> | null, options?: ZuStatementOptions | null): Promise<ZuProfile>
/**
* Another connection to the same database, made from this one.
*
* This is how a pool is written. `connect()` opens the file again
* and looks the database up by path; this forks off the one this
* connection already holds, which costs a schema load and no
* lookup, and works on a database in memory, where there is no
* path to open a second time.
*
* ```js
* await using other = await conn.duplicate()
* const rows = await other.query('MATCH (p:person) RETURN p.name AS name')
* ```
*
* The two are connections in every sense rather than two names
* for one. Each has its own prepared statements, its own caches
* and its own transaction, so a task taking one from a pool is not
* in whatever transaction the last borrower left open, and closing
* one does not close the other. What they share is the write side:
* they queue behind each other to write and each sees what the
* other has committed, which is what two connections to one file
* have always done.
*
* Other clients call this `cursor()`, after the way every
* embedded database has spelled it for thirty years. That name is
* taken here by [`Connection::cursor`], which is a cursor over the
* rows of one statement and a different thing entirely, so this
* one says what it does.
*
* The switches this connection was opened with come across,
* including how it spells the values it gives back, because a pool
* handing out connections that answered differently from the one it
* was seeded with would be a trap nobody would look for.
*/
duplicate(): Promise<Connection>
/**
* Runs one statement and gives back a cursor over its rows.
*
Expand Down
1 change: 1 addition & 0 deletions etc/zudb.api.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ export class Connection {
columnar(statement: string, params?: Record<string, ZuParam> | null, options?: ZuStatementOptions | null): Promise<ZuColumnar>
cursor(statement: string, params?: Record<string, ZuParam> | null, options?: ZuStreamOptions | null): ZuCursor
dispose(): Promise<void>
duplicate(): Promise<Connection>
exec(statement: string, params?: Record<string, ZuParam> | null, options?: ZuStatementOptions | null): Promise<void>
explain(statement: string): Promise<ZuPlan>
get inTransaction(): boolean
Expand Down
101 changes: 101 additions & 0 deletions src/conn.rs
Original file line number Diff line number Diff line change
Expand Up @@ -365,6 +365,57 @@ fn memory(config: Config) -> std::result::Result<Opened, ZuError> {
})
}

/// Forking a second connection off the database one already holds.
///
/// It takes the connection's lock like any statement, because the fork
/// reads the schema through the write side, and it is a task like any
/// statement for the same reason: a schema load on the runtime's
/// thread is the loop stopped for the length of one.
pub struct DuplicateTask {
inner: Arc<Mutex<Option<zudb::Connection>>>,
alive: Arc<AtomicBool>,
in_txn: Arc<AtomicBool>,
spelling: Spelling,
path: String,
read_only: bool,
memory: bool,
/// Why this is not going to run, when it is not.
refused: Option<String>,
}

impl<'task> ScopedTask<'task> for DuplicateTask {
type Output = std::result::Result<zudb::Connection, Failure>;
type JsValue = ClassInstance<'task, Connection>;

fn compute(&mut self) -> Result<Self::Output> {
if let Some(message) = self.refused.take() {
return Ok(Err(Failure::Usage(message)));
}
Ok(with(&self.inner, &self.alive, &self.in_txn, |conn| {
conn.duplicate().map_err(Failure::from)
}))
}

fn resolve(&mut self, env: &'task Env, output: Self::Output) -> Result<Self::JsValue> {
let made = output.map_err(|failure| failed(env, failure, None))?;
let mut instance = Connection {
interrupt: made.interrupt(),
inner: Arc::new(Mutex::new(Some(made))),
alive: Arc::new(AtomicBool::new(true)),
// Its own, and false: a fork is outside whatever
// transaction the connection it came from is in.
in_txn: Arc::new(AtomicBool::new(false)),
spelling: self.spelling,
path: self.path.clone(),
read_only: self.read_only,
memory: self.memory,
}
.into_instance(env)?;
wire_disposal(env, &mut instance, "dispose")?;
Ok(instance)
}
}

/// Opens or creates, then connects.
///
/// A read-only open of a path that holds nothing fails as an open
Expand Down Expand Up @@ -797,6 +848,56 @@ impl Connection {
))
}

/// Another connection to the same database, made from this one.
///
/// This is how a pool is written. `connect()` opens the file again
/// and looks the database up by path; this forks off the one this
/// connection already holds, which costs a schema load and no
/// lookup, and works on a database in memory, where there is no
/// path to open a second time.
///
/// ```js
/// await using other = await conn.duplicate()
/// const rows = await other.query('MATCH (p:person) RETURN p.name AS name')
/// ```
///
/// The two are connections in every sense rather than two names
/// for one. Each has its own prepared statements, its own caches
/// and its own transaction, so a task taking one from a pool is not
/// in whatever transaction the last borrower left open, and closing
/// one does not close the other. What they share is the write side:
/// they queue behind each other to write and each sees what the
/// other has committed, which is what two connections to one file
/// have always done.
///
/// Other clients call this `cursor()`, after the way every
/// embedded database has spelled it for thirty years. That name is
/// taken here by [`Connection::cursor`], which is a cursor over the
/// rows of one statement and a different thing entirely, so this
/// one says what it does.
///
/// The switches this connection was opened with come across,
/// including how it spells the values it gives back, because a pool
/// handing out connections that answered differently from the one it
/// was seeded with would be a trap nobody would look for.
#[napi(ts_return_type = "Promise<Connection>")]
pub fn duplicate(&self) -> AsyncTask<DuplicateTask> {
let refused = match self.alive.load(Ordering::Acquire) {
true => None,
false => Some(CLOSED.to_string()),
};
AsyncTask::new(DuplicateTask {
inner: Arc::clone(&self.inner),
alive: Arc::clone(&self.alive),
in_txn: Arc::clone(&self.in_txn),
spelling: self.spelling,
path: self.path.clone(),
read_only: self.read_only,
memory: self.memory,
refused,
})
}

/// Runs one statement and gives back a cursor over its rows.
///
/// The pull underneath `stream`, which is what a program uses. The
Expand Down
Loading
Loading