diff --git a/docs/superpowers/specs/2026-07-08-kv-store-design.md b/docs/superpowers/specs/2026-07-08-kv-store-design.md index 83e527b..2b50d1d 100644 --- a/docs/superpowers/specs/2026-07-08-kv-store-design.md +++ b/docs/superpowers/specs/2026-07-08-kv-store-design.md @@ -4,6 +4,21 @@ **Date:** 2026-07-08 **Author:** Stefan Steiner (with Claude) +> **Superseded (2026-07-09):** two design points below were settled differently +> during implementation and are retained here only as historical record — the +> authoritative sources are the M1/M2 plans +> (`docs/superpowers/plans/2026-07-08-kv-store-m1-api.md`, +> `docs/superpowers/plans/2026-07-09-kv-store-m2-mcp.md`): +> +> 1. **No PRIMARY KEY.** The PK-enforcement probe found Hyper *rejects* an +> index-backed `PRIMARY KEY` on this table (`0A000`), so the shipped +> `_hyperdb_kv_store` has **no PK/index** at all; `(store_name, key)` +> uniqueness is enforced entirely by the tool layer's UPDATE-then-INSERT +> upsert. Every "composite PK" reference below is stale. +> 2. **M2 PR title uses `feat:`, not `fix:`.** M1 (retitled `chore:`) and M2 +> bundle into a single `v0.6.0` minor release, so the MCP milestone ships +> under `feat:`. + ## Context The `hyperdb-api` crate is a pure-Rust client for the Hyper database (PostgreSQL @@ -100,6 +115,10 @@ static SQL, no runtime DDL, no `format!`-built table names. ### PRIMARY KEY enforcement — verify empirically +> **Superseded:** the probe found Hyper *rejects* the index-backed `PRIMARY KEY` +> (`0A000`); the shipped table has no PK at all. See the note at the top of this +> doc. The section below is the original, pre-probe reasoning. + The Hyper grammar supports enforced, index-backed `PRIMARY KEY` (default index is an Adaptive Radix Tree; see `hyper/cts/infra/RelationOptions.hpp`). However, a comment in `hyperdb-mcp/src/saved_queries.rs` asserts "Hyper has no indexes" and @@ -364,6 +383,9 @@ Add a bullet under `## [Unreleased]` in `hyperdb-mcp/CHANGELOG.md`. ## Milestones, branches, PR titles +> **Superseded:** M2 ships under **`feat:`**, not `fix:` — M1 and M2 bundle into a +> single `v0.6.0` minor release. See the note at the top of this doc. + | Milestone | Crate | Branch | PR title prefix | |---|---|---|---| | M1 — API | `hyperdb-api` | current branch family | **`feat:`** (the real feature) | diff --git a/hyperdb-api/CHANGELOG.md b/hyperdb-api/CHANGELOG.md index 4bf4043..a201666 100644 --- a/hyperdb-api/CHANGELOG.md +++ b/hyperdb-api/CHANGELOG.md @@ -13,6 +13,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/). `KvStore` / `AsyncKvStore` handles over a fixed `_hyperdb_kv_store` table, with `get`/`set`/`get_as`/`set_as`/`delete`/`exists`/`size`/`keys`/`pop`/`clear`/`set_batch`, plus `kv_list_stores`. Adds the `Error::Serialization` variant. +- `Connection::kv_store_in(database, name)` / `kv_list_stores_in(database)` (plus the + `AsyncConnection` twins) to open and enumerate KV stores in a specific attached + database. The database name is identifier-escaped internally. ## [0.1.1] - 2026-05-13 diff --git a/hyperdb-api/src/async_kv_store.rs b/hyperdb-api/src/async_kv_store.rs index 76f91e5..0724ca4 100644 --- a/hyperdb-api/src/async_kv_store.rs +++ b/hyperdb-api/src/async_kv_store.rs @@ -5,7 +5,7 @@ use crate::async_connection::AsyncConnection; use crate::error::{Error, Result}; -use crate::kv_store::{kv_create_table_sql, validate_kv_name, KV_TABLE}; +use crate::kv_store::{kv_create_table_sql, kv_target_prefix, validate_kv_name, KV_TABLE}; /// A handle to one named key-value store over an [`AsyncConnection`]. /// @@ -56,13 +56,11 @@ impl<'conn> AsyncKvStore<'conn> { /// Async twin of [`KvStore::with_target`](crate::KvStore::with_target). /// - /// `target` is interpolated into SQL — the caller must supply a - /// pre-validated / identifier-escaped, SQL-safe qualifier (M2 must escape - /// it before calling). - #[allow( - dead_code, - reason = "M2 (hyperdb-mcp) consumer; kept here so M1 needs no later API change" - )] + /// Crate-internal low-level constructor behind + /// [`AsyncConnection::kv_store_in`](crate::AsyncConnection::kv_store_in). + /// `target` is interpolated into SQL — the caller must supply a pre-escaped, + /// SQL-safe qualifier (public callers go through `kv_store_in`, which + /// escapes for them). pub(crate) async fn with_target( connection: &'conn AsyncConnection, name: &str, @@ -370,16 +368,45 @@ impl AsyncConnection { AsyncKvStore::new(self, name).await } + /// Async twin of [`Connection::kv_store_in`](crate::Connection::kv_store_in). + /// + /// # Errors + /// + /// See [`Connection::kv_store_in`](crate::Connection::kv_store_in). + pub async fn kv_store_in(&self, database: &str, name: &str) -> Result> { + AsyncKvStore::with_target(self, name, &kv_target_prefix(database)?).await + } + /// Lists the names of every KV store that currently holds at least one key. /// /// # Errors /// /// See [`Connection::kv_list_stores`](crate::Connection::kv_list_stores). pub async fn kv_list_stores(&self) -> Result> { - self.execute_command(&kv_create_table_sql(KV_TABLE)).await?; + self.kv_list_stores_impl(KV_TABLE).await + } + + /// Async twin of + /// [`Connection::kv_list_stores_in`](crate::Connection::kv_list_stores_in). + /// + /// # Errors + /// + /// See [`Connection::kv_list_stores_in`](crate::Connection::kv_list_stores_in). + pub async fn kv_list_stores_in(&self, database: &str) -> Result> { + let table_ref = format!("{}.{KV_TABLE}", kv_target_prefix(database)?); + self.kv_list_stores_impl(&table_ref).await + } + + /// Shared body for [`kv_list_stores`](Self::kv_list_stores) / + /// [`kv_list_stores_in`](Self::kv_list_stores_in): the async twin of the sync + /// `kv_list_stores_impl`. Factored out so the default and location-aware + /// paths cannot drift. + async fn kv_list_stores_impl(&self, table_ref: &str) -> Result> { + self.execute_command(&kv_create_table_sql(table_ref)) + .await?; let mut result = self .execute_query(&format!( - "SELECT DISTINCT store_name FROM {KV_TABLE} ORDER BY store_name ASC" + "SELECT DISTINCT store_name FROM {table_ref} ORDER BY store_name ASC" )) .await?; let mut names = Vec::new(); diff --git a/hyperdb-api/src/kv_store.rs b/hyperdb-api/src/kv_store.rs index 83d60da..f141c04 100644 --- a/hyperdb-api/src/kv_store.rs +++ b/hyperdb-api/src/kv_store.rs @@ -74,6 +74,23 @@ pub(crate) fn kv_create_table_sql(table_ref: &str) -> String { ) } +/// Builds the escaped `""."public"` qualifier prefix for the KV table. +/// +/// Single source of truth for the location shape used by +/// [`Connection::kv_store_in`](crate::Connection::kv_store_in) and +/// [`Connection::kv_list_stores_in`](crate::Connection::kv_list_stores_in) (and +/// their async twins): the KV table always lives in the target database's +/// `public` schema. `database` is identifier-escaped via +/// [`escape_name`](crate::escape_name); the fixed `public` schema name is +/// escaped identically for symmetry. +pub(crate) fn kv_target_prefix(database: &str) -> Result { + Ok(format!( + "{}.{}", + crate::escape_name(database)?, + crate::escape_name("public")? + )) +} + use crate::connection::Connection; /// A handle to one named key-value store, backed by `KV_TABLE`. @@ -120,17 +137,14 @@ impl<'conn> KvStore<'conn> { Self::open(connection, name, KV_TABLE.to_string()) } - /// Opens a handle targeting an explicit, already-escaped table reference. - /// - /// Crate-internal seam for the MCP milestone (routes into an attached - /// database). `target` is interpolated directly into SQL, so the **caller - /// must supply a pre-validated / identifier-escaped, SQL-safe qualifier** - /// (M2 must escape it via the crate's identifier-quoting before calling — - /// `store_name`/`key`/`value` are always bound params, but `target` is not). - #[allow( - dead_code, - reason = "M2 (hyperdb-mcp) consumer; kept here so M1 needs no later API change" - )] + /// Opens a handle targeting an explicit, already-escaped table-qualifier prefix. + /// + /// Crate-internal low-level constructor behind + /// [`Connection::kv_store_in`](crate::Connection::kv_store_in). `target` is + /// interpolated directly into SQL, so the **caller must supply a pre-escaped, + /// SQL-safe qualifier** — public callers go through `kv_store_in`, which + /// escapes for them (`store_name` / `key` / `value` are always bound params, + /// but `target` is not). pub(crate) fn with_target( connection: &'conn Connection, name: &str, @@ -438,6 +452,35 @@ impl Connection { KvStore::new(self, name) } + /// Opens a handle to a KV store in a specific database, rather than the + /// default (search-path) location. + /// + /// `database` is the **unescaped** name of an attached database; the store's + /// backing table is created (if absent) in that database's `public` schema. + /// The name is identifier-escaped internally, so it is safe to pass an + /// arbitrary attachment alias. The store name, keys, and values are always + /// bound parameters. + /// + /// # Examples + /// + /// ```no_run + /// # use hyperdb_api::{Connection, CreateMode, Result}; + /// # fn example(conn: &Connection) -> Result<()> { + /// let kv = conn.kv_store_in("persistent", "settings")?; + /// kv.set("theme", "dark")?; + /// # Ok(()) + /// # } + /// ``` + /// + /// # Errors + /// + /// - [`Error::InvalidName`] if `database` or `name` is invalid. + /// - [`Error::FeatureNotSupported`] on gRPC transport. + /// - [`Error::Server`] if creating the backing table fails. + pub fn kv_store_in(&self, database: &str, name: &str) -> Result> { + KvStore::with_target(self, name, &kv_target_prefix(database)?) + } + /// Lists the names of every KV store that currently holds at least one key. /// /// Creates the backing table first (via `kv_create_table_sql`) so calling @@ -449,9 +492,35 @@ impl Connection { /// - [`Error::FeatureNotSupported`] on gRPC transport. /// - [`Error::Server`] if the query fails. pub fn kv_list_stores(&self) -> Result> { - self.execute_command(&kv_create_table_sql(KV_TABLE))?; + self.kv_list_stores_impl(KV_TABLE) + } + + /// Lists the KV stores that hold at least one key in a specific database. + /// + /// The location-aware companion to [`kv_list_stores`](Self::kv_list_stores): + /// `database` is the unescaped name of an attached database (escaped + /// internally). Creates the backing table in that database's `public` schema + /// first, so an empty database returns `[]` rather than erroring. + /// + /// # Errors + /// + /// - [`Error::InvalidName`] if `database` is invalid. + /// - [`Error::FeatureNotSupported`] on gRPC transport. + /// - [`Error::Server`] if the query fails. + pub fn kv_list_stores_in(&self, database: &str) -> Result> { + let table_ref = format!("{}.{KV_TABLE}", kv_target_prefix(database)?); + self.kv_list_stores_impl(&table_ref) + } + + /// Shared body for [`kv_list_stores`](Self::kv_list_stores) / + /// [`kv_list_stores_in`](Self::kv_list_stores_in): create the backing table + /// at `table_ref` (so a fresh location returns `[]`), then list the distinct + /// store names there. Factored out so the default and location-aware paths + /// cannot drift. + fn kv_list_stores_impl(&self, table_ref: &str) -> Result> { + self.execute_command(&kv_create_table_sql(table_ref))?; let mut result = self.execute_query(&format!( - "SELECT DISTINCT store_name FROM {KV_TABLE} ORDER BY store_name ASC" + "SELECT DISTINCT store_name FROM {table_ref} ORDER BY store_name ASC" ))?; let mut names = Vec::new(); while let Some(chunk) = result.next_chunk()? { diff --git a/hyperdb-api/tests/kv_store_in_tests.rs b/hyperdb-api/tests/kv_store_in_tests.rs new file mode 100644 index 0000000..050aee4 --- /dev/null +++ b/hyperdb-api/tests/kv_store_in_tests.rs @@ -0,0 +1,114 @@ +// Copyright (c) 2026, Salesforce, Inc. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 OR MIT + +//! Integration tests for `Connection::kv_store_in` / `kv_list_stores_in` +//! (targeted KV location), sync and async. + +mod common; + +use common::{test_hyper_params, test_result_path}; +use hyperdb_api::{escape_sql_path, AsyncConnection, Connection, CreateMode, HyperProcess, Result}; + +/// A KV store opened in an attached database is isolated from the primary DB, +/// round-trips set/get through the attached location, and is enumerated by the +/// location-aware store listing. +#[test] +fn kv_store_in_targets_attached_database() -> Result<()> { + let params = test_hyper_params("kv_store_in_sync")?; + let hyper = HyperProcess::new(None, Some(¶ms))?; + + // Primary (ephemeral-style) DB. + let primary = test_result_path("kv_store_in_sync_primary", "hyper")?; + let conn = Connection::new(&hyper, &primary, CreateMode::CreateAndReplace)?; + let primary_stem = primary + .file_stem() + .and_then(|s| s.to_str()) + .expect("primary stem"); + + // Attach a second DB under alias "aux". + let aux_path = test_result_path("kv_store_in_sync_aux", "hyper")?; + let _ = std::fs::remove_file(&aux_path); + let aux_str = aux_path.to_string_lossy(); + conn.execute_command(&format!( + "CREATE DATABASE IF NOT EXISTS {}", + escape_sql_path(&aux_str) + ))?; + conn.execute_command(&format!( + "ATTACH DATABASE {} AS \"aux\"", + escape_sql_path(&aux_str) + ))?; + // ATTACH shifts schema_search_path; re-pin to the primary so the *default* + // (unqualified) KV location keeps resolving to the primary DB. Mirrors + // `hyperdb-mcp`'s engine, which pins search_path at attach time. + conn.execute_command(&format!( + "SET schema_search_path = '{}'", + primary_stem.replace('\'', "''") + ))?; + + // Open a KV store in the attached DB — pass the BARE alias; escaping is internal. + let kv = conn.kv_store_in("aux", "settings")?; + kv.set("theme", "dark")?; + assert_eq!(kv.get("theme")?, Some("dark".to_string())); + + // The default-location store must NOT see the attached-DB value. + let default_kv = conn.kv_store("settings")?; + assert_eq!(default_kv.get("theme")?, None); + + // The location-aware listing sees the attached-DB store; the primary has none. + assert_eq!(conn.kv_list_stores_in("aux")?, vec!["settings".to_string()]); + assert!(conn.kv_list_stores()?.is_empty()); + + Ok(()) +} + +/// Async twin of the above — proves the async `kv_store_in` / `kv_list_stores_in` +/// route to the attached DB and stay isolated from the primary. +#[tokio::test(flavor = "current_thread")] +async fn async_kv_store_in_targets_attached_database() -> Result<()> { + let params = test_hyper_params("kv_store_in_async")?; + let hyper = HyperProcess::new(None, Some(¶ms))?; + let endpoint = hyper.require_endpoint()?.to_string(); + + let primary = test_result_path("kv_store_in_async_primary", "hyper")?; + let conn = AsyncConnection::connect( + &endpoint, + primary.to_str().expect("primary path"), + CreateMode::CreateAndReplace, + ) + .await?; + let primary_stem = primary + .file_stem() + .and_then(|s| s.to_str()) + .expect("primary stem"); + + let aux_path = test_result_path("kv_store_in_async_aux", "hyper")?; + let _ = std::fs::remove_file(&aux_path); + let aux_str = aux_path.to_string_lossy(); + conn.execute_command(&format!( + "CREATE DATABASE IF NOT EXISTS {}", + escape_sql_path(&aux_str) + )) + .await?; + conn.execute_command(&format!( + "ATTACH DATABASE {} AS \"aux\"", + escape_sql_path(&aux_str) + )) + .await?; + conn.execute_command(&format!( + "SET schema_search_path = '{}'", + primary_stem.replace('\'', "''") + )) + .await?; + + let kv = conn.kv_store_in("aux", "settings").await?; + kv.set("theme", "dark").await?; + assert_eq!(kv.get("theme").await?, Some("dark".to_string())); + assert_eq!(conn.kv_store("settings").await?.get("theme").await?, None); + assert_eq!( + conn.kv_list_stores_in("aux").await?, + vec!["settings".to_string()] + ); + assert!(conn.kv_list_stores().await?.is_empty()); + + Ok(()) +} diff --git a/hyperdb-mcp/CHANGELOG.md b/hyperdb-mcp/CHANGELOG.md index 3fa3b12..826729d 100644 --- a/hyperdb-mcp/CHANGELOG.md +++ b/hyperdb-mcp/CHANGELOG.md @@ -7,6 +7,24 @@ and this project adheres to [Semantic Versioning](https://semver.org/). ## [Unreleased] +### Added + +- **Key-value scratchpad tools** — eight new MCP tools (`kv_set`, `kv_get`, + `kv_delete`, `kv_list`, `kv_list_stores`, `kv_size`, `kv_pop`, `kv_clear`) + let an LLM stash variables, state, summaries, or JSON strings under a + `store`/`key` namespace without creating a table. Each takes the same + optional `database`/`persist` routing as the data tools; **stores are + ephemeral by default** (lost on server restart) and persist only when + routed to `"persistent"` (or `persist: true`) or an attached alias. Each + database keeps its own isolated set of stores. The mutating tools + (`kv_set`, `kv_delete`, `kv_pop`, `kv_clear`) are disabled in read-only + mode; the readers (`kv_get`, `kv_list`, `kv_size`, `kv_list_stores`) + always work. +- **`hyper://schema/kv` resource** describing the `_hyperdb_kv_store` + backing table, its indexless shape, the ephemeral-vs-persistent + durability rule, per-database isolation, and the `LEFT JOIN` pattern for + enriching analytical tables with KV metadata. + ### Fixed - **TCP keepalive on the `hyperd` connection.** Connections to `hyperd` now diff --git a/hyperdb-mcp/src/readme.rs b/hyperdb-mcp/src/readme.rs index 8cdde2d..3ec4133 100644 --- a/hyperdb-mcp/src/readme.rs +++ b/hyperdb-mcp/src/readme.rs @@ -164,6 +164,27 @@ watcher targets the alias; call `unwatch_directory` first. - `unwatch_directory` — stop watching a previously registered directory. +### Key-value store (scratchpad) +- `kv_set` — save a variable / state / summary / JSON string under a + store + key (upsert). +- `kv_get` — read a value by store + key. +- `kv_delete` — delete a key. +- `kv_list` — list keys in a store. +- `kv_list_stores` — list store namespaces that hold data in a database. +- `kv_size` — count keys in a store. +- `kv_pop` — destructively read-and-remove the lowest-keyed entry. +- `kv_clear` — delete all keys in a store. + +Every kv_* tool takes the same optional `database` parameter as the data +tools. Omit it and the store lives in the EPHEMERAL database (lost on +restart); pass `\"persistent\"` (or `persist: true`) to persist across +restarts, or any attached alias to target that database. Each database +has its own isolated set of stores. Enrich analytical tables with KV +metadata via LEFT JOIN — always filter `kv.store_name = ''` +to avoid row multiplication, and keep the KV table in the same database +as the joined table. See the `hyper://schema/kv` resource for the join +template. + ### Introspection - `get_readme` — this document. Call once at the start of a session. @@ -177,7 +198,8 @@ watcher targets the alias; call `unwatch_directory` first. - **Read-only mode** (`--read-only` flag on the server) disables: `execute`, all `load_*`, writable `attach_database`, `save_query`, `delete_query`, `set_table_metadata`, `copy_query`, `watch_directory`, - `unwatch_directory`. `query`, `describe`, `sample`, `inspect_file`, + `unwatch_directory`, and the mutating KV tools (`kv_set`, `kv_delete`, + `kv_pop`, `kv_clear`). `query`, `describe`, `sample`, `inspect_file`, `export`, `chart`, `status`, `list_attached_databases`, and `get_readme` always work. - **Table names** in `load_*` and `query_data` / `query_file` accept diff --git a/hyperdb-mcp/src/server.rs b/hyperdb-mcp/src/server.rs index 5e35770..0e5f0e6 100644 --- a/hyperdb-mcp/src/server.rs +++ b/hyperdb-mcp/src/server.rs @@ -67,6 +67,47 @@ const TABLE_SAMPLE_ROWS: u64 = 5; /// more compact wire format and the extra rows help LLMs see patterns. const TABLE_CSV_SAMPLE_ROWS: u64 = 20; +/// Body of the `hyper://schema/kv` resource: describes the `_hyperdb_kv_store` +/// backing table behind the `kv_*` tools, its (indexless) shape, the +/// ephemeral-vs-persistent durability rule, per-database isolation, and the +/// LEFT JOIN enrichment pattern. Served verbatim as `text/plain`. +const KV_SCHEMA_RESOURCE: &str = "\ +KV store backing table (managed by the kv_* tools): + + CREATE TABLE _hyperdb_kv_store ( + store_name TEXT NOT NULL, -- namespace (the `store` tool argument) + key TEXT NOT NULL, -- key within the store + value TEXT -- value (nullable); may hold JSON + ); + +There is NO PRIMARY KEY (Hyper disables indexes); (store_name, key) uniqueness is +enforced by the tool layer's upsert, which is atomic within a single server +process. Two DIFFERENT server processes writing the same key in a shared +persistent database concurrently could still create duplicate rows (there is no +DB-level constraint to catch it). Do not INSERT into this table directly — use +the kv_* tools, which guarantee uniqueness within a session. + +DATABASE / DURABILITY: each database has its own _hyperdb_kv_store table. Every +kv_* tool takes the same optional `database` parameter as the other tools. Omit it +and the store lives in the EPHEMERAL database — convenient, but LOST when the +server restarts. Pass \"persistent\" (or persist=true) to survive restarts, or any +attached alias to target that database. A store in one database is invisible from +another. + +Enrich an analytical table with KV metadata without ALTER TABLE. The KV table must +be in the SAME database as the joined table (or fully qualify both) — a LEFT JOIN +cannot span databases implicitly: + + SELECT t.*, kv.value AS metadata + FROM my_table t + LEFT JOIN _hyperdb_kv_store kv + ON t.id = kv.key AND kv.store_name = 'your_namespace' + WHERE t.status = 'active'; + +ALWAYS include the `kv.store_name = '...'` filter: omitting it fans out any key +present in multiple stores (row multiplication). +"; + // --- Parameter structs --- // Field-level doc comments become JSON Schema `description` fields in the // MCP `tools/list` response, so they are written for the LLM caller. @@ -780,6 +821,72 @@ pub struct SetTableMetadataParams { pub database: Option, } +/// Parameters for the key-scoped KV tools (`kv_get`, `kv_delete`). +#[derive(Debug, Deserialize, JsonSchema)] +pub struct KvKeyParams { + /// Namespace of the KV store (like a named bag of settings). Created on + /// first write; no need to declare it up front. + pub store: String, + /// Key to look up or delete within the store. + pub key: String, + /// Target database alias. Omit (or pass `"local"`) to use the ephemeral + /// primary. Pass `"persistent"` to use the durable database that survives + /// across sessions. Other values target a user-attached database (must be + /// writable). Each database has its own isolated set of KV stores. + pub database: Option, + /// Shorthand for `database: "persistent"`. When true, the store lives in + /// the persistent database. If both `database` and `persist` are set, + /// `database` wins. + pub persist: Option, +} + +/// Parameters for `kv_set` (write a value under store + key). +#[derive(Debug, Deserialize, JsonSchema)] +pub struct KvSetParams { + /// Namespace of the KV store. Created on first write. + pub store: String, + /// Key to write. Overwrites any existing value for this key (upsert). + pub key: String, + /// Value to store. Any string, including a JSON document. + pub value: String, + /// Target database alias. Omit (or pass `"local"`) to write to the + /// ephemeral primary. Pass `"persistent"` to write to the durable database + /// that survives across sessions. Other values target a user-attached + /// database (must be writable). Each database has its own isolated stores. + pub database: Option, + /// Shorthand for `database: "persistent"`. When true, the value is written + /// to the persistent database. If both `database` and `persist` are set, + /// `database` wins. + pub persist: Option, +} + +/// Parameters for store-scoped KV tools (`kv_list`, `kv_size`, `kv_pop`, +/// `kv_clear`). +#[derive(Debug, Deserialize, JsonSchema)] +pub struct KvStoreParams { + /// Namespace of the KV store to operate on. + pub store: String, + /// Target database alias. Omit (or pass `"local"`) for the ephemeral + /// primary. Pass `"persistent"` for the durable database, or a + /// user-attached alias. Each database has its own isolated stores. + pub database: Option, + /// Shorthand for `database: "persistent"`. If both `database` and + /// `persist` are set, `database` wins. + pub persist: Option, +} + +/// Parameters for `kv_list_stores` (enumerate every store in a database). +#[derive(Debug, Deserialize, JsonSchema)] +pub struct KvListStoresParams { + /// Target database alias. Omit (or pass `"local"`) for the ephemeral + /// primary. Pass `"persistent"` for the durable database, or a + /// user-attached alias. Each database has its own isolated stores. + pub database: Option, + /// Shorthand for `database: "persistent"`. If both `database` and + /// `persist` are set, `database` wins. + pub persist: Option, +} + // --- Prompt argument structs --- /// Arguments for the `analyze-table` prompt. @@ -1070,6 +1177,26 @@ impl HyperMcpServer { Ok(Some(resolved)) } + /// Opens a KV store handle on the engine's connection, targeting the + /// resolved `database` (`None` = the ephemeral primary's default location, + /// `Some(alias)` = the persistent DB or an attached alias). + /// + /// `alias` comes straight from [`resolve_db`](Self::resolve_db); it is not + /// escaped here — [`Connection::kv_store_in`](hyperdb_api::Connection::kv_store_in) + /// escapes the database name internally. The returned handle borrows + /// `engine`, so it must be used inside the same `with_engine` closure. + fn kv_open<'e>( + engine: &'e Engine, + database: Option<&str>, + store: &str, + ) -> Result, McpError> { + match database { + Some(alias) => engine.connection().kv_store_in(alias, store), + None => engine.connection().kv_store(store), + } + .map_err(McpError::from) + } + /// Lazily start the Hyper engine on first use, returning a mutex guard /// that holds a reference to the initialized `Engine`. /// @@ -2970,6 +3097,180 @@ impl HyperMcpServer { } } + /// Read a value from the KV scratchpad by store + key. + #[tool( + description = "Read a value from the KV scratchpad by store + key. Returns {found, value}; `value` is null when the key is absent (not an error). Omit `database` to read the ephemeral store; pass \"persistent\" (or persist=true) or an attached alias to read elsewhere." + )] + fn kv_get( + &self, + Parameters(p): Parameters, + ) -> Result { + let result = self.with_engine(|engine| { + let db = self.resolve_db(engine, p.database.as_deref(), p.persist, true)?; + let kv = Self::kv_open(engine, db.as_deref(), &p.store)?; + kv.get(&p.key).map_err(McpError::from) + }); + match result { + Ok(value) => Self::ok_content(json!({ "found": value.is_some(), "value": value })), + Err(e) => Self::err_content(e), + } + } + + /// Save a value under store + key (upsert). Ephemeral unless routed. + #[tool( + description = "KV scratchpad. Save a variable, state, summary, or JSON config under store + key to remember later without creating a database table. Overwrites any existing value (upsert). IMPORTANT: without `database` the value is written to the EPHEMERAL database and is LOST when the server restarts. To persist across restarts, pass database=\"persistent\" (or persist=true)." + )] + fn kv_set( + &self, + Parameters(p): Parameters, + ) -> Result { + if let Err(e) = self.check_writable("kv_set") { + return Self::err_content(e); + } + let result = self.with_engine(|engine| { + let db = self.resolve_db(engine, p.database.as_deref(), p.persist, true)?; + let kv = Self::kv_open(engine, db.as_deref(), &p.store)?; + kv.set(&p.key, &p.value).map_err(McpError::from) + }); + match result { + Ok(()) => Self::ok_content(json!({ "stored": true, "store": p.store, "key": p.key })), + Err(e) => Self::err_content(e), + } + } + + /// Delete a key from the scratchpad. + #[tool( + description = "Delete a key from the KV scratchpad. Returns {deleted: true} when the key existed, {deleted: false} otherwise (no error). Omit `database` for the ephemeral store, or route with \"persistent\"/persist=true/an attached alias." + )] + fn kv_delete( + &self, + Parameters(p): Parameters, + ) -> Result { + if let Err(e) = self.check_writable("kv_delete") { + return Self::err_content(e); + } + let result = self.with_engine(|engine| { + let db = self.resolve_db(engine, p.database.as_deref(), p.persist, true)?; + let kv = Self::kv_open(engine, db.as_deref(), &p.store)?; + kv.delete(&p.key).map_err(McpError::from) + }); + match result { + Ok(deleted) => { + Self::ok_content(json!({ "deleted": deleted, "store": p.store, "key": p.key })) + } + Err(e) => Self::err_content(e), + } + } + + /// List all keys in a scratchpad store, sorted ascending. + #[tool( + description = "List all keys in a KV scratchpad store, sorted ascending. Omit `database` for the ephemeral store, or route with \"persistent\"/persist=true/an attached alias." + )] + fn kv_list( + &self, + Parameters(p): Parameters, + ) -> Result { + let result = self.with_engine(|engine| { + let db = self.resolve_db(engine, p.database.as_deref(), p.persist, true)?; + let kv = Self::kv_open(engine, db.as_deref(), &p.store)?; + kv.keys().map_err(McpError::from) + }); + match result { + Ok(keys) => { + Self::ok_content(json!({ "store": p.store, "count": keys.len(), "keys": keys })) + } + Err(e) => Self::err_content(e), + } + } + + /// List all scratchpad store namespaces that hold data in a database. + #[tool( + description = "List all KV scratchpad store namespaces that currently hold data in a database. Omit `database` for the ephemeral store, or route with \"persistent\"/persist=true/an attached alias. Each database has its own isolated set of stores." + )] + fn kv_list_stores( + &self, + Parameters(p): Parameters, + ) -> Result { + let result = self.with_engine(|engine| { + let db = self.resolve_db(engine, p.database.as_deref(), p.persist, true)?; + match db.as_deref() { + Some(alias) => engine.connection().kv_list_stores_in(alias), + None => engine.connection().kv_list_stores(), + } + .map_err(McpError::from) + }); + match result { + Ok(stores) => Self::ok_content(json!({ "count": stores.len(), "stores": stores })), + Err(e) => Self::err_content(e), + } + } + + /// Count the keys in a scratchpad store. + #[tool( + description = "Return the number of keys in a KV scratchpad store. Omit `database` for the ephemeral store, or route with \"persistent\"/persist=true/an attached alias." + )] + fn kv_size( + &self, + Parameters(p): Parameters, + ) -> Result { + let result = self.with_engine(|engine| { + let db = self.resolve_db(engine, p.database.as_deref(), p.persist, true)?; + let kv = Self::kv_open(engine, db.as_deref(), &p.store)?; + kv.size().map_err(McpError::from) + }); + match result { + Ok(size) => Self::ok_content(json!({ "store": p.store, "size": size })), + Err(e) => Self::err_content(e), + } + } + + /// Destructively read-and-remove the lowest-keyed entry (atomic). + #[tool( + description = "Destructively read-and-remove the lowest-keyed entry from a KV store (peek+delete in one transaction, atomic within a single server process — useful as a work queue for one session; two separate server processes popping a shared persistent store could double-serve an entry). Returns {found, key, value}; {found: false} on an empty store. Omit `database` for the ephemeral store, or route with \"persistent\"/persist=true/an attached alias." + )] + fn kv_pop( + &self, + Parameters(p): Parameters, + ) -> Result { + if let Err(e) = self.check_writable("kv_pop") { + return Self::err_content(e); + } + let result = self.with_engine(|engine| { + let db = self.resolve_db(engine, p.database.as_deref(), p.persist, true)?; + let kv = Self::kv_open(engine, db.as_deref(), &p.store)?; + kv.pop().map_err(McpError::from) + }); + match result { + Ok(Some((key, value))) => { + Self::ok_content(json!({ "found": true, "key": key, "value": value })) + } + Ok(None) => Self::ok_content(json!({ "found": false })), + Err(e) => Self::err_content(e), + } + } + + /// Delete all keys in a scratchpad store. + #[tool( + description = "Delete all keys in a KV scratchpad store. Returns the number of keys removed. Omit `database` for the ephemeral store, or route with \"persistent\"/persist=true/an attached alias." + )] + fn kv_clear( + &self, + Parameters(p): Parameters, + ) -> Result { + if let Err(e) = self.check_writable("kv_clear") { + return Self::err_content(e); + } + let result = self.with_engine(|engine| { + let db = self.resolve_db(engine, p.database.as_deref(), p.persist, true)?; + let kv = Self::kv_open(engine, db.as_deref(), &p.store)?; + kv.clear().map_err(McpError::from) + }); + match result { + Ok(removed) => Self::ok_content(json!({ "store": p.store, "removed": removed })), + Err(e) => Self::err_content(e), + } + } + /// Returns plugin health, workspace info, table count, total rows, disk /// usage, the backing `hyperd` connection (mode, endpoint, daemon health /// port), and the list of active directory watchers with their stats. @@ -3610,6 +3911,12 @@ impl HyperMcpServer { if uri == "hyper://readme" { return self.build_readme_body().map(Some); } + if uri == "hyper://schema/kv" { + return Ok(Some(ResourceBody::Text { + mime_type: "text/plain".into(), + content: KV_SCHEMA_RESOURCE.to_string(), + })); + } if let Some(name) = uri .strip_prefix("hyper://tables/") .and_then(|rest| rest.strip_suffix("/schema")) @@ -3720,14 +4027,16 @@ impl HyperMcpServer { /// `RequestContext`. Factored out of [`Self::list_resources`] for tests. /// /// Returns one URI for the workspace, one for the full tables list, one - /// for the workspace readme, three per existing table (schema, sample, - /// csv-sample), and two per saved query (definition, result). + /// for the workspace readme, one for the KV store schema, three per + /// existing table (schema, sample, csv-sample), and two per saved query + /// (definition, result). #[must_use] pub fn list_resource_uris(&self) -> Vec { let mut uris = vec![ "hyper://workspace".to_string(), "hyper://tables".to_string(), "hyper://readme".to_string(), + "hyper://schema/kv".to_string(), ]; if let Ok(tables) = self.with_engine(super::engine::Engine::describe_tables) { // `describe_tables` already filters out `_hyperdb_*` meta- @@ -4103,10 +4412,10 @@ Full SQL reference: https://developer.salesforce.com/docs/data/data-cloud-query- } /// List MCP resources: the workspace, the tables list, a markdown - /// readme, and three entries per existing table (schema, JSON sample, - /// CSV sample). Calling this lazily starts the engine, so it doubles - /// as a "wake up" signal for MCP clients that pre-fetch resources at - /// connection time. + /// readme, the KV store schema, and three entries per existing table + /// (schema, JSON sample, CSV sample). Calling this lazily starts the + /// engine, so it doubles as a "wake up" signal for MCP clients that + /// pre-fetch resources at connection time. async fn list_resources( &self, _request: Option, @@ -4150,6 +4459,22 @@ Full SQL reference: https://developer.salesforce.com/docs/data/data-cloud-query- meta: None, } .no_annotation(), + RawResource { + uri: "hyper://schema/kv".into(), + name: "KV store schema".into(), + title: Some("Key-value scratchpad schema".into()), + description: Some( + "Schema of the _hyperdb_kv_store table backing the kv_* tools, the \ + ephemeral-vs-persistent durability rule, and the LEFT JOIN enrichment \ + pattern for joining KV metadata onto analytical tables." + .into(), + ), + mime_type: Some("text/plain".into()), + size: None, + icons: None, + meta: None, + } + .no_annotation(), ]; if let Ok(tables) = self.with_engine(super::engine::Engine::describe_tables) { diff --git a/hyperdb-mcp/tests/kv_tools_tests.rs b/hyperdb-mcp/tests/kv_tools_tests.rs new file mode 100644 index 0000000..16cb8f9 --- /dev/null +++ b/hyperdb-mcp/tests/kv_tools_tests.rs @@ -0,0 +1,624 @@ +// Copyright (c) 2026, Salesforce, Inc. All rights reserved. +// SPDX-License-Identifier: Apache-2.0 OR MIT + +//! End-to-end coverage for the `kv_*` scratchpad tools. +//! +//! Like [`end_to_end_mcp_tests`], these spin up a `HyperMcpServer` and a +//! minimal client on opposite halves of an in-memory `tokio::io::duplex` +//! pair, then invoke the tools through the real rmcp dispatch path +//! (params deserialization → handler → `CallToolResult`). This exercises +//! the handlers exactly as an MCP client would, including the +//! `database`/`persist` routing, the read-only guard, and durability. + +use rmcp::model::{CallToolRequestParams, CallToolResult, ClientInfo}; +use rmcp::service::{RoleClient, RunningService}; +use rmcp::{ClientHandler, ServiceExt}; +use std::path::PathBuf; +use std::sync::Arc; +use tempfile::TempDir; + +use hyperdb_mcp::server::HyperMcpServer; + +type TestResult = Result<(), Box>; + +/// Minimal client handler — only exists to satisfy `ServiceExt`. +#[derive(Debug, Clone)] +struct DummyClientHandler; + +impl ClientHandler for DummyClientHandler { + fn get_info(&self) -> ClientInfo { + ClientInfo::default() + } +} + +/// In-memory client+server pair backed by a `tokio::io::duplex`. +struct TestHarness { + client: RunningService, + server_handle: tokio::task::JoinHandle>>, + /// Held so the workspace temp dir outlives the server. The persistence + /// test drives the reopen path via [`start_at`](TestHarness::start_at) + /// with an explicit path instead of reading this back. + _temp_dir: Arc, +} + +impl TestHarness { + /// Spin up a server with a fresh persistent workspace + an in-memory + /// client. `read_only=false` is the typical case; `ephemeral_only=true` + /// skips the persistent attachment. + async fn start( + read_only: bool, + ephemeral_only: bool, + ) -> Result> { + let temp_dir = Arc::new(TempDir::new()?); + let persistent_path = temp_dir.path().join("workspace.hyper"); + Self::start_at(read_only, ephemeral_only, persistent_path, temp_dir).await + } + + /// Like [`start`](Self::start) but reuses a caller-provided workspace + /// path + temp dir, so a second server can reopen the same on-disk file + /// (used by the persistence-across-restart test). + async fn start_at( + read_only: bool, + ephemeral_only: bool, + persistent_path: PathBuf, + temp_dir: Arc, + ) -> Result> { + let (server_io, client_io) = tokio::io::duplex(64 * 1024); + + let workspace = if ephemeral_only { + None + } else { + Some(persistent_path.to_string_lossy().to_string()) + }; + let server = HyperMcpServer::with_no_daemon(workspace, read_only, true); + + let server_handle = tokio::spawn(async move { + let running = server + .serve(server_io) + .await + .map_err(|e| -> Box { Box::new(e) })?; + running + .waiting() + .await + .map_err(|e| -> Box { Box::new(e) })?; + Ok(()) + }); + + let client = DummyClientHandler + .serve(client_io) + .await + .map_err(|e| -> Box { Box::new(e) })?; + + Ok(Self { + client, + server_handle, + _temp_dir: temp_dir, + }) + } + + async fn shutdown(self) -> TestResult { + self.client + .cancel() + .await + .map_err(|e| -> Box { Box::new(e) })?; + self.server_handle.await??; + Ok(()) + } +} + +/// Invoke a tool by name, building request params from a JSON object. +async fn call_tool( + client: &RunningService, + name: &'static str, + args: serde_json::Value, +) -> Result> { + let arguments = args.as_object().cloned(); + let params = match arguments { + Some(args) => CallToolRequestParams::new(name).with_arguments(args), + None => CallToolRequestParams::new(name), + }; + let result = client + .call_tool(params) + .await + .map_err(|e| -> Box { Box::new(e) })?; + Ok(result) +} + +/// First text-content block of a tool result (used for error messages). +fn first_text(result: &CallToolResult) -> Option { + result + .content + .first() + .and_then(|c| c.raw.as_text()) + .map(|t| t.text.clone()) +} + +/// Did the tool return an `is_error: true` result? +fn is_error(result: &CallToolResult) -> bool { + result.is_error.unwrap_or(false) +} + +/// The tool's JSON payload as a `serde_json::Value`. Every `kv_*` handler +/// returns via `ok_content`, which serializes the body into the single +/// text content block (pretty-printed) as well as `structuredContent`. +/// Parsing the text block keeps this robust across rmcp versions (the +/// `structured_content` field type varies) and matches how the sibling +/// e2e tests read tool output. +fn structured(result: &CallToolResult) -> serde_json::Value { + let text = first_text(result).unwrap_or_default(); + serde_json::from_str(&text).unwrap_or(serde_json::Value::Null) +} + +// ===================================================================== +// Core CRUD lifecycle (default / ephemeral database). +// ===================================================================== + +/// set → get round-trips; get on an absent key returns `{found:false, +/// value:null}` (not an error); overwrite returns the new value. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn kv_set_get_roundtrip_and_overwrite() -> TestResult { + let h = TestHarness::start(false, false).await?; + + // Absent key first — must be a clean miss, not an error. + let miss = call_tool( + &h.client, + "kv_get", + serde_json::json!({ "store": "cfg", "key": "theme" }), + ) + .await?; + assert!( + !is_error(&miss), + "kv_get miss must not error: {:?}", + first_text(&miss) + ); + assert_eq!(structured(&miss)["found"], serde_json::json!(false)); + assert_eq!(structured(&miss)["value"], serde_json::Value::Null); + + // Set, then read back. + let set = call_tool( + &h.client, + "kv_set", + serde_json::json!({ "store": "cfg", "key": "theme", "value": "dark" }), + ) + .await?; + assert!(!is_error(&set), "kv_set failed: {:?}", first_text(&set)); + assert_eq!(structured(&set)["stored"], serde_json::json!(true)); + + let got = call_tool( + &h.client, + "kv_get", + serde_json::json!({ "store": "cfg", "key": "theme" }), + ) + .await?; + assert_eq!(structured(&got)["found"], serde_json::json!(true)); + assert_eq!(structured(&got)["value"], serde_json::json!("dark")); + + // Overwrite (upsert) → new value. + call_tool( + &h.client, + "kv_set", + serde_json::json!({ "store": "cfg", "key": "theme", "value": "light" }), + ) + .await?; + let got2 = call_tool( + &h.client, + "kv_get", + serde_json::json!({ "store": "cfg", "key": "theme" }), + ) + .await?; + assert_eq!(structured(&got2)["value"], serde_json::json!("light")); + + h.shutdown().await +} + +/// list returns keys sorted ascending; size counts them; list_stores +/// includes the store namespace once it holds data. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn kv_list_size_and_list_stores() -> TestResult { + let h = TestHarness::start(false, false).await?; + + for (k, v) in [("gamma", "3"), ("alpha", "1"), ("beta", "2")] { + call_tool( + &h.client, + "kv_set", + serde_json::json!({ "store": "s", "key": k, "value": v }), + ) + .await?; + } + + let list = call_tool(&h.client, "kv_list", serde_json::json!({ "store": "s" })).await?; + assert_eq!(structured(&list)["count"], serde_json::json!(3)); + assert_eq!( + structured(&list)["keys"], + serde_json::json!(["alpha", "beta", "gamma"]), + "keys must be sorted ascending" + ); + + let size = call_tool(&h.client, "kv_size", serde_json::json!({ "store": "s" })).await?; + assert_eq!(structured(&size)["size"], serde_json::json!(3)); + + let stores = call_tool(&h.client, "kv_list_stores", serde_json::json!({})).await?; + let names = structured(&stores)["stores"].clone(); + assert!( + names.as_array().is_some_and(|a| a.iter().any(|n| n == "s")), + "list_stores must include 's'; got: {names}" + ); + + h.shutdown().await +} + +/// delete returns `{deleted:true}` when the key existed and +/// `{deleted:false}` on a second delete (idempotent, not an error). +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn kv_delete_reports_whether_key_existed() -> TestResult { + let h = TestHarness::start(false, false).await?; + + call_tool( + &h.client, + "kv_set", + serde_json::json!({ "store": "s", "key": "k", "value": "v" }), + ) + .await?; + + let first = call_tool( + &h.client, + "kv_delete", + serde_json::json!({ "store": "s", "key": "k" }), + ) + .await?; + assert_eq!(structured(&first)["deleted"], serde_json::json!(true)); + + let second = call_tool( + &h.client, + "kv_delete", + serde_json::json!({ "store": "s", "key": "k" }), + ) + .await?; + assert!(!is_error(&second), "second delete must not error"); + assert_eq!(structured(&second)["deleted"], serde_json::json!(false)); + + h.shutdown().await +} + +/// pop returns the lowest-keyed entry and removes it; on an empty store +/// it returns `{found:false}`. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn kv_pop_removes_lowest_key_then_empty() -> TestResult { + let h = TestHarness::start(false, false).await?; + + for (k, v) in [("b", "2"), ("a", "1")] { + call_tool( + &h.client, + "kv_set", + serde_json::json!({ "store": "q", "key": k, "value": v }), + ) + .await?; + } + + let pop1 = call_tool(&h.client, "kv_pop", serde_json::json!({ "store": "q" })).await?; + assert_eq!(structured(&pop1)["found"], serde_json::json!(true)); + assert_eq!( + structured(&pop1)["key"], + serde_json::json!("a"), + "lowest key first" + ); + assert_eq!(structured(&pop1)["value"], serde_json::json!("1")); + + let pop2 = call_tool(&h.client, "kv_pop", serde_json::json!({ "store": "q" })).await?; + assert_eq!(structured(&pop2)["key"], serde_json::json!("b")); + + // Store now empty. + let pop3 = call_tool(&h.client, "kv_pop", serde_json::json!({ "store": "q" })).await?; + assert_eq!(structured(&pop3)["found"], serde_json::json!(false)); + + h.shutdown().await +} + +/// clear returns the number of keys removed; the store is empty afterward. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn kv_clear_empties_the_store() -> TestResult { + let h = TestHarness::start(false, false).await?; + + for k in ["x", "y", "z"] { + call_tool( + &h.client, + "kv_set", + serde_json::json!({ "store": "s", "key": k, "value": "1" }), + ) + .await?; + } + + let cleared = call_tool(&h.client, "kv_clear", serde_json::json!({ "store": "s" })).await?; + assert_eq!(structured(&cleared)["removed"], serde_json::json!(3)); + + let size = call_tool(&h.client, "kv_size", serde_json::json!({ "store": "s" })).await?; + assert_eq!(structured(&size)["size"], serde_json::json!(0)); + + h.shutdown().await +} + +// ===================================================================== +// Database routing + isolation. +// ===================================================================== + +/// A value written to the persistent database is invisible from the +/// default (ephemeral) database, and vice-versa. `kv_list_stores` routes +/// per-database too (proves `kv_list_stores_in`). +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn kv_database_routing_isolates_stores() -> TestResult { + let h = TestHarness::start(false, false).await?; + + // Write into persistent. + let set = call_tool( + &h.client, + "kv_set", + serde_json::json!({ "store": "s", "key": "k", "value": "persisted", "database": "persistent" }), + ) + .await?; + assert!( + !is_error(&set), + "kv_set to persistent failed: {:?}", + first_text(&set) + ); + + // Visible when reading persistent. + let got_persist = call_tool( + &h.client, + "kv_get", + serde_json::json!({ "store": "s", "key": "k", "database": "persistent" }), + ) + .await?; + assert_eq!( + structured(&got_persist)["value"], + serde_json::json!("persisted") + ); + + // NOT visible from the default (ephemeral) database. + let got_default = call_tool( + &h.client, + "kv_get", + serde_json::json!({ "store": "s", "key": "k" }), + ) + .await?; + assert_eq!( + structured(&got_default)["found"], + serde_json::json!(false), + "ephemeral DB must not see the persistent store's value" + ); + + // list_stores is per-database: persistent has 's', default has none. + let stores_persist = call_tool( + &h.client, + "kv_list_stores", + serde_json::json!({ "database": "persistent" }), + ) + .await?; + assert!( + structured(&stores_persist)["stores"] + .as_array() + .is_some_and(|a| a.iter().any(|n| n == "s")), + "persistent list_stores must include 's'" + ); + + let stores_default = call_tool(&h.client, "kv_list_stores", serde_json::json!({})).await?; + assert_eq!( + structured(&stores_default)["count"], + serde_json::json!(0), + "default DB must have no stores" + ); + + h.shutdown().await +} + +/// `persist:true` routes to the persistent database, equivalent to +/// `database:"persistent"`. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn kv_persist_flag_routes_to_persistent() -> TestResult { + let h = TestHarness::start(false, false).await?; + + call_tool( + &h.client, + "kv_set", + serde_json::json!({ "store": "s", "key": "k", "value": "v", "persist": true }), + ) + .await?; + + // Readable via database:"persistent" (proving persist:true == persistent). + let got = call_tool( + &h.client, + "kv_get", + serde_json::json!({ "store": "s", "key": "k", "database": "persistent" }), + ) + .await?; + assert_eq!(structured(&got)["value"], serde_json::json!("v")); + + h.shutdown().await +} + +/// A store written into a writable *attached* database is reachable via +/// that alias and invisible from the default database. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn kv_routes_to_attached_database() -> TestResult { + let h = TestHarness::start(false, false).await?; + + // Attach a fresh writable .hyper under alias "aux". + let aux_dir = TempDir::new()?; + let aux_path = aux_dir.path().join("aux.hyper"); + let attach = call_tool( + &h.client, + "attach_database", + serde_json::json!({ + "alias": "aux", + "kind": "local_file", + "path": aux_path.to_string_lossy(), + "writable": true, + "on_missing": "create", + }), + ) + .await?; + assert!( + !is_error(&attach), + "attach failed: {:?}", + first_text(&attach) + ); + + // Write into the attached DB and read it back via the same alias. + call_tool( + &h.client, + "kv_set", + serde_json::json!({ "store": "s", "key": "k", "value": "in_aux", "database": "aux" }), + ) + .await?; + let got_aux = call_tool( + &h.client, + "kv_get", + serde_json::json!({ "store": "s", "key": "k", "database": "aux" }), + ) + .await?; + assert_eq!(structured(&got_aux)["value"], serde_json::json!("in_aux")); + + // Invisible from the default database. + let got_default = call_tool( + &h.client, + "kv_get", + serde_json::json!({ "store": "s", "key": "k" }), + ) + .await?; + assert_eq!(structured(&got_default)["found"], serde_json::json!(false)); + + h.shutdown().await +} + +// ===================================================================== +// Guard rails: ephemeral-only + read-only server. +// ===================================================================== + +/// On an ephemeral-only server, `kv_set` with `database:"persistent"` +/// returns an error content (not a panic) naming the cause. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn kv_set_persistent_on_ephemeral_only_errors() -> TestResult { + let h = TestHarness::start(false, true).await?; + + let result = call_tool( + &h.client, + "kv_set", + serde_json::json!({ "store": "s", "key": "k", "value": "v", "database": "persistent" }), + ) + .await?; + assert!( + is_error(&result), + "must reject persistent in ephemeral-only mode" + ); + let msg = first_text(&result).unwrap_or_default(); + assert!( + msg.contains("ephemeral-only") || msg.contains("persistent"), + "error must name the cause; got: {msg}" + ); + + h.shutdown().await +} + +/// On a `--read-only` server the mutating KV tools are blocked by +/// `check_writable`, while the readers still work against the writable +/// target (they issue only `CREATE TABLE IF NOT EXISTS` + SELECT). This +/// settles the create-on-open question empirically: a reader opening a +/// store in a read-only *server* against a writable engine target +/// succeeds, so readers are intentionally NOT gated by `check_writable`. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn kv_read_only_server_blocks_mutators_allows_readers() -> TestResult { + let h = TestHarness::start(true, false).await?; + + // Mutators are blocked with a read-only violation. + for (tool, args) in [ + ( + "kv_set", + serde_json::json!({ "store": "s", "key": "k", "value": "v" }), + ), + ("kv_delete", serde_json::json!({ "store": "s", "key": "k" })), + ("kv_pop", serde_json::json!({ "store": "s" })), + ("kv_clear", serde_json::json!({ "store": "s" })), + ] { + let r = call_tool(&h.client, tool, args).await?; + assert!(is_error(&r), "{tool} must be blocked in read-only mode"); + let msg = first_text(&r).unwrap_or_default(); + assert!( + msg.contains("read-only"), + "{tool} error must mention read-only mode; got: {msg}" + ); + } + + // Readers succeed (create-on-open is a no-op CREATE IF NOT EXISTS that + // the writable engine target accepts even under a read-only server). + let got = call_tool( + &h.client, + "kv_get", + serde_json::json!({ "store": "s", "key": "missing" }), + ) + .await?; + assert!( + !is_error(&got), + "kv_get must work on a read-only server: {:?}", + first_text(&got) + ); + assert_eq!(structured(&got)["found"], serde_json::json!(false)); + + let list = call_tool(&h.client, "kv_list", serde_json::json!({ "store": "s" })).await?; + assert!( + !is_error(&list), + "kv_list must work: {:?}", + first_text(&list) + ); + assert_eq!(structured(&list)["count"], serde_json::json!(0)); + + let size = call_tool(&h.client, "kv_size", serde_json::json!({ "store": "s" })).await?; + assert!( + !is_error(&size), + "kv_size must work: {:?}", + first_text(&size) + ); + assert_eq!(structured(&size)["size"], serde_json::json!(0)); + + h.shutdown().await +} + +// ===================================================================== +// Durability: persistent values survive a server restart. +// ===================================================================== + +/// A value written with `database:"persistent"` survives dropping the +/// server and reopening a fresh one against the same workspace file. +#[tokio::test(flavor = "multi_thread", worker_threads = 2)] +async fn kv_persistent_value_survives_server_restart() -> TestResult { + let temp_dir = Arc::new(TempDir::new()?); + let workspace = temp_dir.path().join("workspace.hyper"); + + // First server: write into persistent, then shut down cleanly. + { + let h = + TestHarness::start_at(false, false, workspace.clone(), Arc::clone(&temp_dir)).await?; + let set = call_tool( + &h.client, + "kv_set", + serde_json::json!({ "store": "s", "key": "k", "value": "durable", "database": "persistent" }), + ) + .await?; + assert!(!is_error(&set), "kv_set failed: {:?}", first_text(&set)); + h.shutdown().await?; + } + + // Second server on the same workspace path: the value is still there. + let h2 = TestHarness::start_at(false, false, workspace, Arc::clone(&temp_dir)).await?; + let got = call_tool( + &h2.client, + "kv_get", + serde_json::json!({ "store": "s", "key": "k", "database": "persistent" }), + ) + .await?; + assert_eq!( + structured(&got)["value"], + serde_json::json!("durable"), + "persistent value must survive a server restart" + ); + + h2.shutdown().await +} diff --git a/hyperdb-mcp/tests/readme_tests.rs b/hyperdb-mcp/tests/readme_tests.rs index daf088f..a21d7bc 100644 --- a/hyperdb-mcp/tests/readme_tests.rs +++ b/hyperdb-mcp/tests/readme_tests.rs @@ -53,6 +53,14 @@ fn readme_mentions_every_tool() { "list_attached_databases", "watch_directory", "unwatch_directory", + "kv_get", + "kv_set", + "kv_delete", + "kv_list", + "kv_list_stores", + "kv_size", + "kv_pop", + "kv_clear", "get_readme", ]; for tool in tools {