feat(mcp): key-value scratchpad tools + database-targeted KV API#185
Merged
Conversation
4f04f5e to
5a27e14
Compare
This was referenced Jul 10, 2026
Merged
StefanSteiner
added a commit
that referenced
this pull request
Jul 10, 2026
…follow-up to #185) (#188) * fix(mcp): map caller-fixable identifier errors to INVALID_ARGUMENT An invalid KV `store`/`key` (a disallowed byte or over the 512-byte limit) surfaced from the hyperdb-api layer as `Error::InvalidName` and fell through the catch-all arm to `INTERNAL_ERROR` — telling an LLM caller "server bug, give up" for something it supplied and can fix. Map `InvalidName` and `InvalidTableDefinition` to `INVALID_ARGUMENT`, which carries a self-correction suggestion; the human-readable message naming the offending byte or length is unchanged. `InvalidOperation` is deliberately left out: it is hyperdb-api caller-API misuse where the caller is this MCP's own Rust code, not the LLM, so it correctly stays `INTERNAL_ERROR`. A regression test locks in that distinction. Caught during the KV MCP smoke run. * docs(mcp): add on-demand KV smoke-test guide Capture the manual, tool-driven smoke tests that exercise the eight `kv_*` tools against a live MCP server: CRUD/upsert, listing and discovery, value fidelity, destructive semantics, input validation (now INVALID_ARGUMENT), read-only mode, database routing/isolation, the LEFT JOIN enrichment pattern, the hidden backing table, and single-process atomicity. Includes a safety section: the persistent database may hold real data, so scratch names are `smoke_`-prefixed and every run ends by verifying the persistent DB is untouched. The existence-check safety rule is scoped to the target database — a bare `describe` inspects only the ephemeral primary and `status` never lists table names, so neither alone protects a persistent write. Cross-linked from DEVELOPMENT.md's Test section. * docs(mcp): surface the KV store across all doc surfaces The key-value scratchpad tools shipped without documentation in the user-facing MCP README, the LLM-facing get_readme, or the workspace READMEs. Add coverage in the right places so an LLM told "remember X" considers kv_set instead of always reaching for CREATE TABLE: - hyperdb-mcp/README.md: new "### Key-Value Store" tool section; a "Table or key-value store?" decision rule in Queryable Memory for AI; the hyper://schema/kv Resources-table row; KV tools in the Features bullet; and the KV mutator/reader split across all three read-only enumerations (flag table, Allowed, Blocked). - src/readme.rs (get_readme): "KV store vs. a custom table" decision subsection; kv_pop lexicographic-order clarification; a describe persistent-DB signpost (status reports counts, not names); a back-link completing the bidirectional cross-link. - src/server.rs: kv_pop and kv_list_stores #[tool(description)] strings clarify lexicographic pop order and the no-store-registry behavior. - README.md (top-level): KvStore / AsyncKvStore Key Features bullet. Also fix a pre-existing false claim on the --read-only flag-table row (carried forward on a line this change edited): Hyper-format export is NOT disabled in read-only mode (export has no writable gate; it's a read-only file copy), matching the Read-Only "Allowed" list and the HyperMcpServer::new doc comment.
Contributor
Author
|
Follow-up: filed #192 with 5 improvements to these KV tools, surfaced by dogfooding them in a real Claude Code session (storing + querying a 70 KB |
StefanSteiner
added a commit
that referenced
this pull request
Jul 11, 2026
…value_path, JSON docs (#193) ## Summary Makes the HyperDB MCP KV store dramatically more ergonomic for LLMs, closing issue #192 — five dogfooding-driven improvements plus two adjacent API gaps they exposed. The throughline: an LLM using the KV store should get a clear signal on every write, batch efficiently, size a store before reading it, load a file's contents without inlining, and never be surprised by a silent clobber. Closes #192. ## Motivation Dogfooding the M2 KV tools (#185) surfaced concrete friction: `kv_set` gave no signal whether a write *created* a key or *overwrote* one (silent data loss risk), there was no batch write (N tool calls for N keys), no way to size a store before pulling it, no way to load a file server-side, and the JSON-query path was under-documented. This PR addresses all five, and reshapes the underlying `hyperdb-api` KV write surface so the "created vs overwritten" signal is available to Rust callers too. ## What's in it ### `hyperdb-api` (public API) — the created/overwritten signal - **BREAKING:** `KvStore::set` / `set_as` / `set_batch` (and the `AsyncKvStore` twins) now return `SetOutcome { created }` / `BatchSetOutcome { created, overwritten }` instead of `Result<()>`. Callers that ignored the result (`set("k","v")?;`, `let _ = set(...)?;`) still compile unchanged; only callers that *named* the unit return must adapt. Under pre-1.0 semver this is the minor slot, driving the 0.7.0 bump. - **Added:** `set_if_absent` / `set_batch_if_absent` guarded writes (`INSERT ... WHERE NOT EXISTS`; returns `BatchGuardOutcome { written, skipped }` for the batch), `byte_size` (`SUM(OCTET_LENGTH(value))`), and `entries` (all `(key, value)` pairs, sorted). Sync and async twins in lockstep. - New public outcome types `SetOutcome`, `BatchSetOutcome`, `BatchGuardOutcome` re-exported from `hyperdb_api`. ### `hyperdb-mcp` (KV tool surface) | Change | Tool | Detail | |---|---|---| | created signal | `kv_set` | returns `{stored, created, value_bytes}`; `overwrite:false` skips clobbers (`{stored:false, existed:true}`) | | load from file | `kv_set` | `value_path:<abs path>` stores a file's contents server-side (exactly one of `value`/`value_path`); files over 64 MiB rejected before reading | | batch write | `kv_set_many` | atomic `entries` array; all keys validated up front; `{stored, created, overwritten, total_bytes}` (`skipped` under `overwrite:false`) | | size reporting | `kv_size` | now returns `{size, bytes}` (total value bytes) | | whole-store read | `kv_list` | `values:true` returns `{entries:[{key,value}]}` — eliminates N×`kv_get` | | docs | `get_readme` | documents JSON `::json`-cast querying, the `::numeric` scale-0 truncation gotcha, and all new params | ## Design notes - The backing table `_hyperdb_kv_store` has no unique constraint, so `set_if_absent` uses a single `INSERT ... SELECT ... WHERE NOT EXISTS` — race- free within a connection/process (the MCP daemon serializes engine access); the rustdoc documents the cross-process caveat honestly. - `kv_set_many` runs inside one transaction: an invalid key aborts the whole batch without writing anything. `total_bytes` counts all submitted values — an upper bound on bytes persisted when keys are skipped or duplicated. - `value_path` reads any path the server process can read (no sandbox — matches the existing `load_file` posture); the 64 MiB cap is checked via `std::fs::metadata` *before* reading, so an oversized file is rejected without allocating. ## Test plan - [x] `cargo fmt --all --check` — clean - [x] `cargo clippy --workspace --all-targets --all-features -- -D warnings` — 0 warnings - [x] `cargo test --workspace --exclude hyperdb-api-node --exclude hyperdb-bootstrap` — **1179 passed / 0 failed** (114 suites), incl. new sync/async KV outcome tests, `kv_set_many`, `value_path` size-cap unit tests, and `kv_list values:true` - [x] `hyperdb-mcp/tests/readme_tests.rs` structural tool-name coverage — green - Post-merge: manual npm smoke test of the published `hyperdb-mcp` exercising the new KV signals. ## Changelog Per-crate `## [Unreleased]` bullets added to `hyperdb-api/CHANGELOG.md` and `hyperdb-mcp/CHANGELOG.md`. Workspace version bump + root CHANGELOG are left to release-please.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a key-value scratchpad to the HyperDB MCP so an LLM can stash
variables, state, summaries, or JSON strings under a
store/keynamespace without creating a table — and promotes the underlying
hyperdb-apiKV API to a public, database-targetable surface to back it.This is Milestone 2 of the KV-store work; Milestone 1 (the
hyperdb-apiKvStoreitself) shipped in #182. Both land together as onefeat:release.What's in it
hyperdb-api(public API)KvStore::kv_store_in(database, name)andConnection::kv_list_stores_in(alias)— open / enumerate a KV store in aspecific database.
kv_store_intakes an unescaped database nameand quotes it internally, so the API is safe by construction (the caller
can't forget to escape). The async client gets the matching
AsyncConnection::kv_store_in.hyperdb-mcp(8 new tools)kv_setkv_get{found, value})kv_delete{deleted})kv_listkv_list_storeskv_sizekv_popkv_cleardatabaseand the store lives in theephemeral primary DB (lost on restart). Pass
database:"persistent"(orpersist:true) to persist across restarts, or any writable attachedalias to target that database.
kv_set's description carries a louddurability warning. Each database keeps its own isolated set of stores.
kv_set,kv_delete,kv_pop,kv_clear) are blocked with a read-only violation when the server runs--read-only; the four readers keep working.hyper://schema/kvresource documents the_hyperdb_kv_storebacking table, its indexless shape, the ephemeral-vs-persistent
durability rule, per-database isolation, and a
LEFT JOINtemplate forenriching analytical tables with KV metadata.
Testing
hyperdb-mcp/tests/kv_tools_tests.rs— 11 end-to-end testsdriving the tools through the real rmcp dispatch path: CRUD lifecycle,
key sorting, delete/pop/clear semantics, database routing + isolation
(persistent vs ephemeral vs attached alias), the read-only guard
(mutators blocked, readers allowed), the ephemeral-only guard
(
database:"persistent"errors, not panics), and persistence across aserver restart.
hyperdb-api/tests/kv_store_in_tests.rs— covers thedatabase-targeted API.
readme_tests.rsextended so all 8 tool names are enforced present inthe LLM-facing README.
cargo clippy --workspace --all-targets --all-features -- -D warningsclean;make docintroduces no newwarnings; real-tooling stdio smoke test confirms the 8 tools appear in
tools/list,hyper://schema/kvis listed and readable, and akv_set→kv_get→kv_listround-trip works against the release binary.Changelog
hyperdb-api/CHANGELOG.md— thekv_store_in/kv_list_stores_inpublic API under
### Added.hyperdb-mcp/CHANGELOG.md— the 8kv_*tools and thehyper://schema/kvresource under### Added.