Skip to content

feat(kv)!: #192 KV MCP LLM ergonomics — created signal, batch, size, value_path, JSON docs#193

Merged
StefanSteiner merged 20 commits into
tableau:mainfrom
StefanSteiner:feat/kv-mcp-llm-ergonomics
Jul 11, 2026
Merged

feat(kv)!: #192 KV MCP LLM ergonomics — created signal, batch, size, value_path, JSON docs#193
StefanSteiner merged 20 commits into
tableau:mainfrom
StefanSteiner:feat/kv-mcp-llm-ergonomics

Conversation

@StefanSteiner

Copy link
Copy Markdown
Contributor

Summary

Makes the HyperDB MCP KV store dramatically more ergonomic for LLMs, closing
issue #192 — five dogfooding-driven improvements plus two adjacent API gaps
they exposed. The throughline: an LLM using the KV store should get a clear
signal on every write, batch efficiently, size a store before reading it, load
a file's contents without inlining, and never be surprised by a silent
clobber.

Closes #192.

Motivation

Dogfooding the M2 KV tools (#185) surfaced concrete friction: kv_set gave no
signal whether a write created a key or overwrote one (silent data loss
risk), there was no batch write (N tool calls for N keys), no way to size a
store before pulling it, no way to load a file server-side, and the JSON-query
path was under-documented. This PR addresses all five, and reshapes the
underlying hyperdb-api KV write surface so the "created vs overwritten"
signal is available to Rust callers too.

What's in it

hyperdb-api (public API) — the created/overwritten signal

  • BREAKING: KvStore::set / set_as / set_batch (and the AsyncKvStore
    twins) now return SetOutcome { created } / BatchSetOutcome { created, overwritten } instead of Result<()>. Callers that ignored the result
    (set("k","v")?;, let _ = set(...)?;) still compile unchanged; only
    callers that named the unit return must adapt. Under pre-1.0 semver this
    is the minor slot, driving the 0.7.0 bump.
  • Added: set_if_absent / set_batch_if_absent guarded writes
    (INSERT ... WHERE NOT EXISTS; returns BatchGuardOutcome { written, skipped } for the batch), byte_size (SUM(OCTET_LENGTH(value))), and
    entries (all (key, value) pairs, sorted). Sync and async twins in
    lockstep.
  • New public outcome types SetOutcome, BatchSetOutcome, BatchGuardOutcome
    re-exported from hyperdb_api.

hyperdb-mcp (KV tool surface)

Change Tool Detail
created signal kv_set returns {stored, created, value_bytes}; overwrite:false skips clobbers ({stored:false, existed:true})
load from file kv_set value_path:<abs path> stores a file's contents server-side (exactly one of value/value_path); files over 64 MiB rejected before reading
batch write kv_set_many atomic entries array; all keys validated up front; {stored, created, overwritten, total_bytes} (skipped under overwrite:false)
size reporting kv_size now returns {size, bytes} (total value bytes)
whole-store read kv_list values:true returns {entries:[{key,value}]} — eliminates N×kv_get
docs get_readme documents JSON ::json-cast querying, the ::numeric scale-0 truncation gotcha, and all new params

Design notes

  • The backing table _hyperdb_kv_store has no unique constraint, so
    set_if_absent uses a single INSERT ... SELECT ... WHERE NOT EXISTS — race-
    free within a connection/process (the MCP daemon serializes engine access);
    the rustdoc documents the cross-process caveat honestly.
  • kv_set_many runs inside one transaction: an invalid key aborts the whole
    batch without writing anything. total_bytes counts all submitted values —
    an upper bound on bytes persisted when keys are skipped or duplicated.
  • value_path reads any path the server process can read (no sandbox — matches
    the existing load_file posture); the 64 MiB cap is checked via
    std::fs::metadata before reading, so an oversized file is rejected
    without allocating.

Test plan

  • cargo fmt --all --check — clean
  • cargo clippy --workspace --all-targets --all-features -- -D warnings — 0 warnings
  • cargo test --workspace --exclude hyperdb-api-node --exclude hyperdb-bootstrap1179 passed / 0 failed (114 suites), incl. new sync/async KV outcome tests, kv_set_many, value_path size-cap unit tests, and kv_list values:true
  • hyperdb-mcp/tests/readme_tests.rs structural tool-name coverage — green
  • Post-merge: manual npm smoke test of the published hyperdb-mcp exercising the new KV signals.

Changelog

Per-crate ## [Unreleased] bullets added to hyperdb-api/CHANGELOG.md and
hyperdb-mcp/CHANGELOG.md. Workspace version bump + root CHANGELOG are left to
release-please.

…au#192)

Design and 13-task TDD implementation plan for the five issue-tableau#192 KV-store
MCP improvements plus two adjacent LLM-ergonomics gaps:

- insert-vs-overwrite signal + set_if_absent write guard (no silent data loss)
- server-side value_path for kv_set
- byte-size reporting + soft 1 MiB warning
- kv_set_many atomic batch write + values flag on kv_list
- JSON-in-TEXT query docs + ::numeric truncation gotcha
- PermissionDenied I/O-error fidelity + corrected misleading JSON-error suggestion

Planning docs only; no code changes. The 0.6.1 -> 0.7.0 bump is carried by the
feat!: commits the plan produces, per release-please.
Apply the 6 confirmed adversarial-review findings from the plan-vet phase:

- F1: fold the breaking-change MCP caller fix (server.rs:3136 Ok(())->Ok(_))
  into Task 1's commit + add cargo build -p hyperdb-mcp gate and README refresh
  (Step 7b), so hyperdb-mcp compiles through Tasks 2-7 instead of only at Task 8.
- F2: revert the silent require_writable true->false flip in the kv_size (Task 9)
  and kv_list (Task 11) rewrites, restoring reader parity with kv_get/kv_list_stores.
- F3: correct all three Error::server test calls to the real 4-arg signature
  server(Option<String>, impl Into<String>, Option<String>, Option<String>).
- F4: Task 13 Step 3 CHANGELOG range 8-26 -> 8-53 so the existing hyperdb-mcp
  ### Fixed bullets are carried into the replacement, not duplicated/orphaned.
- F5: sync spec 4 total_bytes to the implemented all-submitted upper-bound
  semantics (BatchGuardOutcome returns counts only, not which keys were written).
- F6: README refresh gap folded into Task 1 Step 7b; premise corrected.

No feature-scope change; these make the plan compilable and internally consistent.
Mirror the sync return-type changes (SetOutcome/BatchSetOutcome) in the
async twin to maintain the sync/async lockstep invariant (AGENTS.md).

Changes:
- AsyncKvStore::upsert now returns Result<bool> (created)
- AsyncKvStore::set returns Result<SetOutcome>
- AsyncKvStore::set_as returns Result<SetOutcome>
- AsyncKvStore::set_batch returns Result<BatchSetOutcome>
- Import SetOutcome/BatchSetOutcome from crate::kv_store
- Update existing test calls to ignore outcomes with `let _`
- Add async_set_reports_created_and_batch_outcome test

All async KV tests pass (5/5) and sync tests remain green (14/14).
Restores sync/async twin lockstep (AGENTS.md rule). The sync KvStore
received `set_if_absent` (iteration 3) and `set_batch_if_absent`
(iteration 5), but AsyncKvStore was missing both. This adds:

- `AsyncKvStore::set_if_absent` (line 195): conditional-INSERT guard
  (INSERT ... WHERE NOT EXISTS), mirroring the sync version.
- `AsyncKvStore::set_batch_if_absent` (line 464): transaction-wrapped
  batch insert that skips existing keys, returning BatchGuardOutcome
  (written/skipped counts).

Both methods use the same SQL patterns as their sync twins. All KV tests
(sync + async) pass.

Fixes CRITICAL-001 (sync/async drift). Closes iteration 6 (plan Task 6).
…lue warning

Two critical test gaps identified in iteration 8 review:

1. kv_set_reports_created_and_bytes: Added missing assertion for
   second.value_bytes = 2. Test name promised both created and value_bytes
   checks but only verified created=false on the overwrite path.

2. New test kv_set_large_value_warns: Verifies kv_set returns a warning
   when value_bytes exceeds KV_SOFT_SIZE_WARN_BYTES (1_048_576). Boundary
   check confirms exactly 1MB triggers no warning, 1MB+1 does. Validates
   warning text contains "1048576" and mentions "soft limit"/"recommended".

All 15 KV tests pass; full workspace clippy+fmt green.

Refs: tableau#192 iteration 8 blocking review findings ITER8-001, ITER8-002
Per-crate CHANGELOG.md bullets only — release-please owns the root
CHANGELOG.md, all Cargo.toml versions, and .release-please-manifest.json.
The 0.6.1 → 0.7.0 bump is carried by the `feat!:` commits in this branch.

hyperdb-api: BREAKING — set/set_as/set_batch return SetOutcome/BatchSetOutcome
instead of Result<()>; added set_if_absent, set_batch_if_absent, byte_size,
entries (sync + async twins).

hyperdb-mcp: added kv_set_many tool, value_path + overwrite on kv_set, values
flag on kv_list, byte reporting on kv_set/kv_size; fixed PermissionDenied
I/O-error mapping and misleading JSON-error suggestion (now advises ::json cast
instead of statement split).
Reject a value_path file larger than 64 MiB against its metadata
BEFORE reading it into memory, so a stray path to a multi-gigabyte
file returns INVALID_ARGUMENT instead of OOMing the server by
slurping the whole thing into a single KV TEXT value. The soft
1 MiB warning still fires post-write for smaller-but-large values;
this adds a hard, enforced ceiling for the file-backed path.

Pure check_value_path_size helper is unit-tested at the boundary
(at-cap passes, cap+1 rejects) without materializing a huge fixture.
…sweep

Final-sweep review reconciliation — doc-only, no behavior change:

- hyperdb-mcp CHANGELOG: kv_set_many under overwrite:false returns
  `created` = keys newly inserted, NOT the literal `created: 0` the
  bullet claimed (the code returns o.written; kv_set_many_guard test
  asserts created==1). Also note total_bytes is an upper bound.
- hyperdb-api CHANGELOG: `let _ = set(...)?;` already compiles
  unchanged — it is NOT a breaking case; name the genuinely-breaking
  shapes instead. Drop the hardcoded "0.7.0" that presumed the version
  release-please will assign.
- set_if_absent rustdoc (sync + async twins): qualify "no
  check-then-write race" with the cross-process caveat (no unique
  constraint on (store_name,key); exact within one process/the
  serialized MCP daemon), mirroring kv_pop's wording.
- readme.rs: note the 64 MiB value_path cap and total_bytes
  upper-bound semantics for the LLM.
@StefanSteiner StefanSteiner merged commit 52a3ba0 into tableau:main Jul 11, 2026
12 checks passed
StefanSteiner added a commit that referenced this pull request Jul 11, 2026
🤖 I have created a release *beep* *boop*
---


<details><summary>0.7.0</summary>

##
[0.7.0](v0.6.1...v0.7.0)
(2026-07-11)


### ⚠ BREAKING CHANGES

* **kv:** #192 KV MCP LLM ergonomics — created signal, batch, size,
value_path, JSON docs
([#193](#193))
* **kv:** async set/set_as/set_batch return SetOutcome/BatchSetOutcome
* **kv:** async twin — reshape AsyncKvStore set/set_as/set_batch
* **kv:** sync set/set_as/set_batch return SetOutcome/BatchSetOutcome
* commits the plan produces, per release-please.

### Features

* commits the plan produces, per release-please.
([f6e5789](f6e5789))
* **kv:** [#192](#192)
KV MCP LLM ergonomics — created signal, batch, size, value_path, JSON
docs ([#193](#193))
([52a3ba0](52a3ba0))
* **kv:** add async set_if_absent/byte_size/entries/set_batch_if_absent
([a166813](a166813))
* **kv:** add set_if_absent and set_batch_if_absent to AsyncKvStore
([d116b12](d116b12))
* **kv:** add sync KvStore::byte_size and entries
([aea9ea5](aea9ea5))
* **kv:** add sync KvStore::set_batch_if_absent atomic guard
([4a88aa3](4a88aa3))
* **kv:** add sync KvStore::set_if_absent write guard
([4f1ca85](4f1ca85))
* **kv:** async set/set_as/set_batch return SetOutcome/BatchSetOutcome
([6af0649](6af0649))
* **kv:** async twin — reshape AsyncKvStore set/set_as/set_batch
([89a7767](89a7767))
* **kv:** sync set/set_as/set_batch return SetOutcome/BatchSetOutcome
([5aeab78](5aeab78))
* **mcp:** add kv_set_many atomic batch write tool
([34f15f7](34f15f7))
* **mcp:** add values flag to kv_list for whole-store reads
([f0b3527](f0b3527))
* **mcp:** kv_set reports created/value_bytes, adds overwrite guard +
value_path
([fd89084](fd89084))
* **mcp:** kv_size reports total value bytes
([0967c05](0967c05))


### Bug Fixes

* **mcp:** [#192](#192)
cap kv_set value_path file size before reading
([0fda8ec](0fda8ec))
* **mcp:** preserve PermissionDenied and steer JSON errors to ::json
cast
([e1acbcd](e1acbcd))
</details>

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

MCP KV tools: 5 improvements from dogfooding (value_path, overwrite signal, size reporting, batch ops, JSON-query docs)

1 participant