chore: reconcile shared catalog against ephemeral primary + persistent (#195)#196
Merged
StefanSteiner merged 4 commits intoJul 12, 2026
Merged
Conversation
- CLAUDE.md: thin auto-load shim that re-exports AGENTS.md as the cross-tool source of truth for Claude Code. - AGENTS.md: document the docs/superpowers/ convention — design specs and implementation plans kept in-repo and committed alongside the code that implements them, so the rationale is reviewable with the change. - docs/superpowers/plans/2026-07-09-kv-store-m2-mcp.md: the KV Store Milestone 2 (MCP tool surface) implementation plan artifact.
Cover the ephemeral-primary catalog gap and its metadata-loss trap: - set_metadata_finds_execute_created_ephemeral_primary_table (stub-gap, tableau#195) - reconcile_preserves_ephemeral_primary_metadata_across_unrelated_ddl (delete-trap) - reconcile_does_not_false_rename_persistent_row_onto_new_ephemeral_table (C1 guard) plus three guards that pass today (persistent rows survive, dropped ephemeral row still reaped, persistent wins a cross-DB name collision). Tests 1, 2, and the false-rename guard fail on current code because reconcile_in enumerates only the persistent attachment; the union-reconcile fix in the following commit turns them green.
tableau#195) The persistent _table_catalog is the single store for catalog rows of tables that physically live in EITHER the ephemeral primary (unqualified DDL, e.g. an `execute` CREATE on the default workspace) OR the persistent attachment. reconcile_in enumerated its live-table set from the persistent attachment only, so an execute-created primary table was never stubbed (set_table_metadata → TABLE_NOT_FOUND), and any pre-existing catalog row for a primary table was reaped as "disappeared" by a later unrelated structural execute — silently losing user-set purpose/source_url. Enumerate the UNION of persistent + ephemeral-primary public tables when reconciling the shared catalog, mapping each live table to its origin alias for row counting. Storage stays in the persistent catalog; only the live-set enumeration and COUNT(*) became origin-aware. - reconcile_live_tables(engine, target_db) -> BTreeMap<name, alias>: ephemeral primary inserted first, persistent second, so persistent wins a name collision (authoritative for a name-keyed catalog). - public_tables_in / row_count_qualified: per-alias helpers replacing user_tables_in / row_count_of_in (both deleted; no other callers). - Rename heuristic TARGET pool restricted to persistent-origin new tables, so a disappeared persistent row can never be false-renamed onto a new ephemeral scratch table by coincident row count. Known limitation (documented on reconcile_in): the catalog has no origin column and each MCP process owns a distinct ephemeral primary, so a peer process's ephemeral rows can still be reaped under concurrent multi-process use against one shared persistent catalog. Out of scope here. Type is `chore:` deliberately: releasing this in a v0.7.1 bump is being held so other fixes land first; the fix ships without a CHANGELOG line.
The tableau#195 shared-catalog union restricted the reconcile rename-target pool to a literal PERSISTENT_ALIAS. For a per-DB reconcile (target_db = Some(user_alias)) every live table maps to the user alias, so the pool was always empty and the rename heuristic never fired — a rename inside a user-attached DB deleted + re-stubbed the row, losing purpose/source_url/ notes. Same silent prose-loss class as tableau#195, on a different path. Generalize the pool to the catalog's OWN resolved alias: persistent for the shared catalog (byte-for-byte the prior behavior, C1 guard intact — ephemeral tables still excluded), the user alias for a per-DB reconcile. Count each candidate against live[t] rather than a hard-coded persistent alias. Adds reconcile_in_user_db_preserves_metadata_on_rename (RED before, GREEN after). Caught by the final-sweep deep review.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #195.
set_table_metadatareturnedTABLE_NOT_FOUNDfor a table created via theexecutetool on the default (ephemeral) primary — and, more seriously, an already-catalogued ephemeral-primary table (registered via theload_*ingest path with user-setpurpose/source_url) had its catalog row silently deleted by any later unrelated structuralexecute.Root cause: the persistent
_table_catalogis the single shared store of catalog rows for tables that physically live in either the ephemeral primary DB (where unqualified DDL lands) or the persistent attachment. Butreconcile_in(engine, None)— run after every structuralexecuteand at bootstrap — computed its "live table set" from the persistent attachment only. So ephemeral-primary tables were (a) never stubbed and (b) classified "disappeared" and reaped.Fix
Confined to
hyperdb-mcp/src/table_catalog.rs. When reconciling the shared persistent catalog (target_dbisNoneor"persistent"), enumerate the union of persistent-attachment tables ∪ ephemeral-primary tables, mapping each live table to the DB itsrow_countshould be computed against. Catalog rows are still stored in the persistent catalog; only the live-set enumeration andCOUNT(*)became origin-aware.reconcile_live_tables(engine, target_db) -> BTreeMap<name, alias>— ephemeral primary inserted first, persistent second, so on a name collision the durable persistent table wins the count (authoritative for a name-keyed catalog).public_tables_in/row_count_qualified— per-alias helpers that replace the deleteduser_tables_in/row_count_of_in.execute-driven renames inside a user DB still preserve prose).Known limitation (documented, not fixed here)
The catalog is name-keyed with no origin-DB column, and each MCP process owns a distinct ephemeral primary while sharing one persistent catalog. This fix is correct for the realistic single-MCP-process-per-persistent-workspace deployment; under multiple concurrent processes against one shared persistent catalog, a peer's ephemeral-table rows can still be reaped. A principled fix needs an
origin_dbcolumn and is deferred.Why
chore:and notfix:This is a real (arguably data-loss) bug fix. It is intentionally typed
chore:— and this PR is titledchore:so release-please does not cut a v0.7.1 patch — because other fixes are being held to land before v0.7.1 is released. Consequence: this change will not appear as afix:line in the eventual v0.7.1 CHANGELOG. Filed here so the reasoning is on record for future archaeology.Tests
Six regression tests in
hyperdb-mcp/tests/table_catalog_tests.rs:set_metadata_finds_execute_created_ephemeral_primary_table— the set_table_metadata returns TABLE_NOT_FOUND for tables created via execute on the ephemeral primary #195 stub-gap (was RED:TABLE_NOT_FOUND).reconcile_preserves_ephemeral_primary_metadata_across_unrelated_ddl— the metadata-loss trap (was RED: row deleted).reconcile_does_not_false_rename_persistent_row_onto_new_ephemeral_table— cross-DB false-rename guard (was RED: ephemeral table unstubbed / would inherit prose under a naive union).reconcile_does_not_delete_persistent_rows_when_reconciling_shared_catalog— persistent rows survive reconcile of the shared catalog.reconcile_reaps_dropped_ephemeral_primary_table— a genuinely-dropped ephemeral table's row is still reaped (no over-correction).reconcile_persistent_wins_name_collision_across_dbs— persistent table is authoritative for the count on a name collision.Plus one in
hyperdb-mcp/tests/per_tool_database_tests.rs:reconcile_in_user_db_preserves_metadata_on_rename— a rename inside a user-attached DB preserves prose (was RED: the persistent-only rename-target restriction emptied the candidate pool for user DBs, deleting + re-stubbing the row).Verification
cargo fmt --all --check— clean.cargo clippy --workspace --all-targets --all-features -- -D warnings— clean.cargo test --workspace --exclude hyperdb-api-node --exclude hyperdb-bootstrap— all green.