Filter internal __pgsqlite_* tables from client sqlite_master queries#81
Closed
robertjbass wants to merge 3 commits into
Closed
Filter internal __pgsqlite_* tables from client sqlite_master queries#81robertjbass wants to merge 3 commits into
robertjbass wants to merge 3 commits into
Conversation
pgsqlite keeps its own bookkeeping in tables named __pgsqlite_* (schema, migrations, enum metadata, string/numeric constraints, datetime cache, etc.). The catalog handlers that pgsqlite synthesizes itself already exclude these (e.g. src/catalog/pg_class.rs appends "AND name NOT LIKE '__pgsqlite_%'"), but a sqlite_master / sqlite_schema query written directly by a client is passed through verbatim, so tools that introspect the native SQLite catalog see pgsqlite's internal tables alongside the user's own. Add a branch to CatalogInterceptor::intercept_query that runs before the pg_catalog / information_schema gate (a sqlite_master query contains none of those substrings). It parses the query with the bundled sqlparser, finds every sqlite_master / sqlite_schema relation in the FROM clause (alias-aware and JOIN-safe, matching the SQLite sqlite_schema alias too), ANDs "<relation>.name NOT LIKE '__pgsqlite_%'" into the WHERE clause, and executes the rewritten query. It fails open (runs the original query unchanged) on any parse failure or when sqlite_master is not an actual FROM relation, so ordinary queries are untouched. Only the __pgsqlite_% prefix is filtered; sqlite_% internal objects are left as-is. Covered by new unit tests (rewrite: plain / aliased / joined / sqlite_schema / fail-open / literal-not-table / similarly-named-table) and an integration test (catalog_sqlite_master_filter_test) asserting the internal tables are hidden over the wire while user tables, indexes and views are still returned.
…lite_master The sqlite_master / sqlite_schema filter previously hid only the __pgsqlite_* bookkeeping tables. Extend the rewritten predicate to also exclude the four pg_catalog tables that are physically materialized as real SQLite tables (pg_attrdef, pg_constraint, pg_depend, pg_index) and any index or trigger owned by an internal table, by filtering on tbl_name as well as name. This keeps an unqualified SELECT * FROM sqlite_master clean, not just a type = 'table' scan. The four table names are a closed set defined next to the __pgsqlite_ prefix as PG_CATALOG_INTERNAL_TABLES, with src/migration/registry.rs cited as the source of truth. Matching is exact-set, not a pg_% wildcard, so a user table such as pg_indexes is still returned. The rewrite keeps its alias/JOIN awareness and fail-open-on-parse-failure behavior.
…e_master Extends the client sqlite_master/sqlite_schema filter to exclude the 24 pg_catalog and information_schema compatibility views pgsqlite materializes as real SQLite views (source of truth: migration/registry.rs) in addition to the __pgsqlite_* tables, the four materialized pg_catalog tables, and their internal indexes. A raw type='view' scan or an unqualified SELECT * now shows only the user's own schema. The exclusion is an exact-name set, never a pg_% / information_schema_% wildcard, so a user-owned view or table such as pg_my_report is still returned. Filter applies to both name and tbl_name for symmetry, keeps the same alias/JOIN awareness, and fails open on any parse failure.
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Filter internal
__pgsqlite_*tables from clientsqlite_masterqueriesWhat
pgsqlite keeps its own bookkeeping in tables named
__pgsqlite_*(
__pgsqlite_schema,__pgsqlite_migrations,__pgsqlite_enum_types,__pgsqlite_string_constraints,__pgsqlite_numeric_constraints,__pgsqlite_datetime_cache, ...). The catalog queries pgsqlite synthesizesitself already exclude these (for example
src/catalog/pg_class.rsappendsAND name NOT LIKE '__pgsqlite_%'), but asqlite_master/sqlite_schemaquery written directly by a client is passed through verbatim, so tools that
introspect the native SQLite catalog see pgsqlite's internal tables alongside
the user's own.
How
This adds a branch to
CatalogInterceptor::intercept_query(
src/catalog/query_interceptor.rs) that runs before the existingpg_catalog/information_schemagate, because asqlite_masterquerycontains none of those substrings and would otherwise fall straight through.
The new branch:
sqlparser;FROMclause (including JOINs) for everysqlite_master/sqlite_schemarelation, honoring table aliases;<relation>.name NOT LIKE '__pgsqlite_%'into theWHEREclause,qualified by the relation's alias (or bare name) so it stays unambiguous
across joins;
It fails open (runs the original query unchanged) on any parse failure, on
anything that is not a single
SELECT, and whensqlite_masterappears onlyas a string literal or column name rather than as a real
FROMrelation. Onlythe unambiguous
__pgsqlite_%prefix is filtered;sqlite_%internal objectsare left untouched (they are part of SQLite's own catalog and are what a normal
SQLite client expects to see).
The core rewrite is factored into a pure
rewrite_sqlite_master_query(&str) -> Option<String>so it can be unit-tested without a database.Tests
src/catalog/query_interceptor.rs: plain select, existingWHEREpreserved viaAND, aliased relation uses the alias qualifier,JOINed relation stays join-safe,
sqlite_schemaalias, unparseable SQL failsopen,
sqlite_masteras a literal is left untouched, and a similarly-nameduser table (
sqlite_master_backup) is not matched.tests/catalog_sqlite_master_filter_test.rs: over thewire, internal tables are hidden from plain / aliased /
sqlite_schemaqueries while user tables, indexes and views are still returned, and a
non-
sqlite_masterquery is untouched.Scope note
pgsqlite also materializes its
pg_catalog/information_schemaemulation asreal SQLite tables (
pg_class,information_schema_tables, ...), which alsoappear in a client
sqlite_masterlisting. Filtering those is a broader policyquestion (a user could name a table
pg_*), so this PR intentionally limitsitself to the
__pgsqlite_*prefix. See the linked issue for details.Closes #80.