Skip to content

feat(sqlserver): part 5 add catalog introspection - #15

Merged
axellpadilla merged 1 commit into
sqlserver-v2-portfrom
part-5-sqlserver-catalog
Aug 3, 2026
Merged

feat(sqlserver): part 5 add catalog introspection#15
axellpadilla merged 1 commit into
sqlserver-v2-portfrom
part-5-sqlserver-catalog

Conversation

@axellpadilla

Copy link
Copy Markdown
Collaborator

Part 5 of the SQL Server Fusion adapter series. Part of dbt-labs#15714.

Stacked on #14 (Part 4). Base is part-4-sqlserver-relation, so this diff shows only Part 5's four files. Merge #11#12#13#14 first; this retargets automatically as each lands.

#5 says Part 5 is independent of #4. It isn't — list_relations and get_relation both construct Relation::new_sqlserver, which Part 4 adds. The dependency is real, not just a stacking convenience.

Not a copy of Fabric's module

The plan said "modeled on src/metadata/fabric/mod.rs", and it is, but Fabric drives sp_tables / sp_columns and those are the wrong tool here. Two measurements against SQL Server 2022 (16.0.4265.3), from a connection attached to a different database:

1. The procedures are single-database.

EXEC sys.sp_tables @table_qualifier = 'master', @table_owner = 'dbo', @table_name = 'spt_values'
→ ERROR 15250: The database name component of the object qualifier must be
  the name of the current database.

Fabric never hits this — a warehouse is the database, so @table_qualifier is always the current one. On SQL Server a project that reads a relation in another database on the same server is ordinary, and v1 supports it by emitting USE [db]; before every metadata query.

2. @table_name is a LIKE pattern. With probe_table and probeXtable both present:

EXEC sys.sp_tables @table_qualifier='probe_db', @table_owner='probe_schema', @table_name='probe_table'
→ 2 rows: probe_table, probeXtable

dbt model names are snake_case, so this is not an edge case. In get_relation the second row trips the len() != 1 check and turns a valid lookup into an error; in sp_columns it merges another table's columns into the schema. @table_name = 'probe\_table' returns zero rows — the backslash isn't an escape here. Only @fUsePattern = 0 fixes it (verified: 1 row).

Both apply to metadata/fabric/mod.rs as written. I haven't touched it — I can't test whether Fabric's T-SQL surface accepts @fUsePattern, and cross-database is genuinely impossible there so only the pattern half is a live bug. Worth a separate issue by someone who can run against Fabric.

What this uses instead: three-part sys.objects / sys.schemas / sys.columns / sys.types. Resolves cross-database, matches exactly, and unlike v1's USE it leaves the connection's current database alone — which matters when the connection goes back to a pool. Verified: select db_name() after the query still returns the original database.

Type text

sp_columns.TYPE_NAME is the bare type name. decimal(18,4) arrives as decimal, nvarchar(50) as nvarchar — Fabric passes that straight to make_arrow_field, dropping precision, scale and length. sys.columns carries them, so compose_type_text rebuilds the declared text, following v1's sqlserver__get_columns_in_relation:

category rule measured
char varchar binary varbinary (max_length), -1(max) varchar(50), varchar(max)
nchar nvarchar (max_length / 2) — bytes, two per char nvarchar(100)nvarchar(50)
decimal numeric (precision,scale) decimal(18,4)
datetime2 datetimeoffset time (scale) datetime2(7), time(3)
everything else bare datetime reports scale 3 and rejects datetime(3)

float(24) needs no case: SQL Server stores it as real, and float/real each have exactly one precision.

The narrow integer columns are cast: max_length comes back smallint and precision/scale tinyint, which Arrow surfaces as int16/uint8. cast(... as int) makes all three int32 so the Rust reads one array type.

Comments

build_schemas_from_stats_sql and build_columns_from_get_columns read table_comment and column_comment, following Postgres rather than Fabric, which drops both. v1's sqlserver__get_catalog returns them and v1 supports persist_docs. This fixes the column contract the catalog macro has to satisfy in §5.6.

Arms outside #5's checklist

metadata_adapter and list_relations in adapter_impl.rs are two of Part 7's arms. Taken here because they're what makes this module reachable — without them it's dead code that never runs. #7's checklist should lose metadata_adapter.

Left as todo!()

freshness_inner and list_relations_schemas_by_patterns_inner, matching both Fabric and Postgres. v1 has sqlserver__get_relation_last_modified, so freshness is a known port, not an unknown.

Verification

Ten unit tests over the SQL builders and the type composer — the builders take the quoted database as a parameter, so they're testable without a connection.

Separately, every query was run against a live SQL Server 2022 in the exact text the Rust emits (dumped from the builders, executed verbatim), from a connection attached to master while reading probe_db:

  • list_relations — 6 relations, tables and views classified correctly
  • get_relationTABLE for a table, VIEW for a view, 0 rows for a missing one, and exactly 1 row for probe_table with probeXtable present
  • columns — all 23 types above, returning string/int32 exactly as the Rust reads them
  • a database named probe "odd" db — Part 4's delimiter doubling holds inside a three-part name

dbt-adapter still has 40 E0004s (Parts 6–7), so cargo test -p dbt-adapter can't run on this branch as-is. As in Part 4 I stubbed those arms locally with todo!(), ran the suite, and reverted before committing: 891 passed / 0 failed (881 + these 10).

Workspace E0004 count drops 44 → 41: adapter_impl.rs (33), sql_types.rs (5), column_builder.rs (2), and dbt-df-providers seed_io.rs (1), which no issue in the series claims.

Closes #5

Modeled on `metadata/fabric/mod.rs`, but not a copy of it. Fabric drives
`sp_tables` / `sp_columns`, and two things measured against SQL Server 2022
rule that out here:

- The procedures read the connection's current database only. Any other
  `@table_qualifier` is error 15250, "The database name component of the
  object qualifier must be the name of the current database". A dbt project
  that reaches across databases on one server is ordinary on SQL Server and
  impossible on Fabric, where a warehouse is its own database.
- `@table_name` is a `LIKE` pattern unless `@fUsePattern = 0` is passed, so
  `stg_orders` also matches `stgXorders`. Underscores are everywhere in dbt
  model names, and the second row makes `get_relation` fail the
  one-row check rather than return the wrong type. `\_` does not escape it;
  only `@fUsePattern = 0` does.

The catalog views take a three-part name, which resolves cross-database and,
unlike the `USE` that v1 emits before every metadata query, leaves the
connection's current database alone. That matters for a pooled connection.

`sys.columns` also carries the type detail `sp_columns` drops: `TYPE_NAME`
is the bare name, so `decimal(18,4)` arrives as `decimal` and `nvarchar(50)`
as `nvarchar`. `compose_type_text` rebuilds the declared text from
`max_length` / `precision` / `scale`, following v1's `get_columns_in_relation`
-- including its halving of `max_length` for the national types, which
`sys.columns` reports in bytes. `datetime` reports a scale but rejects one.

`build_schemas_from_stats_sql` and `build_columns_from_get_columns` read
`table_comment` and `column_comment`, as Postgres does and Fabric does not;
v1's `get_catalog` returns both, and v1 supports `persist_docs`.
`freshness_inner` stays `todo!()`, matching Fabric and Postgres, though v1
has `get_relation_last_modified` to port.

`metadata_adapter` and `list_relations` in `adapter_impl.rs` are two of
Part 7's arms, taken here because they are what makes this module
reachable.

Ten tests over the SQL builders and the type composer. Every query was also
run against a live SQL Server 2022 from a connection attached to a different
database, in the exact text the Rust emits.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@axellpadilla
axellpadilla force-pushed the part-5-sqlserver-catalog branch from 5cdbd82 to 5396ae6 Compare August 2, 2026 18:03
@axellpadilla axellpadilla changed the title feat(sqlserver): add catalog introspection feat(sqlserver): part 5 add catalog introspection Aug 2, 2026
@axellpadilla
axellpadilla changed the base branch from part-4-sqlserver-relation to sqlserver-v2-port August 3, 2026 01:18
@axellpadilla
axellpadilla merged commit 58b7149 into sqlserver-v2-port Aug 3, 2026
@axellpadilla axellpadilla linked an issue Aug 3, 2026 that may be closed by this pull request
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(sqlserver): Part 5 — catalog introspection (get_relation, metadata module)

1 participant