Skip to content

feat: add sqlserver__openquery macro and SQL Server best-practices guide - #790

Open
axellpadilla wants to merge 2 commits into
masterfrom
feat/openquery-macro-and-best-practices
Open

feat: add sqlserver__openquery macro and SQL Server best-practices guide#790
axellpadilla wants to merge 2 commits into
masterfrom
feat/openquery-macro-and-best-practices

Conversation

@axellpadilla

Copy link
Copy Markdown
Collaborator

Adds a sqlserver__openquery helper for linked-server pass-through queries, its
tests, and a SQL Server configuration guide. 3 files, +515, no existing code
touched.

Macro

dbt/include/sqlserver/macros/utils/openquery.sql

{{ sqlserver__openquery('LOCALLOOP', "select id from RemoteDb.dbo.customers where active = 1") }}
-- OPENQUERY([LOCALLOOP], 'select id from RemoteDb.dbo.customers where active = 1')
  • doubles single quotes and strips carriage returns in the remote SQL
  • brackets the server name
  • rejects empty arguments, and queries past the 8 KB OPENQUERY limit — the
    error reports the actual escaped length and points at EXEC(...) AT or a
    remote view

Validation is execution-phase ({% if execute %}) rather than parse-time, so one
run reports every bad model instead of aborting on the first.

Tests

tests/functional/adapter/mssql/test_openquery.py — eight models built by a
single dbt run. Three execute OPENQUERY against a LOCALLOOP loopback linked
server (MSOLEDBSQL) and assert both the emitted SQL and the rows the server
returned; the rest cover the length boundary and the validation errors. The
linked server is created and dropped inside the fixture.

Requires a live SQL Server and sp_addlinkedserver rights. Verified 9 passed
on SQL Server 2022 (16.0.4265.3), mssql-python backend.

Guide

docs/sqlserver-best-practices.md — SQL Server defaults that are wrong for a dbt
workload, each with Microsoft's recommended value and why it matters for this
adapter:

Setting Ships as Why it matters here
RCSI off Readers block behind every writer; also the documented prerequisite for table_refresh_method: dml
tempdb data files one, 8 MB Allocation contention under threads > 1; RCSI's version store lives here too
max server memory unlimited Starves the OS
MAXDOP 0 Competes with the model-level parallelism dbt already runs via threads
Recovery model FULL SELECT INTO, how the adapter materializes tables, is only minimally logged outside FULL

Plus OPENQUERY over 4-part names: stage remote reads as models rather than
joining across the boundary, which also avoids error 3988 under the adapter's
default XACT_ABORT ON.

A closing section covers cost threshold for parallelism and optimize for ad hoc workloads — both marked as having no published target value and unmeasured
against a dbt workload, framed as candidates to test rather than settings to
apply.

All claims cited to Microsoft Learn; adapter-specific claims link to the macro or
materialization they come from.

Adds sqlserver__openquery for linked-server pass-through queries: escapes
quotes, strips carriage returns, brackets the server name, and enforces the
8 KB OPENQUERY limit with a compiler error. Validation runs at execution
phase so one invocation surfaces every bad model.

Adds functional tests covering emitted SQL and live OPENQUERY execution
against a loopback linked server in a single dbt run, and
docs/sqlserver-best-practices.md covering RCSI, tempdb sizing, max server
memory/MAXDOP, recovery model, and OPENQUERY vs 4-part-name joins.
@axellpadilla

Copy link
Copy Markdown
Collaborator Author

@joshmarkovic , @Benjamin-Knight please review this best-practices guide

The openquery tests set up a loopback linked server, hardcoded for the engines
it was written against. CI passed on SQL Server 2019 and 2022 and failed on
2017 and 2025, for two unrelated reasons:

- 2017 rejected sp_addlinkedserver with Msg 7222, "Only a SQL Server provider
  is allowed on this instance". MSOLEDBSQL only became a valid linked-server
  provider on Linux in 2019; 2017 needs the native SQLNCLI name.

- 2025 created the linked server fine, then failed every query through it with
  "SSL Provider: The handle specified is invalid (-2146893055)" - the engine's
  own outbound connection as an OLE DB client, not dbt's. From 2025 the
  provider negotiates encryption by default and the loopback presents the
  instance's self-signed certificate, so the handshake fails unless the
  certificate is trusted explicitly.

Both now derive from SERVERPROPERTY('ProductMajorVersion'): SQLNCLI below 2019,
and @provstr = 'TrustServerCertificate=Yes' from 2025 on. The provider string
is sent only where needed, so 2019 and 2022 generate byte-identical SQL and
cannot regress. The SQLNCLI branch carries a note to drop it, with the 2017 CI
leg, once 2017 leaves extended support on 2027-10-12.

The failure also took out the six tests that need no linked server - the
compiled-SQL assertions and the four validation errors - because they hang off
the same class-scoped fixture. Left as is: with the setup fixed they run
everywhere, and a real linked-server regression should fail loudly, not skip.
@axellpadilla
axellpadilla force-pushed the feat/openquery-macro-and-best-practices branch from 64cc887 to ac6732a Compare August 1, 2026 05:42
@axellpadilla axellpadilla added this to the v1.12.0 milestone Aug 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant