Skip to content

Implements dbt_sqlserver_use_dbt_transactions behavior flag#710

Open
axellpadilla wants to merge 1 commit into
masterfrom
fix/708-dbt-transactions
Open

Implements dbt_sqlserver_use_dbt_transactions behavior flag#710
axellpadilla wants to merge 1 commit into
masterfrom
fix/708-dbt-transactions

Conversation

@axellpadilla

@axellpadilla axellpadilla commented Jun 15, 2026

Copy link
Copy Markdown
Collaborator

Adds an opt-in flag that makes dbt's transaction hooks real at the SQL Server level, replacing the current no-op behavior, couldn't find the exact reason this was removed and this stopsome issues we currently have with left over temp tables, dirty reads of empty tables, non-standard behaviors.

What it does

When dbt_sqlserver_use_dbt_transactions: true is set in dbt_project.yml:

  • add_begin_query() emits BEGIN TRANSACTION instead of being a no-op
  • add_commit_query() emits IF @@TRANCOUNT > 0 COMMIT TRANSACTION instead of being a no-op
  • _rollback_handle() emits IF @@TRANCOUNT > 0 ROLLBACK TRANSACTION with error handling matching the base class pattern (fires RollbackFailed event on failure)
  • After-commit hooks — the run_hooks macro uses the flag to decide whether hooks run after a real commit or after the previous autocommit-based cleanup

When false (default), behavior is unchanged — begin/commit remain no-ops and each statement is auto-committed by the ODBC driver.

Why opt-in

Enabling real transactions changes the atomicity guarantees and maps to default dbt standard behavior. Without the flag, each SQL statement commits individually — a failure mid-materialization doesn't roll back earlier DDL/DML. With the flag, the entire model run is wrapped in a single T-SQL transaction, and failures roll back everything.

This includes the DML refresh materialization (table_dml_refresh.sql), which has its own explicit BEGIN TRANSACTION for the atomic DELETE+INSERT swap. With the flag on, this nests inside dbt's outer transaction. SQL Server handles this correctly — only the outermost COMMIT actually commits, and a rollback always rolls back all nesting levels — so scratch tables from a partially-executed materialization are cleaned up on failure.

Other changes

  • Updated the stale comment in table_dml_refresh.sql to reflect the flag
  • Fixed a line length issue in test files
  • Added functional tests for table/view/incremental materializations, rollback behavior (with and without the flag), after-commit post-hooks, and recovery after a failed model including xfail temporal variants that will never pass without the flag.
  • Added unit tests for add_begin_query/add_commit_query flag gating

Closes #708

dbt_sqlserver_use_dbt_transactions flag

@Benjamin-Knight Benjamin-Knight left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The DML work I did has an explicit transaction in it, this probably should only be emitted when this flag is false because when DBT is handling the transaction all it does it create a nested transaction that doesn't add anything to the process.

I can't see the XACT_ABORT work that @joshmarkovic identified as being an issue. If we want DBT to handle transaction should be have this flag emit the XACT_ABORT setting as well?

I think we need a test to cover snapshots as they call adapter.commit in their code path.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hooks.sql reads the flag from a different source (flags) than the connection manager (adapter.behavior), and the two agree only while the default is False

@Benjamin-Knight

Copy link
Copy Markdown
Collaborator

I remember someone stating that there are parts of DBT which we may want to not run inside the transaction which is now DBT managed, mostly post hooks, are we planning on shipping this without that resolved? Right now this work would break online and resumable post hook indices that are declared.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support opt-in dbt-managed SQL Server transactions

2 participants