Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
- **Behavior change:** `dbt_sqlserver_use_dbt_transactions` now defaults to `True`: dbt-managed transaction hooks (begin/commit) emit real `BEGIN TRANSACTION` / `COMMIT TRANSACTION` T-SQL instead of no-ops, so a failed model rolls back its own statements instead of leaving a partial result behind on autocommit. The `False` (legacy autocommit) behavior is deprecated and will be removed in a future release.
- **Behavior change:** `dbt_sqlserver_use_native_string_types` now defaults to `True`: `STRING` maps to `VARCHAR(MAX)`, `NCHAR` to `NCHAR(1)`, and `NVARCHAR` to `NVARCHAR(4000)`, instead of the legacy `VARCHAR(8000)`/`CHAR(1)` mappings. The `False` (legacy) behavior is deprecated and will be removed in a future release.
- Add an experimental `adbc` backend (`backend: adbc`), an alternative to `pyodbc`/`mssql-python` built on [ADBC](https://arrow.apache.org/adbc/) that talks to SQL Server via `go-mssqldb` instead of an ODBC/DB-API bridge. Install with `dbt-sqlserver[adbc]` plus the separate `dbc` CLI-installed driver binary; supports SQL Server (user/password) authentication only for now. See [docs/adbc_backend.md](docs/adbc_backend.md). [#771](https://github.com/dbt-msft/dbt-sqlserver/issues/771)
- Add a model-level `denies` config that re-applies object-level `DENY` permissions after each build, diffed against `sys.database_permissions`, so an object DENY survives dbt's drop-and-recreate. A SQL Server object DENY is the only way to carve an exception out of a schema-level GRANT, but it is stored against `object_id` and was silently discarded on every rebuild (every run for a view), leaving a fail-open posture. Shaped like `grants` (`{privilege: [principals]}`); covers `table`, `view`, `incremental` and `snapshot`; emits `DENY`/`REVOKE` only for what changed; warns-and-skips an absent principal; and is a no-op on other adapters. Mirrors the existing `masks` re-application. See the README for details.

#### Bugfixes

Expand Down
17 changes: 17 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -304,6 +304,23 @@ Behaviour:

**Version notes.** All masking DDL the adapter emits (`ADD MASKED`, `MASKED WITH`, `DROP MASKED`) and the functions `default()`, `email()`, `random(a,b)` and `partial(...)` work on 2016+. The `datetime()` partial-date function and granular column/schema/table-scoped `UNMASK` are SQL Server 2022+ only; the adapter never emits them, but mask-function strings are passed through verbatim, so using a 2022-only function on an older server will be rejected by SQL Server.

### Object-level DENY (`denies`)

In SQL Server, an object-level `DENY` is the only way to carve an exception out of a schema-level `GRANT` — grant a principal `SELECT` on a whole schema, then `DENY SELECT` on the individual PII-bearing models. But a `DENY` is stored against the object's `object_id`, so dbt destroys it every time it drops and recreates the relation (which is *every* run for a view). The schema grant survives; its exceptions silently evaporate, leaving a **fail-open** posture that no run reports.

The `denies` config re-applies object-level DENYs after each build, diffed against `sys.database_permissions`, the same way `masks` re-applies Dynamic Data Masking. Its shape mirrors `grants` — a `{privilege: [principals]}` map — and it is settable in the in-file `{{ config() }}`, the model's `.yml` `config:` block, or a directory-wide default in `dbt_project.yml` (merging key-wise across those levels, like `masks`):

```sql
{{ config(denies={'select': ['Restricted_Read_Only']}) }}
```

- **Survives rebuilds:** the DENY is present after `dbt run` and *still present after the next run*, for `table`, `view`, `incremental` (append and full-refresh) and `snapshot`. Unlike masks, this **includes views** — a view is a valid securable and is recreated on every run, which is where a DENY is lost most often.
- **Reconciled:** the adapter diffs the config against the live DENY state and emits only what changed — `DENY` for configured-not-present, `REVOKE` for present-not-configured (a `REVOKE` removes a `DENY`). Removing a principal from `denies` revokes its DENY on the next run; a converged, persisted relation issues no DDL.
- **Scope:** object-level DENY for the table privileges `select`, `insert`, `update`, `delete`, `references`. An unsupported privilege is warned and skipped (the warning surfaces the likely typo without taking down the build). Column-, database- and server-scoped permissions are out of scope (the latter survive a rebuild already).
- **Absent principal:** a `DENY` to a non-existent database principal is warned-and-skipped (the run still succeeds), so one config runs unchanged across dev, CI and prod where the principal set differs.
- **Ordering vs `grants`:** grants are applied first, then denies, so the final state is unambiguous. A `(privilege, principal)` pair appearing in both `grants` and `denies` is warned as a likely mistake (a DENY overrides a GRANT in SQL Server).
- **Other adapters:** `denies` is a no-op on non-SQL-Server adapters, not an error. This is SQL-Server-specific by design — `DENY` is absent from the SQL standard and does not port.

## Contributing

[![Unit tests](https://github.com/dbt-msft/dbt-sqlserver/actions/workflows/unit-tests.yml/badge.svg)](https://github.com/dbt-msft/dbt-sqlserver/actions/workflows/unit-tests.yml)
Expand Down
48 changes: 48 additions & 0 deletions dbt/adapters/sqlserver/sqlserver_adapter.py
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,8 @@
from dbt.adapters.sqlserver.sqlserver_column import SQLServerColumn, SQLServerColumnNative
from dbt.adapters.sqlserver.sqlserver_configs import SQLServerConfigs
from dbt.adapters.sqlserver.sqlserver_connections import SQLServerConnectionManager
from dbt.adapters.sqlserver.sqlserver_deny import deny_changes as _deny_changes
from dbt.adapters.sqlserver.sqlserver_deny import resolve_denies as _resolve_denies
from dbt.adapters.sqlserver.sqlserver_mask import ColumnMask
from dbt.adapters.sqlserver.sqlserver_mask import mask_changes as _mask_changes
from dbt.adapters.sqlserver.sqlserver_mask import resolve_masks as _resolve_masks
Expand Down Expand Up @@ -663,6 +665,52 @@ def mask_changes(
existing_columns=(list(existing_columns) if existing_columns is not None else None),
)

@available
def resolve_denies(self, model: Any, model_denies: Optional[dict] = None) -> Dict[str, list]:
"""Normalise the model-level `denies` config into a `{privilege:
[principals]}` map for `apply_denies`.

`model` is the Jinja `model` dict (`node.to_dict()`); its `grants` config
(under `model['config']`) is read only to warn when a principal is both
granted and denied the same privilege. `model_denies` is
`config.get('denies')`, already surface-merged by dbt.

Unsupported privileges (anything other than the object-level table
privileges) are warned and skipped rather than failing the run — the
warning surfaces the likely typo without taking down the build.
"""
model = model or {}
grant_config = (model.get("config") or {}).get("grants")
model_name = model.get("name") or model.get("alias") or "<unknown>"
resolved, warnings, unsupported = _resolve_denies(model_denies, grant_config, model_name)
for warning in warnings:
logger.warning(warning)
if unsupported:
from dbt.adapters.sqlserver.sqlserver_deny import SUPPORTED_PRIVILEGES

logger.warning(
f"On model '{model_name}', the `denies` config lists unsupported "
f"privilege(s): {', '.join(sorted(unsupported))}; skipping them. "
f"Object-level DENY is supported only for the table privileges: "
f"{', '.join(SUPPORTED_PRIVILEGES)}."
)
return resolved

@available
def deny_changes(self, existing_denies: Any, deny_config: Optional[dict]) -> dict:
"""Diff a resolved deny map against current `sys.database_permissions`.

`existing_denies` is the agate table from `get_show_deny_sql` (columns
`grantee`, `privilege_type`). Returns plain lists for jinja: `denies` and
`revokes`, each a list of `[privilege, principal]` pairs. The macro emits
`DENY` for the former and `REVOKE` for the latter."""
rows = []
if existing_denies is not None:
column_names = existing_denies.column_names
for row in existing_denies.rows:
rows.append(dict(zip(column_names, row)))
return _deny_changes(rows, deny_config or {})


COLUMNS_EQUAL_SQL = """
with diff_count as (
Expand Down
8 changes: 8 additions & 0 deletions dbt/adapters/sqlserver/sqlserver_configs.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,11 @@ class SQLServerConfigs(AdapterConfig):
masks: Optional[Dict[str, Any]] = field(
default_factory=dict, metadata=MergeBehavior.Update.meta()
)
# privilege -> [principals] map for the model-level `denies` surface, shaped
# like `grants`. Re-applied after each build because an object-level DENY is
# stored against object_id and discarded on drop-and-recreate. Same key-wise
# MergeBehavior.Update as `masks`, so a directory-level default and a
# per-model tweak combine instead of one clobbering the whole dict.
denies: Optional[Dict[str, Any]] = field(
default_factory=dict, metadata=MergeBehavior.Update.meta()
)
141 changes: 141 additions & 0 deletions dbt/adapters/sqlserver/sqlserver_deny.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
"""Pure resolution + diff logic for object-level ``DENY`` permissions.

Kept free of any database or dbt-context dependency so it can be unit tested
in isolation, mirroring ``sqlserver_mask.py``. The adapter's ``@available``
wrappers extract plain data from the model / ``sys.database_permissions`` and
delegate here; the Jinja ``sqlserver__apply_denies`` macro turns the diff into
DDL.

A SQL Server object-level ``DENY`` is stored against ``object_id``, so dbt
destroys it every time it drops and recreates the relation. The ``denies:``
config re-applies it after materialization, diffed against the live state, the
same way ``masks:`` re-applies Dynamic Data Masking.

One config surface feeds a ``{privilege: [principals]}`` map:

* the model-level ``denies`` dict, shaped exactly like ``grants``.

Privilege and principal identifiers are compared case-insensitively, matching
SQL Server's default collation. The user's original spelling is the one emitted
in DDL.
"""

from typing import Dict, List, Optional, Sequence, Tuple

# Object-level, table-scoped privileges that a DENY can carry. Column-level
# DENY (minor_id > 0) and database/server-scoped permissions are out of scope:
# the schema-grant-with-object-exceptions pattern only needs these.
SUPPORTED_PRIVILEGES = ("select", "insert", "update", "delete", "references")


def _normalize(name: str) -> str:
return name.strip().lower()


def _dedupe_ci(principals: Sequence[str]) -> List[str]:
"""Drop case-insensitive duplicates, preserving first-seen spelling/order."""
seen: Dict[str, str] = {}
for principal in principals or []:
key = _normalize(principal)
if key and key not in seen:
seen[key] = principal
return list(seen.values())


def _normalize_grant_map(grant_config: Optional[Dict[str, Sequence[str]]]) -> Dict[str, set]:
"""``{privilege: [grantee]}`` -> ``{privilege_lower: {grantee_lower}}``."""
result: Dict[str, set] = {}
for privilege, grantees in (grant_config or {}).items():
result.setdefault(_normalize(privilege), set()).update(
_normalize(g) for g in (grantees or [])
)
return result


def resolve_denies(
deny_config: Optional[Dict[str, Sequence[str]]],
grant_config: Optional[Dict[str, Sequence[str]]],
model_name: str,
) -> Tuple[Dict[str, List[str]], List[str], List[str]]:
"""Normalise the ``denies`` config into a clean ``{privilege: [principals]}``.

``deny_config`` is already surface-merged by dbt across ``dbt_project.yml`` /
``.yml`` / in-file ``config()``. This:

* lower-cases privilege names and case-insensitively de-duplicates the
principal list under each (preserving the first spelling seen);
* collects any privilege that is **not** an object-level table privilege into
``unsupported`` so the caller can fail loudly — a silently-skipped DENY is
the fail-open regression this feature exists to prevent;
* warns when a ``(privilege, principal)`` pair appears in **both** ``denies``
and ``grants``, which would emit contradictory DDL and is almost certainly
a mistake.

Returns ``(resolved, warnings, unsupported)``.
"""
warnings: List[str] = []
unsupported: List[str] = []
resolved: Dict[str, List[str]] = {}

for privilege, principals in (deny_config or {}).items():
norm_priv = _normalize(privilege)
if norm_priv not in SUPPORTED_PRIVILEGES:
unsupported.append(privilege)
continue
deduped = _dedupe_ci(principals)
if not deduped:
continue
# Merge case-variant privilege keys ("select" and "SELECT") into one.
existing = resolved.get(norm_priv, [])
resolved[norm_priv] = _dedupe_ci(existing + deduped)

grant_map = _normalize_grant_map(grant_config)
for privilege, principals in resolved.items():
granted = grant_map.get(privilege, set())
for principal in principals:
if _normalize(principal) in granted:
warnings.append(
f"On model '{model_name}', principal '{principal}' is both granted "
f"and denied '{privilege}'. A DENY overrides a GRANT in SQL Server; "
f"remove the principal from one of `grants` or `denies` to avoid "
f"contradictory permissions."
)

return resolved, warnings, unsupported


def deny_changes(
existing_denies: Sequence[Dict[str, str]],
desired: Dict[str, Sequence[str]],
) -> Dict[str, List[Tuple[str, str]]]:
"""Diff the desired deny map against current ``sys.database_permissions``.

``existing_denies`` is a sequence of ``{"grantee", "privilege_type"}`` rows
(state ``DENY``, class 1, minor_id 0). ``desired`` is the resolved
``{privilege: [principals]}`` map.

Returns lists of ``(privilege, principal)`` pairs keyed:

* ``denies`` – configured but not yet present (emit ``DENY``), keeping the
config's spelling;
* ``revokes`` – present but no longer configured (emit ``REVOKE``, which
removes a DENY as well as a GRANT), keeping the database's spelling.

Both dimensions are compared case-insensitively on ``(privilege, principal)``.
A converged relation returns two empty lists.
"""
existing_pairs: Dict[Tuple[str, str], Tuple[str, str]] = {}
for row in existing_denies:
privilege = row["privilege_type"]
principal = row["grantee"]
existing_pairs[(_normalize(privilege), _normalize(principal))] = (privilege, principal)

desired_pairs: Dict[Tuple[str, str], Tuple[str, str]] = {}
for privilege, principals in (desired or {}).items():
for principal in principals:
desired_pairs[(_normalize(privilege), _normalize(principal))] = (privilege, principal)

denies = [pair for key, pair in desired_pairs.items() if key not in existing_pairs]
revokes = [pair for key, pair in existing_pairs.items() if key not in desired_pairs]

return {"denies": denies, "revokes": revokes}
Loading