feat: add catalog_database support for Unity catalogs (catalogs.yml v2)#1590
Merged
Conversation
Route models to an explicit physical Unity catalog via catalog_database, decoupled from the dbt catalog label. Mirrors the Fusion (fs) implementation: generate_database_name returns catalog_relation.catalog_database first, taking precedence over the model database config and catalog_name. - DatabricksCatalogRelation: add catalog_database field - UnityCatalogIntegration: carry catalog_database onto the relation - databricks__generate_database_name: prefer catalog_database - bump dbt-adapters lower bound to 1.24.5 (base catalog_database support) - unit tests + changelog
…e catalog_database)
Renders the jinja macro to verify catalog_database > model database > catalog_name > target.database (the precedence added for catalog_database).
…ogs path) Manual testing (dbt parse + manifest) showed catalog_database was ignored: on the dbt-core versions dbt-databricks runs on (1.11.x), catalogs.yml write- integration extras arrive via adapter_properties, and CatalogWriteIntegrationConfig has no catalog_database field, so config.catalog_database was always None. Read it from adapter_properties (like location_root/use_uniform), falling back to the base attribute for newer dbt-core. Verified a model now routes to catalog_database over its own database config and catalog_name.
…er_properties catalog_database is a v2 catalogs feature; the base CatalogIntegration surfaces it on the config from the catalogs.yml config block. Read config.catalog_database (via getattr for version-safety) rather than adapter_properties. On dbt-core without v2 catalog support it resolves to None (no-op), which is expected for a v2 feature.
…lly on <1.24.4 Revert pyproject/requirements.lowest-direct/uv.lock back to dbt-adapters>=1.22.0. No runtime guard needed: UnityCatalogIntegration reads getattr(config, 'catalog_database', None) and the relation always carries the field, so on dbt-adapters <1.24.4 (no catalog_database on the config) it resolves to None and catalog_database is a graceful no-op. Add a regression test with a config lacking the attribute, and note the 1.24.4 requirement in the CHANGELOG instead of pinning.
…=1.24.4 (v2 catalogs)
sd-db
changed the title
sd-db
reviewed
Jul 16, 2026
| @@ -19,9 +19,20 @@ class _Config: | |||
| table_format: Optional[str] = "iceberg" | |||
Collaborator
There was a problem hiding this comment.
Note for later to add functional tests once support for dbt-core v1.12 is added in the adapter
sd-db
approved these changes
Jul 17, 2026
sd-db
left a comment
sd-db
left a comment
Collaborator
There was a problem hiding this comment.
Changes look good, thx for the PR !!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Resolves #
Description
Adds
catalog_databasesupport for Unity catalogs in catalogs.yml v2, so a model can be routed to an explicit physical Unity catalog independent of the dbt catalog label (name). This mirrors the Fusion implementation (dbt-labs/fs#11436):databricks__generate_database_namenow returnscatalog_relation.catalog_databasefirst, taking precedence over the modeldatabaseconfig andcatalog_name.Precedence (matches Fusion):
catalog_database> modeldatabaseconfig >catalog_name>target.database.Config (v2 catalogs.yml):
Changes
DatabricksCatalogRelation: addcatalog_databasefield.UnityCatalogIntegration: read the first-classcatalog_databasefield (surfaced on the config by the baseCatalogIntegrationfrom the v2 catalogs.yml config block) and carry it onto the relation. Hive-metastore is unaffected (no catalog layer).databricks__generate_database_name: prefercatalog_database, mirroring fs.catalog_databaseis read viagetattr(config, "catalog_database", None)and the relation always carries the field, so on dbt-adapters<1.24.4(which has nocatalog_databaseon the catalog config) it resolves toNone— a graceful no-op, no crash. It's honored ondbt-adapters>=1.24.4.No behavior change for existing projects that don't set
catalog_database.Requirements:
catalog_databasetakes effect only under v2 catalogs — needs dbt-core ≥ 1.12 (use_catalogs_v2, which triggersbridge_v2_catalog) and dbt-adapters ≥ 1.24.4 (bridge surfacescatalog_databaseonto the catalog config). Below that it resolves toNone(graceful no-op). Note: dbt-core's ownCatalogWriteIntegrationConfigdoes not carrycatalog_databaseeven on 1.13.0a1, so the value flows only through the dbt-adapters v2 bridge — the v1write_integrationspath can't carry it.pyproject.tomlcurrently pinsdbt-core <1.11.13, which excludes 1.12 — so the feature can't be exercised with the adapter's current pin. Merging as-is lands it dormant/forward-looking; activating it needs a dbt-core pin bump to allow ≥1.12 (separate/maintainer decision).Testing: unit-tested — the
catalog_databaseconfig→integration→relation flow (test_catalogs_v2.py, incl. a graceful-degradation case for adapters without the field) and the macro precedence via a rendereddatabricks__generate_database_name(test_generate_database_name.py:catalog_database> modeldatabase>catalog_name>target.database). End-to-enddbt parseon a v2-capable dbt-core is not run here (the pin caps below 1.12).Checklist
CHANGELOG.mdand added information about my change to the "dbt-databricks next" section./dbt-databricks-pr-ready(AI agent skill in.claude/skills/) and addressed its merge-readiness feedback