feat: enable Delta Lake catalog attachment by default#553
Merged
Conversation
Flips delta_catalog_enabled to true everywhere: CLI/env/YAML defaults in DefaultServerConfig and the duckgres-worker flag, plus column-level gorm:"default:true" on the configstore ManagedWarehouseS3 and DuckLakeConfig fields. A one-shot migration tracked in duckgres_schema_migrations backfills existing configstore rows so upgraded clusters pick up the new default without admin intervention; admins can still opt out per-warehouse via the admin API afterward without being re-flipped on subsequent restarts. To keep plain standalone deployments (no DuckLake metadata_store / object_store / data_path) bootable under the new default, AttachDeltaCatalog now silently no-ops when no catalog path is derivable instead of erroring. The catalog is still attached as 'delta' (ATTACH '...' AS delta (TYPE delta), unchanged). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Under the always-on Delta default, a fresh DuckLake tenant has no Delta data at the sibling delta/ prefix yet, and ATTACH returns "No files in log segment" from DeltaKernel. That's the expected state, not a failure — log it and continue without the catalog attached. Fixes the k8s tenant isolation test where billing's worker activation rejected client queries before the tenant ever wrote Delta data. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ATTACH '...' (TYPE delta) is lazy — it succeeds against an empty path, then every subsequent query against DuckLake fails at prepare time because the planner walks all attached catalogs and Delta blows up trying to read a missing _delta_log/. Earlier the empty-path check was only at ATTACH time; that wasn't enough because ATTACH never errored. Now we probe with SHOW TABLES FROM delta immediately after ATTACH and DETACH if the catalog is unusable, leaving the connection clean for ducklake-only queries until Delta data lands. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
delta_catalog_enableddefault totrueacross all input layers:DefaultServerConfiginconfigresolve, the--ducklake-delta-catalog-enabledCLI flag in both the all-in-one binary andduckgres-worker, andgorm:"default:true"onManagedWarehouseS3.DeltaCatalogEnabled/DuckLakeConfig.DeltaCatalogEnabledin the configstore.2026_05_delta_catalog_default_enabledtracked in a newduckgres_schema_migrationstable backfills existing configstore rows. Idempotent — admins can later disable per-warehouse via the admin API without being re-flipped on subsequent restarts.AttachDeltaCatalognow silently no-ops when no catalog path is derivable (no DuckLakeobject_store/data_pathand no explicitdelta_catalog_path), so plain standalone DuckDB instances stay bootable under the new default.deltaalready viaATTACH '...' AS delta (TYPE delta)inserver/ducklake/migration.go— no change needed for naming.Test plan
go build ./...go build -tags kubernetes ./...go test ./configresolve/...— added subtests asserting default-on with and without DuckLake configured, and explicit YAML opt-outgo test ./server/ducklake/... ./server/...go test -tags kubernetes ./controlplane/...— covers the new configstore migration pathdelta_catalog_enabled = truedelta catalog path is emptyerror)🤖 Generated with Claude Code