Skip to content

feat: DuckLake Live Connector + Docs#9267

Open
royendo wants to merge 14 commits intomainfrom
worktree-ducklake-ui-support
Open

feat: DuckLake Live Connector + Docs#9267
royendo wants to merge 14 commits intomainfrom
worktree-ducklake-ui-support

Conversation

@royendo
Copy link
Copy Markdown
Contributor

@royendo royendo commented Apr 20, 2026

Adding DuckLake as OLAP Live Connector
This updates the UI flow for DuckLake with advanced config settings or ATTACH SQL
has ducklake icon in data explorer;
Deployment page updates OLAP Engine to DuckLake

Screenshot 2026-04-20 at 17 14 31 Screenshot 2026-04-20 at 17 15 08 Screenshot 2026-04-20 at 17 14 36 Screenshot 2026-04-20 at 17 14 51

Checklist:

  • Covered by tests
  • Ran it and it works as intended
  • Reviewed the diff before requesting a review
  • Checked for unhandled edge cases
  • Linked the issues it closes
  • Checked if the docs need to be updated. If so, create a separate Linear DOCS issue
  • Intend to cherry-pick into the release branch
  • I'm proud of this work!

royendo and others added 13 commits April 20, 2026 14:59
Adds a DuckLake entry to the Add Data connector grid, backed by the DuckDB
driver via `x-driver: "duckdb"`. The schema exposes a single required `attach`
clause (with the full DuckLake ATTACH syntax) plus an advanced readwrite mode
toggle. Including DuckLake in OLAP_ENGINES mirrors MotherDuck's behavior: on
save the new connector is set as the default `olap_connector` in rill.yaml.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…Lake in explorer/deployments

- Plumb schemaName through createConnector so the DuckLake schema (not the duckdb driver schema) is used when compiling the connector YAML, ensuring `attach` is written to ducklake.yaml.
- Detect DuckLake in getConnectorIconKey so the data explorer shows the DuckLake SVG for connectors configured with a `ducklake:` attach clause.
- Detect DuckLake in getOlapEngineLabel so deployments display "DuckLake" for the OLAP engine, and extend formatConnectorName with DuckLake/MotherDuck.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Check projectConfig/presetConfig for the `attach` clause and fall back to
the connector name so the DuckLake SVG still renders when the resolved
config is unavailable (e.g. when the runtime returns only non-resolved
config) or when the attach value differs slightly.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Fall back to connector name when detecting DuckLake on the deployment page so the label still shows "DuckLake (name)" when the merged config is redacted.
- Update the DuckLake form placeholder to the documented ATTACH format and expose advanced options: CREATE_IF_NOT_EXISTS, DATA_INLINING_ROW_LIMIT, DATA_PATH, ENCRYPTED, META_PARAMETER_NAME, METADATA_CATALOG, METADATA_PARAMETERS, METADATA_PATH, METADATA_SCHEMA, AUTOMATIC_MIGRATION, OVERRIDE_DATA_PATH, SNAPSHOT_TIME, SNAPSHOT_VERSION.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Mirrors the ClickHouse Parameters vs. Connection string pattern. The SQL tab keeps the raw ATTACH string plus the write-mode toggle. The Parameters tab exposes the documented DuckLake options (catalog path, alias, DATA_PATH, OVERRIDE_DATA_PATH, and advanced settings like CREATE_IF_NOT_EXISTS, ENCRYPTED, METADATA_*, SNAPSHOT_*). When Parameters is selected, the field values are composed into a single `attach` YAML key before the connector is written.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The tab-group filter strips `attach` when the "parameters" tab is active (since `attach` belongs to the "sql" tab), so the YAML preview stayed empty while the user filled in parameter fields. Add an `injectDuckLakeAttach` helper that restores the composed value after filtering.

Also show the write-mode toggle under advanced on both tabs.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
On the Parameters tab, only `catalog`, `alias`, and `data_path` render
inline. All other fields (mode, override_data_path, metadata_*, etc.)
now live outside the tab group so the form renderer surfaces them under
the "Advanced options" toggle, matching the ClickHouse pattern.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Previously boolean options equal to their DuckLake default were
suppressed and `DATA_INLINING_ROW_LIMIT` only emitted when > 0. That
meant users would open the Advanced toggle, set several fields, and
only see a subset of their selections reflected in the ATTACH clause.

Emit every configured advanced value so the generated clause matches
what the user sees in the form.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Fill in placeholders for `meta_parameter_name`, `metadata_path`,
`metadata_schema`, `snapshot_time`, and `snapshot_version` so every
text field in the Parameters tab shows an example value.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ceholder

- Display "DuckLake" without the connector instance suffix on the
  deployment status page.
- Match DuckLake connector names case-insensitively so "Ducklake_1"
  is still detected when the runtime redacts `config.attach`.
- Fill in a placeholder for `DATA_INLINING_ROW_LIMIT` to match the
  other text fields.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Move `getOlapEngineLabel` to `web-common/features/resources/display-utils.ts`
and re-export from web-admin. Relax the helper so callers that only have a
connector name (like web-local's `instance.olapConnector`) still get DuckLake
and basic type formatting. Web-local's status page now uses the same helper,
so a DuckLake connector displays as "DuckLake" in both Rill Developer and
Rill Cloud.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@royendo royendo requested a review from nishantmonu51 April 20, 2026 20:50
- Detect DuckLake schema by module reference instead of `schema.title`
- Gate name-based DuckLake OLAP detection on duckdb type to avoid
  mislabelling unrelated connectors like `ducklake_meta`
- Drop `default: 0` on `data_inlining_row_limit` so ATTACH only emits
  the option when the user sets a value
- Update web-admin display-utils specs to match current label output

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant