Skip to content

Glue DataLakeCatalog allows dropping an active Iceberg partition column #2084

Description

@alsugiliazova

Problem

ClickHouse allows DROP COLUMN for an active Iceberg partition source column when using a Glue-backed DataLakeCatalog.

The same operation through the REST catalog is correctly rejected:

Code: 736. DB::Exception: Iceberg alter: catalog commit failed ...
(DATALAKE_DATABASE_ERROR)

Tested with https://altinity-build-artifacts.s3.amazonaws.com/PRs/1841/78d272e18d7641866ac4f4ded573708cf005a3f9/build_arm_release/clickhouse-common-static_26.3.13.20001.altinityantalya_arm64.deb

Reproduction

  1. Create an Iceberg v2 table with columns name, double, and integer.
  2. Add an identity partition field sourced from name.
  3. Register the table in a Glue-backed DataLakeCatalog.
  4. Run:
SET allow_insert_into_iceberg = 1;
ALTER TABLE datalake_db.`namespace.table` DROP COLUMN name;

Actual behavior

The Glue-backed alteration succeeds with exit code 0.

The resulting metadata removes field ID 1 (name) from the current schema but leaves the active partition spec referencing source-id: 1. This creates an incompatible schema and partition specification.

Expected behavior

ClickHouse should reject the alteration before writing or committing metadata, matching the REST catalog behavior. The schema, partition specification, and existing data should remain unchanged.

Impact

Glue can accept invalid Iceberg metadata with a dangling partition source reference. This may make partition planning, metadata queries, future writes, or reads fail.

Related upstream PyIceberg fix:

Metadata

Metadata

Assignees

Labels

antalyabugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions