Glue DataLakeCatalog: dropping an Iceberg sort-order column makes the table unreadable

[alsugiliazova]

Dropping an Iceberg sort-order source column through a Glue-backed DataLakeCatalog makes the table unreadable.

Describe the situation

ALTER TABLE ... DROP COLUMN succeeds when the column is referenced by the current Iceberg sort order and the table is accessed through a Glue-backed DataLakeCatalog. The next SELECT from the table fails with UNKNOWN_IDENTIFIER for a non-existent column named ASC.

The same operation through the REST-backed DataLakeCatalog succeeds safely: ClickHouse removes the column, resets the current sort order to unsorted, preserves the old sort order historically, and continues to read existing data.

This issue:

• Is specific to the Glue-backed DataLakeCatalog path in the tested configuration.
• Commits the schema change successfully before the read failure is detected.
• Leaves the Iceberg table unreadable through ClickHouse immediately after the alteration.
• May leave the current schema and sort-order metadata incompatible.

---

How to reproduce the behavior

Environment

• Version: 26.3.13.20001.altinityantalya
• Build type: from pr Fixed iceberg alter table modify rename #1841 https://altinity-build-artifacts.s3.amazonaws.com/PRs/1841/78d272e18d7641866ac4f4ded573708cf005a3f9/build_arm_release/clickhouse-common-static_26.3.13.20001.altinityantalya_arm64.deb
• Catalog: Glue through LocalStack
• Object storage: rustfs
• Iceberg format version: 2

Initial Iceberg table

Create an Iceberg table in the Glue catalog with:

• name String, field ID 1
• double Float64, field ID 2
• integer Int64, field ID 3
• Identity partitioning on name
• Ascending identity sort order on double

Insert one row:

Alice, 195.23, 20

Steps

1. Create a Glue-backed DataLakeCatalog database:

SET allow_experimental_database_glue_catalog = 1;

CREATE DATABASE datalake_db
ENGINE = DataLakeCatalog('http://localstack:4566')
SETTINGS
    catalog_type = 'glue',
    storage_endpoint = 'http://minio:9000/warehouse',
    region = 'us-east-1',
    aws_access_key_id = '<access-key>',
    aws_secret_access_key = '<secret-key>';

2. Drop the current sort-order source column:

SET allow_insert_into_iceberg = 1;

ALTER TABLE datalake_db.`namespace.table`
DROP COLUMN double;

The command succeeds with exit code 0.

3. Read the table:

SELECT *
FROM datalake_db.`namespace.table`;

---

Expected behavior

ClickHouse should use one of the following safe behaviors:

1. Atomically reset the current sort order to unsorted and drop the column, matching the REST catalog behavior; or
2. Reject DROP COLUMN before writing metadata because the column is referenced by the active sort order.

The operation must not commit metadata that makes the table unreadable.

After the REST catalog performs the operation successfully:

• The current schema contains name and integer.
• default-sort-order-id is 0.
• The current sort order is unsorted.
• The old sort order remains available in historical metadata.
• SELECT * returns:

Alice	20

---

Actual behavior

The Glue-backed alteration succeeds, but the subsequent query fails:

Received exception from server (version 26.3.13):
Code: 47. DB::Exception: Received from localhost:9000.
DB::Exception: Missing columns: 'ASC' while processing: 'ASC',
required columns: 'ASC',
available columns: 'name.null' 'integer.null' 'integer' 'name.size' 'name'.
(UNKNOWN_IDENTIFIER)