Skip to content

fix(dataset): fail-fast casting for columns with attached indices#7158

Open
WenDing-Y wants to merge 3 commits into
lance-format:mainfrom
WenDing-Y:fix/cast-with-index-fail-fast
Open

fix(dataset): fail-fast casting for columns with attached indices#7158
WenDing-Y wants to merge 3 commits into
lance-format:mainfrom
WenDing-Y:fix/cast-with-index-fail-fast

Conversation

@WenDing-Y
Copy link
Copy Markdown
Contributor

Summary

Currently, calling alter_columns with a cast on a column that has an attached
index silently drops the index without any warning or error. This has caused
production incidents where vector search silently regressed to brute-force scan.

This PR adds a fail-fast check before any cast is performed: if a target
column has any index attached, the operation is rejected with a clear error
message that names the affected column(s), the index(es), and the required
remediation (drop_index() → cast → rebuild index).

Changes

  • Added index-awareness check in alter_columns that scans cast target columns
    for attached indices
  • Returns a descriptive Error::invalid_input with column names, index names,
    and remediation steps when a conflict is detected
  • Updated existing tests that previously relied on silent index-drop behavior
    to explicitly drop_index() before casting
  • Added new test test_alter_columns_cast_fails_with_attached_index covering
    the IVF_PQ index path: verifies the error message, schema integrity, index
    preservation, and that the cast succeeds after a manual drop

Breaking Change ⚠️

Callers that previously depended on the silent index-drop behavior must now
explicitly call drop_index() before alter_columns with a cast, then rebuild
the index afterward.

Implement fail-fast behavior for casting columns with attached indices to
prevent silent index drops. Update tests to ensure proper error messages and
dataset integrity when attempting to cast indexed columns.
Copy link
Copy Markdown

@claude claude Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@github-actions github-actions Bot added the bug Something isn't working label Jun 8, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented Jun 8, 2026

Codecov Report

❌ Patch coverage is 96.42857% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
rust/lance/src/dataset/schema_evolution.rs 96.42% 0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant