Skip to content

fix(spec): validate sequence.field against table schema#615

Open
jackylee-ch wants to merge 1 commit into
apache:mainfrom
jackylee-ch:fix/validate-sequence-field
Open

fix(spec): validate sequence.field against table schema#615
jackylee-ch wants to merge 1 commit into
apache:mainfrom
jackylee-ch:fix/validate-sequence-field

Conversation

@jackylee-ch

Copy link
Copy Markdown

Purpose

Creating a table with sequence.field referencing a column that does not exist was silently accepted. The write path then resolved sequence fields with a lenient lookup and fell back to the auto-increment sequence, so merge results ignored the user's ordering intent instead of failing loud. Repeated fields and merge-engine=first-row were likewise unchecked.

Brief change log

  • Add Schema::validate_sequence_field, mirroring Java SchemaValidation#validateSequenceField: every listed field must exist in the schema, must not be listed more than once, and merge-engine=first-row rejects user-defined sequence fields.
  • Wire it into both the create path (Schema::new) and the alter path (TableSchema::apply_changes).

Tests

  • test_create_schema_rejects_unknown_sequence_field, test_create_schema_rejects_repeated_sequence_field, test_create_schema_rejects_sequence_field_with_first_row, test_alter_set_unknown_sequence_field_rejected.
  • cargo test -p paimon --lib, cargo clippy -p paimon --all-targets -- -D warnings, cargo fmt --all -- --check pass.

API and Format

No API or storage format change; only rejects schemas that already violate the sequence.field contract.

Documentation

No documentation change required.

@JingsongLi

Copy link
Copy Markdown
Contributor

Claims to be aligned with Java’s validateSequenceField, but omits validation for cross-partition updates. sequence.field can still be configured even when not all partition keys are included in the primary key, whereas Java explicitly rejects this combination. Both the create and alter paths should include partition keys and primary keys in the validation.

@jackylee-ch
jackylee-ch force-pushed the fix/validate-sequence-field branch 2 times, most recently from f32bd08 to 23bc86f Compare July 27, 2026 02:46
Creating a table with `sequence.field` referencing a missing column was
silently accepted, then the write path resolved sequence fields with a
lenient lookup and fell back to the auto-increment sequence, ignoring the
user's ordering intent. Repeated fields, `merge-engine=first-row` and
cross-partition update tables were likewise unchecked.

Validate `sequence.field` at create and alter time, mirroring Java
`SchemaValidation#validateSequenceField`: every listed field must exist,
must not repeat, first-row merge engine rejects user-defined sequence
fields, and cross-partition update tables (primary key constraint not
including all partition fields) reject `sequence.field` because partition
migration retracts old rows with generated DELETEs whose ordering a
user-provided sequence value could break.
@jackylee-ch
jackylee-ch force-pushed the fix/validate-sequence-field branch from 23bc86f to 3f063d0 Compare July 27, 2026 02:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants