[core][flink] Detect overwrite conflicts for index commits

[leaves12138]

Purpose

Prevent stale global index commits after later OVERWRITE snapshots change row-id/index safety.

Changes

• Add index commit overwrite conflict checking via commit.overwrite-conflict-with-index.last-safe-snapshot.
• Mark row-id reassignment OVERWRITE snapshots with transient overwrite-barrier.
• Abort index commits on later overwrite barrier snapshots or overlapping ordinary overwrite snapshots.

Validation

• git diff --check
• mvn -pl paimon-api,paimon-docs -Pfast-build -DfailIfNoTests=false -Dtest=ConfigOptionsDocsCompletenessITCase#testCompleteness test

[JingsongLi]

Can we just reuse setRowIdCheckFromSnapshot?

[leaves12138]

Thanks for the suggestion. I think these two snapshot markers should stay separate because they guard different invariants. rowIdCheckFromSnapshot is used by row-id range / column-level conflict checks, while this one is driven by commit.row-id-reassign.last-safe-snapshot and scans later OVERWRITE snapshots for row-id reassign or overlapped partition overwrite conflicts during index build/compact commits. Reusing the existing field would couple two independent checks and make the COMPACT/index path harder to reason about.

[jerry-024]

Overall LGTM. Three minor optimization suggestions below — none are blockers.

[jerry-024]

nit: Can be simplified to:

Long rowIdReassignCheckFromSnapshot = indexBuilder.scanSnapshotId().orElse(null);

Optional.orElse(null) is idiomatic and avoids the double method call.

[jerry-024]

nit: This method is identical to BTreeIndexTopoBuilder.withRowIdReassignCheck. Consider extracting to a shared utility to avoid drift if the option key or logic changes later. Not blocking — the method is small.

[jerry-024]

suggestion: Consider adding a test that verifies compactManifestOnce also strips ROW_ID_REASSIGN_PROPERTY. Current tests cover replaceManifestList but not the compact path.

[JingsongLi]

Reviewed commit 36f6f1a. The overwrite-conflict check looks scoped to index commits via the scan snapshot from the index builders, the overwrite barrier is treated as a one-shot marker and stripped from derived snapshots, and the added tests cover barrier, partition-overlap, non-overwrite, and data-only cases. I didn't find blocking issues from my side.

[JingsongLi]

I cannot get why introducing OVERWRITE_BARRIER_PROPERTY. Can you explain more?

[JingsongLi]

After discussing, you should only need to check whether the RowID data exists