fix: clean up CometCast support-level reporting (#4501)#4595
Merged
Conversation
86e9580 to
d6d1ea4
Compare
Address the cast expression audit follow-ups from apache#4493. Several of the issue's line references were stale; the resolutions reflect current main: - canCastFromFloat/Double/Decimal need no evalMode parameter. The native code branches on eval mode for these narrowing casts but matches Spark in every mode (CometCastSuite passes in LEGACY, ANSI, and TRY), so there is no per-mode support-level divergence to report. No change needed. - Remove the vestigial getIncompatibleReasons/getUnsupportedReasons overrides. For Cast they are never consumed: cast.md is generated directly from the per-pair isSupported matrix and EXPLAIN surfaces the per-pair reason via getSupportLevel. - Fill in the previously empty Incompatible() reason for negative-scale decimal to string, sharing the string with the test via a constant to avoid drift. - Clarify the literal short-circuit comment in getSupportLevel: literal casts are folded by Spark at planning time, so they never execute natively.
d6d1ea4 to
b3d4339
Compare
Member
Author
|
@coderfender could you review? |
coderfender
approved these changes
Jun 4, 2026
Contributor
|
Thank you @andygrove |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Closes #4501.
Rationale for this change
#4501 collected support-level and serde consistency follow-ups deferred from the cast expression audit in #4493. While working through them, several of the issue's line references turned out to be stale against current
main, so the fixes here reflect the current state ofCometCast.scalaandGenerateDocs.scala.Findings per item:
The issue noted that
canCastFromFloat/canCastFromDouble/canCastFromDecimalignoreevalModewhile every other arm ofisSupportedthreads it through, and suggested the support level should reflect the eval mode. Investigation shows there is nothing to reflect: the native code does branch on eval mode for these narrowing casts (ANSI throws on overflow/NaN, LEGACY/TRY wrap), but Comet matches Spark in every mode, andCometCastSuiteexercises them in LEGACY, ANSI, and TRY and passes. So there is no per-mode support-level divergence, and these three helpers (likecanCastFromTimestamp) correctly take noevalMode. No code change needed.For
Cast,getIncompatibleReasons/getUnsupportedReasonsare never consumed.cast.mdis generated directly from the per-pairisSupportedmatrix (GenerateDocs.writeCastMatrixForMode, including per-pair note annotations), and EXPLAIN surfaces the per-pair reason viagetSupportLevel.Castis not incategoryPages, the only caller of those static methods. The generic overrides were dead, so they are removed in favor of the matrix as the single source of truth.The array-of-binary
Incompatible()branch described in the issue no longer exists; binary to string (including arrays) is currentlyCompatible(). The only remaining bareIncompatible()was the negative-scale decimal to string branch, which now carries a concise reason. The separate#4488from_utf8_uncheckedquestion for binary to string is a behavior change and is left to its own issue.The literal short-circuit in
getSupportLevelis correct:convert()folds literal casts viacast.eval()in Spark at planning time, so the cast never executes natively and the result matches Spark by definition. Only the comment was misleading; it is clarified.What changes are included in this PR?
getIncompatibleReasons/getUnsupportedReasonsoverrides fromCometCastand document that the per-pair matrix is the canonical reason source.Incompatible()reason for negative-scale decimal to string, sharing the string with the test via a constant to avoid drift.getSupportLevel.How are these changes tested?
Covered by existing
CometCastSuitecoverage, which exercises the float, double, and decimal cast paths in LEGACY, ANSI, and TRY modes. The negative-scale decimal to string test's support-level assertion is updated to reference the shared reason constant. No behavior changes, so no golden file or generated doc regeneration is required.