Skip to content

Spark 4.2: UnionCodegenSuite partitioning-aware test inspects UnionExec that Comet replaces #4965

Description

@andygrove

Spark 4.2's UnionCodegenSuite (SPARK-56482) test "partitioning-aware union falls back to non-codegen" inspects Spark's UnionExec whole-stage-codegen internals via executedPlan.collectFirst { case u: UnionExec => u }.get.

Comet replaces UnionExec with CometUnionExec, so collectFirst returns None and the test fails with NoSuchElementException: None.get.

Currently marked IgnoreComet in dev/diffs/4.2.0.diff (WholeStageCodegenSuite is already IgnoreCometSuite).

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpriority:lowMinor issues, test failures, tooling, cosmeticspark 4.2spark sql testsSpark SQL test failures

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions