Skip to content

allow interleaveExec to support Range partioning#23623

Merged
gabotechs merged 5 commits into
apache:mainfrom
Rich-T-kid:rich-T-kid/Implement-Range-Partioning-interleaveExec
Jul 16, 2026
Merged

allow interleaveExec to support Range partioning#23623
gabotechs merged 5 commits into
apache:mainfrom
Rich-T-kid:rich-T-kid/Implement-Range-Partioning-interleaveExec

Conversation

@Rich-T-kid

@Rich-T-kid Rich-T-kid commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

see #23455

What changes are included in this PR?

The goal of this PR is to allow range partitioning to propagate through InterleaveExec. Updated can_interleave() to accept Partitioning::Range when all children share an identical RangePartitioning (same ordering and split points), matching the existing behavior for Partitioning::Hash.

Updated range_partitioning.slt to expect InterleaveExec where it previously expected UnionExec, since can_interleave() now accepts Partitioning::Range.

Are these changes tested?

yes, the range_partioning.slt file as well as the union.slt file sql logic test. This PR also includes three test.

Are there any user-facing changes?

physical plans may look different now.

@github-actions github-actions Bot added sqllogictest SQL Logic Tests (.slt) physical-plan Changes to the physical-plan crate labels Jul 15, 2026
@Rich-T-kid Rich-T-kid force-pushed the rich-T-kid/Implement-Range-Partioning-interleaveExec branch from ddf65fb to 8242bfa Compare July 15, 2026 20:45
@Rich-T-kid Rich-T-kid changed the title alloww interleaveExec to support Range partioning allow interleaveExec to support Range partioning Jul 15, 2026

@gene-bordegaray gene-bordegaray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some comments on tests but overall looks good

could we also add smoke test in enforce_distribution.rs for smoke tests (see other related PR)s

thank you 🙇

Comment thread datafusion/physical-plan/src/union.rs Outdated
}

#[test]
fn test_can_interleave_range_matching() -> Result<()> {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you think about making this a general distribution since that seems to be lacking in the file. A matrixs test case like:

  1. matching hash
  2. matching range
  3. subset hash
  4. subset range

then equivalently below we can have the ngative test for distributions that interleave shouldnt accept:

  1. mixing hash and range
  2. incompatible hashes
  3. incompatible ranges
  4. sort and null order variants for ranges
  5. etc

just a thought lmk

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This sounds great. Ill update the PR

----
physical_plan
01)UnionExec
01)InterleaveExec

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we also add:

  1. a sub set test
  2. negative test (fallback to union)
  3. union into another operator test to ensure propagation

@Rich-T-kid

Copy link
Copy Markdown
Contributor Author

subset range

@gene-bordegaray are you referring to a case like
[10,20]
[10,15]

this shouldn't be accepted right? Interleave should fall back to unionExec in this case.

@gene-bordegaray

Copy link
Copy Markdown
Contributor

subset range

@gene-bordegaray are you referring to a case like [10,20] [10,15]

this shouldn't be accepted right? Interleave should fall back to unionExec in this case.

@Rich-T-kid yes exactly 🙇

@Rich-T-kid

Copy link
Copy Markdown
Contributor Author

@gene-bordegaray this is ready for review now

@codecov-commenter

codecov-commenter commented Jul 16, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 72.05882% with 19 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@2e78508). Learn more about missing BASE report.

Files with missing lines Patch % Lines
datafusion/physical-plan/src/union.rs 72.05% 2 Missing and 17 partials ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main   #23623   +/-   ##
=======================================
  Coverage        ?   80.65%           
=======================================
  Files           ?     1086           
  Lines           ?   366165           
  Branches        ?   366165           
=======================================
  Hits            ?   295317           
  Misses          ?    53236           
  Partials        ?    17612           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@gene-bordegaray gene-bordegaray left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

apprved once last things addressed. thakn you 👍

cc: @gabotechs

Comment thread datafusion/physical-plan/src/union.rs
Comment thread datafusion/sqllogictest/test_files/range_partitioning.slt
Comment thread datafusion/physical-plan/src/union.rs
@Rich-T-kid

Copy link
Copy Markdown
Contributor Author

updated the PR to address your comments.
Thanks @gene-bordegaray 🚀

@Rich-T-kid Rich-T-kid force-pushed the rich-T-kid/Implement-Range-Partioning-interleaveExec branch from e2b0786 to 68ea81a Compare July 16, 2026 17:04
@gabotechs

Copy link
Copy Markdown
Contributor

Thanks @Rich-T-kid and @gene-bordegaray, this looks good to me.

@Rich-T-kid Rich-T-kid force-pushed the rich-T-kid/Implement-Range-Partioning-interleaveExec branch from 68ea81a to f6f7a64 Compare July 16, 2026 17:37
@gabotechs gabotechs added this pull request to the merge queue Jul 16, 2026
Merged via the queue into apache:main with commit ac17a7b Jul 16, 2026
40 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

physical-plan Changes to the physical-plan crate sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow InterleaveExec to preserve Partitioning::Range

4 participants