Skip to content

fix: Handle input_file_name() pushdown into ParquetSource with filter pushdown enabled#23638

Open
AdamGS wants to merge 3 commits into
apache:mainfrom
AdamGS:adamg/fix-23531
Open

fix: Handle input_file_name() pushdown into ParquetSource with filter pushdown enabled#23638
AdamGS wants to merge 3 commits into
apache:mainfrom
AdamGS:adamg/fix-23531

Conversation

@AdamGS

@AdamGS AdamGS commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

input_file_name() is FileSource dependent like file_row_index(), and therefore shouldn't be pushed down into a filter.

What changes are included in this PR?

PushdownChecker now handles both UDFs consistently.

If we keep adding this sort of metadata functions, we might want a better API to detect them, but for now I think this is a reasonable approach that isn't very invasive.

Are these changes tested?

Additional SLT test that verifies that both function behave correctly with pushdown either enabled or disabled.

Are there any user-facing changes?

None

AdamGS added 2 commits July 16, 2026 13:47
Signed-off-by: Adam Gutglick <adamgsal@gmail.com>
Signed-off-by: Adam Gutglick <adamgsal@gmail.com>
@github-actions github-actions Bot added sqllogictest SQL Logic Tests (.slt) datasource Changes to the datasource crate labels Jul 16, 2026
@codecov-commenter

codecov-commenter commented Jul 16, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@95de385). Learn more about missing BASE report.

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #23638   +/-   ##
=======================================
  Coverage        ?   80.65%           
=======================================
  Files           ?     1086           
  Lines           ?   366100           
  Branches        ?   366100           
=======================================
  Hits            ?   295269           
  Misses          ?    53236           
  Partials        ?    17595           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Signed-off-by: Adam Gutglick <adamgsal@gmail.com>
@AdamGS
AdamGS force-pushed the adamg/fix-23531 branch from 47b68a4 to bc4a25f Compare July 16, 2026 15:15
@AdamGS

AdamGS commented Jul 16, 2026

Copy link
Copy Markdown
Contributor Author

I think this is a reasonable fix especially with the target date for the release getting closer.
I'm experimenting with something more powerful - while we can't push it down, maybe we can treat it similarly to table partitions/literal columns, so a predicate like input_file_name() LIKE '%first.parquet' can be used by the FilePruner.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

datasource Changes to the datasource crate sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

input_file_name() in a WHERE clause errors when parquet pushdown_filters is enabled

2 participants