Skip to content

Support declared output partitioning for file/listing scans #22645

@gene-bordegaray

Description

@gene-bordegaray

Is your feature request related to a problem or challenge?

range_partitioning.slt currently needs a custom TableProvider / DataSource to declare that a scan has known Range output partitioning. This incurred a lot of test-only boilerplate, and there is no general external file/listing table API to declare scan output partitioning.

Describe the solution you'd like

Add a general output_partitioning: Option<Partitioning> API for file scans and listing tables:

  • Add FileScanConfig::with_output_partitioning, similar to with_output_ordering
  • Update range_partitioning.slt as first use case by registering a listing table with output_partitioning instead of a custom provider

Describe alternatives you've considered

Keep the current custom range-partitioned sqllogictest or add a one-off MemorySourceConfig output-partitioning builder. Both seem like bandaids and don't provide an external value.

Additional context

Follow-up from #22607.
Relevant discussions: #22607 (comment) #22607 (comment)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions