Currently we don't support these, since we have a custom delta Parquet writer (i.e. plan_to_object_store) as a way to mitigate memory performance of writes.
This could be achieved in two ways:
- Ditch our custom writing logic and use delta-rs one, which has support for partitioning columns at the expense of getting sub-optimal memory profiles.
- Implement partitioning into our own writing logic.
Currently we don't support these, since we have a custom delta Parquet writer (i.e.
plan_to_object_store) as a way to mitigate memory performance of writes.This could be achieved in two ways: