GH-3679: Allow TimestampType to annotate FLBA(12)#3680
Conversation
stevomitric
left a comment
There was a problem hiding this comment.
Should the converted_type be suppressed for the FLBA(12) carrier, the same way NANOS already returns empty? Those legacy types are defined as INT64-only, so emitting them here gives a contradictory footer.
divjotarora
left a comment
There was a problem hiding this comment.
Should the
converted_typebe suppressed for the FLBA(12) carrier, the same way NANOS already returns empty? Those legacy types are defined as INT64-only, so emitting them here gives a contradictory footer.
Thanks for catching @stevomitric, the existing tests didn't catch this because they were focusing on using FLBA(12) with NANOS and it doesn't have a converted type already. The code is now fixed and there are more tests to validate FLBA(12) with MILLIS and MICROS as well.
Rationale for this change
See parquet-format issue apache/parquet-format#600 and the linked proposal document for rationale
What changes are included in this PR?
This PR implements support for the
TimestampTypelogical type annotation onFIXED_LEN_BYTE_ARRAYphysical type values withtype_length=12(96 bits).Are these changes tested?
Yes, several test files are updated along with the source.
Are there any user-facing changes?
Users will be able to declare schemas with this physical/logical type combination. No API changes.
Closes #3679