Skip to content

GH-3679: Allow TimestampType to annotate FLBA(12)#3680

Open
divjotarora wants to merge 2 commits into
apache:masterfrom
divjotarora:flba12-ts
Open

GH-3679: Allow TimestampType to annotate FLBA(12)#3680
divjotarora wants to merge 2 commits into
apache:masterfrom
divjotarora:flba12-ts

Conversation

@divjotarora

Copy link
Copy Markdown
Contributor

Rationale for this change

See parquet-format issue apache/parquet-format#600 and the linked proposal document for rationale

What changes are included in this PR?

This PR implements support for the TimestampType logical type annotation on FIXED_LEN_BYTE_ARRAY physical type values with type_length=12 (96 bits).

Are these changes tested?

Yes, several test files are updated along with the source.

Are there any user-facing changes?

Users will be able to declare schemas with this physical/logical type combination. No API changes.

Closes #3679

@stevomitric stevomitric left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the converted_type be suppressed for the FLBA(12) carrier, the same way NANOS already returns empty? Those legacy types are defined as INT64-only, so emitting them here gives a contradictory footer.

Comment thread parquet-column/src/main/java/org/apache/parquet/schema/Types.java Outdated
Comment thread parquet-column/src/main/java/org/apache/parquet/schema/PrimitiveComparator.java Outdated

@divjotarora divjotarora left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the converted_type be suppressed for the FLBA(12) carrier, the same way NANOS already returns empty? Those legacy types are defined as INT64-only, so emitting them here gives a contradictory footer.

Thanks for catching @stevomitric, the existing tests didn't catch this because they were focusing on using FLBA(12) with NANOS and it doesn't have a converted type already. The code is now fixed and there are more tests to validate FLBA(12) with MILLIS and MICROS as well.

Comment thread parquet-column/src/main/java/org/apache/parquet/schema/PrimitiveComparator.java Outdated
Comment thread parquet-column/src/main/java/org/apache/parquet/schema/Types.java Outdated
@divjotarora
divjotarora requested a review from emkornfield July 23, 2026 12:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support extended precision for nanosecond timestamps

3 participants