isthmus: Substrait→Calcite conversion rejects precision_time / precision_timestamp / precision_timestamp_tz precision > 6

[nielspardon]

Description

Isthmus's Substrait → Calcite conversion rejects any precision_time, precision_timestamp, or precision_timestamp_tz whose precision is greater than 6, even though the Substrait spec allows sub-second precision from 0 to 12 (0 = seconds, 3 = milliseconds, 6 = microseconds, 9 = nanoseconds, 12 = picoseconds — see type.proto).

Converting a plan that uses e.g. nanosecond precision fails with:

java.lang.IllegalArgumentException: unsupported precision_timestamp precision 9, max precision in Calcite type system is set to 6

The cap comes from SubstraitTypeSystem.getMaxPrecision, which returns 6 for TIME, TIMESTAMP, and TIMESTAMP_WITH_LOCAL_TIME_ZONE. The precision is then validated against that maximum in two places:

• TypeConverter.ToRelDataType#visit(Type.PrecisionTime | PrecisionTimestamp | PrecisionTimestampTZ)
• ExpressionRexConverter (the corresponding literal-conversion paths)

The Substrait POJO model itself does not cap the precision, so such a plan is perfectly representable in substrait-java and can be produced by other engines; only the conversion to Calcite fails.

Relationship to #93

This is the reverse direction of #93. That issue reported that the Calcite → Substrait direction rejected valid TIME(6)/TIMESTAMP(6) inputs. It was resolved by the migration to PrecisionTime/PrecisionTimestamp (#721) and the removal of the deprecated Time/Timestamp/TimestampTZ types (#950): the Calcite → Substrait direction now passes the precision straight through without a cap. The Substrait → Calcite direction still caps at 6, which this issue tracks.

Notes / open questions

• Current Calcite dependency is 1.42.0. It may be possible to raise SubstraitTypeSystem.getMaxPrecision for TIMESTAMP/TIME beyond 6 (worth checking what this Calcite version actually supports).
• Calcite very likely cannot represent picosecond (12) precision, so full 0–12 support may not be achievable. At minimum the supported range and the failure behavior should be documented, and the error should make clear this is a Calcite-side limitation rather than a Substrait one.

🤖 Generated with AI