`Decimal` Add/Sub kernels by mhk197 · Pull Request #8724 · vortex-data/vortex

mhk197 · 2026-07-11T00:59:06Z

Adds shared infrastructure for decimal arithmetic and native Add/Sub kernels.

Operand alignment

The execution kernel requires both operands to have the same precision and scale (nullability may differ). Addition and subtraction can then operate directly and exactly on the unscaled integers because both values use the same power-of-ten factor.

Vortex's optional coercion pass can align operands with different decimal dtypes before execution. Query-engine integrations may instead insert their own planner casts. If neither layer aligns the operands, the kernel rejects them rather than performing an implicit per-kernel rescale.

Result precision

Adding or subtracting two p-digit integers can require one additional carry digit. This PR follows the Arrow/Hive result-type rule for already aligned operands, capped at Vortex's maximum decimal precision:

decimal(p, s) +/- decimal(p, s) -> decimal(min(p + 1, 76), s)

The scale remains unchanged because the operation is performed on integers at the same scale. For example:

decimal(2, 0): 60 + 60 = 120 -> decimal(3, 0)
decimal(5, 2): 123.45 + 0.55 = 124.00 -> decimal(6, 2)

Widening the logical precision can also widen the native storage used for the result. The kernel executes at the smallest native width capable of representing the result dtype, including the important decimal(18, s) -> decimal(19, s) (i64 -> i128) and decimal(38, s) -> decimal(39, s) (i128 -> i256) transitions. Inputs may already use a wider physical representation; they are converted into the selected working width before arithmetic.

At precision 76 the result dtype cannot widen to precision 77, so it stays at precision 76 and the kernel explicitly checks the mathematical result against the precision-76 bound. Let M = 10^76 - 1:

Nulls, constants, and errors

Output nullability is the union of the input nullabilities, while a lane is valid only when both input lanes are valid.
Validity is combined and materialized once. Arithmetic runs only on valid lanes, so arbitrary values underneath null lanes cannot trigger an overflow error.
Any native-width overflow or result-precision overflow on a valid lane returns an error for the operation.
Constant and all-null operands are handled directly instead of first materializing full decimal buffers. Constant folding is preserved, and empty Add/Sub inputs produce a correctly widened empty dtype without evaluating a constant value.
Decimal Mul/Div remain unsupported because they require separate result-type and rescaling rules.

Implementation

Splits numeric execution into numeric/{mod,primitive,decimal,tests}.rs, mirroring the compare implementation.
Canonicalizes encoded decimal operands through execute::<DecimalArray>, while extracting constant and null operands directly.
Uses a checked lane loop at the result's working width, followed by an explicit precision-bound check.
Shares widened_buffer from arrays/decimal between comparison and numeric execution.

Coverage

The conformance suite exercises Decimal, Chunked, and DecimalByteParts arrays, including:

Add/Sub with array and constant operands.
Carry-digit widening and native-width boundary transitions.
Mixed physical storage widths.
Nullable overflow cases, verifying that invalid lanes are skipped.
Empty and all-null inputs.
Precision-76 success and overflow boundaries.

binary_ops also includes non-null i64 and nullable i128 decimal Add benchmarks.

Mechanical move mirroring the compare split: dtype-generic lane machinery stays in numeric/mod.rs (bounds relaxed to T: Default so wider decimal types can reuse it), primitive-specific code moves to numeric/primitive.rs, tests to numeric/tests.rs. Signed-off-by: Matt Katz <mhkatz97@gmail.com>

Add and Sub over decimal arrays sharing a decimal dtype, applied to the unscaled stored integers (exact at a shared scale) in a working width wide enough that in-precision inputs cannot spuriously overflow. Precision violations error only on valid lanes, matching primitive semantics. Mul and Div need rescaling and are gated off until follow-up PRs. widened_buffer moves to arrays/decimal so compare and numeric share it. Signed-off-by: Matt Katz <mhkatz97@gmail.com>

The conformance harness now accepts decimal arrays, checking Add/Sub results element-wise against DecimalScalar::checked_binary_numeric for representative constants. Wire it into the decimal, chunked, and decimal-byte-parts compute tests, and add decimal Add cases to the binary_ops benchmark. Signed-off-by: Matt Katz <mhkatz97@gmail.com>

codspeed-hq · 2026-07-11T01:03:00Z

Merging this PR will not alter performance

✅ 1660 untouched benchmarks
🆕 2 new benchmarks
⏩ 47 skipped benchmarks¹

Performance Changes

	Mode	Benchmark	`BASE`	`HEAD`	Efficiency
🆕	Simulation	`add_decimal_i128_nullable`	N/A	3.5 ms	N/A
🆕	Simulation	`add_decimal_i64_nonnull`	N/A	1.9 ms	N/A

_{Comparing mk/decimal-add-sub (3a56ba9) with develop (6b27c45)}

47 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

Signed-off-by: Matt Katz <mhkatz97@gmail.com>

joseph-isaacs · 2026-07-16T11:05:43Z

 pub struct Binary;

+/// Derive the result type for Add/Sub over operands that already share a decimal dtype.
+pub(crate) fn decimal_add_sub_result_dtype(input: DecimalDType) -> DecimalDType {


please can you explain why this is large enough?

This follows arrow rules. The sum of two positive/negative operands will have max 1 extra digit from a carry. Either that or we've crossed max precision, in which case we error

Will put in comment

joseph-isaacs · 2026-07-16T11:06:26Z

+    let valid_rows = validity.execute_mask(len, ctx)?;
+
+    match DecimalType::smallest_decimal_value_type(&result_decimal_dtype) {
+        DecimalType::I8 => execute_decimal_at_widths::<i8, i8>(


use the for each type macro?

joseph-isaacs · 2026-07-16T11:07:56Z

+enum DecimalOperand {
+    Array {
+        values: DecimalArray,
+        validity: Validity,
+    },
+    Constant {
+        value: DecimalValue,
+        len: usize,
+        validity: Validity,
+    },
+    Null(usize),
+}
+
+impl DecimalOperand {
+    fn try_new(array: &ArrayRef, ctx: &mut ExecutionCtx) -> VortexResult<Self> {
+        if let Some(constant) = array.as_opt::<Constant>() {
+            return Ok(match constant.scalar().as_decimal().decimal_value() {
+                Some(value) => Self::Constant {
+                    value,
+                    len: array.len(),
+                    validity: if constant.scalar().dtype().is_nullable() {
+                        Validity::AllValid
+                    } else {
+                        Validity::NonNullable
+                    },
+                },
+                None => Self::Null(array.len()),
+            });
+        }
+
+        let values = array.clone().execute::<DecimalArray>(ctx)?;
+        let validity = values.validity()?;


why add this type, do you think we could instead have a type columnar type elsewhere?

like a general TypedColumnar<CanonicalVTable>?

The Operand pattern is in numeric/primitive.rs as well so was just matching that

joseph-isaacs · 2026-07-16T11:08:45Z

+/// Per-execution constants for checked decimal lane operations at working width `W`.
+struct DecimalOpPlan<W> {
+    /// Inclusive stored-value bounds implied by the result precision.
+    prec_min: W,
+    prec_max: W,
+}
+
+impl<W: NativeDecimalType> DecimalOpPlan<W> {
+    fn new(dtype: DecimalDType) -> Self {
+        let precision = dtype.precision() as usize;
+        Self {
+            prec_min: W::MIN_BY_PRECISION[precision],
+            prec_max: W::MAX_BY_PRECISION[precision],
+        }
+    }
+
+    /// Bounds-check a candidate result against the result precision.
+    #[inline(always)]
+    fn in_precision(&self, value: W) -> Option<W> {
+        (self.prec_min <= value && value <= self.prec_max).then_some(value)
+    }
+}
+
+/// A checked fixed-point decimal operation on unscaled values at working width `W`.
+trait CheckedDecimalOp {
+    const ERROR: &'static str;
+
+    fn checked<W>(lhs: W, rhs: W, plan: &DecimalOpPlan<W>) -> Option<W>
+    where
+        W: NativeDecimalType + CheckedAdd + CheckedSub;
+}
+
+struct DecimalAdd;
+
+struct DecimalSub;
+
+impl CheckedDecimalOp for DecimalAdd {
+    const ERROR: &'static str = "decimal overflow in checked add";
+
+    #[inline(always)]
+    fn checked<W>(lhs: W, rhs: W, plan: &DecimalOpPlan<W>) -> Option<W>
+    where
+        W: NativeDecimalType + CheckedAdd + CheckedSub,
+    {
+        plan.in_precision(lhs.checked_add(&rhs)?)
+    }
+}
+
+impl CheckedDecimalOp for DecimalSub {
+    const ERROR: &'static str = "decimal overflow in checked sub";
+
+    #[inline(always)]
+    fn checked<W>(lhs: W, rhs: W, plan: &DecimalOpPlan<W>) -> Option<W>
+    where


why does this do?

checked_add/sub may not overflow, but may exceed max/min precision

joseph-isaacs · 2026-07-16T11:09:33Z

+    match op {
+        NumericOperator::Add => execute_decimal_typed::<W, O, DecimalAdd>(
+            lhs,
+            rhs,
+            result_decimal_dtype,
+            result_dtype,
+            validity,
+            valid_rows,
+        ),
+        NumericOperator::Sub => execute_decimal_typed::<W, O, DecimalSub>(
+            lhs,
+            rhs,
+            result_decimal_dtype,
+            result_dtype,
+            validity,
+            valid_rows,
+        ),
+        NumericOperator::Mul | NumericOperator::Div => vortex_bail!(
+            "numeric operator {:?} is not yet supported for decimal arrays",
+            op
+        ),


could macro this?

joseph-isaacs · 2026-07-16T11:11:35Z

+        len: usize,
+        validity: Validity,
+    },
+    Null(usize),


you can bail early here?

mhk197 added 3 commits July 10, 2026 17:54

This was referenced Jul 11, 2026

Add native decimal arithmetic kernels #8722

Closed

Fix decimal scalar Mul/Div to rescale at the shared scale #8721

Closed

mhk197 added changelog/feature A new feature changelog/performance A performance improvement labels Jul 14, 2026

mhk197 added 2 commits July 14, 2026 15:53

Improve decimal arithmetic execution

1345b5d

Signed-off-by: Matt Katz <mhkatz97@gmail.com>

Match Arrow decimal Add/Sub result typing

3a56ba9

Signed-off-by: Matt Katz <mhkatz97@gmail.com>

mhk197 marked this pull request as ready for review July 15, 2026 16:53

mhk197 changed the title ~~Add native decimal Add/Sub kernels~~ Decimal Add/Sub kernels Jul 15, 2026

mhk197 removed the changelog/performance A performance improvement label Jul 15, 2026

joseph-isaacs reviewed Jul 16, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

`Decimal` Add/Sub kernels#8724

`Decimal` Add/Sub kernels#8724
mhk197 wants to merge 5 commits into
developfrom
mk/decimal-add-sub

mhk197 commented Jul 11, 2026 •

edited

Loading

Uh oh!

codspeed-hq Bot commented Jul 11, 2026 •

edited

Loading

Uh oh!

joseph-isaacs Jul 16, 2026

Uh oh!

mhk197 Jul 16, 2026

Uh oh!

mhk197 Jul 16, 2026

Uh oh!

joseph-isaacs Jul 16, 2026

Uh oh!

joseph-isaacs Jul 16, 2026

Uh oh!

mhk197 Jul 16, 2026

Uh oh!

mhk197 Jul 16, 2026

Uh oh!

joseph-isaacs Jul 16, 2026

Uh oh!

mhk197 Jul 16, 2026

Uh oh!

joseph-isaacs Jul 16, 2026

Uh oh!

joseph-isaacs Jul 16, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

mhk197 commented Jul 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Operand alignment

Result precision

Nulls, constants, and errors

Implementation

Coverage

Uh oh!

codspeed-hq Bot commented Jul 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will not alter performance

Performance Changes

Footnotes

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mhk197 commented Jul 11, 2026 •

edited

Loading

codspeed-hq Bot commented Jul 11, 2026 •

edited

Loading