Skip to content

sql: fix variance and stddev aggregates with DISTINCT#37647

Open
ggevay wants to merge 1 commit into
MaterializeInc:mainfrom
ggevay:gabor/sql-362-distinct-variance-aggregates
Open

sql: fix variance and stddev aggregates with DISTINCT#37647
ggevay wants to merge 1 commit into
MaterializeInc:mainfrom
ggevay:gabor/sql-362-distinct-variance-aggregates

Conversation

@ggevay

@ggevay ggevay commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Fixes SQL-362

Motivation

var_samp, var_pop, variance, stddev_samp, stddev_pop, and stddev with DISTINCT returned wrong results whenever the input contained values differing only in sign. For example, over {-2.0, 2.0, 2.0}, var_pop(DISTINCT x) returned 2 instead of 4. They now match PostgreSQL.

Description

These aggregates are planned in transform_ast.rs by decomposing into sum(x*x), sum(x), and count(x). The DISTINCT flag was pushed into each component aggregate. That is correct for sum(DISTINCT x) and count(DISTINCT x), which deduplicate on x, but sum(DISTINCT x*x) deduplicates on x², collapsing values that differ only in sign (e.g. -2 and 2 both square to 4, which was then counted once instead of twice).

The fix computes the sum of squares with deduplication on x by splitting it by sign. Squaring is injective on the non-negative values and, separately, on the negative values, so

COALESCE(sum(DISTINCT CASE WHEN x >= 0 THEN x*x END), 0)
  + COALESCE(sum(DISTINCT CASE WHEN x < 0 THEN x*x END), 0)

deduplicates exactly like the other two component aggregates. COALESCE handles an empty sign class. For empty or all-NULL input the overall result is still NULL because sum(DISTINCT x) is NULL. The non-DISTINCT expansion is unchanged. All six function names and all input types are covered, since they all funnel through the same plan_variance expansion.

Verification

New regression tests in test/sqllogictest/aggregates.slt: the exact repro from the issue, all six functions, float/numeric/int/smallint/bigint/real inputs, NULLs (mixed and all-NULL), empty input, a single distinct value, DISTINCT combined with FILTER and with GROUP BY, special float values (NaN, -Infinity, -0.0), the window-function error case, and unchanged non-DISTINCT results. The new tests fail without the fix (9 failures) and pass with it. Expected values were verified against PostgreSQL.

Additionally ran a locally generated differential test of 720 randomized checks (60 random datasets x 6 functions x DISTINCT/plain, biased toward sign-mirrored values, duplicates, and NULLs) with expected values taken from a live PostgreSQL instance: all pass. Also ran test/sqllogictest/window_funcs.slt and test/sqllogictest/github-5797.slt to cover the unchanged non-DISTINCT paths.

🤖 Generated with Claude Code

var_samp, var_pop, variance, stddev_samp, stddev_pop, and stddev are
planned by decomposing into sum(x*x), sum(x), and count(x). The DISTINCT
flag was pushed into each component aggregate, but sum(DISTINCT x*x)
deduplicates on x*x rather than on x, wrongly collapsing input values
that differ only in sign. For example, over {-2.0, 2.0, 2.0} var_pop
(DISTINCT x) returned 2 instead of 4.

Fix by splitting the sum of squares by the sign of x when DISTINCT is
specified. Squaring is injective on the non-negative values and,
separately, on the negative values, so

    sum(DISTINCT CASE WHEN x >= 0 THEN x*x END)
      + sum(DISTINCT CASE WHEN x < 0 THEN x*x END)

deduplicates exactly like sum(DISTINCT x) and count(DISTINCT x) do. The
non-DISTINCT expansion is unchanged.

Fixes SQL-362.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@ggevay ggevay added the A-ADAPTER Topics related to the ADAPTER layer label Jul 14, 2026
@ggevay ggevay marked this pull request as ready for review July 14, 2026 19:03
@ggevay ggevay requested a review from a team as a code owner July 14, 2026 19:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-ADAPTER Topics related to the ADAPTER layer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant