Skip to content

Ungrouped (count (distinct col)) in a select projection returns the row count (STR/SYM) or a type error (I64) #405

Description

@alexey-milovidov

Summary

(count (distinct col)) used as an ungrouped select projection (no by:)
does not compute a distinct count. On a STR / SYM column it silently returns
the row count; on an integer column it fails with a type error. The same
expression is correct under by:, and the whole-vector form
(count (distinct (at t 'col))) is correct too — so only the ungrouped
select-projection path is wrong.

Reproduction

(set t (table [i s g] (list [1 1 2 2 3] ["aa" "aa" "bb" "bb" "cc"] [9 9 9 9 9])))

(count (distinct (at t 'i)))                    ; 3  correct
(count (distinct (at t 's)))                    ; 3  correct
(select {from: t by: g u: (count (distinct i))}) ; u = 3  correct

(select {from: t u: (count (distinct s))})      ; u = 5  <-- wrong, that is (count s)
(select {from: t u: (count (distinct i))})      ; error: type: distinct: argument must be a list, got i64

Expected

(select {from: t u: (count (distinct s))})u = 3, matching both the
grouped form and the whole-vector reducer. An ungrouped aggregate select is just
the one-group case of by:.

Notes

  • The STR/SYM case is the dangerous one: no error, and the answer is a
    plausible-looking number. On a 2M-row table the same query returned 1999939
    (the row count) where the correct distinct count was 100665, so it does not
    look obviously wrong at a glance either.
  • The integer case at least fails loudly, and the message
    (distinct: argument must be a list, got i64) suggests distinct is being
    applied per row rather than to the column, i.e. the projection is evaluated
    element-wise instead of being lowered to the count-distinct aggregate that
    the by: path uses (ops/idiom.c rw_count_distinct
    exec_count_distinct).
  • Whichever way it is resolved, an error would be better than the current
    silent row count.

Versions

Reproduced on v2.5.14 (release tag, make release) and on dev at
95defb7aef2ff0863e70ab3b79785036cd529b1d, Linux aarch64.

Found while porting Rayforce to
ClickBench, where several queries
are SELECT COUNT(DISTINCT c) FROM hits.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions