Skip to content

Add STATS and EXTENDED_STATS compound aggregate expansion#5497

Open
varun-st wants to merge 1 commit into
opensearch-project:mainfrom
varun-st:sql-stats-bundle-aggregates
Open

Add STATS and EXTENDED_STATS compound aggregate expansion#5497
varun-st wants to merge 1 commit into
opensearch-project:mainfrom
varun-st:sql-stats-bundle-aggregates

Conversation

@varun-st
Copy link
Copy Markdown

@varun-st varun-st commented Jun 2, 2026

Description

Adds parser-level expansion for the STATS and EXTENDED_STATS compound aggregates, matching legacy V1 SQL plugin output. A query like SELECT STATS(price) FROM orders now expands to its primitive
components at AST-building time, so the analyzer/planner sees only standard primitives.

Compound Expanded columns
STATS(f) count, sum, avg, min, max
EXTENDED_STATS(f) above + sumOfSquares, variance, stdDeviation

variance and stdDeviation use population variants (var_pop, stddev_pop), matching V1. sumOfSquares lowers to SUM(f * f).

Display-name format matches V1: STATS(price).count, or p.count when an explicit AS p is given. FILTER (WHERE …) propagates to every primitive.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 2, 2026

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🧪 PR contains tests
🔒 No security concerns identified
✅ No TODO sections
🔀 No multiple PR themes
⚡ Recommended focus areas for review

Possible Issue

The loop at line 132 iterates over ctx.selectElements().selectElement() but never adds non-compound items to the builder when ctx.selectElements().star is null and there are no compound aggregates. If all select elements are non-compound, the builder remains empty after the star check, resulting in an empty Project. The old code always added every field; this change only adds items inside the loop body, which may skip additions if the condition at line 134 is false for all items.

for (SelectElementContext element : ctx.selectElements().selectElement()) {
  UnresolvedExpression item = visitSelectItem(element);
  if (CompoundAggregateExpander.isCompoundAggregateAlias(item)) {
    builder.addAll(expandCompoundAggregate(item));
  } else {
    builder.add(item);
  }
}
return new Project(builder.build());
Possible Issue

The SQUARE transform at line 46-47 constructs new Function("*", List.of(field, field)) by passing the same field reference twice. If field is a mutable expression or if downstream code mutates one of the references, both operands could be affected. While UnresolvedExpression implementations may be immutable, the contract is not enforced here, and shared references can lead to unexpected aliasing issues if the expression tree is modified later.

private static final UnaryOperator<UnresolvedExpression> SQUARE =
    field -> new Function("*", List.of(field, field));

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Jun 2, 2026

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
General
Validate empty string input

The resolveComponents method should validate that the compoundName is not empty
before attempting to look it up. An empty string will pass the null check but fail
to match any registered aggregate, resulting in a less informative error message.

sql/src/main/java/org/opensearch/sql/sql/parser/CompoundAggregateExpander.java [161-170]

 private static List<Component> resolveComponents(String compoundName) {
-  if (compoundName == null) {
-    throw new IllegalArgumentException("compoundName is null");
+  if (compoundName == null || compoundName.isEmpty()) {
+    throw new IllegalArgumentException("compoundName is null or empty");
   }
   List<Component> components = COMPOUND_AGGREGATES.get(compoundName.toUpperCase(Locale.ROOT));
   if (components == null) {
     throw new IllegalArgumentException("Not a compound aggregate: " + compoundName);
   }
   return components;
 }
Suggestion importance[1-10]: 4

__

Why: Adding an empty string check improves error handling by providing a more specific error message. However, the impact is minimal since the existing code already handles this case by throwing an exception when the lookup fails. The improvement is primarily in error message clarity rather than correctness.

Low

Copy link
Copy Markdown
Collaborator

@ahkcs ahkcs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grammar changes shouldn't be made in language-grammar directory, that directory is used for shared grammar with spark repo.

Copy link
Copy Markdown
Collaborator

@dai-chen dai-chen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please fix DCO and add IT to verify. I assume this will enable in SQL V2 engine too (previously query with such function falls back to legacy engine), I recall many legacy IT was ignored after 3.0

@dai-chen dai-chen added enhancement New feature or request SQL labels Jun 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request SQL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants