refactor(perf): offload metric ratio computation to native ClickHouse CTE

[Aryainguz]

Summary

the metric ratios computation in @hyperdx/common-utils to execute natively inside ClickHouse via CTEs instead of fetching two separate streams and zipping them in memory using Node.js. Updated tests, changeset file as per the changes.

Why this change was made:
Currently, when evaluating metric ratio charts (e.g. dividing the sum of one metric by the sum of another), the backend executes two separate queries concurrently. The results are then loaded entirely into Node.js memory, zipped by their respective series, and the mathematical ratio calculation is executed sequentially in JavaScript.

For heavy, high-cardinality charts such as time series distributions across thousands of service.names this leads to:

• Significant serialization overhead to and from ClickHouse over the network.
• An unnecessarily bloated memory footprint in the Node backend due to retaining massive parallel arrays in the V8 heap.
• Higher latency and CPU cycle waste for mathematical zipping that the database is better optimized for.
• When a multi-series query specifies seriesReturnType: 'ratio', the AST compiler now produces a single composite SQL query with an ANY LEFT JOIN on the requested grouping keys and time buckets. This severely decreases latency by letting the database execute the math directly in the SELECT projection, drastically reducing Node API memory footprint and serialization overhead. It fully preserves backwards compatibility for legacy column alias outputs (/).

Fixes #2680

Local Validations:

[changeset-bot]

🦋 Changeset detected

Latest commit: afe1fd7

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@hyperdx/common-utils	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[vercel]

@Aryainguz is attempting to deploy a commit to the HyperDX Team on Vercel.

A member of the Team first needs to authorize it.

[greptile-apps]

Greptile Summary

This PR moves metric ratio calculation into a native ClickHouse query path. The main changes are:

• Adds a CTE-based ratio query for two-series builder charts.
• Normalizes ratio group-by aliases before rendering operand queries.
• Keeps share-of-total ratios on the existing merge path.
• Adds integration coverage and a patch changeset.

Confidence Score: 5/5

This looks safe to merge.

• No blocking issues found in the changed code.

Important Files Changed

Filename	Overview
packages/common-utils/src/clickhouse/index.ts	Adds the native ClickHouse ratio query path and keeps share-of-total ratios on the existing merge path.
packages/common-utils/src/tests/queryChartConfig.int.test.ts	Adds integration coverage for grouped native ratio results.
.changeset/refactor-metric-ratios-cte.md	Adds a patch changeset for the common-utils ratio refactor.

_{Reviews (9): Last reviewed commit: "Merge branch 'main' into feat/native-cli..." | Re-trigger Greptile}

[github-actions]

Deep Review

✅ No critical issues found. The new native-CTE ratio path is SQL-injection-safe — every user-influenceable value (q0Alias, q1Alias, the <q0>/<q1> ratio alias, and each groupBy-derived join key) is bound through the { Identifier: ... } form of chSql and sent via query_params, never concatenated into raw SQL. The ratio column is selected first, matching the meta[0]-keyed assumption in useChartNumberFormats, so the column-format contract is preserved. The recommendations below are correctness-divergence and coverage concerns, not ship-blockers.

🟡 P2 -- recommended

• packages/common-utils/src/clickhouse/index.ts:897 -- The native ratio COALESCE(q0.<q0Alias>, 0) / q1.<q1Alias> guards only the numerator, so its zero/missing-denominator behavior diverges from the legacy computeRatio, which returns NaN when the denominator is 0 or absent.
  • Fix: Normalize the denominator path (e.g. return null/NaN when the denominator is 0 or NULL) so a zero or one-sided group in a grouped ratio chart renders the same gap the in-memory path produced instead of inf/NULL.
  • _{correctness, api-contract, reliability}
• packages/common-utils/src/__tests__/queryChartConfig.int.test.ts:250 -- The single new test covers only the time-series + single-column-groupBy + distinct-alias case and asserts merely that the ratio parses as a number, leaving the CROSS JOIN (no groupBy), non-time-series (Table), duplicate-alias, zero-denominator, and share_of_total→legacy branches unverified and the computed ratio value unchecked.
  • Fix: Add cases for the un-grouped CROSS JOIN path, a Table display type, an identical-alias ratio, and a zero/one-sided denominator, and assert an expected numeric ratio rather than only !Number.isNaN.
  • _testing

🔵 P3 nitpicks (4)

• packages/common-utils/src/clickhouse/index.ts:837 -- queryChartConfig mutates its config argument in place (reassigning config.groupBy and, via setChartSelectsAlias, config.select), a side effect visible to the caller's object.
  • Fix: Clone the config (or the mutated sub-fields) before rewriting them.
• packages/common-utils/src/clickhouse/index.ts:838 -- The guard config.seriesReturnType === 'ratio' && config.ratioMode !== 'share_of_total' is repeated three times and the string/array groupBy normalization branching is duplicated between normalization and join-key extraction.
  • Fix: Extract the ratio-eligibility predicate and the join-key derivation into named helpers, mirroring how mergeResultSets is factored out.
• packages/common-utils/src/clickhouse/index.ts:923 -- The branch returns resp.json<any>(), widening the method's declared ResponseJSON<Record<string, string | number>> contract to any.
  • Fix: Type the response to the declared return type instead of any.
• packages/common-utils/src/clickhouse/index.ts:875 -- config.select[0].alias and config.select[1].alias are read after only an Array.isArray(config.select) check, relying on the separate queries.length === 2 guard for length rather than checking select length directly.
  • Fix: Add an explicit config.select.length === 2 guard alongside Array.isArray so the indexing is locally sound.

---

Reviewers (12): correctness, testing, maintainability, project-standards, security, performance, api-contract, reliability, kieran-typescript, adversarial, agent-native, learnings.

Testing gaps: No test asserts the new native-CTE output equals the legacy in-memory output for the same input; zero/one-sided-denominator behavior (inf/NULL vs NaN) is unverified; groupBy values containing SQL metacharacters are not exercised to confirm identifier binding.

[Aryainguz]

@pulpdrew please review once