From 7bb7830443db3f850fcd9fbd52b2cd68f10d2a08 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 27 Jul 2026 12:53:05 +0000 Subject: [PATCH 1/3] feat(analytics): honour widget dateGranularity, sortBy/sortOrder, and limit in dataset queries (#3588) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three presentation options were accepted by the metadata layer and then dropped by the analytics query builder — no SQL, no error. The only symptom was the `sql` a dataset response echoes, so a dashboard could declare `dateGranularity: 'month'` and quietly render one bar per record. - `DatasetSelection.dateGranularity` buckets every selected date dimension. Precedence per dimension: an explicit `timeDimensions` granularity, then the selection's, then the dataset dimension's own default — so a widget can bucket a trend by month without the dataset committing every consumer to it. - `order`/`limit`/`offset` are applied to the ASSEMBLED grid: after measure-scoped sub-queries merge, after `compareTo` columns attach, and after derived measures are computed. A derived measure is therefore a valid sort key, and the ObjectQL aggregate path — which has no ordering grammar, and which native SQL hands every date-bucketed query to — orders identically to native SQL. A single-query selection still pushes the window into the statement. An order key naming nothing the selection projects is rejected (400) rather than ignored. - A `limit` without an `order` orders by the selected dimensions first, so it truncates a reproducible window instead of an arbitrary subset. - The four query-affecting widget `options` keys plus `stageOrder` are declared on `DashboardWidgetOptionsSchema`, making a typo an author-time error. The bag stays open, so renderer extras pass through untouched. Two latent bugs surfaced while fixing the above: - `order`/`limit` were forwarded to EVERY sub-query. A measure-scoped supplementary query selects one measure, so an inherited ORDER BY named a column it never selected and an inherited LIMIT truncated it before the merge, dropping rows from the grid. Nothing hit this only because nothing passed `order`. - The `compareTo` pass hand-built its query and skipped granularity resolution, merging a month-bucketed primary grid against raw-timestamp comparison rows. No dimension key matched, so every `__compare` column came back empty. `ObjectQLStrategy` now also echoes a representative `sql` (date_trunc, WHERE, ORDER BY, LIMIT; filter values parameterized, never inlined). The `sql` field previously vanished whenever a query was date-bucketed, leaving an author unable to tell "not implemented" from "this strategy doesn't report". Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01SEZLBGyNoJMLvtGPqsMPRC --- .changeset/analytics-widget-query-options.md | 53 +++ packages/rest/src/rest-server.ts | 2 +- .../dataset-selection-window.test.ts | 364 ++++++++++++++++++ .../service-analytics/src/dataset-executor.ts | 249 +++++++++++- .../src/strategies/objectql-strategy.ts | 99 ++++- packages/spec/json-schema.manifest.json | 1 + .../spec/src/contracts/analytics-service.ts | 28 ++ packages/spec/src/ui/dashboard.zod.ts | 60 ++- 8 files changed, 826 insertions(+), 30 deletions(-) create mode 100644 .changeset/analytics-widget-query-options.md create mode 100644 packages/services/service-analytics/src/__tests__/dataset-selection-window.test.ts diff --git a/.changeset/analytics-widget-query-options.md b/.changeset/analytics-widget-query-options.md new file mode 100644 index 0000000000..15ef341ce0 --- /dev/null +++ b/.changeset/analytics-widget-query-options.md @@ -0,0 +1,53 @@ +--- +"@objectstack/spec": patch +"@objectstack/service-analytics": patch +"@objectstack/rest": patch +--- + +feat(analytics): honour widget `dateGranularity`, `sortBy`/`sortOrder`, and `limit` in the dataset query (#3588) + +Three presentation options were accepted by the metadata layer and then dropped +by the analytics query builder. They reached no SQL, produced no error, and the +only way to notice was to read the `sql` a dataset response echoes — so a +dashboard could declare `dateGranularity: 'month'` and quietly render one bar +per record. + +- **`dateGranularity` now buckets.** `DatasetSelection` gained an optional + `dateGranularity`, applied to every selected `date` dimension. Precedence per + dimension: an explicit `timeDimensions` granularity, then the selection's, + then the dataset dimension's own default. A widget can bucket a trend by month + without the dataset committing every other consumer to that granularity. +- **`order` / `limit` / `offset` now apply on every path.** They are applied to + the ASSEMBLED grid — after measure-scoped sub-queries merge, after `compareTo` + columns attach, and after derived measures are computed — so a derived measure + is a valid sort key and the ObjectQL aggregate path (which has no ordering + grammar, and which native SQL hands every date-bucketed query to) orders + identically to native SQL. A single-query selection still pushes the window + down into the statement. An `order` key that names nothing the selection + projects is now rejected (400) rather than silently ignored. +- **`limit` is deterministic.** Without an `order`, a limit orders by the + selected dimensions first, so it truncates a reproducible window instead of an + arbitrary subset. +- **Widget `options` is a contract again.** The four query-affecting keys + (`dateGranularity`, `sortBy`, `sortOrder`, `limit`) plus `stageOrder` are + declared on `DashboardWidgetOptionsSchema`, so a typo like `sortDirection` is + an author-time error. The bag stays open — renderer extras (`icon`, `columns`, + `striped`, …) pass through untouched. + +Two latent bugs surfaced while fixing the above and are fixed here too: + +- `order`/`limit` were forwarded to EVERY sub-query. A measure-scoped + supplementary query selects one measure, so an inherited `ORDER BY` named a + column it never selected, and an inherited `LIMIT` truncated it before the + merge — dropping rows from the assembled grid. Nothing hit this only because + nothing passed `order`. +- The `compareTo` pass built its query by hand and skipped granularity + resolution, so a month-bucketed primary grid was merged against raw-timestamp + comparison rows. No dimension key matched and every `__compare` + column came back empty. + +`ObjectQLStrategy` now also echoes a representative `sql` (with `date_trunc`, +`WHERE`, `ORDER BY`, and `LIMIT`; filter values parameterized, never inlined). +Previously the `sql` field simply vanished from the response whenever a query +was date-bucketed, leaving an author unable to tell "not implemented" from "this +strategy doesn't report". diff --git a/packages/rest/src/rest-server.ts b/packages/rest/src/rest-server.ts index 2e9df4245f..dcbe1e281b 100644 --- a/packages/rest/src/rest-server.ts +++ b/packages/rest/src/rest-server.ts @@ -5260,7 +5260,7 @@ export class RestServer { const msg = String(error?.message ?? error ?? ''); // Dataset-compiler D-C / unsupported-aggregate / read-scope // errors are client-side mistakes — surface as 400. - if (/not declared in the dataset|not backed by a declared relationship|not supported by the v1 dataset runtime|read-scope-sql/.test(msg)) { + if (/not declared in the dataset|not backed by a declared relationship|not supported by the v1 dataset runtime|read-scope-sql|not a selected dimension or measure|is not a subset of the selected dimensions/.test(msg)) { return res.status(400).json({ code: 'DATASET_INVALID', message: msg.slice(0, 1000) }); } logError('[REST] Analytics dataset query error:', error); diff --git a/packages/services/service-analytics/src/__tests__/dataset-selection-window.test.ts b/packages/services/service-analytics/src/__tests__/dataset-selection-window.test.ts new file mode 100644 index 0000000000..87bb028ece --- /dev/null +++ b/packages/services/service-analytics/src/__tests__/dataset-selection-window.test.ts @@ -0,0 +1,364 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * #3588 — presentation-scope query options reach the generated query. + * + * Three widget options were accepted by the metadata layer and then dropped on + * the floor by the query builder: `dateGranularity` never became a date + * truncation, `sortBy`/`sortOrder` never became an ordering, and `limit` + * truncated an arbitrary subset because nothing ordered the rows first. These + * tests pin the query the executor now builds, on BOTH strategy paths: + * + * - native SQL — asserts the emitted statement (pushdown), and + * - the ObjectQL aggregate path — which native SQL hands every date-bucketed + * query to, and which has no ordering grammar of its own, so ordering there + * must come from the executor's post-pass. + */ + +import { describe, it, expect } from 'vitest'; +import { DatasetSchema } from '@objectstack/spec/ui'; +import type { ExecutionContext } from '@objectstack/spec/kernel'; +import { AnalyticsService } from '../analytics-service.js'; +import { applyOrdering, applyWindow, resolveOrdering } from '../dataset-executor.js'; + +const CTX = { tenantId: 'org_A' } as ExecutionContext; + +/** A date dimension with NO dataset-level granularity — the widget must supply it. */ +const accounts = DatasetSchema.parse({ + name: 'account_metrics', + label: 'Accounts', + object: 'crm_account', + include: [], + dimensions: [ + { name: 'created_at', field: 'created_at', type: 'date' }, + { name: 'industry', field: 'industry', type: 'string' }, + ], + measures: [ + { name: 'account_count', aggregate: 'count' }, + { name: 'annual_revenue_sum', aggregate: 'sum', field: 'annual_revenue' }, + ], +}); + +type AggCall = { object: string; options: Record }; + +/** ObjectQL-aggregate service; captures every `executeAggregate` call. */ +function aggService(rows: Record[], calls: AggCall[] = []) { + const svc = new AnalyticsService({ + queryCapabilities: () => ({ nativeSql: false, objectqlAggregate: true, inMemory: false }), + executeAggregate: async (object, options) => { + calls.push({ object, options: options as Record }); + return rows; + }, + }); + return { svc, calls }; +} + +/** Native-SQL service; captures every statement it is asked to run. */ +function sqlService(rows: Record[], captured: string[] = []) { + const svc = new AnalyticsService({ + queryCapabilities: () => ({ nativeSql: true, objectqlAggregate: false, inMemory: false }), + executeRawSql: async (_o, sql) => { captured.push(sql); return rows; }, + }); + return { svc, captured }; +} + +const groupByOf = (call: AggCall) => call.options.groupBy as Array; + +describe('#3588 — selection.dateGranularity reaches the GROUP BY', () => { + it('buckets a date dimension the DATASET left ungranulated', async () => { + // The `new_accounts_by_month` case: the dataset declares `created_at` as a + // plain date dimension, so before this change the query grouped on the raw + // timestamp — one bucket per record, five identical bars. + const { svc, calls } = aggService([{ created_at: '2026-06', account_count: 4 }]); + const result = await svc.queryDataset( + accounts, + { dimensions: ['created_at'], measures: ['account_count'], dateGranularity: 'month' }, + CTX, + ); + expect(groupByOf(calls[0])).toEqual([{ field: 'created_at', dateGranularity: 'month' }]); + expect(result.rows).toEqual([{ created_at: '2026-06', account_count: 4 }]); + }); + + it('leaves the dimension unbucketed when the selection asks for nothing', async () => { + const { svc, calls } = aggService([{ created_at: '2026-06-01', account_count: 1 }]); + await svc.queryDataset(accounts, { dimensions: ['created_at'], measures: ['account_count'] }, CTX); + expect(groupByOf(calls[0])).toEqual(['created_at']); + }); + + it('overrides the dataset dimension default for this widget only', async () => { + const monthly = DatasetSchema.parse({ + name: 'opp', label: 'Opp', object: 'opportunity', include: [], + dimensions: [{ name: 'close_date', field: 'close_date', type: 'date', dateGranularity: 'month' }], + measures: [{ name: 'revenue', aggregate: 'sum', field: 'amount' }], + }); + const { svc, calls } = aggService([{ close_date: '2026', revenue: 1 }]); + await svc.queryDataset( + monthly, + { dimensions: ['close_date'], measures: ['revenue'], dateGranularity: 'year' }, + CTX, + ); + expect(groupByOf(calls[0])).toEqual([{ field: 'close_date', dateGranularity: 'year' }]); + }); + + it('never overrides an explicit timeDimensions entry', async () => { + const { svc, calls } = aggService([{ created_at: '2026-06-08', account_count: 1 }]); + await svc.queryDataset( + accounts, + { + dimensions: ['created_at'], + measures: ['account_count'], + dateGranularity: 'month', + timeDimensions: [{ dimension: 'created_at', granularity: 'week' }], + }, + CTX, + ); + expect(groupByOf(calls[0])).toEqual([{ field: 'created_at', dateGranularity: 'week' }]); + }); + + it('applies only to DATE dimensions, never to a string dimension', async () => { + const { svc, calls } = aggService([{ industry: 'Tech', account_count: 2 }]); + await svc.queryDataset( + accounts, + { dimensions: ['industry'], measures: ['account_count'], dateGranularity: 'month' }, + CTX, + ); + expect(groupByOf(calls[0])).toEqual(['industry']); + }); + + it('buckets the compareTo pass identically, so the grids actually merge', async () => { + // The comparison query used to be hand-built without granularity resolution: + // a month-bucketed primary grid was merged against raw-timestamp comparison + // rows, no dimension key matched, and every `__compare` column came back + // empty. + const calls: AggCall[] = []; + const svc = new AnalyticsService({ + queryCapabilities: () => ({ nativeSql: false, objectqlAggregate: true, inMemory: false }), + executeAggregate: async (object, options) => { + calls.push({ object, options: options as Record }); + return [{ created_at: '2026-06', account_count: calls.length }]; + }, + }); + const result = await svc.queryDataset( + accounts, + { + dimensions: ['created_at'], + measures: ['account_count'], + dateGranularity: 'month', + timeDimensions: [{ dimension: 'created_at', dateRange: ['2026-06-01', '2026-06-30'] }], + compareTo: { kind: 'previousPeriod', dimension: 'created_at' }, + }, + CTX, + ); + // Both passes bucket by month… + expect(groupByOf(calls[0])).toEqual([{ field: 'created_at', dateGranularity: 'month' }]); + expect(groupByOf(calls[1])).toEqual([{ field: 'created_at', dateGranularity: 'month' }]); + // …so the comparison column lands on the same row instead of a phantom bucket. + expect(result.rows).toHaveLength(1); + expect(result.rows[0]).toMatchObject({ created_at: '2026-06', account_count: 1, account_count__compare: 2 }); + }); +}); + +describe('#3588 — sortBy/sortOrder become a real ordering', () => { + it('orders a measure descending on the ObjectQL path (which has no ORDER BY of its own)', async () => { + // The `accounts_by_industry` case: rows arrive in whatever order the group-by + // produced (alphabetical here, which was ascending revenue — the exact + // opposite of the requested `desc`). + const { svc } = aggService([ + { industry: 'Finance', annual_revenue_sum: 3_500_000 }, + { industry: 'Healthcare', annual_revenue_sum: 8_000_000 }, + { industry: 'Manufacturing', annual_revenue_sum: 12_000_000 }, + { industry: 'Technology', annual_revenue_sum: 30_000_000 }, + ]); + const result = await svc.queryDataset( + accounts, + { dimensions: ['industry'], measures: ['annual_revenue_sum'], order: { annual_revenue_sum: 'desc' } }, + CTX, + ); + expect(result.rows.map((r) => r.industry)).toEqual(['Technology', 'Manufacturing', 'Healthcare', 'Finance']); + }); + + it('pushes ordering + window into the SQL when the selection is a single query', async () => { + const { svc, captured } = sqlService([{ industry: 'Tech', annual_revenue_sum: 1 }]); + await svc.queryDataset( + accounts, + { dimensions: ['industry'], measures: ['annual_revenue_sum'], order: { annual_revenue_sum: 'desc' }, limit: 10 }, + CTX, + ); + expect(captured[0]).toContain('ORDER BY "annual_revenue_sum" DESC'); + expect(captured[0]).toContain('LIMIT 10'); + }); + + it('rejects an order key the selection does not project', async () => { + const { svc } = aggService([]); + await expect( + svc.queryDataset( + accounts, + { dimensions: ['industry'], measures: ['account_count'], order: { annual_revenue_sum: 'desc' } }, + CTX, + ), + ).rejects.toThrow(/not a selected dimension or measure/); + }); + + it('sorts nulls last in both directions', () => { + const rows = [{ v: 5 }, { v: null }, { v: 9 }]; + expect(applyOrdering(rows, { v: 'asc' }).map((r) => r.v)).toEqual([5, 9, null]); + expect(applyOrdering(rows, { v: 'desc' }).map((r) => r.v)).toEqual([9, 5, null]); + }); + + it('compares numeric strings numerically, not lexicographically', () => { + const rows = [{ v: '9' }, { v: '10' }]; + expect(applyOrdering(rows, { v: 'asc' }).map((r) => r.v)).toEqual(['9', '10']); + }); + + it('orders by a DERIVED measure — a column no single SQL statement computes', async () => { + const withRatio = DatasetSchema.parse({ + name: 'ratio_ds', label: 'Ratio', object: 'opportunity', include: [], + dimensions: [{ name: 'stage', field: 'stage', type: 'string' }], + measures: [ + { name: 'won', aggregate: 'sum', field: 'won_amount' }, + { name: 'total', aggregate: 'sum', field: 'amount' }, + { name: 'win_rate', derived: { op: 'ratio', of: ['won', 'total'] } }, + ], + }); + const { svc } = aggService([ + { stage: 'a', won: 1, total: 10 }, // 0.1 + { stage: 'b', won: 9, total: 10 }, // 0.9 + { stage: 'c', won: 5, total: 10 }, // 0.5 + ]); + const result = await svc.queryDataset( + withRatio, + { dimensions: ['stage'], measures: ['win_rate'], order: { win_rate: 'desc' } }, + CTX, + ); + expect(result.rows.map((r) => r.stage)).toEqual(['b', 'c', 'a']); + }); +}); + +describe('#3588 — limit depends on a deterministic ordering', () => { + it('orders by the selected dimensions when a bare limit is requested', () => { + expect(resolveOrdering({ measures: ['m'], limit: 3 }, ['a', 'b'])).toEqual({ a: 'asc', b: 'asc' }); + // …and stays out of the way when nothing asked for a window. + expect(resolveOrdering({ measures: ['m'] }, ['a'])).toBeUndefined(); + }); + + it('truncates the TOP rows, not an arbitrary subset', async () => { + const { svc } = aggService([ + { industry: 'Finance', annual_revenue_sum: 3 }, + { industry: 'Technology', annual_revenue_sum: 30 }, + { industry: 'Healthcare', annual_revenue_sum: 8 }, + ]); + const result = await svc.queryDataset( + accounts, + { dimensions: ['industry'], measures: ['annual_revenue_sum'], order: { annual_revenue_sum: 'desc' }, limit: 2 }, + CTX, + ); + expect(result.rows.map((r) => r.industry)).toEqual(['Technology', 'Healthcare']); + }); + + it('applies offset after ordering', () => { + const rows = [{ v: 1 }, { v: 2 }, { v: 3 }, { v: 4 }]; + expect(applyWindow(rows, 2, 1).map((r) => r.v)).toEqual([2, 3]); + expect(applyWindow(rows, undefined, 2).map((r) => r.v)).toEqual([3, 4]); + expect(applyWindow(rows, undefined, undefined)).toBe(rows); + }); +}); + +describe('#3588 — ordering never corrupts a multi-query selection', () => { + const scoped = DatasetSchema.parse({ + name: 'scoped_ds', label: 'Scoped', object: 'opportunity', include: [], + dimensions: [{ name: 'stage', field: 'stage', type: 'string' }], + measures: [ + { name: 'total', aggregate: 'sum', field: 'amount' }, + { name: 'won', aggregate: 'sum', field: 'amount', filter: { stage: 'closed_won' } }, + ], + }); + + it('keeps ORDER BY/LIMIT out of measure-scoped sub-queries, ordering the merged grid instead', async () => { + // A supplementary query selects ONE measure. Forwarding `ORDER BY "won"` to + // the primary (which never selects `won`) is invalid SQL, and forwarding + // `LIMIT` to either truncates it before the merge — rows would silently + // disappear from the assembled grid. + const captured: string[] = []; + const svc = new AnalyticsService({ + queryCapabilities: () => ({ nativeSql: true, objectqlAggregate: false, inMemory: false }), + executeRawSql: async (_o, sql) => { + captured.push(sql); + // Dispatch on the projected alias — the scoped filter's value is bound + // as a parameter, so it never appears in the statement text. + return sql.includes('AS "won"') + ? [{ stage: 'proposal', won: 5 }, { stage: 'negotiation', won: 50 }] + : [{ stage: 'proposal', total: 10 }, { stage: 'negotiation', total: 100 }]; + }, + }); + const result = await svc.queryDataset( + scoped, + { dimensions: ['stage'], measures: ['total', 'won'], order: { won: 'desc' }, limit: 1 }, + CTX, + ); + // No sub-query carried the window… + expect(captured).toHaveLength(2); + for (const sql of captured) { + expect(sql).not.toContain('ORDER BY'); + expect(sql).not.toContain('LIMIT'); + } + // …and the assembled grid is ordered by the merged column and then cut. + expect(result.rows).toEqual([{ stage: 'negotiation', total: 100, won: 50 }]); + }); + + it('leaves totals covering the whole selection, unaffected by the window', async () => { + const { svc } = aggService([ + { industry: 'Finance', annual_revenue_sum: 3 }, + { industry: 'Technology', annual_revenue_sum: 30 }, + ]); + const result = await svc.queryDataset( + accounts, + { + dimensions: ['industry'], + measures: ['annual_revenue_sum'], + order: { annual_revenue_sum: 'desc' }, + limit: 1, + totals: { groupings: [[]] }, + }, + CTX, + ); + expect(result.rows).toHaveLength(1); + // The grand total still saw BOTH rows (mock returns the full set per call). + expect(result.totals?.[0].rows).toHaveLength(2); + }); +}); + +describe('#3588 — the echoed SQL tells the truth on the ObjectQL path', () => { + it('renders date_trunc for a bucketed dimension instead of the bare column', async () => { + const { svc } = aggService([{ created_at: '2026-06', account_count: 4 }]); + const result = await svc.queryDataset( + accounts, + { dimensions: ['created_at'], measures: ['account_count'], dateGranularity: 'month' }, + CTX, + ); + expect(result.sql).toContain(`date_trunc('month', created_at)`); + expect(result.sql).toContain('GROUP BY'); + expect(result.sql).toContain('COUNT(*) AS "account_count"'); + }); + + it('renders the ordering and window that the response rows actually reflect', async () => { + const { svc } = aggService([{ industry: 'Tech', annual_revenue_sum: 1 }]); + const result = await svc.queryDataset( + accounts, + { dimensions: ['industry'], measures: ['annual_revenue_sum'], order: { annual_revenue_sum: 'desc' }, limit: 10 }, + CTX, + ); + expect(result.sql).toContain('ORDER BY "annual_revenue_sum" DESC'); + expect(result.sql).toContain('LIMIT 10'); + }); + + it('parameterizes filter values rather than inlining them into the echoed statement', async () => { + const { svc } = aggService([{ industry: 'Tech', account_count: 1 }]); + const result = await svc.queryDataset( + accounts, + { dimensions: ['industry'], measures: ['account_count'], runtimeFilter: { industry: 'Tech' } }, + CTX, + ); + expect(result.sql).toContain('WHERE industry = $1'); + expect(result.sql).not.toContain('Tech'); + }); +}); diff --git a/packages/services/service-analytics/src/dataset-executor.ts b/packages/services/service-analytics/src/dataset-executor.ts index e649f75241..7ef10d8608 100644 --- a/packages/services/service-analytics/src/dataset-executor.ts +++ b/packages/services/service-analytics/src/dataset-executor.ts @@ -29,7 +29,28 @@ export type CompareTo = DatasetCompareTo; * attaches `__compare` columns, * - computes server-side totals (`selection.totals.groupings`, #1753) by * re-running the selection per dimension subset, so matrix subtotals and - * the grand total use each measure's true aggregate. + * the grand total use each measure's true aggregate, + * - orders and windows the final grid (`order` / `limit` / `offset`, #3588). + * + * **Where ordering happens, and why here.** `order`/`limit`/`offset` are applied + * to the ASSEMBLED grid — after measure-scoped sub-queries are merged in, after + * `compareTo` columns are attached, and after derived measures are computed — + * never by forwarding them blindly to every sub-query. Two reasons: + * + * 1. **Correctness.** A supplementary measure-scoped query selects ONE measure; + * forwarding `ORDER BY ` to it emits SQL referencing a column + * that query never selects, and forwarding `LIMIT` truncates it before the + * merge, so rows silently vanish from the grid. A derived measure has no SQL + * column at all, yet is a perfectly reasonable sort key. + * 2. **Coverage.** Only `NativeSQLStrategy` honours `order`/`limit`; the + * ObjectQL aggregate path has nowhere to put them (`EngineAggregateOptions` + * has no ordering grammar), and date-bucketed queries are *forced* down that + * path because native SQL declines granularity. Sorting here makes ordering + * work identically on every driver and strategy. + * + * The single-query case still pushes `order`/`limit`/`offset` DOWN into the SQL + * (see `canPushDownWindow`) so the database does the work and the echoed `sql` + * shows it; the post-pass is then a no-op re-sort of already-sorted rows. * * RLS/tenant scoping is NOT handled here — it is enforced inside the strategy * via the StrategyContext read-scope hook (D-C). This layer is pure query @@ -89,6 +110,120 @@ function computeDerived(d: DerivedMeasureSpec, row: Record): nu } } +// ── ordering + windowing (#3588) ───────────────────────────────────────────── + +/** + * Compare two grouped-cell values for ORDER BY, ascending. + * + * Nulls sort LAST regardless of direction (the SQL `NULLS LAST` convention, and + * the one users expect: an empty bucket shouldn't win a "top 10 by revenue"). + * The caller negates the result for `desc`, so the null branch deliberately + * returns its verdict BEFORE that negation can flip it — see `compareRows`. + * + * Numbers (and numeric strings, which is how some drivers return SUM results) + * compare numerically so 9 sorts below 10; everything else compares as a string. + * Dates arrive here already bucketed to sort-stable keys ("2026-04", "2026-Q2"), + * so lexicographic ordering is chronological for them too. + */ +function compareValues(a: unknown, b: unknown): number { + const aNull = a == null || a === ''; + const bNull = b == null || b === ''; + if (aNull || bNull) return aNull && bNull ? 0 : aNull ? 1 : -1; + if (a instanceof Date || b instanceof Date) { + return Number(a instanceof Date ? a.getTime() : a) - Number(b instanceof Date ? b.getTime() : b); + } + if (typeof a === 'boolean' || typeof b === 'boolean') { + return Number(a) - Number(b); + } + const an = typeof a === 'number' ? a : Number(a); + const bn = typeof b === 'number' ? b : Number(b); + if (Number.isFinite(an) && Number.isFinite(bn)) return an - bn; + return String(a).localeCompare(String(b)); +} + +/** + * Order rows by each key in `order`, in the object's own key order (first key is + * the primary sort). Returns a NEW array; the input is not mutated. Null/empty + * cells stay last in both directions (see {@link compareValues}). + */ +export function applyOrdering( + rows: Record[], + order: Record | undefined, +): Record[] { + const keys = Object.entries(order ?? {}); + if (keys.length === 0 || rows.length < 2) return rows; + // Array.prototype.sort is stable (ES2019+), so equal rows keep the order the + // grouping produced — an important property for reproducible LIMITs. + return [...rows].sort((ra, rb) => { + for (const [key, dir] of keys) { + const av = ra[key]; + const bv = rb[key]; + const aNull = av == null || av === ''; + const bNull = bv == null || bv === ''; + // Nulls last in BOTH directions — decided before `desc` negation. + if (aNull || bNull) { + if (aNull && bNull) continue; + return aNull ? 1 : -1; + } + const c = compareValues(av, bv); + if (c !== 0) return dir === 'desc' ? -c : c; + } + return 0; + }); +} + +/** Apply `offset`/`limit` to an already-ordered grid. */ +export function applyWindow( + rows: Record[], + limit?: number, + offset?: number, +): Record[] { + const start = offset != null && offset > 0 ? offset : 0; + if (start === 0 && limit == null) return rows; + return rows.slice(start, limit != null ? start + limit : undefined); +} + +/** + * Validate `order` keys and resolve the EFFECTIVE ordering for a selection. + * + * A key must name something the caller actually selected — a dimension, a + * measure, or a `__compare` column. An unknown key throws rather than + * being dropped: silently ignoring `sortBy` is precisely the failure mode this + * change exists to remove (#3588), and a mistyped sort key that quietly returns + * arbitrarily-ordered rows is worse than a loud 400. + * + * When `limit`/`offset` is requested WITHOUT an order, the selected dimensions + * ascending become the implicit ordering, so the truncated window is + * reproducible instead of "whatever the group-by happened to emit". + */ +export function resolveOrdering( + selection: DatasetSelection, + dimensions: string[], +): Record | undefined { + const order = selection.order; + if (order && Object.keys(order).length > 0) { + const selectable = new Set([ + ...dimensions, + ...selection.measures, + ...selection.measures.map((m) => `${m}__compare`), + ]); + const unknown = Object.keys(order).filter((k) => !selectable.has(k)); + if (unknown.length) { + throw new Error( + `[dataset-executor] order key(s) ${unknown.map((k) => `"${k}"`).join(', ')} — ` + + `not a selected dimension or measure. Selectable here: ` + + `${[...selectable].join(', ') || '(none)'}.`, + ); + } + return order; + } + // Implicit, deterministic ordering so a bare `limit` is reproducible. + if ((selection.limit != null || selection.offset != null) && dimensions.length > 0) { + return Object.fromEntries(dimensions.map((d) => [d, 'asc' as const])); + } + return undefined; +} + // ── compareTo date math (deterministic — no Date.now) ──────────────────────── function parseUTC(date: string): number { @@ -205,6 +340,24 @@ export class DatasetExecutor { const baseFilter = combineFilters(compiled.filter, selection.runtimeFilter); const dimensions = selection.dimensions ?? []; + // Effective ordering — validated against what this selection projects, with + // a deterministic dimension order synthesized for a bare `limit` (#3588). + const order = resolveOrdering(selection, dimensions); + + // Push `order`/`limit`/`offset` down into the SQL only when this selection + // is ONE query whose columns can satisfy them. With supplementary + // measure-scoped queries, a compareTo pass, or derived measures in play, the + // grid is assembled from several results — a sub-query LIMIT would drop rows + // before the merge and an ORDER BY would name a column that sub-query never + // selects. Those cases order in memory below instead. + const singleQuery = filtered.length === 0 && !selection.compareTo && selectedDerived.length === 0; + const pushDownKeys = new Set([...dimensions, ...unfiltered]); + const canPushDownWindow = + singleQuery && Object.keys(order ?? {}).every((k) => pushDownKeys.has(k)); + const windowQuery = canPushDownWindow + ? { order, limit: selection.limit, offset: selection.offset } + : undefined; + // Primary query: all unfiltered base measures in one pass. When every base // measure is filter-scoped, the supplementary queries below build the grid. let result: AnalyticsResult; @@ -215,6 +368,7 @@ export class DatasetExecutor { where: baseFilter, selection, contextTimezone: context?.timezone, + window: windowQuery, }), context); } else { result = { rows: [], fields: [] }; @@ -247,6 +401,16 @@ export class DatasetExecutor { result.rows = evaluateDerivedMeasures(result.rows, selectedDerived); for (const d of selectedDerived) result.fields.push({ name: d.name, type: 'number' }); + // Order + window the assembled grid (#3588). Every column the caller may + // sort by exists by now — merged measure-scoped values, `__compare` + // columns, and derived measures included. When the window was already + // pushed into SQL this re-sorts an already-sorted grid (a no-op) and + // re-slices an already-sliced one; when it could not be (the ObjectQL + // aggregate path has no ordering grammar, and date-bucketed queries are + // forced down it), this is what makes `sortBy` work at all. + result.rows = applyOrdering(result.rows, order); + result.rows = applyWindow(result.rows, selection.limit, selection.offset); + return result; } @@ -258,6 +422,13 @@ export class DatasetExecutor { where?: FilterCondition; selection: DatasetSelection; contextTimezone?: string; + /** + * Ordering/window to push DOWN into this query. Set only for a selection + * the caller proved is a single self-sufficient query (see + * `canPushDownWindow`); omitted for supplementary/compare sub-queries, + * which must return their full grid for the merge. + */ + window?: { order?: Record; limit?: number; offset?: number }; }, ): AnalyticsQuery { const q: AnalyticsQuery = { @@ -269,25 +440,56 @@ export class DatasetExecutor { timezone: opts.selection.timezone ?? opts.contextTimezone ?? 'UTC', }; if (opts.where) q.where = opts.where as Record; - // Bucket selected date dimensions that declare an explicit `dateGranularity` - // (the dataset compiled a single-entry `granularities`). Without this a date - // dimension groups by the raw timestamp — one bucket per row, rendering epoch - // millis on trend charts. A dimension already carried by `selection.timeDimensions` - // (e.g. compareTo) keeps its entry; we never override it. + // Bucket selected date dimensions. Without this a date dimension groups by + // the raw timestamp — one bucket per ROW, which is why a "new accounts by + // month" bar chart drew one bar per account instead of one per month + // (#3588). + // + // Granularity precedence, per dimension: + // 1. a `granularity` already stated on that dimension's + // `selection.timeDimensions` entry — never overridden; + // 2. `selection.dateGranularity` — the PRESENTATION's choice, so a widget + // can bucket by month without the dataset committing every consumer to + // that granularity; + // 3. the dataset dimension's own default (the compiler lowers an explicit + // `dateGranularity` to a single-entry `granularities`; the 5-entry + // "all granularities" list means the dataset stated no default). + // + // Note the unit of precedence is the GRANULARITY, not the entry. A + // `timeDimensions` entry that only carries a `dateRange` (which is exactly + // what `compareTo` needs) states a WINDOW, not a bucket size — letting its + // mere presence suppress bucketing left the compared pass grouping raw + // timestamps while the primary pass grouped months, so the two grids shared + // no dimension key and every `__compare` column came back empty. const selTimeDims = opts.selection.timeDimensions ?? []; const selDims = new Set(selTimeDims.map((t) => t.dimension)); + const selectionGranularity = opts.selection.dateGranularity; + const granularityFor = (name: string): string | undefined => { + const cd = compiled.cube.dimensions[name]; + if (cd?.type !== 'time') return undefined; + const datasetDefault = cd.granularities?.length === 1 ? String(cd.granularities[0]) : undefined; + return selectionGranularity ?? datasetDefault; + }; + // Fill in a bucket size for caller-supplied entries that named none. + const resolvedTimeDims = selTimeDims.map((t) => { + if (t.granularity) return t; + const granularity = granularityFor(t.dimension); + return granularity ? { ...t, granularity } : t; + }); const explicitTimeDims: Array<{ dimension: string; granularity: string }> = []; for (const name of opts.dimensions) { - const cd = compiled.cube.dimensions[name]; - if (cd?.type === 'time' && cd.granularities?.length === 1 && !selDims.has(name)) { - explicitTimeDims.push({ dimension: name, granularity: String(cd.granularities[0]) }); - } + if (selDims.has(name)) continue; + const granularity = granularityFor(name); + if (granularity) explicitTimeDims.push({ dimension: name, granularity }); } - const mergedTimeDims = [...selTimeDims, ...explicitTimeDims]; + const mergedTimeDims = [...resolvedTimeDims, ...explicitTimeDims]; if (mergedTimeDims.length > 0) q.timeDimensions = mergedTimeDims as AnalyticsQuery['timeDimensions']; - if (opts.selection.order) q.order = opts.selection.order; - if (opts.selection.limit != null) q.limit = opts.selection.limit; - if (opts.selection.offset != null) q.offset = opts.selection.offset; + // Ordering/window: pushed down ONLY when the caller vouched for it. The + // executor always re-applies both over the assembled grid, so omitting them + // here costs correctness nothing — it only moves the work to memory. + if (opts.window?.order && Object.keys(opts.window.order).length > 0) q.order = opts.window.order; + if (opts.window?.limit != null) q.limit = opts.window.limit; + if (opts.window?.offset != null) q.offset = opts.window.offset; return q; } @@ -313,14 +515,21 @@ export class DatasetExecutor { const shiftedTd = (selection.timeDimensions ?? []).map((t) => t.dimension === cmp.dimension ? { ...t, dateRange: shifted } : t, ); - const sub = await this.service.query({ - cube: compiled.cube.name, + // Built through `buildQuery` so the comparison pass buckets its date + // dimensions EXACTLY like the primary pass. Hand-rolling the query here + // skipped granularity resolution, so a bucketed primary grid ("2026-04") + // was merged against raw-timestamp comparison rows and no dimension key + // ever matched — every `__compare` column came back empty. The shifted + // `timeDimensions` still win for their own dimension (rule 1 of the + // precedence chain); `window` is deliberately omitted — the comparison grid + // must stay whole for the merge. + const sub = await this.service.query(this.buildQuery(compiled, { measures, dimensions, - where: baseFilter as Record | undefined, - timeDimensions: shiftedTd, - timezone: selection.timezone ?? context?.timezone ?? 'UTC', - }, context); + where: baseFilter, + selection: { ...selection, timeDimensions: shiftedTd }, + contextTimezone: context?.timezone, + }), context); // Rename measure columns to `__compare` so they merge alongside primary. return sub.rows.map((row) => { const out: Record = {}; diff --git a/packages/services/service-analytics/src/strategies/objectql-strategy.ts b/packages/services/service-analytics/src/strategies/objectql-strategy.ts index 8436e53d84..84698197ba 100644 --- a/packages/services/service-analytics/src/strategies/objectql-strategy.ts +++ b/packages/services/service-analytics/src/strategies/objectql-strategy.ts @@ -116,41 +116,126 @@ export class ObjectQLStrategy implements AnalyticsStrategy { }); const fields = this.buildFieldMeta(query, cube); - return { rows: mappedRows, fields }; + // Echo a representative SQL alongside the rows (#3588). `NativeSQLStrategy` + // returns the statement it actually ran, and dataset responses surface that + // string — it is how an author checks what their widget compiled to. This + // path builds an AST, so it had nothing to echo, and the `sql` field simply + // vanished from the response whenever a query was date-bucketed (native SQL + // declines granularity, handing those queries here). An author reading the + // response then couldn't tell "bucketing is not implemented" from "this + // strategy doesn't report". Best-effort: rendering is a debugging aid and + // must never fail a query that already ran. + let sql: string | undefined; + try { + sql = (await this.generateSql(query, ctx)).sql; + } catch { + sql = undefined; + } + return sql ? { rows: mappedRows, fields, sql } : { rows: mappedRows, fields }; } + /** + * Render a REPRESENTATIVE SQL string for an ObjectQL aggregate query. + * + * This path executes through `engine.aggregate()`, not raw SQL, so the string + * is documentation rather than the literal statement — but it must be an + * honest account of what the query does, because dataset responses echo it + * and authors read it to verify their widget options landed (#3588). It + * therefore renders date bucketing (`date_trunc`), the WHERE predicate, + * ordering, and the row window. + * + * Filter VALUES are rendered as `$n` placeholders and returned in `params`, + * never inlined: the echoed statement travels to the browser, and a filter + * comparand can carry tenant data. + */ async generateSql(query: AnalyticsQuery, ctx: StrategyContext): Promise<{ sql: string; params: unknown[] }> { const cube = ctx.getCube(query.cube!); if (!cube) { throw new Error(`Cube not found: ${query.cube}`); } - // Generate a representative SQL even though ObjectQL uses AST internally const selectParts: string[] = []; const groupByParts: string[] = []; + const params: unknown[] = []; + + // Date-bucketed dimensions render as `date_trunc('', col)` — + // the SQL shape the driver's own bucketing implements — so a `month` trend + // no longer reads as if it grouped by the raw column. + const granByDim = new Map(); + for (const td of query.timeDimensions ?? []) { + if (td.granularity) granByDim.set(td.dimension, td.granularity); + } + const dimExpr = (dim: string): string => { + const col = this.resolveFieldName(cube, dim, 'dimension'); + const gran = granByDim.get(dim); + return gran ? `date_trunc('${gran}', ${col})` : col; + }; if (query.dimensions) { for (const dim of query.dimensions) { - const col = this.resolveFieldName(cube, dim, 'dimension'); - selectParts.push(`${col} AS "${dim}"`); - groupByParts.push(col); + const expr = dimExpr(dim); + selectParts.push(`${expr} AS "${dim}"`); + groupByParts.push(expr); } } + // A time dimension that is bucketed but not also listed in `dimensions` + // still groups (see `execute`), so it belongs in the rendered GROUP BY too. + for (const [dim] of granByDim) { + if (query.dimensions?.includes(dim)) continue; + const expr = dimExpr(dim); + selectParts.push(`${expr} AS "${dim}"`); + groupByParts.push(expr); + } if (query.measures) { for (const m of query.measures) { const { field, method } = this.resolveMeasureAggregation(cube, m); - const aggSql = method === 'count' ? 'COUNT(*)' : `${method.toUpperCase()}(${field})`; + const aggSql = method === 'count' + ? 'COUNT(*)' + : method === 'count_distinct' + ? `COUNT(DISTINCT ${field})` + : `${method.toUpperCase()}(${field})`; selectParts.push(`${aggSql} AS "${m}"`); } } const tableName = this.extractObjectName(cube); let sql = `SELECT ${selectParts.join(', ')} FROM "${tableName}"`; + + const SQL_OPERATORS: Record = { + equals: '=', notEquals: '<>', gt: '>', gte: '>=', lt: '<', lte: '<=', + contains: 'LIKE', in: 'IN', notIn: 'NOT IN', set: 'IS NOT NULL', notSet: 'IS NULL', + }; + const whereParts: string[] = []; + for (const f of normalizeAnalyticsFilters(query)) { + const col = this.resolveFieldName(cube, f.member, 'any'); + const op = SQL_OPERATORS[f.operator] ?? '='; + if (f.operator === 'set' || f.operator === 'notSet') { + whereParts.push(`${col} ${op}`); + continue; + } + const values = f.values ?? []; + if (values.length === 0) continue; + if (f.operator === 'in' || f.operator === 'notIn') { + const placeholders = values.map((v) => { params.push(v); return `$${params.length}`; }); + whereParts.push(`${col} ${op} (${placeholders.join(', ')})`); + } else { + params.push(values[0]); + whereParts.push(`${col} ${op} $${params.length}`); + } + } + if (whereParts.length > 0) sql += ` WHERE ${whereParts.join(' AND ')}`; + if (groupByParts.length > 0) { sql += ` GROUP BY ${groupByParts.join(', ')}`; } + if (query.order && Object.keys(query.order).length > 0) { + const orderClauses = Object.entries(query.order).map(([f, d]) => `"${f}" ${d.toUpperCase()}`); + sql += ` ORDER BY ${orderClauses.join(', ')}`; + } + if (query.limit != null) sql += ` LIMIT ${query.limit}`; + if (query.offset != null) sql += ` OFFSET ${query.offset}`; - return { sql, params: [] }; + return { sql, params }; } // ── Helpers ────────────────────────────────────────────────────── diff --git a/packages/spec/json-schema.manifest.json b/packages/spec/json-schema.manifest.json index 3c4ddd66b6..60e9a13b21 100644 --- a/packages/spec/json-schema.manifest.json +++ b/packages/spec/json-schema.manifest.json @@ -1602,6 +1602,7 @@ "ui/DashboardHeaderAction", "ui/DashboardNavItem", "ui/DashboardWidget", + "ui/DashboardWidgetOptions", "ui/Dataset", "ui/DatasetDimension", "ui/DatasetMeasure", diff --git a/packages/spec/src/contracts/analytics-service.ts b/packages/spec/src/contracts/analytics-service.ts index 4a92559e1a..64c9880c1f 100644 --- a/packages/spec/src/contracts/analytics-service.ts +++ b/packages/spec/src/contracts/analytics-service.ts @@ -129,7 +129,35 @@ export interface DatasetSelection { runtimeFilter?: FilterCondition; /** Optional time-dimension windows passed through to the runtime. */ timeDimensions?: AnalyticsQuery['timeDimensions']; + /** + * Presentation-scope date bucketing (framework#3588). Applies to every + * selected dimension the dataset declares as a `date` dimension, so a + * widget can bucket a trend by month without the dataset having to declare + * that granularity for every consumer. + * + * Precedence, per dimension: an explicit `timeDimensions` entry for that + * dimension wins, then this selection-level granularity, then the dataset + * dimension's own `dateGranularity` default. Unset leaves each dimension on + * its dataset default (which may be no bucketing at all — grouping by the + * raw column). + */ + dateGranularity?: 'day' | 'week' | 'month' | 'quarter' | 'year'; + /** + * Result ordering, applied by key in insertion order (`{ revenue: 'desc' }`). + * + * Every key must be a selected dimension, a selected measure, or a + * `__compare` column; anything else is rejected rather than + * silently ignored. Ordering is applied AFTER measure-scoped filters are + * merged, `compareTo` columns are attached, and derived measures are + * evaluated — so a derived measure (e.g. a win-rate ratio) is a valid sort + * key even though no single SQL statement computes it. + */ order?: Record; + /** + * Max rows to return, applied after `order`. When `limit` is set without + * `order`, rows are ordered by the selected dimensions ascending first, so + * the truncated window is deterministic rather than an arbitrary subset. + */ limit?: number; offset?: number; /** Compare-to directive — runs a shifted query and attaches `__compare`. */ diff --git a/packages/spec/src/ui/dashboard.zod.ts b/packages/spec/src/ui/dashboard.zod.ts index 3cae90aab5..8fd18f409e 100644 --- a/packages/spec/src/ui/dashboard.zod.ts +++ b/packages/spec/src/ui/dashboard.zod.ts @@ -4,6 +4,7 @@ import { z } from 'zod'; import { ProtectionSchema } from '../shared/protection.zod'; import { MetadataProtectionFields } from '../kernel/metadata-protection.zod'; import { FilterConditionSchema } from '../data/filter.zod'; +import { DateGranularity } from '../data/query.zod'; import { ChartTypeSchema, ChartConfigSchema } from './chart.zod'; import { SnakeCaseIdentifierSchema } from '../shared/identifiers.zod'; import { I18nLabelSchema, AriaPropsSchema } from './i18n.zod'; @@ -122,6 +123,60 @@ const strictWidgetAnalyticsError: z.core.$ZodErrorMap = (issue) => { return base; }; +/** + * Widget `options` — the renderer-extras escape hatch, with the keys that + * reach the ANALYTICS QUERY declared explicitly (framework#3588). + * + * The bag stays open (`passthrough`): presentation-only settings the renderer + * understands — `icon`, `trend`, `columns`, `striped`, `density`, … — are + * carried through untouched and are none of the spec's business. What the + * declared keys below buy is a real contract for the four that are NOT + * presentation-only: they change the SQL the dataset query compiles to, so a + * typo (`sortDirection`, `granularity`) is now an author-time type error + * instead of an option that reads as if it works and silently does nothing. + * + * The dashboard/report renderer lowers these into the `DatasetSelection` it + * posts (`dateGranularity`, `order`, `limit`); see the field docs on + * `DatasetSelection` in `contracts/analytics-service.ts` for their exact + * runtime semantics and precedence. + */ +export const DashboardWidgetOptionsSchema = lazySchema(() => z.object({ + /** + * Bucket this widget's selected DATE dimensions (e.g. group a daily + * `close_date` into months for a trend line). Overrides the dataset + * dimension's own `dateGranularity` default for this widget only. + */ + dateGranularity: DateGranularity.optional() + .describe('Bucket selected date dimensions (day/week/month/quarter/year)'), + + /** + * Order rows by this dimension or measure name — must be one this widget + * actually selects (a `dimensions` entry or a `values` entry). + */ + sortBy: z.string().optional().describe('Dimension/measure name to order by'), + + /** Sort direction for `sortBy` (default ascending). */ + sortOrder: z.enum(['asc', 'desc']).optional().describe('Sort direction for sortBy'), + + /** + * Max rows to render. Applied AFTER ordering, so a "top 10" needs `sortBy` + * to be meaningful; without one the runtime orders by the selected + * dimensions so the truncated window is at least deterministic. + */ + limit: z.number().int().positive().optional().describe('Max rows (applied after ordering)'), + + /** + * Explicit category order for ordered-sequence charts — `funnel` / `pyramid` + * stages above all. Values are the dimension's STORED values (e.g. + * `['qualification', 'needs_analysis', 'proposal', 'negotiation']`), not + * display labels. Omit to let the renderer fall back to the dimension + * field's own picklist option order, which is the pipeline order an author + * already declared on the object. + */ + stageOrder: z.array(z.union([z.string(), z.number(), z.boolean()])).optional() + .describe('Explicit category order for funnel/pyramid stages (stored values)'), +}).passthrough().describe('Widget configuration — declared query keys + open renderer extras')); + /** * Dashboard Widget Schema * A single component on the dashboard grid. @@ -235,8 +290,8 @@ export const DashboardWidgetSchema = lazySchema(() => z.object({ h: z.number(), }).optional().describe('Grid layout position (auto-flowed when omitted)'), - /** Widget specific options (colors, legend, etc.) */ - options: z.unknown().optional().describe('Widget specific configuration'), + /** Widget specific options (colors, legend, etc.) — see {@link DashboardWidgetOptionsSchema}. */ + options: DashboardWidgetOptionsSchema.optional().describe('Widget specific configuration'), /** * Per-widget bindings from a dashboard-level filter (referenced by its @@ -420,6 +475,7 @@ export const DashboardSchema = lazySchema(() => z.object({ export type Dashboard = z.infer; export type DashboardInput = z.input; export type DashboardWidget = z.infer; +export type DashboardWidgetOptions = z.infer; export type DashboardHeader = z.infer; export type DashboardHeaderAction = z.infer; export type WidgetColorVariant = z.infer; From 04792618b6864ce7ab30cc93219ab6db4afe9e9f Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 27 Jul 2026 12:56:25 +0000 Subject: [PATCH 2/3] docs(spec,ui): regenerate dashboard reference + document the widget query options (#3588) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `content/docs/references/ui/dashboard.mdx` is generated from the spec and went stale when `DashboardWidgetOptionsSchema` was added — CI's generated-artifact drift check caught it. Also rewrites the hand-written `options` row in the dashboards guide, which described the bag as "Additional display options". That description is what let this bug happen: four of its keys change the compiled query, and an author had no way to know which. The guide now lists the five declared keys with their effects and the behavioural notes that are easy to get wrong (an unprojectable `sortBy` is rejected, a bare `limit` orders by dimensions first, a derived measure is a valid sort key, a funnel without a declared stage order falls back to value-descending). Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01SEZLBGyNoJMLvtGPqsMPRC --- content/docs/references/ui/dashboard.mdx | 23 +++++++++++-- content/docs/ui/dashboards.mdx | 43 +++++++++++++++++++++++- 2 files changed, 62 insertions(+), 4 deletions(-) diff --git a/content/docs/references/ui/dashboard.mdx b/content/docs/references/ui/dashboard.mdx index 082284608c..f1f4aa0511 100644 --- a/content/docs/references/ui/dashboard.mdx +++ b/content/docs/references/ui/dashboard.mdx @@ -14,8 +14,8 @@ Color variant for dashboard widgets (e.g., KPI cards). ## TypeScript Usage ```typescript -import { Dashboard, DashboardHeader, DashboardHeaderAction, DashboardWidget, GlobalFilter, GlobalFilterOptionsFrom, WidgetActionType, WidgetColorVariant } from '@objectstack/spec/ui'; -import type { Dashboard, DashboardHeader, DashboardHeaderAction, DashboardWidget, GlobalFilter, GlobalFilterOptionsFrom, WidgetActionType, WidgetColorVariant } from '@objectstack/spec/ui'; +import { Dashboard, DashboardHeader, DashboardHeaderAction, DashboardWidget, DashboardWidgetOptions, GlobalFilter, GlobalFilterOptionsFrom, WidgetActionType, WidgetColorVariant } from '@objectstack/spec/ui'; +import type { Dashboard, DashboardHeader, DashboardHeaderAction, DashboardWidget, DashboardWidgetOptions, GlobalFilter, GlobalFilterOptionsFrom, WidgetActionType, WidgetColorVariant } from '@objectstack/spec/ui'; // Validate data const result = Dashboard.parse(data); @@ -107,13 +107,30 @@ Dashboard header action | **dimensions** | `string[]` | optional | Dimension names — X/group/split | | **values** | `string[]` | ✅ | Measure names — Y (at least one) | | **layout** | `{ x: number; y: number; w: number; h: number }` | optional | Grid layout position (auto-flowed when omitted) | -| **options** | `any` | optional | Widget specific configuration | +| **options** | `Record` | optional | Widget specific configuration | | **filterBindings** | `Record` | optional | Per-widget dashboard-filter bindings: filter name → this widget's field, or false to opt out | | **suppressWarnings** | `string[]` | optional | Build diagnostic rule ids suppressed on this widget | | **responsive** | `{ breakpoint?: Enum<'xs' \| 'sm' \| 'md' \| 'lg' \| 'xl' \| '2xl'>; hiddenOn?: Enum<'xs' \| 'sm' \| 'md' \| 'lg' \| 'xl' \| '2xl'>[]; columns?: object; order?: object }` | optional | Responsive layout configuration | | **aria** | `{ ariaLabel?: string; ariaDescribedBy?: string; role?: string }` | optional | ARIA accessibility attributes | +--- + +## DashboardWidgetOptions + +Widget configuration — declared query keys + open renderer extras + +### Properties + +| Property | Type | Required | Description | +| :--- | :--- | :--- | :--- | +| **dateGranularity** | `Enum<'day' \| 'week' \| 'month' \| 'quarter' \| 'year'>` | optional | Bucket selected date dimensions (day/week/month/quarter/year) | +| **sortBy** | `string` | optional | Dimension/measure name to order by | +| **sortOrder** | `Enum<'asc' \| 'desc'>` | optional | Sort direction for sortBy | +| **limit** | `integer` | optional | Max rows (applied after ordering) | +| **stageOrder** | `string \| number \| boolean[]` | optional | Explicit category order for funnel/pyramid stages (stored values) | + + --- ## GlobalFilter diff --git a/content/docs/ui/dashboards.mdx b/content/docs/ui/dashboards.mdx index c02eb7e7b4..3337bf52e6 100644 --- a/content/docs/ui/dashboards.mdx +++ b/content/docs/ui/dashboards.mdx @@ -104,9 +104,50 @@ selects `dimensions` (X / group / split) and `values` (the measures to plot): | `chartConfig` | `object` | optional | Advanced chart configuration | | `colorVariant` | `enum` | optional | KPI/card accent color | | `compareTo` | `enum \| object` | optional | Period-over-period comparison window | -| `options` | `object` | optional | Additional display options | +| `options` | `object` | optional | Renderer extras **plus** the query keys below | | `responsive` | `object` | optional | Responsive behavior | +### Widget `options` + +`options` is an open bag: keys the renderer understands (`icon`, `columns`, +`striped`, `density`, …) pass through untouched. Five keys are **declared**, +because they are not presentation-only — four of them change the query the +dataset compiles to: + +| Key | Type | Effect | +| :--- | :--- | :--- | +| `dateGranularity` | `day \| week \| month \| quarter \| year` | Buckets this widget's selected **date** dimensions. Overrides the dataset dimension's own default for this widget only. | +| `sortBy` | `string` | Orders rows by a dimension or measure **this widget selects**. | +| `sortOrder` | `asc \| desc` | Direction for `sortBy` (default `asc`). | +| `limit` | `number` | Max rows, applied **after** ordering. | +| `stageOrder` | `(string \| number \| boolean)[]` | Explicit stage order for `funnel` / `pyramid`, as the dimension's **stored values**. Omit to use the dimension field's picklist option order. | + +Because they are declared, a misspelling (`sortDirection`, `granularity`) is an +author-time error rather than an option that reads as if it works. + +```ts +{ + id: 'revenue_trend', + type: 'area', + dataset: 'opportunity_metrics', + dimensions: ['close_date'], + values: ['total_amount'], + // Group by month rather than by raw timestamp. + options: { dateGranularity: 'month' }, +} +``` + +Notes on behaviour: + +- `sortBy` must name something the widget selects; the server rejects an order + key it cannot resolve rather than ignoring it. +- A `limit` with no `sortBy` orders by the selected dimensions first, so it + truncates a reproducible window instead of an arbitrary subset. +- Ordering is applied to the finished grid, so a **derived measure** is a valid + `sortBy` even though no single SQL statement computes it. +- A `funnel` with no declared stage order falls back to sorting by value + descending. + ## Datasets Every widget binds to a **dataset** — the single semantic layer (ADR-0021). From d3f745ac02f4b824fa924979c1127aa579968f95 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 27 Jul 2026 13:04:05 +0000 Subject: [PATCH 3/3] chore(spec): refresh the public API surface snapshot for DashboardWidgetOptions (#3588) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The api-surface gate reports 0 breaking, 2 added — the new `DashboardWidgetOptionsSchema` const and its inferred `DashboardWidgetOptions` type, both purely additive. Regenerated and committed as the gate asks. Verified the whole spec generated-artifact set locally this time rather than one gate per CI round-trip: check:docs, check:api-surface, check:skill-refs, check:skill-docs, check:spec-changes, check:upgrade-guide, check:react-blocks — all pass. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01SEZLBGyNoJMLvtGPqsMPRC --- packages/spec/api-surface.json | 2 ++ 1 file changed, 2 insertions(+) diff --git a/packages/spec/api-surface.json b/packages/spec/api-surface.json index 29ddafa93b..6a1f985ecc 100644 --- a/packages/spec/api-surface.json +++ b/packages/spec/api-surface.json @@ -3092,6 +3092,8 @@ "DashboardNavItemSchema (const)", "DashboardSchema (const)", "DashboardWidget (type)", + "DashboardWidgetOptions (type)", + "DashboardWidgetOptionsSchema (const)", "DashboardWidgetSchema (const)", "Dataset (type)", "DatasetDimension (type)",