Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions .changeset/analytics-effective-granularity.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
---
"@objectstack/service-analytics": patch
---

fix(analytics): apply the EFFECTIVE date granularity to bucket labels and drill ranges (#3588 follow-up)

`selection.dateGranularity` (shipped in #3652) reached the `GROUP BY` but not the
post-processing: the bucket-label formatter and the drill-range inverter both
kept reading the DATASET dimension's default. A query was grouped one way and
described another. Found by driving a real dashboard query in a browser against
a dataset whose dimension declares `dateGranularity: 'month'`:

- selection `year` → the row came back labelled **`1970-01`** — a year bucket
re-formatted with the dataset's month granularity, its `"2026"` key re-read as
2026 *milliseconds* past the epoch;
- selection `day` → day buckets were re-labelled as months, so ten distinct days
collapsed into two duplicated keys;
- selection `quarter` / `year` / `day` / `week` → `drillRanges` came back empty,
silently removing drill-through from every bucketed chart.

Granularity precedence now lives in one exported function,
`resolveDimensionGranularity`, called from all three sites that must agree — the
query's `GROUP BY`, the label formatter, and the range inverter. The drift was
possible only because each site resolved it independently.

Two consequences beyond the override case:

- A dataset dimension that declares **no** granularity but is bucketed by the
widget now gets drill ranges too. Previously the range sidecar keyed off the
dataset's own `dateGranularity`, so this case — the one #3588 is actually
about — could never drill.
- `formatDateBucket` no longer mistakes a bare year key for an epoch timestamp.
A year bucket's canonical key IS `"2026"`, which is the only bucket key that
collides with the pure-digit epoch heuristic (`"2026-Q2"`, `"2026-07"` and
`"2026-07-15"` all fail it). Being idempotent over already-formatted keys is
that function's stated contract; the year case just never held.
Original file line number Diff line number Diff line change
@@ -0,0 +1,194 @@
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.

/**
* #3588 follow-up — the EFFECTIVE granularity must reach the post-processing,
* not just the GROUP BY.
*
* `selection.dateGranularity` correctly overrode the dataset dimension's default
* when compiling the query, but two post-processing steps kept reading the
* DATASET default: the bucket-label formatter and the drill-range inverter. The
* result was a query grouped one way and described another. Caught by driving a
* real dashboard query in a browser against a dataset whose dimension declares
* `dateGranularity: 'month'`:
*
* selection 'year' → row labelled "1970-01" (a year key re-parsed as epoch ms)
* selection 'day' → day buckets re-labelled as months, so rows collapsed
* to duplicate keys
* selection 'quarter' → correct label, but `drillRanges` empty — the chart
* silently lost drill-through
*
* These pin the three sites (query, labels, ranges) to one resolution.
*/

import { describe, it, expect } from 'vitest';
import { DatasetSchema } from '@objectstack/spec/ui';
import type { ExecutionContext } from '@objectstack/spec/kernel';
import { AnalyticsService } from '../analytics-service.js';
import { resolveDimensionGranularity } from '../dataset-executor.js';

const CTX = { tenantId: 'org_A' } as ExecutionContext;

/** Mirrors the showcase dataset: the dimension declares a MONTH default. */
const monthly = DatasetSchema.parse({
name: 'task_metrics', label: 'Tasks', object: 'showcase_task', include: [],
dimensions: [{ name: 'created_at', field: 'created_at', type: 'date', dateGranularity: 'month' }],
measures: [{ name: 'task_count', aggregate: 'count' }],
});

/** A dataset that declares NO granularity — the widget supplies it (the hotcrm case). */
const bare = DatasetSchema.parse({
name: 'account_metrics', label: 'Accounts', object: 'crm_account', include: [],
dimensions: [{ name: 'created_at', field: 'created_at', type: 'date' }],
measures: [{ name: 'account_count', aggregate: 'count' }],
});

/**
* Service whose aggregate returns the bucket key the DRIVER would produce for
* the granularity actually requested — so a mismatch between the query's
* granularity and the post-processing's shows up exactly as it did in the browser.
*/
function service(bucketByGranularity: Record<string, string>) {
const seen: { granularity?: string } = {};
const svc = new AnalyticsService({
queryCapabilities: () => ({ nativeSql: false, objectqlAggregate: true, inMemory: false }),
executeAggregate: async (_o, options) => {
const gb = (options.groupBy ?? []) as Array<string | { field: string; dateGranularity?: string }>;
const g = gb.map((x) => (typeof x === 'string' ? undefined : x.dateGranularity)).find(Boolean);
seen.granularity = g;
return [{ created_at: bucketByGranularity[g ?? 'none'], task_count: 10, account_count: 10 }];
},
// `created_at` is a tz-naive calendar date → ranges are exact under any tz.
measureCurrency: (_o, f) => (f === 'created_at' ? { type: 'date' } : undefined),
});
return { svc, seen };
}

const BUCKETS = { day: '2026-07-15', week: '2026-W29', month: '2026-07', quarter: '2026-Q3', year: '2026' };

describe('#3588 — effective granularity reaches labels and drill ranges', () => {
it('labels a YEAR bucket as the year, not as epoch month "1970-01"', async () => {
const { svc, seen } = service(BUCKETS);
const r = await svc.queryDataset(
monthly,
{ dimensions: ['created_at'], measures: ['task_count'], dateGranularity: 'year' },
CTX,
) as any;
expect(seen.granularity).toBe('year');
expect(r.rows[0].created_at).toBe('2026');
expect(r.rows[0].created_at).not.toBe('1970-01');
});

it('drills a YEAR bucket into the whole year', async () => {
const { svc } = service(BUCKETS);
const r = await svc.queryDataset(
monthly,
{ dimensions: ['created_at'], measures: ['task_count'], dateGranularity: 'year' },
CTX,
) as any;
expect(r.drillRanges).toEqual([
{ created_at: { field: 'created_at', gte: '2026-01-01', lt: '2027-01-01' } },
]);
});

it('drills a QUARTER bucket into that quarter — previously the range was dropped', async () => {
const { svc } = service(BUCKETS);
const r = await svc.queryDataset(
monthly,
{ dimensions: ['created_at'], measures: ['task_count'], dateGranularity: 'quarter' },
CTX,
) as any;
expect(r.drillRanges).toEqual([
{ created_at: { field: 'created_at', gte: '2026-07-01', lt: '2026-10-01' } },
]);
});

it('keeps a DAY bucket at day resolution instead of collapsing it to a month', async () => {
const { svc } = service(BUCKETS);
const r = await svc.queryDataset(
monthly,
{ dimensions: ['created_at'], measures: ['task_count'], dateGranularity: 'day' },
CTX,
) as any;
expect(r.rows[0].created_at).toBe('2026-07-15');
expect(r.drillRanges).toEqual([
{ created_at: { field: 'created_at', gte: '2026-07-15', lt: '2026-07-16' } },
]);
});

it('still honours the dataset default when the selection asks for nothing', async () => {
const { svc, seen } = service(BUCKETS);
const r = await svc.queryDataset(monthly, { dimensions: ['created_at'], measures: ['task_count'] }, CTX) as any;
expect(seen.granularity).toBe('month');
expect(r.rows[0].created_at).toBe('2026-07');
expect(r.drillRanges).toEqual([
{ created_at: { field: 'created_at', gte: '2026-07-01', lt: '2026-08-01' } },
]);
});

it('gives a dataset that declares NO granularity both labels and ranges (#3588 case 1)', async () => {
const { svc, seen } = service(BUCKETS);
const r = await svc.queryDataset(
bare,
{ dimensions: ['created_at'], measures: ['account_count'], dateGranularity: 'month' },
CTX,
) as any;
expect(seen.granularity).toBe('month');
expect(r.rows[0].created_at).toBe('2026-07');
// Before, a bare dataset dimension had no `dateGranularity` at all, so the
// range sidecar was skipped and a bucketed chart could not drill.
expect(r.drillRanges).toEqual([
{ created_at: { field: 'created_at', gte: '2026-07-01', lt: '2026-08-01' } },
]);
});
});

describe('resolveDimensionGranularity — the single source of precedence', () => {
it('a stated timeDimensions granularity wins over everything', () => {
expect(resolveDimensionGranularity(
{ timeDimensions: [{ dimension: 'd', granularity: 'week' }], dateGranularity: 'year' }, 'd', 'month',
)).toBe('week');
});

it('the selection beats the dataset default', () => {
expect(resolveDimensionGranularity({ dateGranularity: 'year' }, 'd', 'month')).toBe('year');
});

it('the dataset default applies when the selection is silent', () => {
expect(resolveDimensionGranularity({}, 'd', 'month')).toBe('month');
});

it('a dateRange-only entry states a WINDOW, not a bucket — it must not suppress bucketing', () => {
expect(resolveDimensionGranularity(
{ timeDimensions: [{ dimension: 'd', dateRange: ['2026-01-01', '2026-12-31'] }], dateGranularity: 'month' },
'd', undefined,
)).toBe('month');
});

it('returns undefined when nothing declares one', () => {
expect(resolveDimensionGranularity({}, 'd', undefined)).toBeUndefined();
});
});

describe('formatDateBucket — a year key must survive its own re-labeling', () => {
it('keeps a bare year key as the year, not epoch 1970', async () => {
const { formatDateBucket } = await import('../dimension-labels.js');
// Both forms drivers return for a `date_trunc('year', …)` bucket.
expect(formatDateBucket('2026', 'year')).toBe('2026');
expect(formatDateBucket(2026, 'year')).toBe('2026');
// Idempotent: re-labeling an already-labelled key is a no-op.
expect(formatDateBucket(formatDateBucket('2026', 'year'), 'year')).toBe('2026');
});

it('still reads a real epoch value under year granularity', async () => {
const { formatDateBucket } = await import('../dimension-labels.js');
expect(formatDateBucket(Date.UTC(2026, 5, 15), 'year')).toBe('2026');
expect(formatDateBucket('2026-06-15T00:00:00Z', 'year')).toBe('2026');
});

it('leaves the other granularity keys alone (they never collided)', async () => {
const { formatDateBucket } = await import('../dimension-labels.js');
expect(formatDateBucket('2026-Q2', 'quarter')).toBe('2026-Q2');
expect(formatDateBucket('2026-07', 'month')).toBe('2026-07');
expect(formatDateBucket('2026-07-15', 'day')).toBe('2026-07-15');
});
});
35 changes: 26 additions & 9 deletions packages/services/service-analytics/src/analytics-service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ import type { AnalyticsStrategy, DriverCapabilities, StrategyContext } from './s
import { NativeSQLStrategy } from './strategies/native-sql-strategy.js';
import { ObjectQLStrategy } from './strategies/objectql-strategy.js';
import { compileDataset, type CompiledDataset, type RelationshipResolver } from './dataset-compiler.js';
import { DatasetExecutor } from './dataset-executor.js';
import { DatasetExecutor, resolveDimensionGranularity, type DateGranularityValue } from './dataset-executor.js';
import { resolveDimensionLabels, type DimensionLabelDeps } from './dimension-labels.js';
import { evaluateAnalyticsQueryOverRows } from './preview-evaluator.js';

Expand Down Expand Up @@ -567,22 +567,30 @@ export class AnalyticsService implements IAnalyticsService {
// - unknown field type → safe only under UTC (where the calendar day and
// its instant coincide); under a non-UTC tz we can't tell whether to
// shift, so the dim is omitted and the host drills a superset.
const rangeDims: Array<{ d: (typeof selectedDims)[number]; instant: boolean }> = [];
// The bucket size to invert MUST be the one the query actually grouped by —
// `selection.dateGranularity` overrides the dataset dimension's default
// (#3588). Reading `d.dateGranularity` here meant a widget that bucketed by
// quarter or year got its ranges computed from the dataset's month (or,
// when the dataset declared none, dropped entirely) — so drilling a bucket
// opened the wrong span, or the chart lost drill-through altogether.
const rangeDims: Array<{ d: (typeof selectedDims)[number]; granularity: DateGranularityValue; instant: boolean }> = [];
for (const d of selectedDims) {
if (!d.field || d.type !== 'date' || !d.dateGranularity) continue;
if (!d.field || d.type !== 'date') continue;
const granularity = resolveDimensionGranularity(selection, d.name, d.dateGranularity);
if (!granularity) continue;
const ftype = this.measureCurrency?.(dataset.object, d.field as string)?.type;
if (ftype === 'datetime') rangeDims.push({ d, instant: true });
else if (ftype === 'date') rangeDims.push({ d, instant: false });
else if (rangeTz === 'UTC') rangeDims.push({ d, instant: false });
if (ftype === 'datetime') rangeDims.push({ d, granularity, instant: true });
else if (ftype === 'date') rangeDims.push({ d, granularity, instant: false });
else if (rangeTz === 'UTC') rangeDims.push({ d, granularity, instant: false });
// else: unknown field type under a non-UTC reference tz → omit (superset).
}
if (rangeDims.length && result.rows.length) {
const bound = (ymd: string, instant: boolean): string =>
instant ? new Date(zonedDateStartToUtcMs(ymd, rangeTz)).toISOString() : ymd;
(result as AnalyticsResultWithDrill).drillRanges = result.rows.map((row) => {
const ranges: Record<string, { field: string; gte: string; lt: string }> = {};
for (const { d, instant } of rangeDims) {
const cal = bucketKeyToCalendarRange(row[d.name] as string, d.dateGranularity!);
for (const { d, granularity, instant } of rangeDims) {
const cal = bucketKeyToCalendarRange(row[d.name] as string, granularity);
if (cal) {
ranges[d.name] = { field: d.field as string, gte: bound(cal.start, instant), lt: bound(cal.end, instant) };
}
Expand All @@ -600,9 +608,18 @@ export class AnalyticsService implements IAnalyticsService {
// dimension key verbatim, so this is the single place that turns a stored
// value / FK id into the text a user expects to read.
if (this.labelResolver && selectedDims.length) {
// Same single-source rule as the drill ranges above: a date bucket must be
// LABELLED with the granularity it was actually grouped by (#3588).
// Formatting a `year` bucket with the dataset's `month` default rendered
// it as "1970-01" — the year key re-parsed as an epoch millisecond count.
const dims = selectedDims
.filter((d) => !!d.field)
.map((d) => ({ name: d.name, field: d.field, type: d.type, dateGranularity: d.dateGranularity }));
.map((d) => ({
name: d.name,
field: d.field,
type: d.type,
dateGranularity: resolveDimensionGranularity(selection, d.name, d.dateGranularity),
}));
if (dims.length) {
// #3602 — bind the referenced object's read scope to THIS request so the
// label lookup (a per-record read of the related object) cannot surface a
Expand Down
49 changes: 47 additions & 2 deletions packages/services/service-analytics/src/dataset-executor.ts
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,52 @@ function computeDerived(d: DerivedMeasureSpec, row: Record<string, unknown>): nu
}
}

// ── date bucketing (#3588) ───────────────────────────────────────────────────

/** The date-bucket vocabulary shared by the dataset, the selection, and the
* bucketing utilities in `@objectstack/core`. */
export type DateGranularityValue = NonNullable<DatasetSelection['dateGranularity']>;

/**
* The EFFECTIVE bucket size for one date dimension of a selection — the single
* source of truth for granularity precedence.
*
* Precedence, per dimension:
* 1. a `granularity` already stated on that dimension's `timeDimensions`
* entry — never overridden;
* 2. `selection.dateGranularity` — the presentation's choice, so a widget can
* bucket by month without the dataset committing every consumer to it;
* 3. `datasetDefault` — the dataset dimension's own `dateGranularity`.
*
* The unit of precedence is the GRANULARITY, not the entry: a `timeDimensions`
* entry carrying only a `dateRange` (what `compareTo` needs) states a WINDOW,
* not a bucket size, and must not suppress bucketing.
*
* **Why this is exported.** The bucket size chosen here decides three things
* that MUST agree: the `GROUP BY` the query compiles to, the humanized label
* each bucket key is rendered as, and the half-open `[gte, lt)` range a bucket
* drills into. When the query layer resolved granularity and the post-processing
* in `analytics-service` read the dataset default instead, they silently
* disagreed for every selection that overrode it — a `year` query came back
* labelled `1970-01` (a year bucket re-formatted as a month), a `day` query
* collapsed to duplicate month labels, and `quarter`/`year` lost their drill
* ranges entirely. One function, called from all three sites, is what stops
* that drift recurring.
*/
export function resolveDimensionGranularity(
selection: Pick<DatasetSelection, 'timeDimensions' | 'dateGranularity'>,
dimension: string,
datasetDefault?: string,
): DateGranularityValue | undefined {
// `timeDimensions[].granularity` and the compiled cube's `granularities` are
// both typed as bare strings by their own layers (Cube.js heritage), but the
// only values that reach here come from the dataset/selection granularity
// vocabulary — the same five the bucketing utilities accept.
const stated = (selection.timeDimensions ?? []).find((t) => t.dimension === dimension)?.granularity;
if (stated) return stated as DateGranularityValue;
return selection.dateGranularity ?? (datasetDefault as DateGranularityValue | undefined);
}

// ── ordering + windowing (#3588) ─────────────────────────────────────────────

/**
Expand Down Expand Up @@ -463,12 +509,11 @@ export class DatasetExecutor {
// no dimension key and every `__compare` column came back empty.
const selTimeDims = opts.selection.timeDimensions ?? [];
const selDims = new Set(selTimeDims.map((t) => t.dimension));
const selectionGranularity = opts.selection.dateGranularity;
const granularityFor = (name: string): string | undefined => {
const cd = compiled.cube.dimensions[name];
if (cd?.type !== 'time') return undefined;
const datasetDefault = cd.granularities?.length === 1 ? String(cd.granularities[0]) : undefined;
return selectionGranularity ?? datasetDefault;
return resolveDimensionGranularity(opts.selection, name, datasetDefault);
};
// Fill in a bucket size for caller-supplied entries that named none.
const resolvedTimeDims = selTimeDims.map((t) => {
Expand Down
Loading
Loading