diff --git a/.changeset/temporal-time-axis.md b/.changeset/temporal-time-axis.md new file mode 100644 index 0000000000..bca8ab3187 --- /dev/null +++ b/.changeset/temporal-time-axis.md @@ -0,0 +1,45 @@ +--- +"@objectstack/spec": minor +"@objectstack/driver-memory": patch +"@objectstack/driver-mongodb": patch +--- + +feat(spec,drivers): the temporal conformance matrix gains its `Field.time` axis — and `time` finally gets a storage form off SQL (ADR-0053 D-A3.2) + +`@objectstack/spec/data` gains `TEMPORAL_TIME_ROWS` / `TEMPORAL_TIME_CASES`, +the wall-clock half of the shared matrix. A time gets its own table rather than +a third `kind` on the existing one because it shares no comparand vocabulary +with the other two: no relative token resolves to a wall clock, and the +bare-day whole-day rule (#3777) must **not** reach it — which the table now +asserts rather than assumes, since "the rule leaked into the wrong field type" +is exactly what a conformance matrix is for. The fixture is a business day +carrying the boundaries #3994 measured: both window edges, the pair straddling +the millisecond-suffix width change, midnight and `23:59:59.999`. + +**The axis found a real gap on its first run.** ADR-0053 D-C gave `Field.time` +a canonical form on every SQL dialect, but `driver-memory` and +`driver-mongodb` were never extended — both declared +`TemporalFieldKind = 'datetime' | 'date'`, so a `time` column was never +classified and never coerced. It therefore held whatever each writer produced, +and both stores compare across types by bracket: a text bound matched no +`Date`-written row, in either direction, for every operator. Measured on +`driver-memory`, **8 of the 9 shared cases** returned only the text-written +half — a business-hours window answering `[d_mid, f_close]` instead of +`[c_open, d_mid, e_mid_ms, f_close]`. This is #4047's failure one field type +over, and it survived #4047 because that work extended `datetime` and `date` +without revisiting `time`. On mongo it was also a documentation failure: that +module's canon table has listed `time` as `HH:MM:SS[.fff]` text since #3994, +and nothing implemented it. + +Both drivers now carry `storageTimeValue`, mirroring the SQL +`canonicalTimeOfDay`: `HH:MM:SS`, `.fff` only when the milliseconds are +non-zero, a `Date` / epoch / full-timestamp folding to its **UTC** time-of-day +(never the host's), and totality — an out-of-range wall clock like `'25:00'` +passes through rather than being silently rewritten. Text on both, mongo +included: a wall clock is not an instant, so a BSON `Date` would invent a +calendar day and a zone the author never wrote. + +If you have existing `time` data on either driver, values written as `Date` +objects converge to canonical text on their next write; reads of un-migrated +documents are unchanged. Filters were already unable to reach the mixed half, +so no query that worked before stops working. diff --git a/docs/adr/0053-date-and-datetime-semantics.md b/docs/adr/0053-date-and-datetime-semantics.md index 3873a6dbb9..0a272a160f 100644 --- a/docs/adr/0053-date-and-datetime-semantics.md +++ b/docs/adr/0053-date-and-datetime-semantics.md @@ -911,3 +911,70 @@ Two things, which is the argument for having built it: (#1874's original shape), a `driver-sqlite-wasm` consumer pinning the inheritance seam, and a legacy-storage sweep in the `driver-sql` consumer so the un-backfilled read-repair path answers the same table. + +--- + +## Addendum — D-A3.2: the `Field.time` axis, and the two backends that had no time convention at all + +> **Status:** landed. Extends the D-A3 matrix to the third temporal field type, +> and closes the gap that extension measured. + +### The axis + +`TEMPORAL_TIME_ROWS` / `TEMPORAL_TIME_CASES` sit beside the existing table. A +wall clock gets its **own** table rather than a third `kind` on the first, +because it shares no comparand vocabulary with the other two: no relative token +resolves to a time (`{today}` is a calendar day), and the bare-day whole-day +rule (D-D) must NOT reach it. That last point is asserted rather than assumed — +`nextUtcCalendarDay` widens only a `YYYY-MM-DD` string, and "the rule leaked +into the wrong field type" is exactly the class of defect this matrix exists to +catch, so an inclusive time upper bound is pinned as EXACT. + +The fixture is a business day whose boundaries are the ones #3994 measured: +both window edges, the two rows straddling the millisecond-suffix width change +(`14:30:00` and `14:30:00.500`), midnight, and `23:59:59.999`. Same scope rule +as the parent table — a minutes-only comparand (`'14:30'`) is canonicalised by +a typed backend and compared raw by a type-blind one, so that cell is +schema-aware-only and stays in the typed drivers' own suites, the exact twin of +the `$gt`-on-`datetime` limit above. + +### What the axis found: D-C never reached the non-SQL drivers + +D-C gave `Field.time` a canonical form on every SQL dialect. `driver-memory` +and `driver-mongodb` were never extended: both declared +`TemporalFieldKind = 'datetime' | 'date'`, so `indexTemporalFields` never +classified a time column and `coerceTemporalValue` never touched one. A `time` +field kept whatever each writer produced — and mingo (memory) and BSON (mongo) +both compare across types by bracket, so a text bound matched no `Date`-written +row in either direction, for every operator. + +Measured on `driver-memory`: **8 of the 9 shared time cases** returned only the +text-written half of the fixture. A business-hours window answered +`[d_mid, f_close]` instead of `[c_open, d_mid, e_mid_ms, f_close]`. This is +#4047's failure, one field type over, and it survived #4047 precisely because +that work extended `datetime` and `date` without revisiting `time`. + +Worth recording as a documentation failure too: `mongodb-temporal.ts`'s own +canon table has listed `time` as `HH:MM:SS[.fff]` text since #3994. The +document was right and the code never implemented it — a declared-but-unenforced +claim of the kind Prime Directive #10 is about, invisible until something +executed it. + +### The resolution + +`storageTimeValue` in each driver, mirroring `canonicalTimeOfDay`: `HH:MM:SS`, +`.fff` only when non-zero, a `Date`/epoch/full-timestamp folding to its **UTC** +time-of-day, and totality (an out-of-range wall clock passes through rather +than being rewritten). Text on both, including mongo — a wall clock is not an +instant, so a BSON `Date` would invent a calendar day and a zone the author +never wrote, the same reasoning that keeps `date` as text there. + +The variable width is what makes text storage correct rather than merely +convenient: `.` sorts below every digit, so lexicographic order — which is +exactly what mingo performs and what a mongo text range uses — stays +chronological across both widths. + +Coverage note: the mongo end-to-end sweep needs a real server +(`mongodb-memory-server`), so it runs in CI and skips where no binary is +available. The conversion itself is pinned by `mongodb-time-storage.test.ts`, +which is pure and runs everywhere. diff --git a/packages/formula/src/matches-filter-temporal-conformance.test.ts b/packages/formula/src/matches-filter-temporal-conformance.test.ts index 49b0746416..b97e15bc39 100644 --- a/packages/formula/src/matches-filter-temporal-conformance.test.ts +++ b/packages/formula/src/matches-filter-temporal-conformance.test.ts @@ -15,7 +15,12 @@ */ import { describe, it, expect } from 'vitest'; -import { TEMPORAL_CASES, TEMPORAL_ROWS } from '@objectstack/spec/data'; +import { + TEMPORAL_CASES, + TEMPORAL_ROWS, + TEMPORAL_TIME_CASES, + TEMPORAL_TIME_ROWS, +} from '@objectstack/spec/data'; import { matchesFilterCondition } from './matches-filter'; @@ -48,4 +53,20 @@ describe('matchesFilterCondition — temporal conformance', () => { * cannot be handed. If that ever changes — if a `check` gains a token-bearing * filter form — this comment is the thing to delete, and the axis is already * sitting in the shared table ready to be consumed. + * + * The same reasoning covers the wall-clock sweep below: `TemporalTimeCase` + * carries no token spelling at all, because no relative-date macro resolves to + * a time of day. */ + +describe('matchesFilterCondition — Field.time conformance', () => { + // Type-blind, so the records carry the canonical wall-clock text a converged + // column presents. That the variable-width canon still orders correctly under + // a plain string comparison is the assertion. + for (const c of TEMPORAL_TIME_CASES) { + it(c.name, () => { + const got = TEMPORAL_TIME_ROWS.filter((r) => matchesFilterCondition(r, c.filter)).map((r) => r.id); + expect(got, c.note).toEqual(c.expected); + }); + } +}); diff --git a/packages/plugins/driver-memory/src/memory-temporal-conformance.test.ts b/packages/plugins/driver-memory/src/memory-temporal-conformance.test.ts index 02fee7d5d1..28cb73b35b 100644 --- a/packages/plugins/driver-memory/src/memory-temporal-conformance.test.ts +++ b/packages/plugins/driver-memory/src/memory-temporal-conformance.test.ts @@ -17,7 +17,13 @@ */ import { describe, it, expect, beforeAll } from 'vitest'; -import { TEMPORAL_CASES, TEMPORAL_NOW, TEMPORAL_ROWS } from '@objectstack/spec/data'; +import { + TEMPORAL_CASES, + TEMPORAL_NOW, + TEMPORAL_ROWS, + TEMPORAL_TIME_CASES, + TEMPORAL_TIME_ROWS, +} from '@objectstack/spec/data'; import { resolveFilterTokens } from '@objectstack/core'; import { InMemoryDriver } from './memory-driver.js'; @@ -63,3 +69,33 @@ describe('driver-memory — temporal conformance', () => { } } }); + +describe('driver-memory — Field.time conformance', () => { + let driver: InMemoryDriver; + + beforeAll(async () => { + driver = new InMemoryDriver({}); + await driver.connect(); + await driver.syncSchema('time_conformance', { + name: 'time_conformance', + fields: { at: { type: 'time' }, why: { type: 'string' } }, + }); + for (const r of TEMPORAL_TIME_ROWS) { + await driver.create('time_conformance', { + id: r.id, + // The mixed-writer axis (D-E4) for wall clocks: a `Date` write and a + // canonical-text write of the SAME wall clock must converge. + at: r.writerForm === 'native' ? new Date(`1970-01-01T${r.at}Z`) : r.at, + why: r.why, + }); + } + }); + + for (const c of TEMPORAL_TIME_CASES) { + it(c.name, async () => { + const rows = await driver.find('time_conformance', { where: c.filter } as any); + const got = (rows as any[]).map((r) => r.id).sort(); + expect(got, c.note).toEqual([...c.expected].sort()); + }); + } +}); diff --git a/packages/plugins/driver-memory/src/memory-temporal.ts b/packages/plugins/driver-memory/src/memory-temporal.ts index 7aecd8b1ab..a7f7cd9e1e 100644 --- a/packages/plugins/driver-memory/src/memory-temporal.ts +++ b/packages/plugins/driver-memory/src/memory-temporal.ts @@ -21,6 +21,7 @@ * |---|---|---| * | `datetime` | canonical UTC ISO text (`…T…Z`, ms precision) | this store has no native instant type; ISO-8601 UTC sorts chronologically under the plain string comparison mingo performs, and it is the wire form, so it survives JSON persistence unchanged. | * | `date` | `YYYY-MM-DD` text | timezone-naive by ADR-0053 Phase 1 — an instant would re-couple it to a zone. | + * | `time` | `HH:MM:SS`, `.fff` only when non-zero | a timezone-naive wall clock (ADR-0053 D-C1); the variable width still sorts chronologically because `.` sorts below every digit. | * * The rule is applied on write ({@link toStorageForms}) and to filter * comparands, which is the pairing that keeps the two sides from disagreeing. @@ -89,8 +90,56 @@ export function storageDateValue(value: unknown): unknown { return value; } +/** + * Collapse a `Field.time` value to a canonical timezone-naive wall clock — + * `HH:MM:SS`, extended to `HH:MM:SS.fff` exactly when the milliseconds are + * non-zero. Mirrors `SqlDriver`'s `canonicalTimeOfDay` (ADR-0053 D-C1), so a + * fixture that moves between the two backends compares identically. + * + * This driver had no time rule at all, which is the same meta-problem #3994 + * found on SQL and #4047 found here for `datetime`: with nothing normalising + * the column, a `Date` write and a `'09:00:00'` write sat side by side, and + * mingo's cross-type comparison meant a text bound matched no `Date` row in + * either direction. Measured: 8 of the 9 shared time cases returned only the + * text-written half of the fixture. + * + * Why variable width rather than a fixed `.000`: `.` sorts below every digit, + * so lexicographic order — which is exactly what mingo performs on strings — + * stays chronological across the two widths (`'14:30:00.100' < '14:30:01'`), + * and the zero-millisecond spelling stays the `HH:MM:SS` every dialect's + * native TIME emits. + * + * A `Date` / epoch-ms / full-timestamp value folds to its **UTC** time-of-day, + * never the host's, matching the platform's instant semantics everywhere else. + * Total: an out-of-range wall clock (`'25:00'`) or unparseable junk passes + * through untouched rather than being silently rewritten. + */ +export function storageTimeValue(value: unknown): unknown { + if (value == null) return value; + if (typeof value === 'string') { + const s = value.trim(); + if (s === '') return value; + const m = /^(\d{2}):(\d{2})(?::(\d{2})(?:\.(\d+))?)?$/.exec(s); + if (m) { + const [, hh, mm, ss = '00', frac] = m; + if (Number(hh) > 23 || Number(mm) > 59 || Number(ss) > 59) return value; + const ms = frac ? `${frac}000`.slice(0, 3) : '000'; + return ms === '000' ? `${hh}:${mm}:${ss}` : `${hh}:${mm}:${ss}.${ms}`; + } + } + // Not a bare wall clock — a `Date`, epoch ms, or a full/zone-naive timestamp + // string. Delegate to the one function that owns instants, then keep its UTC + // time-of-day. Same delegation `canonicalTimeOfDay` performs. + const instant = storageDatetimeValue(value); + if (typeof instant === 'string' && /^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}Z$/.test(instant)) { + const time = instant.slice(11, 23); + return time.endsWith('.000') ? time.slice(0, 8) : time; + } + return value; +} + /** Which temporal rule a declared field takes, if any. */ -export type TemporalFieldKind = 'datetime' | 'date'; +export type TemporalFieldKind = 'datetime' | 'date' | 'time'; /** * Put a value into the storage form of a field of `kind`. `undefined` kind — @@ -100,7 +149,9 @@ export type TemporalFieldKind = 'datetime' | 'date'; export function coerceTemporalValue(value: unknown, kind: TemporalFieldKind | undefined): unknown { if (kind === undefined) return value; if (Array.isArray(value)) return value.map((v) => coerceTemporalValue(v, kind)); - return kind === 'datetime' ? storageDatetimeValue(value) : storageDateValue(value); + if (kind === 'datetime') return storageDatetimeValue(value); + if (kind === 'time') return storageTimeValue(value); + return storageDateValue(value); } /** @@ -114,6 +165,7 @@ export function indexTemporalFields( for (const [name, def] of Object.entries(fields ?? {})) { if (def?.type === 'datetime') out.set(name, 'datetime'); else if (def?.type === 'date') out.set(name, 'date'); + else if (def?.type === 'time') out.set(name, 'time'); } return out; } diff --git a/packages/plugins/driver-mongodb/src/mongodb-filter.ts b/packages/plugins/driver-mongodb/src/mongodb-filter.ts index 297a9fcb59..6dc7382854 100644 --- a/packages/plugins/driver-mongodb/src/mongodb-filter.ts +++ b/packages/plugins/driver-mongodb/src/mongodb-filter.ts @@ -18,7 +18,11 @@ import type { Filter } from 'mongodb'; import { nextUtcCalendarDay } from '@objectstack/core'; -import { coerceTemporalValue, type TemporalFieldKindResolver } from './mongodb-temporal.js'; +import { + coerceTemporalValue, + type TemporalFieldKind, + type TemporalFieldKindResolver, +} from './mongodb-temporal.js'; /** * Translate an ObjectStack `where` clause into a MongoDB filter document. @@ -132,7 +136,11 @@ function translateCondition( */ function translateFieldOperators( ops: Record, - kind?: 'datetime' | 'date', + // The shared type, not a hand-copy of its members. This signature spelled + // the union out literally, so widening the canon to include `time` + // (ADR-0053 D-C1) left the two out of step and the call site stopped + // compiling. One definition means the next temporal type is added once. + kind?: TemporalFieldKind, ): Record { const result: Record = {}; const store = (v: unknown) => coerceTemporalValue(v, kind); diff --git a/packages/plugins/driver-mongodb/src/mongodb-temporal-conformance.test.ts b/packages/plugins/driver-mongodb/src/mongodb-temporal-conformance.test.ts index cf6736757f..5cd05bf8fc 100644 --- a/packages/plugins/driver-mongodb/src/mongodb-temporal-conformance.test.ts +++ b/packages/plugins/driver-mongodb/src/mongodb-temporal-conformance.test.ts @@ -20,7 +20,13 @@ import { describe, it, expect, beforeAll, afterAll } from 'vitest'; import { MongoMemoryServer } from 'mongodb-memory-server'; -import { TEMPORAL_CASES, TEMPORAL_NOW, TEMPORAL_ROWS } from '@objectstack/spec/data'; +import { + TEMPORAL_CASES, + TEMPORAL_NOW, + TEMPORAL_ROWS, + TEMPORAL_TIME_CASES, + TEMPORAL_TIME_ROWS, +} from '@objectstack/spec/data'; import { resolveFilterTokens } from '@objectstack/core'; import { MongoDBDriver } from './mongodb-driver.js'; @@ -61,9 +67,12 @@ describe.skipIf(!sharedMongod)('driver-mongodb — temporal conformance', () => } }, 90_000); + // Only this suite's own connection — the SERVER is shared with the time + // suite below and is stopped once, at file level, after both have run. + // Stopping it here left `getUri()` throwing `Incorrect State … gotState: + // 'new'` for the second suite, whose cases then never executed at all. afterAll(async () => { if (driver) await driver.disconnect(); - if (mongod) await mongod.stop(); }); for (const c of TEMPORAL_CASES) { @@ -82,3 +91,45 @@ describe.skipIf(!sharedMongod)('driver-mongodb — temporal conformance', () => } } }); + +describe.skipIf(!sharedMongod)('driver-mongodb — Field.time conformance', () => { + const mongod = sharedMongod as MongoMemoryServer; + let driver: MongoDBDriver; + + beforeAll(async () => { + driver = new MongoDBDriver({ url: mongod.getUri(), database: 'time_conformance' }); + await driver.connect(); + await driver.syncSchema('time_conformance', { + name: 'time_conformance', + fields: { at: { type: 'time' }, why: { type: 'string' } }, + }); + for (const r of TEMPORAL_TIME_ROWS) { + await driver.create('time_conformance', { + id: r.id, + // The mixed-writer axis for wall clocks: a `Date` write and a + // canonical-text write of the SAME wall clock must converge, or BSON + // type-bracket comparison hides one half from every text bound. + at: r.writerForm === 'native' ? new Date(`1970-01-01T${r.at}Z`) : r.at, + why: r.why, + }); + } + }, 90_000); + + afterAll(async () => { + if (driver) await driver.disconnect(); + }); + + for (const c of TEMPORAL_TIME_CASES) { + it(c.name, async () => { + const rows = await driver.find('time_conformance', { where: c.filter } as any); + const got = (rows as any[]).map((r) => r.id).sort(); + expect(got, c.note).toEqual([...c.expected].sort()); + }); + } +}); + +// The shared server outlives both suites, so it is torn down once here rather +// than by whichever suite happens to finish first. +afterAll(async () => { + if (sharedMongod) await sharedMongod.stop(); +}); diff --git a/packages/plugins/driver-mongodb/src/mongodb-temporal.ts b/packages/plugins/driver-mongodb/src/mongodb-temporal.ts index bb5a42b989..c83ba66b7f 100644 --- a/packages/plugins/driver-mongodb/src/mongodb-temporal.ts +++ b/packages/plugins/driver-mongodb/src/mongodb-temporal.ts @@ -92,8 +92,58 @@ export function storageDateValue(value: unknown): unknown { return value; } +/** + * Collapse a `Field.time` value to a canonical timezone-naive wall clock — + * `HH:MM:SS`, extended to `HH:MM:SS.fff` exactly when the milliseconds are + * non-zero. Mirrors `SqlDriver`'s `canonicalTimeOfDay` (ADR-0053 D-C1). + * + * Text rather than a BSON `Date`, for the same reason `date` is text: a wall + * clock is not an instant (#2004), and storing one as a `Date` would invent a + * calendar day and a zone the author never wrote. The canon table at the top of + * this module has claimed this form since #3994 — but nothing implemented it, + * so a `time` column was left holding whatever each writer produced, and BSON + * type-bracket comparison made a text bound miss every `Date`-written row. The + * shared conformance table measures it: 8 of its 9 cases returned only the + * text-written half of the fixture. + * + * Why variable width: `.` sorts below every digit, so lexicographic order stays + * chronological across the two widths (`'14:30:00.100' < '14:30:01'`), and the + * zero-millisecond spelling stays the `HH:MM:SS` every dialect's native TIME + * emits. + * + * A `Date` / epoch-ms / full-timestamp value folds to its **UTC** time-of-day, + * never the server's. Total: an out-of-range wall clock (`'25:00'`) or + * unparseable junk passes through untouched. + */ +export function storageTimeValue(value: unknown): unknown { + if (value == null) return value; + if (typeof value === 'string') { + const s = value.trim(); + if (s === '') return value; + const m = /^(\d{2}):(\d{2})(?::(\d{2})(?:\.(\d+))?)?$/.exec(s); + if (m) { + const [, hh, mm, ss = '00', frac] = m; + if (Number(hh) > 23 || Number(mm) > 59 || Number(ss) > 59) return value; + const ms = frac ? `${frac}000`.slice(0, 3) : '000'; + return ms === '000' ? `${hh}:${mm}:${ss}` : `${hh}:${mm}:${ss}.${ms}`; + } + } + // Not a bare wall clock — a `Date`, epoch ms, or a full/zone-naive timestamp. + // Delegate to the one function that owns instants, then keep its UTC + // time-of-day. (That one returns a BSON `Date` here, so read the parts off it + // rather than off an ISO string.) + const instant = storageDatetimeValue(value); + if (instant instanceof Date && !Number.isNaN(instant.getTime())) { + const p = (n: number, w = 2) => String(n).padStart(w, '0'); + const base = `${p(instant.getUTCHours())}:${p(instant.getUTCMinutes())}:${p(instant.getUTCSeconds())}`; + const ms = instant.getUTCMilliseconds(); + return ms === 0 ? base : `${base}.${p(ms, 3)}`; + } + return value; +} + /** Which temporal rule a declared field takes, if any. */ -export type TemporalFieldKind = 'datetime' | 'date'; +export type TemporalFieldKind = 'datetime' | 'date' | 'time'; /** * Resolves the temporal kind of `field` on the object being queried, or @@ -114,7 +164,9 @@ export function coerceTemporalValue( ): unknown { if (kind === undefined) return value; if (Array.isArray(value)) return value.map((v) => coerceTemporalValue(v, kind)); - return kind === 'datetime' ? storageDatetimeValue(value) : storageDateValue(value); + if (kind === 'datetime') return storageDatetimeValue(value); + if (kind === 'time') return storageTimeValue(value); + return storageDateValue(value); } /** @@ -129,6 +181,7 @@ export function indexTemporalFields( for (const [name, def] of Object.entries(fields ?? {})) { if (def?.type === 'datetime') out.set(name, 'datetime'); else if (def?.type === 'date') out.set(name, 'date'); + else if (def?.type === 'time') out.set(name, 'time'); } return out; } diff --git a/packages/plugins/driver-mongodb/src/mongodb-time-storage.test.ts b/packages/plugins/driver-mongodb/src/mongodb-time-storage.test.ts new file mode 100644 index 0000000000..cf7cc0daef --- /dev/null +++ b/packages/plugins/driver-mongodb/src/mongodb-time-storage.test.ts @@ -0,0 +1,159 @@ +// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license. + +/** + * `Field.time` gets a storage form on MongoDB — the D-C1 canon transplanted + * (ADR-0053; the SQL original is #3994, the cross-driver twin for `datetime` + * is #4047). + * + * The module's own canon table has declared `time` as `HH:MM:SS[.fff]` text + * since #3994, but nothing implemented it: `TemporalFieldKind` was + * `'datetime' | 'date'`, so `indexTemporalFields` never classified a time + * column and `coerceTemporalValue` never touched one. A `time` field therefore + * kept whatever each writer produced, and BSON compares across types by + * bracket — so a text bound matched no `Date`-written row, in either + * direction, for every operator. The shared conformance table measures the + * result: 8 of its 9 cases returned only the text-written half of the fixture. + * + * `mongodb-temporal-conformance.test.ts` runs those cases against a real + * server; this file pins the pure conversion, which needs no server and so + * runs everywhere the suite does — including the environments where + * `mongodb-memory-server` cannot fetch a binary and the conformance sweep + * skips. + */ + +import { describe, it, expect } from 'vitest'; +import { + coerceTemporalValue, + indexTemporalFields, + storageTimeValue, +} from './mongodb-temporal.js'; +import { translateFilter } from './mongodb-filter.js'; + +describe('storageTimeValue — the canonical wall clock (D-C1)', () => { + it('keeps a canonical value byte-identical', () => { + expect(storageTimeValue('09:00:00')).toBe('09:00:00'); + expect(storageTimeValue('14:30:00.500')).toBe('14:30:00.500'); + expect(storageTimeValue('23:59:59.999')).toBe('23:59:59.999'); + }); + + it('completes a minutes-only wall clock so one time has one spelling', () => { + // Determinism is what equality and distinct() rest on. + expect(storageTimeValue('14:30')).toBe('14:30:00'); + }); + + it('drops a zero-millisecond suffix to the native TIME spelling', () => { + expect(storageTimeValue('14:30:00.000')).toBe('14:30:00'); + }); + + it('truncates sub-millisecond precision rather than rounding it', () => { + // Matches `Date` resolution, and rounding would move the stored wall clock + // — the defect MySQL's bare TIME had (#3994). + expect(storageTimeValue('14:30:00.5009')).toBe('14:30:00.500'); + }); + + it('folds a Date to its UTC time-of-day, not the host zone', () => { + // The pg failure this mirrors: a bound Date serialised in the process's + // local zone, so the stored wall clock depended on the host's TZ. + expect(storageTimeValue(new Date('1970-01-01T14:30:00.500Z'))).toBe('14:30:00.500'); + expect(storageTimeValue(new Date('2026-07-28T09:00:00.000Z'))).toBe('09:00:00'); + expect(storageTimeValue(new Date('1970-01-01T00:00:00.000Z'))).toBe('00:00:00'); + }); + + it('folds epoch ms and a full ISO string the same way', () => { + expect(storageTimeValue(Date.parse('1970-01-01T18:00:00.000Z'))).toBe('18:00:00'); + expect(storageTimeValue('2026-07-28T23:59:59.999Z')).toBe('23:59:59.999'); + }); + + it('is total: uninterpretable input passes through untouched', () => { + // Never silently rewritten — junk keeps failing the comparison instead of + // becoming a plausible wrong wall clock. + expect(storageTimeValue('25:00')).toBe('25:00'); + expect(storageTimeValue('12:60:00')).toBe('12:60:00'); + expect(storageTimeValue('not a time')).toBe('not a time'); + expect(storageTimeValue('')).toBe(''); + expect(storageTimeValue(null)).toBeNull(); + expect(storageTimeValue(undefined)).toBeUndefined(); + }); + + it('orders chronologically under plain string comparison across both widths', () => { + // The property the whole variable-width canon rests on: `.` sorts below + // every digit, which is what makes a range filter correct on a text column. + const sorted = ['14:30:01', '14:30:00.100', '14:30:00', '09:00:00'] + .map((v) => storageTimeValue(v) as string) + .sort(); + expect(sorted).toEqual(['09:00:00', '14:30:00', '14:30:00.100', '14:30:01']); + }); +}); + +describe('the time kind reaches the coercion seam', () => { + it('classifies a declared time field', () => { + const kinds = indexTemporalFields({ + at: { type: 'time' }, + when: { type: 'datetime' }, + on: { type: 'date' }, + name: { type: 'string' }, + }); + expect(kinds.get('at')).toBe('time'); + expect(kinds.get('when')).toBe('datetime'); + expect(kinds.get('on')).toBe('date'); + expect(kinds.has('name')).toBe(false); + }); + + it('coerces a time comparand — including every element of an $in set', () => { + expect(coerceTemporalValue(new Date('1970-01-01T09:00:00Z'), 'time')).toBe('09:00:00'); + expect(coerceTemporalValue(['14:30', new Date('1970-01-01T18:00:00Z')], 'time')).toEqual([ + '14:30:00', + '18:00:00', + ]); + }); + + it('leaves an undeclared field alone — the driver never guesses from a value', () => { + const value = new Date('1970-01-01T09:00:00Z'); + expect(coerceTemporalValue(value, undefined)).toBe(value); + }); +}); + +describe('the time canon reaches the FILTER path, not just the write path', () => { + // The two halves must apply the same rule or they disagree again — the + // #3912/#4047 lesson. The end-to-end sweep needs a real server + // (`mongodb-temporal-conformance.test.ts`); this pins the comparand + // translation, which is pure, so the filter half is verified everywhere. + const timeKind = (field: string) => (field === 'at' ? ('time' as const) : undefined); + + it('canonicalises a bound comparand on both sides of a window', () => { + expect( + translateFilter( + { at: { $gte: new Date('1970-01-01T09:00:00Z'), $lte: new Date('1970-01-01T18:00:00Z') } }, + timeKind, + ), + ).toEqual({ at: { $gte: '09:00:00', $lte: '18:00:00' } }); + }); + + it('completes a minutes-only comparand so it can match a stored wall clock', () => { + expect(translateFilter({ at: { $eq: '14:30' } }, timeKind)).toEqual({ at: { $eq: '14:30:00' } }); + }); + + it('leaves an inclusive upper bound EXACT — the calendar-day rule must not reach a time', () => { + // `nextUtcCalendarDay` widens only a bare `YYYY-MM-DD`; a wall clock never + // matches that shape, so `$lte` stays `$lte` rather than becoming `$lt`. + expect(translateFilter({ at: { $lte: '14:30:00' } }, timeKind)).toEqual({ + at: { $lte: '14:30:00' }, + }); + }); + + it('canonicalises every element of an $in set', () => { + expect( + translateFilter({ at: { $in: ['09:00', new Date('1970-01-01T18:00:00Z')] } }, timeKind), + ).toEqual({ at: { $in: ['09:00:00', '18:00:00'] } }); + }); + + it('canonicalises an implicit-equality comparand', () => { + expect(translateFilter({ at: new Date('1970-01-01T14:30:00.500Z') }, timeKind)).toEqual({ + at: '14:30:00.500', + }); + }); + + it('does not touch a field the resolver does not call temporal', () => { + expect(translateFilter({ why: '14:30' }, timeKind)).toEqual({ why: '14:30' }); + }); +}); diff --git a/packages/plugins/driver-sql/src/sql-driver-temporal-conformance.test.ts b/packages/plugins/driver-sql/src/sql-driver-temporal-conformance.test.ts index 321a60d2f5..fded3f1809 100644 --- a/packages/plugins/driver-sql/src/sql-driver-temporal-conformance.test.ts +++ b/packages/plugins/driver-sql/src/sql-driver-temporal-conformance.test.ts @@ -32,7 +32,13 @@ */ import { describe, it, expect, beforeAll, afterAll } from 'vitest'; -import { TEMPORAL_CASES, TEMPORAL_NOW, TEMPORAL_ROWS } from '@objectstack/spec/data'; +import { + TEMPORAL_CASES, + TEMPORAL_NOW, + TEMPORAL_ROWS, + TEMPORAL_TIME_CASES, + TEMPORAL_TIME_ROWS, +} from '@objectstack/spec/data'; import { resolveFilterTokens } from '@objectstack/core'; import { SqlDriver } from '../src/index.js'; import { LegacyStorageDriver } from './legacy-datetime-storage.testkit.js'; @@ -140,3 +146,43 @@ describe('sql-driver — temporal conformance on un-backfilled legacy storage', }); } }); + +describe('sql-driver — Field.time conformance', () => { + let driver: SqlDriver; + + beforeAll(async () => { + driver = new SqlDriver({ + client: 'better-sqlite3', + connection: { filename: ':memory:' }, + useNullAsDefault: true, + }); + await driver.initObjects([ + { name: 'time_conformance', fields: { at: { type: 'time' }, why: { type: 'string' } } }, + ]); + for (const r of TEMPORAL_TIME_ROWS) { + await driver.create( + 'time_conformance', + { + id: r.id, + // The mixed-writer axis for wall clocks (#3994 measured this exact + // column): a bound `Date` and a canonical-text write must converge. + at: r.writerForm === 'native' ? new Date(`1970-01-01T${r.at}Z`) : r.at, + why: r.why, + }, + { bypassTenantAudit: true } as any, + ); + } + }); + + afterAll(async () => { + await driver.disconnect?.(); + }); + + for (const c of TEMPORAL_TIME_CASES) { + it(c.name, async () => { + const rows = await driver.find('time_conformance', { where: c.filter } as any); + const got = (rows as any[]).map((r) => r.id).sort(); + expect(got, c.note).toEqual([...c.expected].sort()); + }); + } +}); diff --git a/packages/plugins/driver-sqlite-wasm/src/sqlite-wasm-temporal-conformance.test.ts b/packages/plugins/driver-sqlite-wasm/src/sqlite-wasm-temporal-conformance.test.ts index 45705370f1..1e8bb2857c 100644 --- a/packages/plugins/driver-sqlite-wasm/src/sqlite-wasm-temporal-conformance.test.ts +++ b/packages/plugins/driver-sqlite-wasm/src/sqlite-wasm-temporal-conformance.test.ts @@ -13,7 +13,13 @@ */ import { describe, it, expect, beforeAll, afterAll } from 'vitest'; -import { TEMPORAL_CASES, TEMPORAL_NOW, TEMPORAL_ROWS } from '@objectstack/spec/data'; +import { + TEMPORAL_CASES, + TEMPORAL_NOW, + TEMPORAL_ROWS, + TEMPORAL_TIME_CASES, + TEMPORAL_TIME_ROWS, +} from '@objectstack/spec/data'; import { resolveFilterTokens } from '@objectstack/core'; import { SqliteWasmDriver } from './index.js'; @@ -66,3 +72,39 @@ describe('driver-sqlite-wasm — temporal conformance', () => { } } }); + +describe('driver-sqlite-wasm — Field.time conformance', () => { + let driver: SqliteWasmDriver; + + beforeAll(async () => { + driver = new SqliteWasmDriver({ filename: ':memory:' }); + await driver.initObjects([ + { name: 'time_conformance', fields: { at: { type: 'time' }, why: { type: 'string' } } }, + ]); + for (const r of TEMPORAL_TIME_ROWS) { + await driver.create( + 'time_conformance', + { + id: r.id, + // The mixed-writer axis for wall clocks (#3994 measured this exact + // column): a bound `Date` and a canonical-text write must converge. + at: r.writerForm === 'native' ? new Date(`1970-01-01T${r.at}Z`) : r.at, + why: r.why, + }, + { bypassTenantAudit: true } as any, + ); + } + }); + + afterAll(async () => { + await (driver as any).knex.destroy(); + }); + + for (const c of TEMPORAL_TIME_CASES) { + it(c.name, async () => { + const rows = await driver.find('time_conformance', { where: c.filter } as any); + const got = (rows as any[]).map((r) => r.id).sort(); + expect(got, c.note).toEqual([...c.expected].sort()); + }); + } +}); diff --git a/packages/services/service-analytics/src/__tests__/preview-temporal-conformance.test.ts b/packages/services/service-analytics/src/__tests__/preview-temporal-conformance.test.ts index e7f8994fe1..ae47801445 100644 --- a/packages/services/service-analytics/src/__tests__/preview-temporal-conformance.test.ts +++ b/packages/services/service-analytics/src/__tests__/preview-temporal-conformance.test.ts @@ -18,7 +18,13 @@ */ import { describe, it, expect } from 'vitest'; -import { TEMPORAL_CASES, TEMPORAL_NOW, TEMPORAL_ROWS } from '@objectstack/spec/data'; +import { + TEMPORAL_CASES, + TEMPORAL_NOW, + TEMPORAL_ROWS, + TEMPORAL_TIME_CASES, + TEMPORAL_TIME_ROWS, +} from '@objectstack/spec/data'; import type { Cube } from '@objectstack/spec/data'; import { resolveFilterTokens } from '@objectstack/core'; import { evaluateAnalyticsQueryOverRows, matchesWhere } from '../preview-evaluator.js'; @@ -74,3 +80,12 @@ describe('preview-evaluator — timeDimensions.dateRange temporal conformance', }); } }); + +describe('preview-evaluator — Field.time conformance', () => { + for (const c of TEMPORAL_TIME_CASES) { + it(c.name, () => { + const got = TEMPORAL_TIME_ROWS.filter((r) => matchesWhere(r as any, c.filter as any)).map((r) => r.id); + expect(got, c.note).toEqual(c.expected); + }); + } +}); diff --git a/packages/spec/api-surface.json b/packages/spec/api-surface.json index 06a14958a8..35cd28c03d 100644 --- a/packages/spec/api-surface.json +++ b/packages/spec/api-surface.json @@ -532,12 +532,16 @@ "TEMPORAL_CASES (const)", "TEMPORAL_NOW (const)", "TEMPORAL_ROWS (const)", + "TEMPORAL_TIME_CASES (const)", + "TEMPORAL_TIME_ROWS (const)", "TITLE_ELIGIBLE (const)", "TITLE_ELIGIBLE_TYPES (const)", "TITLE_INELIGIBLE_TYPES (const)", "TemporalCase (interface)", "TemporalFieldKind (type)", "TemporalRow (interface)", + "TemporalTimeCase (interface)", + "TemporalTimeRow (interface)", "TemporalWriterForm (type)", "TenancyConfig (type)", "TenancyConfigSchema (const)", diff --git a/packages/spec/src/data/temporal-conformance.ts b/packages/spec/src/data/temporal-conformance.ts index 4b47e81eec..fa5d0b9afc 100644 --- a/packages/spec/src/data/temporal-conformance.ts +++ b/packages/spec/src/data/temporal-conformance.ts @@ -344,3 +344,142 @@ export const TEMPORAL_CASES: readonly TemporalCase[] = [ note: 'The `expires_on: { $in: [daysFromNow(30)] }` template shape from the #1874 family — element-wise, order-independent.', }, ] as const; + +// ── `Field.time` — the third temporal type, its own axis ───────────────────── + +/** + * A row in the time-of-day fixture. + * + * `at` is a `Field.time`: a timezone-naive WALL CLOCK (#2004), not an instant + * and not a calendar day. The canonical form is `HH:MM:SS`, extended to + * `HH:MM:SS.fff` exactly when the milliseconds are non-zero (ADR-0053 D-C1) — + * variable width on purpose, because `.` sorts below every digit, so + * lexicographic order stays chronological order across the two widths. + */ +export interface TemporalTimeRow { + id: string; + /** `Field.time` — canonical wall-clock text. */ + at: string; + /** + * Writer-population seeding hint — the D-E4 axis, one field type over. See + * {@link TemporalWriterForm}. `native` means a driver consumer writes the + * row as a JS `Date` whose UTC time-of-day IS `at` (build it from + * `1970-01-01T${at}Z`), which is what an SDK caller or a bound `NOW()` + * produces; `wire` means the canonical text a REST/JSON write delivers. + * + * Seeding both is the point. A fixture written entirely in canonical text + * passes even on a backend with NO time convention at all, because both + * sides of every comparison happen to be the same string — so it would + * assert nothing. The mixed column is what #3994 measured on SQL, and it is + * the shape that must not survive on any other backend either. + */ + writerForm: TemporalWriterForm; + /** Why the row is in the fixture — surfaced when a case fails. */ + why: string; +} + +/** + * The fixture: a business day, chosen so the boundaries that broke #3994 are + * each decidable. `c_open` and `f_close` sit exactly ON the window edges, + * `d_mid`/`e_mid_ms` straddle the millisecond-suffix width change, and + * `a_midnight`/`g_last` are the extremes of the 24-hour range. + */ +export const TEMPORAL_TIME_ROWS: readonly TemporalTimeRow[] = [ + { id: 'a_midnight', at: '00:00:00', writerForm: 'native', why: 'the day\'s lower extreme, written as a Date — epoch 0 on SQLite, which sorts before every TEXT row' }, + { id: 'b_early', at: '08:00:00', writerForm: 'wire', why: 'before opening — the row a business-hours lower bound must exclude' }, + { id: 'c_open', at: '09:00:00', writerForm: 'native', why: 'exactly the opening bound, which is inclusive' }, + { id: 'd_mid', at: '14:30:00', writerForm: 'wire', why: 'mid-afternoon, zero milliseconds — the `HH:MM:SS` spelling every dialect\'s native TIME emits' }, + { id: 'e_mid_ms', at: '14:30:00.500', writerForm: 'native', why: 'the same wall clock plus 500ms: the width change lexicographic order has to survive, and what MySQL bare TIME rounded away' }, + { id: 'f_close', at: '18:00:00', writerForm: 'wire', why: 'exactly the closing bound, which is inclusive' }, + { id: 'g_last', at: '23:59:59.999', writerForm: 'native', why: 'the day\'s upper extreme — the last representable wall clock' }, +] as const; + +/** One time conformance case: a filter on the wall-clock column. */ +export interface TemporalTimeCase { + /** Stable identifier, usable as a test name. */ + name: string; + filter: FilterCondition; + /** Ids of matching rows, ascending. */ + expected: string[]; + /** Why the case is here — surfaced in failure output. */ + note?: string; +} + +/** + * The cases. + * + * # Why `Field.time` needs its own table rather than a `kind` on the one above + * + * A wall clock shares no comparand vocabulary with the other two: no relative + * token resolves to one (`{today}` is a calendar day), and the bare-day + * whole-day rule (D-D) must NOT reach it — `nextUtcCalendarDay` only widens a + * `YYYY-MM-DD` string, and a time never matches that shape. That last point is + * asserted rather than assumed, because "the rule leaks into the wrong field + * type" is precisely the class of bug this file exists to catch. + * + * # The scope rule is the same: only what EVERY backend must agree on + * + * One cell is deliberately absent. A minutes-only comparand (`{ at: '14:30' }`) + * is canonicalised to `'14:30:00'` by a TYPED backend and compared as raw text + * by a type-blind one, where `'14:30' !== '14:30:00'` — so it matches on the + * drivers and misses in `formula`/the preview evaluator. It is the exact twin + * of the `$gt`-on-`datetime` gap documented above, irreducible for the same + * reason, and stays pinned in the typed drivers' own suites + * (`sql-driver-time-of-day.test.ts`). Every case below uses a fully-spelled + * canonical comparand, which both readings agree on. + */ +export const TEMPORAL_TIME_CASES: readonly TemporalTimeCase[] = [ + { + name: 'time: a business-hours window keeps both bounds', + filter: { at: { $gte: '09:00:00', $lte: '18:00:00' } }, + expected: ['c_open', 'd_mid', 'e_mid_ms', 'f_close'], + note: '#3994, measured: 4 of 7 rows silently dropped. An epoch-ms row failed `>= 09:00:00` outright (INTEGER < TEXT on SQLite) and a full-timestamp row failed `<= 18:00:00` lexicographically.', + }, + { + name: 'time: an inclusive upper bound is EXACT, not widened', + filter: { at: { $lte: '14:30:00' } }, + expected: ['a_midnight', 'b_early', 'c_open', 'd_mid'], + note: 'The bare-day whole-day rule (D-D) must not reach a time column: `nextUtcCalendarDay` widens only a `YYYY-MM-DD` string. If it leaked, e_mid_ms (14:30:00.500) would appear.', + }, + { + name: 'time: the millisecond suffix sorts after its zero-ms twin', + filter: { at: { $gt: '14:30:00' } }, + expected: ['e_mid_ms', 'f_close', 'g_last'], + note: 'The variable-width canon (D-C1) only works because `.` sorts below every digit; d_mid is excluded by strict-greater and e_mid_ms is kept.', + }, + { + name: 'time: a sub-second window bounds the millisecond row', + filter: { at: { $gte: '14:30:00.100', $lt: '14:30:01' } }, + expected: ['e_mid_ms'], + note: 'Guards the same ordering from the other side — `14:30:00.500` must fall inside a window whose bounds differ only after the decimal point.', + }, + { + name: 'time: equality on a canonical wall clock', + filter: { at: '14:30:00' }, + expected: ['d_mid'], + note: 'Determinism is what equality and distinct() rest on: one wall clock has exactly one canonical spelling, so e_mid_ms is a DIFFERENT value, not a near-match.', + }, + { + name: 'time: $in of two canonical wall clocks', + filter: { at: { $in: ['09:00:00', '18:00:00'] } }, + expected: ['c_open', 'f_close'], + }, + { + name: 'time: $between spans the working day inclusively', + filter: { at: { $between: ['08:00:00', '18:00:00'] } }, + expected: ['b_early', 'c_open', 'd_mid', 'e_mid_ms', 'f_close'], + note: 'Both ends inclusive, and — like the $lte case — NOT widened by the calendar-day rule.', + }, + { + name: 'time: midnight is an ordinary wall clock, not a null', + filter: { at: { $lt: '09:00:00' } }, + expected: ['a_midnight', 'b_early'], + note: 'A `00:00:00` write that became INTEGER 0 (a bound `Date` at UTC midnight) sorted before every TEXT row on SQLite and was reachable by no text bound at all.', + }, + { + name: 'time: the last wall clock of the day is reachable', + filter: { at: { $gte: '23:59:59' } }, + expected: ['g_last'], + note: 'The upper extreme, where an implementation that truncates or rounds the millisecond suffix (MySQL bare TIME did) loses the row.', + }, +] as const;