Skip to content

Commit 0c8a22f

Browse files
os-zhuangclaude
os-zhuang
and
claude
authored
feat(spec): one canonical conformance table for the filter logical combinators (#3812)
* feat(spec): one canonical conformance table for the filter logical combinators `FilterCondition` is evaluated by four independent implementations — the SQL compiler in driver-sql, driver-memory's matcher, formula's record-at-a-time evaluator (the RLS write-side `check`), and service-analytics' read-scope SQL lowering — and nothing held them to a shared standard. #3774 was the cost: the SQL compiler OR-ed the contents within a `$or` branch instead of AND-ing them, widening every `$or` filter. The other three were correct by luck rather than by enforcement. That fix left three near-identical shape tables copied across packages and the fourth backend unlocked entirely, which is the same drift setup one step later. `@objectstack/spec/data` now exports FILTER_LOGIC_ROWS and FILTER_LOGIC_CASES, and each backend has a thin test that runs the shared rows through its own evaluator. Adding a case to the table adds it to all four at once. read-scope-sql is now verified by executing its SQL against a real engine and comparing rows, not just by asserting the emitted string; it passes unchanged. The table is a public export, so third-party driver authors can check a new backend against the same standard. Scope is deliberately limited to the logical combinators — null handling, dates, coercion and LIKE escaping legitimately differ between a SQL engine and a JS matcher, and folding them in would make the table unpassable. Co-Authored-By: Claude <noreply@anthropic.com> * test(driver-sql): tighten the nested-filter assertion to exact ids `should handle complex nested filters` asserted `toBeGreaterThan(0)`, which any non-empty result satisfies — it could not distinguish a correct compile from a wrong one. To be precise about what this does and does not buy: the shape it exercises, `{$or:[{$and:[…]},{$and:[…]}]}`, was never miscompiled by #3774. Each `$or` branch holds a single `$and` key, so there were no sibling keys to wrongly OR; I confirmed this by reverting the fix and watching the tightened assertion still pass. The assertion is worth tightening because it was too weak to have caught the difference either way, not because it now catches that bug. Co-Authored-By: Claude <noreply@anthropic.com> * fix(service-analytics): run the read-scope conformance on WASM SQLite The first revision imported `better-sqlite3` and killed the vitest worker on CI — a process-level abort with no JS error to catch, so all 17 conformance cases silently did not run while the suite reported 22/23 files passing. Reproduced locally under Node 20 (CI's version): `better-sqlite3@13` declares `engines: >=22`, and a native binding is only loadable by the exact Node ABI it was built for. `driver-sql` can afford that dependency because it falls back to WASM SQLite when the binding fails to load; a test has no such fallback. Switched to `sql.js` — the same engine that fallback lands on. Pure WASM, no ABI to match, no build step. Verified on both Node 20 and Node 25: 17/17 conformance cases and 286/286 for the package. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Jack Zhuang <277994282+os-zhuang@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com>
1 parent 763931e commit 0c8a22f

11 files changed

Lines changed: 461 additions & 302 deletions

File tree

‎.changeset/filter-logic-conformance-single-source.md‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
---
2+
"@objectstack/spec": minor
3+
---
4+
5+
feat(spec): one canonical conformance table for the filter logical combinators
6+
7+
`FilterCondition` is evaluated by four independent implementations, and nothing
8+
held them to a shared standard:
9+
10+
| Backend | Where |
11+
|---|---|
12+
| SQL compiler | `driver-sql` `applyFilterCondition` |
13+
| In-memory matcher | `driver-memory` `memory-matcher` |
14+
| Record-at-a-time evaluator | `formula` `matchesFilterCondition` (RLS write-side `check`) |
15+
| Read-scope SQL lowering | `service-analytics` `read-scope-sql` |
16+
17+
In #3774 the SQL compiler OR-ed the contents *within* a `$or` branch instead of
18+
AND-ing them, so every `$or` filter matched more rows than it should. The other
19+
three were correct — but that was luck, not enforcement, and the divergence was
20+
invisible until someone ran a real query. The fix for #3774 left three
21+
near-identical shape tables copied across packages and the fourth backend
22+
unlocked entirely, which is the same drift setup one step later.
23+
24+
`@objectstack/spec/data` now exports the table itself:
25+
26+
- `FILTER_LOGIC_ROWS` — a 2x2 truth table over two columns (so a wrongly-OR-ed
27+
pair always shows up as extra ids rather than by luck of the data), plus the
28+
record-scope columns real read scopes are written against.
29+
- `FILTER_LOGIC_CASES` — 17 cases, each a `FilterCondition` and the ids it must
30+
match: keys within a branch, multiple operators on one field, `$and`/`$or`/
31+
`$not` nesting in both key orders, and the scope shapes that occur in shipped
32+
metadata.
33+
34+
Each backend now has a thin test that feeds the rows through its own evaluator
35+
and asserts the shared expectations. **Adding a case to the table adds it to all
36+
four at once** — that is the point.
37+
38+
Two things this bought immediately:
39+
40+
- `read-scope-sql` — the compiler that lowers RLS read scopes for the analytics
41+
path — is now verified by **executing** its SQL against a real engine and
42+
comparing rows. It was previously only checked by asserting the emitted SQL
43+
string, whose ceiling is the author's own reading of SQL. It passes unchanged.
44+
- The table is a public export, so a third-party driver author can check a new
45+
backend against the same standard.
46+
47+
**Deliberate scope:** logical combinators only. The predicates are boring on
48+
purpose — string equality, `$in`, `$ne`, `$gte`/`$lt`. Nothing here exercises
49+
null handling, dates, numeric coercion, `LIKE` escaping or case sensitivity,
50+
because those legitimately differ between a SQL engine and a JS matcher; folding
51+
them in would make the table unpassable rather than more useful. A case belongs
52+
in it only if **every** backend must agree.

‎packages/formula/src/matches-filter-or-semantics.test.ts‎

Lines changed: 19 additions & 66 deletions
Original file line numberDiff line numberDiff line change
@@ -1,78 +1,31 @@
11
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
22

33
/**
4-
* `$or` semantics conformance for the record-at-a-time filter evaluator.
4+
* Filter logical-combinator conformance for the record-at-a-time evaluator.
55
*
6-
* Companion to `driver-sql`'s `sql-driver-or-filter.test.ts` and
7-
* `driver-memory`'s `memory-matcher-or-semantics.test.ts`: same shapes, same
8-
* 2x2 fixture, same expected ids. driver-sql used to OR the field keys within a
9-
* `$or` branch, widening the match set; this evaluator was already correct.
6+
* The cases come from `@objectstack/spec/data` so this backend, `driver-sql`,
7+
* `driver-memory` and `read-scope-sql` are all held to one standard — see
8+
* `filter-logic-conformance.ts` for why that standard exists (#3774). Adding a
9+
* case there adds it to all four at once; that is the point.
1010
*
11-
* Agreement here is load-bearing rather than cosmetic: this evaluator decides
12-
* the RLS `check` clause on writes while the SQL compiler decides the `where`
13-
* on reads. If they disagree on `$or`, a record can be writable but unreadable
14-
* (or worse, readable when the scope says otherwise).
11+
* Agreement matters here beyond tidiness: this evaluator decides the RLS
12+
* `check` clause on writes while the SQL compilers decide the `where` on reads.
13+
* If they disagree, a record can be writable but unreadable — or readable when
14+
* the scope says otherwise.
1515
*/
1616

1717
import { describe, expect, it } from 'vitest';
18+
import { FILTER_LOGIC_CASES, FILTER_LOGIC_ROWS } from '@objectstack/spec/data';
1819

1920
import { matchesFilterCondition as m } from './matches-filter';
2021

21-
const ROWS = [
22-
{ id: '1', a: 'x', b: 'y', c: 'z' },
23-
{ id: '2', a: 'x', b: 'zz', c: 'z' },
24-
{ id: '3', a: 'qq', b: 'y', c: 'z' },
25-
{ id: '4', a: 'qq', b: 'zz', c: 'z' },
26-
];
27-
28-
const ids = (filter: any): string[] => ROWS.filter((r) => m(r, filter)).map((r) => r.id);
29-
30-
describe('matchesFilterCondition — $or semantics', () => {
31-
it('ANDs the keys of a single multi-key branch', () => {
32-
expect(ids({ $or: [{ a: 'x', b: 'y' }] })).toEqual(['1']);
33-
});
34-
35-
it('ANDs the keys of each branch independently', () => {
36-
expect(ids({ $or: [{ a: 'x', b: 'y' }, { a: 'qq', b: 'zz' }] })).toEqual(['1', '4']);
37-
});
38-
39-
it('ANDs operator-object keys within a branch', () => {
40-
expect(ids({ $or: [{ a: { $eq: 'x' }, b: { $ne: 'zz' } }] })).toEqual(['1']);
41-
});
42-
43-
it('ANDs keys inside a $or nested in a $or branch', () => {
44-
expect(ids({ $or: [{ $or: [{ a: 'x', b: 'zz' }] }] })).toEqual(['2']);
45-
});
46-
47-
it('ANDs multiple operators on ONE field within a branch', () => {
48-
expect(ids({ $or: [{ a: { $ne: 'qq', $eq: 'x' }, b: 'y' }] })).toEqual(['1']);
49-
});
50-
51-
it('OR-s a $and branch against a sibling multi-key branch', () => {
52-
expect(ids({ $or: [{ $and: [{ a: 'x' }, { b: 'y' }] }, { a: 'qq', b: 'zz' }] })).toEqual(['1', '4']);
53-
});
54-
55-
it('ANDs a $and with a sibling key in the same branch, either order', () => {
56-
expect(ids({ $or: [{ c: 'nope' }, { $and: [{ a: 'qq' }], b: 'y' }] })).toEqual(['3']);
57-
expect(ids({ $or: [{ c: 'nope' }, { b: 'y', $and: [{ a: 'qq' }] }] })).toEqual(['3']);
58-
});
59-
60-
it('keeps single-key $or branches as a plain OR', () => {
61-
expect(ids({ $or: [{ a: 'x' }, { b: 'y' }] })).toEqual(['1', '2', '3']);
62-
});
63-
64-
it('ANDs a $or with a sibling top-level field key', () => {
65-
expect(ids({ $or: [{ a: 'x' }, { b: 'y' }], b: 'zz' })).toEqual(['2']);
66-
});
67-
68-
it('does not widen an "own AND active, OR shared" read scope', () => {
69-
const docs = [
70-
{ id: 'own-active', owner: 'u1', status: 'active', shared_with: null },
71-
{ id: 'own-archived', owner: 'u1', status: 'archived', shared_with: null },
72-
{ id: 'other-active', owner: 'u2', status: 'active', shared_with: null },
73-
{ id: 'shared', owner: 'u2', status: 'active', shared_with: 'u1' },
74-
];
75-
const scope = { $or: [{ owner: 'u1', status: 'active' }, { shared_with: 'u1' }] };
76-
expect(docs.filter((d) => m(d, scope)).map((d) => d.id)).toEqual(['own-active', 'shared']);
77-
});
22+
describe('matchesFilterCondition — filter logic conformance', () => {
23+
for (const c of FILTER_LOGIC_CASES) {
24+
it(c.name, () => {
25+
const got = FILTER_LOGIC_ROWS.filter((r) => m(r as unknown as Record<string, unknown>, c.filter)).map(
26+
(r) => r.id,
27+
);
28+
expect(got, c.note).toEqual(c.expected);
29+
});
30+
}
7831
});

‎packages/plugins/driver-memory/src/memory-matcher-or-semantics.test.ts‎

Lines changed: 14 additions & 65 deletions
Original file line numberDiff line numberDiff line change
@@ -1,75 +1,24 @@
11
// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.
22

33
/**
4-
* `$or` semantics conformance for the in-memory matcher.
4+
* Filter logical-combinator conformance for the in-memory matcher.
55
*
6-
* Companion to `driver-sql`'s `sql-driver-or-filter.test.ts`: the same filter
7-
* shapes, the same 2x2 fixture, the same expected ids. driver-sql used to
8-
* compile a `$or` branch's own field keys with OR instead of AND, so
9-
* `{$or:[{a,b}]}` matched strictly more rows than the Filter Protocol allows.
10-
* This matcher was already correct — these cases exist so the two backends
11-
* cannot silently drift apart again, since a read scope evaluated by one and
12-
* pushed down by the other must agree.
6+
* The cases come from `@objectstack/spec/data` so this backend, `driver-sql`,
7+
* `formula`'s `matchesFilterCondition` and `read-scope-sql` are all held to one
8+
* standard — see `filter-logic-conformance.ts` for why that standard exists
9+
* (#3774). Adding a case there adds it to all four at once; that is the point.
1310
*/
1411

1512
import { describe, it, expect } from 'vitest';
16-
import { match } from './memory-matcher.js';
17-
18-
const ROWS = [
19-
{ id: '1', a: 'x', b: 'y', c: 'z' },
20-
{ id: '2', a: 'x', b: 'zz', c: 'z' },
21-
{ id: '3', a: 'qq', b: 'y', c: 'z' },
22-
{ id: '4', a: 'qq', b: 'zz', c: 'z' },
23-
];
24-
25-
const ids = (filter: any): string[] => ROWS.filter((r) => match(r, filter)).map((r) => r.id);
26-
27-
describe('memory-matcher $or semantics', () => {
28-
it('ANDs the keys of a single multi-key branch', () => {
29-
expect(ids({ $or: [{ a: 'x', b: 'y' }] })).toEqual(['1']);
30-
});
31-
32-
it('ANDs the keys of each branch independently', () => {
33-
expect(ids({ $or: [{ a: 'x', b: 'y' }, { a: 'qq', b: 'zz' }] })).toEqual(['1', '4']);
34-
});
35-
36-
it('ANDs operator-object keys within a branch', () => {
37-
expect(ids({ $or: [{ a: { $eq: 'x' }, b: { $ne: 'zz' } }] })).toEqual(['1']);
38-
});
13+
import { FILTER_LOGIC_CASES, FILTER_LOGIC_ROWS } from '@objectstack/spec/data';
3914

40-
it('ANDs keys inside a $or nested in a $or branch', () => {
41-
expect(ids({ $or: [{ $or: [{ a: 'x', b: 'zz' }] }] })).toEqual(['2']);
42-
});
43-
44-
it('ANDs multiple operators on ONE field within a branch', () => {
45-
expect(ids({ $or: [{ a: { $ne: 'qq', $eq: 'x' }, b: 'y' }] })).toEqual(['1']);
46-
});
47-
48-
it('OR-s a $and branch against a sibling multi-key branch', () => {
49-
expect(ids({ $or: [{ $and: [{ a: 'x' }, { b: 'y' }] }, { a: 'qq', b: 'zz' }] })).toEqual(['1', '4']);
50-
});
51-
52-
it('ANDs a $and with a sibling key in the same branch, either order', () => {
53-
expect(ids({ $or: [{ c: 'nope' }, { $and: [{ a: 'qq' }], b: 'y' }] })).toEqual(['3']);
54-
expect(ids({ $or: [{ c: 'nope' }, { b: 'y', $and: [{ a: 'qq' }] }] })).toEqual(['3']);
55-
});
56-
57-
it('keeps single-key $or branches as a plain OR', () => {
58-
expect(ids({ $or: [{ a: 'x' }, { b: 'y' }] })).toEqual(['1', '2', '3']);
59-
});
60-
61-
it('ANDs a $or with a sibling top-level field key', () => {
62-
expect(ids({ $or: [{ a: 'x' }, { b: 'y' }], b: 'zz' })).toEqual(['2']);
63-
});
15+
import { match } from './memory-matcher.js';
6416

65-
it('does not widen an "own AND active, OR shared" read scope', () => {
66-
const docs = [
67-
{ id: 'own-active', owner: 'u1', status: 'active', shared_with: null },
68-
{ id: 'own-archived', owner: 'u1', status: 'archived', shared_with: null },
69-
{ id: 'other-active', owner: 'u2', status: 'active', shared_with: null },
70-
{ id: 'shared', owner: 'u2', status: 'active', shared_with: 'u1' },
71-
];
72-
const scope = { $or: [{ owner: 'u1', status: 'active' }, { shared_with: 'u1' }] };
73-
expect(docs.filter((d) => match(d, scope)).map((d) => d.id)).toEqual(['own-active', 'shared']);
74-
});
17+
describe('memory-matcher — filter logic conformance', () => {
18+
for (const c of FILTER_LOGIC_CASES) {
19+
it(c.name, () => {
20+
const got = FILTER_LOGIC_ROWS.filter((r) => match(r, c.filter)).map((r) => r.id);
21+
expect(got, c.note).toEqual(c.expected);
22+
});
23+
}
7524
});

‎packages/plugins/driver-sql/src/sql-driver-advanced.test.ts‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -323,7 +323,14 @@ describe('SqlDriver Advanced Operations (SQLite)', () => {
323323
},
324324
});
325325

326-
expect(results.length).toBeGreaterThan(0);
326+
// Was `toBeGreaterThan(0)`, which any non-empty result satisfies. This
327+
// particular shape was never miscompiled by #3774 — each `$or` branch
328+
// holds a single `$and` key, so there were no sibling keys to wrongly OR
329+
// — but the assertion was too weak to have noticed either way, which is
330+
// the only reason it is worth tightening. Branch one is completed AND
331+
// over 100 (Laptop 1200, Monitor 350); branch two is Alice AND pending
332+
// (Keyboard).
333+
expect(results.map((r: any) => r.id).sort()).toEqual(['1', '3', '4']);
327334
});
328335

329336
it('should handle contains filter', async () => {

0 commit comments

Comments
 (0)