Skip to content

Commit bdd4172

Browse files
os-zhuangclaude
andcommitted
feat(drivers,objectql): $regex 响亮拒收 + SQL 族 $icontains 实现 (#5702)
#4706 裁决 B 案的驱动半边。#5701 已落契约(词表 / RETIRED_FILTER_OPERATORS 处方 / 共享文本 case-set),#5710 已翻掉最后一个活体生产者(plugin-auth 的 ObjectQL 适配器,认证路径),因此拒收现在可以落地而不打断登录。 - 五个拒收点(driver-sql / driver-memory / driver-turso remote / driver-mongodb / objectql `having`)统一逐字打印 `RETIRED_FILTER_OPERATORS[op].why`,并点名 `$icontains`; - `$icontains` 在 SQL 族实现:复用 `applyLike` / `pushLike` 的 `%`/`_`/`\` 转义与显式 ESCAPE,新增 `fold` 参数把 LOWER() 套在两侧; - driver-mongodb 的 `default:` 裸 `new Error` 接进本文件既有的 `INVALID_FILTER` / 400 信封; - conformance LEDGER 五行按实测重写,剩余两半分别记为 #6518 / #6520。 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WyvqvKMG6asi9aXjKE6xtx
1 parent 259459d commit bdd4172

20 files changed

Lines changed: 1305 additions & 222 deletions
Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
---
2+
"@objectstack/driver-sql": minor
3+
"@objectstack/driver-sqlite-wasm": minor
4+
"@objectstack/driver-turso": minor
5+
"@objectstack/driver-memory": minor
6+
"@objectstack/driver-mongodb": minor
7+
"@objectstack/objectql": minor
8+
---
9+
10+
feat(drivers,objectql): `$regex` / `$options` are refused everywhere, and `$icontains` is implemented on the SQL family (#5702)
11+
12+
The driver half of the #4706 ruling. #5701 landed the contract (the vocabulary,
13+
the `RETIRED_FILTER_OPERATORS` prescriptions, the shared text case-set) and
14+
#5710 flipped the last live producer — `plugin-auth`'s ObjectQL adapter, which
15+
emitted `$regex` on the authentication path — so the refusal can now land
16+
without breaking sign-in.
17+
18+
**BREAKING for anyone writing `$regex` or `$options` in a filter.** Both are
19+
refused on every backend with `INVALID_FILTER` / 400 and a message that names
20+
the replacement. `$regex` was never a declared operator: `driver-sql` compiled
21+
it to a LIKE-escaped substring (so `a.b` matched only the literal `a.b`),
22+
`driver-memory` ran it as a real `RegExp` (so the same filter also matched
23+
`axb`, and an *invalid* pattern was caught and answered `false` — zero rows, in
24+
silence), and `objectql`'s `having` did the same. Write `$icontains` for the
25+
case-insensitive substring search this was almost always used for, `$contains`
26+
for a case-sensitive one; a pattern that genuinely needs a regex has no
27+
filter-level replacement.
28+
29+
**`$icontains` now runs on the SQL family**`driver-sql`, `driver-sqlite-wasm`,
30+
and both of `driver-turso`'s transports (the remote one does not go through
31+
knex, so it needed its own). It compiles to `LOWER(col) LIKE LOWER(?) ESCAPE ?`
32+
through the same `applyLike` / `pushLike` that carries the `%` / `_` / `\`
33+
escaping, as a `fold` parameter rather than a second emitter — a copied emitter
34+
is where the escape class would have been dropped, and an unescaped `%` matches
35+
every row. An empty or non-string comparand is refused on the validating walk
36+
(an empty one matches every row, which widens rather than narrows). On SQLite
37+
`lower()` folds ASCII only, which IS the contract (#4706 Q1 = A): `$icontains:
38+
'café'` does not match `CAFÉ`.
39+
40+
`driver-mongodb`'s unknown-operator arm was throwing a bare `Error` with no
41+
`code` and no `status`, three lines from the helper in its own file that sets
42+
`INVALID_FILTER` / 400 — a 500-shaped body for a 400-class client mistake. It
43+
now speaks the same envelope as its three siblings.
44+
45+
Two parts of the ruling are deliberately NOT in this change and stay tracked in
46+
`scripts/check-driver-conformance.mjs`'s ledger: the `$contains` family's
47+
case-sensitivity (#4706 Q2 = A) needs SQLite's `LIKE` replaced by a case-exact
48+
construct in the driver, the RLS lowering and the analytics lowering together,
49+
or one permission rule compiles to two row sets (#6518); and `$icontains` on the
50+
JS evaluation faces needs the spec vocabulary to take the operator, which cannot
51+
happen before `driver-memory` has an arm for it (#6520).

packages/drivers/driver-memory/src/filter-refusal.ts

Lines changed: 79 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@
3030
* vocabulary; it does not get to drop what falls outside it.
3131
*/
3232

33-
import { FILTER_OPERATORS, LOGICAL_OPERATORS } from '@objectstack/spec/data';
33+
import { FILTER_OPERATORS, LOGICAL_OPERATORS, RETIRED_FILTER_OPERATORS } from '@objectstack/spec/data';
3434
import { StandardErrorCode } from '@objectstack/spec/api';
3535

3636
/**
@@ -148,18 +148,27 @@ export function emptyFieldConstraintError(field: string, path: string): Error {
148148
* `convertConditionToMongo`'s alias fold) — a list written out here would agree
149149
* with the spec on the day it was typed and never again.
150150
*
151-
* Two additions the spec's list does not carry, both deliberate and both
152-
* pre-existing behaviour rather than new capability:
151+
* ## [#5702] The two additions are GONE — nothing is added any more
153152
*
154-
* - **`$regex`** — not in `FILTER_OPERATORS`, but really produced: plugin-auth's
155-
* ObjectQL adapter emits `{ field: { $regex: value } }` for a `contains`
156-
* search. `driver-sql` compiles it (to a substring LIKE), `objectql`'s
157-
* `having` allows it, and this driver's matcher implements it. Refusing it
158-
* here would break a live producer.
159-
* - **`$options`** — the regex-flags companion `memory-matcher` reads
160-
* (`new RegExp(target, condition.$options)`) and `objectql`'s `having` skips
161-
* for the same reason. It is a modifier of `$regex`, not a predicate of its
162-
* own.
153+
* This set used to be `[...FILTER_OPERATORS, '$regex', '$options']`. Both extra
154+
* members existed for one reason, recorded here verbatim at the time: *"Refusing
155+
* it here would break a live producer"* — plugin-auth's ObjectQL adapter emitted
156+
* `{ field: { $regex: value } }` for better-auth's `contains` search, on the
157+
* AUTHENTICATION path.
158+
*
159+
* That producer was flipped to `$contains` by #5710 (PR #5812), and a whole-repo
160+
* scan on `origin/main` found no other: every surviving `$regex` occurrence is a
161+
* consumer arm, a retirement prescription, or a refusal assertion. The reason the
162+
* two members existed is therefore gone, and #4706 retired both spellings — so
163+
* they are refused here like any other undeclared operator, with the spec's
164+
* prescription attached (see {@link retiredFilterOperatorError}).
165+
*
166+
* Note what this does NOT do: it does not add `$icontains`. That name is
167+
* declared by `StringOperatorSchema` but deliberately absent from
168+
* `FILTER_OPERATORS` (#5701), and this set is derived, so this driver refuses it
169+
* — fail-closed, an unimplemented capability rather than a silent widening. The
170+
* `$icontains` implementation for the JS faces is #5499-frozen; see the
171+
* `driver-memory` row of `scripts/check-driver-conformance.mjs`.
163172
*
164173
* Everything else is refused. That includes the mingo operators this driver used
165174
* to hand through by accident (`$elemMatch`, `$size`, `$type`, `$mod`, `$where`,
@@ -168,8 +177,6 @@ export function emptyFieldConstraintError(field: string, path: string): Error {
168177
*/
169178
export const SUPPORTED_FIELD_OPERATORS: ReadonlySet<string> = new Set<string>([
170179
...FILTER_OPERATORS,
171-
'$regex',
172-
'$options',
173180
]);
174181

175182
/** The vocabulary as it appears in a refusal message, in declaration order. */
@@ -418,24 +425,53 @@ export function nonBooleanNullComparandError(field: string, value: unknown, path
418425
}
419426

420427
/**
421-
* [#5324] `$options` without the `$regex` it modifies.
422-
*
423-
* `$options` is in {@link SUPPORTED_FIELD_OPERATORS} as a MODIFIER, not a
424-
* predicate — it carries the regex flags (`memory-matcher` reads it as
425-
* `new RegExp(target, condition.$options)`, and objectql's `having` skips it for
426-
* the same reason). On its own it is not a filter at all, and the two faces
427-
* proved it: mingo raised `unknown query operator $options` — uncoded, the very
428-
* escape #5324 is about — while the matcher ignored it and matched EVERY row.
429-
* Allowlisting the key without requiring its partner would have left exactly one
430-
* operator still leaking out of the envelope.
428+
* [#5702] A RETIRED filter operator in a field constraint.
429+
*
430+
* Distinct from {@link unknownFieldOperatorError} on purpose, and the
431+
* distinction is the author's: `$sounds_like` is a name that never meant
432+
* anything, while `$regex` and `$options` are names this driver ANSWERED — with
433+
* a real `RegExp`, the only regex evaluator in the repo — until #4706 retired
434+
* them. Handing that author the fifteen-name vocabulary list is true and
435+
* useless; what they need is `$icontains`.
436+
*
437+
* The prescription is `RETIRED_FILTER_OPERATORS[op].why`, printed VERBATIM. The
438+
* spec table exists precisely so `driver-sql`, this driver, `driver-turso`'s
439+
* remote transport, `driver-mongodb` and `objectql`'s `having` stop each writing
440+
* their own sentence about one retirement (#5701).
441+
*
442+
* This subsumes the `$options`-with-no-`$regex` refusal #5324 added
443+
* (`danglingRegexOptionsError`, deleted with this change): while `$options` was
444+
* an allowlisted MODIFIER, a dangling one needed its own gate; now that both
445+
* spellings are refused outright there is no shape left for that gate to catch,
446+
* and the message it printed — which taught the reader to write
447+
* `{ "$regex": "abc", "$options": "i" }` — would be prescribing the retired form.
448+
*
449+
* `siblings` are the other keys of the SAME field constraint, and every retired
450+
* one among them is named too — `{ $regex: '^acme', $options: 'i' }` is ONE
451+
* mistake with ONE fix, and a message naming only the key iteration reached
452+
* first would send its author back for a second round-trip on the other.
453+
*
454+
* Returns `null` when `op` is not retired, so the caller falls through to the
455+
* ordinary unknown-operator refusal in one expression.
431456
*/
432-
export function danglingRegexOptionsError(field: string, path: string): Error {
457+
export function retiredFilterOperatorError(
458+
op: string,
459+
field: string,
460+
path: string,
461+
siblings: readonly string[] = [],
462+
): Error | null {
463+
const guidance = RETIRED_FILTER_OPERATORS[op];
464+
if (!guidance) return null;
465+
const replacement = guidance.to ? ` Write "${guidance.to}" instead.` : '';
466+
const alsoRetired = siblings.filter((key) => key !== op && RETIRED_FILTER_OPERATORS[key]);
467+
const also = alsoRetired.length
468+
? ` The same field constraint also carries the retired ` +
469+
`${alsoRetired.map((key) => `"${key}"`).join(', ')} — one "${guidance.to}" replaces the whole ` +
470+
`shape, so this is ONE mistake with ONE fix, not one per key.`
471+
: '';
433472
return unsupportedFilterError(
434-
`Operator "$options" on field "${field}" at ${path} has no "$regex" to modify. "$options" ` +
435-
`carries the flags of a regex predicate (e.g. { "${field}": { "$regex": "abc", "$options": "i" } }); ` +
436-
`it is not a predicate on its own. It is refused rather than ignored because the two ` +
437-
`evaluation paths answered it differently — one raised an uncoded engine error, the other ` +
438-
`matched every row (#5324).`,
473+
`Filter operator "${op}" on field "${field}" at ${path} is RETIRED and is no longer evaluated ` +
474+
`by this driver.${replacement} ${guidance.why}${also}`,
439475
);
440476
}
441477

@@ -564,7 +600,13 @@ function assertFieldConstraintShape(
564600
const keys = Object.keys(spec);
565601
if (!keys.some((key) => key.startsWith('$'))) return;
566602
for (const op of keys) {
567-
if (!SUPPORTED_FIELD_OPERATORS.has(op)) throw unknownFieldOperatorError(op, field, path);
603+
if (!SUPPORTED_FIELD_OPERATORS.has(op)) {
604+
// [#5702] A RETIRED spelling gets the prescription; anything else gets
605+
// the vocabulary. Checked in this order because `$regex` satisfies both
606+
// descriptions ("not supported" and "retired") and only the second one
607+
// tells its author what to write.
608+
throw retiredFilterOperatorError(op, field, path, keys) ?? unknownFieldOperatorError(op, field, path);
609+
}
568610
// [#5345] Declared, but not by THIS face. Checked before the comparand-shape
569611
// rules below so a `$between` a face cannot compile is reported as
570612
// unsupported-here rather than as a malformed range the face would refuse
@@ -583,9 +625,12 @@ function assertFieldConstraintShape(
583625
throw nonBooleanNullComparandError(field, spec[op], `${path}.$null`);
584626
}
585627
}
586-
// `$options` is the one entry in the vocabulary that is a modifier rather than
587-
// a predicate, so it is the one that needs a companion.
588-
if (keys.includes('$options') && !keys.includes('$regex')) throw danglingRegexOptionsError(field, path);
628+
// [#5702] The `$options`-without-`$regex` companion check that stood here is
629+
// GONE. It was needed while `$options` was an allowlisted MODIFIER — a key the
630+
// vocabulary accepted but which is not a predicate on its own. Both spellings
631+
// are retired now, so the loop above refuses either of them on sight and there
632+
// is no surviving shape for a companion rule to judge. See
633+
// {@link retiredFilterOperatorError}.
589634
}
590635

591636
/**

packages/drivers/driver-memory/src/memory-analytics-filter-refusal.test.ts

Lines changed: 28 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -141,7 +141,6 @@ describe('[#5345] MemoryAnalyticsService — filters it cannot compile are refus
141141
{ op: '$startsWith', where: { name: { $startsWith: 'al' } } },
142142
{ op: '$endsWith', where: { name: { $endsWith: 'ta' } } },
143143
{ op: '$null', where: { closed_at: { $null: true } } },
144-
{ op: '$regex', where: { name: { $regex: '^al' } } },
145144
];
146145

147146
for (const { op, where } of UNCOMPILABLE) {
@@ -153,6 +152,34 @@ describe('[#5345] MemoryAnalyticsService — filters it cannot compile are refus
153152
});
154153
}
155154

155+
/**
156+
* [#5702] `$regex` was the sixth row of the table above until this change, and
157+
* it was in the WRONG table: its assertion read "declared by the Filter
158+
* Protocol, not compilable by this face", and `$regex` was never declared by
159+
* the Filter Protocol at all — it was an undeclared spelling this package
160+
* evaluated. #4706 retired it outright, so it is no longer a
161+
* declared-but-uncompilable operator on this face; it is a refused one on
162+
* every face, with a prescription attached.
163+
*
164+
* Kept as its own case rather than deleted, because the analytics face is a
165+
* SECOND door into the same walk and "the query path refuses it" is not
166+
* evidence that this one does — the two faces answering one filter differently
167+
* is the divergence class this whole file exists over (#5345).
168+
*/
169+
for (const op of ['$regex', '$options'] as const) {
170+
it(`refuses the retired ${op} on the analytics face too, naming $icontains`, async () => {
171+
const err = await expectRefusal(() => count({ name: { [op]: '^al' } } as FilterCondition), op);
172+
expect(err.message).toContain('RETIRED');
173+
expect(err.message).toContain('$icontains');
174+
// NOT the uncompilable-on-this-face sentence. Asserted against that
175+
// message's own distinctive phrase rather than against "declared by the
176+
// Filter Protocol", which the spec's prescription also contains — in the
177+
// NEGATED form ("was never declared by the Filter Protocol"), so a
178+
// substring test on it passes for both messages and pins nothing.
179+
expect(err.message).not.toContain('Supported operators on this surface');
180+
});
181+
}
182+
156183
it('refuses an unmapped operator reached through the nested-relation branch', async () => {
157184
// `{profile: {verified: …}}` is re-entered as a synthesised `{'profile.verified': …}`
158185
// node the up-front gate never walked — the one path where the lowering's own

packages/drivers/driver-memory/src/memory-driver.test.ts

Lines changed: 24 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -509,12 +509,30 @@ describe('InMemoryDriver', () => {
509509
expect(results.map((r: any) => r.name).sort()).toEqual(['Bob', 'Diana']);
510510
});
511511

512-
it('should filter with $regex operator', async () => {
513-
const results = await driver.find(testTable, {
514-
where: { name: { $regex: /^[AB]/ } },
515-
});
516-
expect(results).toHaveLength(2);
517-
expect(results.map((r: any) => r.name).sort()).toEqual(['Alice', 'Bob']);
512+
// [#5702] REPLACED WHOLESALE, not re-spelled. This case used to read
513+
// `should filter with $regex operator` and asserted that
514+
// `{ name: { $regex: /^[AB]/ } }` returned Alice and Bob — i.e. it pinned
515+
// the real `RegExp` evaluation that made this driver the only backend in
516+
// the repo answering `$regex` as a pattern while every SQL backend answered
517+
// it as a literal substring. That divergence is what #4706 retired the
518+
// operator over, so the limb is gone and its fixture cannot be re-spelled
519+
// into the new world: there is no `$regex` answer left to assert.
520+
//
521+
// What replaces it pins the retirement instead — and pins `code`/`status`,
522+
// not merely that something threw, because a bare `toThrow()` here would
523+
// stay green against any error at all, including the uncoded engine errors
524+
// #5324 spent a whole issue routing back into the ADR-0112 envelope.
525+
it('refuses the retired $regex operator, in the ADR-0112 envelope', async () => {
526+
const err = await driver
527+
.find(testTable, { where: { name: { $regex: '^[AB]' } } })
528+
.then(() => null, (e: any) => e);
529+
expect(err).toBeInstanceOf(Error);
530+
expect(err.code).toBe('INVALID_FILTER');
531+
expect(err.status).toBe(400);
532+
expect(err.message).toContain('$regex');
533+
// The prescription, not just the verdict: an author who wrote `$regex`
534+
// needs the name of what replaces it.
535+
expect(err.message).toContain('$icontains');
518536
});
519537

520538
it('should count with complex filter', async () => {

packages/drivers/driver-memory/src/memory-driver.ts

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1002,9 +1002,16 @@ export class InMemoryDriver implements IDataDriver {
10021002
result[op] = store(val);
10031003
break;
10041004
// Evaluated by mingo under the same name. `$exists` is a presence
1005-
// predicate, `$regex`/`$options` a pattern and its flags — none of them
1006-
// is a comparand, so none takes the field's storage form (#4047).
1007-
case '$exists': case '$regex': case '$options':
1005+
// predicate, not a comparand, so it does not take the field's storage
1006+
// form (#4047).
1007+
//
1008+
// [#5702] `$regex` and `$options` were passed through here too, on the
1009+
// same line, for the same "not a comparand" reason. Both are RETIRED
1010+
// (#4706) and refused by the shape gate before this method runs, so the
1011+
// arm is gone rather than left as an unreachable third name — an
1012+
// evaluation arm for a refused operator is exactly what let this
1013+
// driver's two faces answer one `$regex` differently for so long.
1014+
case '$exists':
10081015
result[op] = val;
10091016
break;
10101017
default:

0 commit comments

Comments
 (0)