A filter with an operator outside VALID_AST_OPERATORS is silently dropped, not rejected — single-condition views return unfiltered results

[os-zhuang]

A bare-triple filter whose operator is not in VALID_AST_OPERATORS is passed through unconverted and then skipped entirely by the SQL driver. No WHERE clause is emitted, no error is raised, and the caller gets every row. The user sees a list they believe is filtered and it isn't.

This is reachable today through an ordinary ObjectUI view-config affordance — it is not hypothetical.

The chain

Each hop verified by reading the code; the spec hop was executed.

1. isFilterAST gates a bare triple on the operator being known — packages/spec/src/data/filter.zod.ts:400:

if (filter.length >= 2 && typeof filter[1] === 'string') {
  return VALID_AST_OPERATORS.has(filter[1].toLowerCase());
}

2. The protocol treats that as a conversion gate, not a validation gate — packages/metadata-protocol/src/protocol.ts:2754-2757:

if (isFilterAST(parsedFilter)) {
  parsedFilter = parseFilterAST(parsedFilter);
}
options.where = parsedFilter;   // <-- unconverted array assigned as-is

3. applyFilters walks that array — packages/plugins/driver-sql/src/sql-driver.ts:3702-3708. For a bare triple the three elements are all strings, so each hits the string branch, matches neither and nor or, and is continued:

for (const item of filters) {
  if (typeof item === 'string') {
    if (item.toLowerCase() === 'or') nextJoin = 'or';
    else if (item.toLowerCase() === 'and') nextJoin = 'and';
    continue;                     // <-- 'close_date', 'before', '2024-01-01' all land here
  }
  if (Array.isArray(item)) { ... }
}

No Array.isArray branch is ever taken. No predicate is emitted.

driver-memory has the same shape (default: return null, memory-driver.ts:768).

Executed evidence

Against published @objectstack/spec:

["close_date","before","2024-01-01"]     isFilterAST=false   *** PASSED THROUGH UNCONVERTED ***
["close_date","after","2024-01-01"]      isFilterAST=false   *** PASSED THROUGH UNCONVERTED ***
["stage","not in",["won","lost"]]        isFilterAST=false   *** PASSED THROUGH UNCONVERTED ***
["status","=","active"]                  isFilterAST=true    converted -> {"status":"active"}

Why those three operators, and why it is reachable

VALID_AST_OPERATORS (filter.zod.ts:352) contains not_in and nin but not 'not in' (with a space), and contains neither before nor after.

ObjectUI's view-config builder emits exactly those three verbatim — packages/plugin-view/src/config/view-config-utils.ts:46:

'notIn':  'not in',     // space — VALID_AST_OPERATORS has not_in / nin
'before': 'before',     // absent
'after':  'after',      // absent

and data-objectstack's normalizeFilterOperator ends ?? op, so they reach the wire unchanged.

The trigger is the single-condition case. toSpecFilter (view-config-utils.ts:158) returns a bare triple when there is exactly one condition with AND logic:

if (triplets.length === 1 && logic === 'and') return triplets[0];

So: an author adds one date filter "before " to a view → the saved view returns every row, silently. Add a second condition and the shape becomes nested, which reaches the driver's wider switch and throws instead — the failure mode flips on how many conditions the user happened to add.

Scope of impact

Not a permission bypass, to be explicit about it. Row-scoping middleware $and-composes onto the same ast.where, so a scoped object yields {$and: [<raw array>, <scope>]}; the $and arm (sql-driver.ts:3987) calls applyFilterCondition on each element, and Object.entries(['close_date','before','2024-01-01']) produces keys "0"/"1"/"2" — it would attempt to filter on columns named 0, 1, 2 and error. The scope arm survives. (This arm is reasoned from the code, not executed.)

So the impact is: on objects without row-scoping, a silently unfiltered result set; on scoped objects, a confusing SQL error. Both are wrong; neither crosses a permission boundary.

Backends disagree on the failure mode

Path	Unknown operator →
driver-sql, $-object	throw (sql-driver.ts:4016)
driver-sql, nested AST	throw (sql-driver.ts:3866)
driver-sql, bare triple	silently drops the filter
driver-memory, AST	silently drops the condition (memory-driver.ts:768)
driver-mongodb, AST	passes $${op} raw to Mongo → server error
mapAnalyticsOperator	silently becomes $eq (protocol.ts:3800)
matches-filter (RLS)	false — fail closed
analytics preview-evaluator	true — fail open (preview-evaluator.ts:44)

Six behaviours for one input class.

Suggested fix

1. In applyFilters (sql-driver.ts:3702) and memory-driver.ts:768: an array item that is neither a recognised join keyword nor an array — and a bare triple whose operator is unrecognised — should throw, matching what the nested and $-object paths already do. Silently emitting no predicate is the one outcome that must not happen.
2. Consider making protocol.ts:2754 reject rather than pass through: if the value looks like a filter array but isFilterAST says no, that is a malformed filter, not an opaque where.
3. Independently, before / after / 'not in' are ObjectUI-side spellings that no spec vocabulary defines — tracked at Vocabulary consolidation (#2901): what shipped, and why Track C items 1–3 are not planned objectui#2945. They should be lowered client-side to < / > / not_in (ObjectUI's own ListView.tsx:75 already does exactly this for before/after; the view-config persistence path does not). Fixing that removes today's trigger but not the underlying silent-drop.

Why this blocks other work

Any narrowing of VALID_AST_OPERATORS — e.g. consolidating the filter-operator vocabularies — converts working bare-triple filters into silently unfiltered queries. Item 1 above should land before any such narrowing is attempted, otherwise the change is untestable: it passes CI, passes os build, and degrades quietly against stored view metadata.

Found while auditing spec-enum ↔ renderer coverage for objectstack-ai/objectui#2901.

[os-zhuang]

Correction to item 3 of the report — I had the attribution backwards, and it makes this issue bigger, not smaller.

I wrote that before/after "are ObjectUI-side spellings that no spec vocabulary defines". That is wrong. They are canonical members of VIEW_FILTER_OPERATORS (packages/spec/src/ui/view.zod.ts:90):

export const VIEW_FILTER_OPERATORS = [
  'equals', 'not_equals',
  'contains', 'not_contains',
  'starts_with', 'ends_with',
  'greater_than', 'less_than',
  'greater_than_or_equal', 'less_than_or_equal',
  'in', 'not_in',
  'is_empty', 'is_not_empty',
  'is_null', 'is_not_null',
  'before', 'after', 'between',
] as const;

So this is not a client emitting junk. It is two spec vocabularies disagreeing — the view layer blesses an operator as canonical authoring vocabulary, and the query layer's AST gate does not accept it.

The disagreement is 8 of 19, not 2

These VIEW_FILTER_OPERATORS members are absent from VALID_AST_OPERATORS:

equals
not_equals
greater_than
less_than
greater_than_or_equal
less_than_or_equal
before
after

equals is on that list — the most common filter operator there is. Every one of them is a spelling ViewFilterRuleSchema validates and defineStack accepts.

They are not all reachable today only because ObjectUI's adapter happens to translate six of them (FILTER_OPERATOR_ALIASES, packages/data-objectstack/src/index.ts:51). It had no entry for before/after — which is the entire reason those two surfaced and the other six did not. The safety of the query path is currently resting on a hand-written table in a different repository being complete, and it wasn't.

What that changes about the fix

Item 3 as written pointed the wrong way. The client-side lowering is worth doing and is done — objectstack-ai/objectui#2974 — but it is a mitigation, not the fix. It leaves the contract still self-contradictory and the next gap still silent.

The durable fix belongs here, and there are two shapes:

1. AST_OPERATOR_MAP folds the view vocabulary. Add the eight, mapping equals→=, before→<, after→>, etc. parseFilterAST then accepts anything the view layer can author. Cheapest, and purely additive to VALID_AST_OPERATORS — no narrowing, so none of the migration risk in the original report applies.
2. Derive one from the other, so a token added to VIEW_FILTER_OPERATORS cannot fail to have an AST lowering. filter.zod.ts:362 already documents that VALID_AST_OPERATORS is kept in sync with ObjectUI's adapter by hand; that comment is the bug. Two vocabularies plus five independent alias tables cannot be held in agreement by convention.

Items 1 and 2 of the original report stand unchanged, and item 1 — making the driver throw rather than skip — is still the prerequisite. It is what turns "8 of 19 canonical operators silently return every row" into "8 of 19 raise an error", which is the difference between a latent data-correctness bug and a caught one.

[os-zhuang]

Reopened — my commit message said "Closes #3948 item 1", which GitHub read as closing the whole issue. Only item 1 is done.

Done — #4029. Both drivers now throw on a filter element that is neither a logical keyword nor a condition array, so the bare-triple silent drop is gone and the three code paths (bare / nested / $-object) finally agree. driver-memory additionally gained the seven VALID_AST_OPERATORS members it had been dropping (not_in, is_null, is_not_null, isnull, isnotnull, is_empty, is_not_empty) — throwing without implementing those would have traded a silent wrong-result for a hard regression on operators the spec blesses.

Verified: driver-sql 487 pass, driver-memory 153 pass, objectql 1171 pass, plus the Dogfood Regression Gate.

Still open:

Item 2 — make protocol.ts:2754 reject rather than pass through. It is currently a conversion gate: when isFilterAST() says no, the raw value is assigned to where unchanged. After #4029 that now surfaces as a driver error rather than silence, which is a big improvement, but the error names a driver-level symptom for what is really a malformed request. Rejecting at the protocol boundary would return a 4xx naming the operator instead.

Item 3 — the two spec vocabularies still disagree. This is the root cause and it is untouched. 8 of the 19 canonical VIEW_FILTER_OPERATORS are absent from VALID_AST_OPERATORS — equals, not_equals, greater_than, less_than, greater_than_or_equal, less_than_or_equal, before, after. An author can declare any of them on a ViewFilterRule, ViewFilterRuleSchema validates them, and defineStack accepts them; whether the query then works depends on a hand-written alias table in a different repository being complete.

After #4029 an incomplete table produces a loud error instead of unfiltered rows, so this is no longer a correctness hazard — it is now a reliability one. The fix is purely additive and carries none of the migration risk in the original report: teach AST_OPERATOR_MAP the eight spellings (equals→=, before→<, after→>, …), or better, derive one vocabulary from the other so a token added to VIEW_FILTER_OPERATORS cannot ship without an AST lowering. filter.zod.ts:362 currently documents the cross-repo hand-sync as the mechanism; that comment is the bug.

[os-zhuang]

Item 3 done — #4039. Only item 2 remains.

AST_OPERATOR_MAP is now the single source of truth: VALID_AST_OPERATORS is derived from its keys rather than restated, so an operator can no longer pass isFilterAST() without also having a lowering. The two were independent hand-written lists that happened to agree, with nothing enforcing it — and both failure directions were invisible.

The eight canonical view spellings are in, plus the squashed/short forms stored metadata carries. New export canonicalAstOperator(op) folds every accepted spelling of one comparison onto a single infix form, and both drivers call it instead of growing private alias lists — which is what let them accept different vocabularies before #4029. like/ilike are deliberately not folded onto contains, since driver-sql passes them to SQL verbatim and folding would silently wrap the value in %…%.

Widening only, so no stored filter stops validating and there is no migration.

Verified: spec 6922, objectql 1171, driver-sql 487, driver-memory 178, plus Check Generated Artifacts, Spec property liveness, Temporal Conformance (live PG + MySQL) and both Dogfood Regression Gate shards. The parity test was confirmed to fail without the fix.

Worth recording how the driver half surfaced: driver-memory's parity test from #4029 iterates VALID_AST_OPERATORS directly, so widening the gate failed it immediately — the drivers' array paths lagged by 20 spellings. driver-sql had the identical gap and no test to catch it. That is the argument for canonicalAstOperator over two more alias lists.

---

Item 2 still open — make protocol.ts:2754 reject rather than pass through.

It remains a conversion gate: when isFilterAST() says no, the raw value is assigned to where unchanged. After #4029 that surfaces as a driver error rather than silence, and after #4039 the 8-of-19 mismatch that made it reachable from ordinary authoring is gone — so this is now the last of the three, and the least urgent.

What it still buys: the error names a driver-level symptom for what is really a malformed request. Rejecting at the protocol boundary would return a 4xx naming the operator, instead of surfacing as a SQL-layer failure. Worth doing, no longer load-bearing.