Skip to content

Commit e5f4f16

Browse files
committed
Merge origin/main (cdfbee2) into claude/issue-5239-filter-logic-identity-batch
Textual conflict: packages/plugins/driver-mongodb/src/mongodb-filter.ts — both sides rewrote translateFilter/translateCondition (#5239 reduction vs #5329 array-dialect deletion + #5368 $null gate/path threading). Resolution keeps both: main's array refusal and path threading, this branch's three-valued reduction and shape gates; the three helpers both sides defined (unsupportedFilterError, describeFilterOperand, safeShapePreview) are de-duplicated onto main's copies. Semantic reconciliation the textual merge could not see (AGENTS.md s10): main's #5347 $null comparand gate sat in the emitter, and this branch's reduction makes emitters skippable by a boolean identity — { $or: [ {}, { stage: { $null: 'yes' } } ] } would have translated to match-all while driver-sql refuses it. The gate's load-bearing copy moved onto the validating walk (reduceFilterKey), mirroring driver-sql's #5368 placement; the emitter arm keeps its local check. Pinned in mongodb-null-comparand-refusal.test.ts (three identity-sibling fixtures). Fixture triage: the 'legacy array dialect is untouched' pin in mongodb-filter-boolean-identity.test.ts pinned a dialect #5329 deleted — replaced wholesale with the surviving boundary ([] = absent filter = match-all, non-empty array refused before the reduction runs). The reduceFilterKey field-arm comment on { field: {} } is recalibrated to current main: #5327 gated the shape on the other four backends; this driver remaining the one still answering it is now tracked by #5376. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ErbEDVAg1No9gdg1pgDAGB
2 parents 7327a94 + cdfbee2 commit e5f4f16

218 files changed

Lines changed: 24352 additions & 4718 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
---
2+
---
3+
4+
docs(adr-0076): 把状态行从 `Proposed` 校准为 `Accepted`,并逐条记录 D1–D12 的真实落地状态(#5063)。参照 #4919 对 ADR-0078 的做法:不整体翻牌,每条决定给出在 `origin/main` 上核对到的证据 —— D3 完全未建、D10/D11 只落一半、D9 已越过 step-1 把联合类型彻底解散、D7 已有 CI 触发指标。同时给 D9 的 back-compat 条目补一条状态注记(组合别名已不存在)。纯文档改动,不发布任何包。
Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
---
2+
"@objectstack/types": minor
3+
"@objectstack/cloud-connection": minor
4+
"@objectstack/lint": minor
5+
"@objectstack/cli": minor
6+
"@objectstack/spec": patch
7+
"@objectstack/driver-sql": patch
8+
---
9+
10+
feat(types,cloud-connection,lint,cli): ADR-0120 17.x 收尾 —— `isolated` 安装期姿态硬门(D5e)、D5c 重拼写 advisory、成文契约扫荡与三姿态 conformance (#5081)
11+
12+
ADR-0120 17.x 波的第三块,也是最后一块。前两块已在 main 上:#5212(driver 侧
13+
D3+D4 —— `COALESCE(organization_id, '__global__')` 物化、drift 两侧同步、重复预检)
14+
#5208(spec 词汇 `'organization'` + D5a/D5b lint)。本次补齐三件事:安装期的
15+
姿态决策点、剩余的成文契约、以及把「一个 app 包跑遍三种姿态」从假设变成测试。
16+
17+
**D5e —— 装进 `isolated` 环境时的硬门。** 词汇本身是姿态无关的:作者说的是业务
18+
边界(`'organization'` 一个组织一份 / `'global'` 整个安装一份),没有任何索引形状
19+
读姿态。唯一的残留在一个方向上:`isolated` 下组织就是**不同客户**,此时 app 业务
20+
对象上的 `'global'` 唯一既跨客户过度约束,又变成跨客户的存在性预言机(S10/S14)。
21+
维护者裁定这是**硬门而非 advisory**:把带 `'global'` 唯一(非 `sys` 对象)的 app
22+
装进 `isolated` 环境会**停下来并逐索引列出**,安装者(通常是 AI agent)要么确认它
23+
确实是平台级的,要么改写为 `'organization'`;确认按 ADR-0104 attestation 风格
24+
留痕在安装清单里(`InstalledManifestEntry.globalUniqueAttestation` —— 确认了什么、
25+
谁确认的、何时、在哪个姿态下问的),**之后不复问**
26+
27+
- 停下的安装**什么都不留**:先于 hot-register 和任何 ledger 写入,所以作者改完
28+
元数据可以直接重试,不需要先卸载。
29+
- 逐索引确认是有牙齿的:`confirmGlobalUniques``true` 或明确的 id 数组,只确认
30+
其中一条仍会在剩下的那条上停住。
31+
- 升级引入的****约束会被问,老的答案继续算数。
32+
- 另一个姿态下给出的确认**不算同意** —— `isolated` 那个问题在 `single` 下从未被
33+
问过,所以按「未确认」处理(唯一不会静默放行跨客户约束的方向)。
34+
-**永不做成启动期告警**(#4884 纪律)。boot 时的 rehydrate 不评估此门;门够不到
35+
的两类存量 —— 门禁上线前的安装、装后姿态变更的环境 —— 由 `os doctor`
36+
`os migrate plan` 的 advisory 形态覆盖。
37+
38+
判定里有三条是承重的,别「简化」掉:声明索引上的裸 `unique: true` ****(D1 说它
39+
就是 `'global'` 的位置式拼写,排除它等于让整个 17.x 可以靠拼写绕过);字段级
40+
`true` **不算**(它是 `'organization'`,永久合法);`sys_`/`base_` 对象**不算**
41+
(S5 那批引擎幂等键天然就是平台级的,每次安装都问一遍就是 #4884 的误报类)。
42+
43+
CLI: `os package install` 新增 `--confirm-global-uniques`,并把 409 渲染成可读的
44+
逐条清单而不是一句 "Install failed (409)"。
45+
46+
**D5c —— 遗留手写组织复合索引的 advisory。** 新规则
47+
`unique/legacy-organization-composite`:声明的唯一索引自己列出了组织列
48+
(`{ fields: ['name','organization_id'], unique: true }`)—— 这是词汇出现之前手写
49+
per-organization 的写法。它读起来像「每组织唯一」,物化出来却是普通复合索引,而
50+
SQL UNIQUE 是 NULL-distinct 的:组织列为 NULL 的行上它**什么都不约束**(#5030),
51+
在单组织部署上那就是每一行。改写成 `unique: 'organization'`(`fields` 原样保留,
52+
driver 会把已列出的组织列**就地**变成 NULL-safe 形式)正是补上这个洞的动作。
53+
**永远只是 advisory,永远不自动修**:老拼写永久合法、零强制 drift,而 opt-in 是
54+
真实的物理收紧,要走 D4 的 `recreate_index` + 重复预检。
55+
56+
**D6 —— 成文契约扫荡。** `content/docs/data-modeling/indexing.mdx`
57+
§Two ways to say "unique" 全节按新词汇重写(含 `os:check` 代码块);
58+
`content/docs/protocol/objectql/schema.mdx` 的 §Uniqueness and tenancy 重写为
59+
§Uniqueness and scope —— 其中那句「单租户部署不受影响,租户列是常量,复合索引
60+
退化为单列索引」是 #5030 **证伪过的原话**,现已替换为 D3 的 NULL-safe 事实;
61+
`content/docs/deployment/cli.mdx``replace_unique_index` / `recreate_index`
62+
条目补上 NULL-safe 形状与重复预检;`content/docs/references/**`
63+
`gen:schema && gen:docs` 再生成,未手改。
64+
65+
按 ADR-0120 Resolved #2 的非规范性引导(官方示例/脚手架/生成器在新代码中输出
66+
显式拼写),`skills/objectstack-data/**` 的索引与校验规则整体扫过:声明索引一律
67+
说清 scope,并新增一节完整讲 `'organization'` 的 NULL-safe 语义与「永远不写姿态」。
68+
顺带修掉那里长期使用的 `tenant_id` —— 平台的列叫 `organization_id`
69+
`examples/**``create-objectstack` 模板与 `os generate` 经核查**根本没有声明任何
70+
唯一约束**,故无可扫;这是核查结论,不是遗漏。
71+
72+
**三姿态 conformance(ADR §Acceptance tests)。** 同一个 fixture app 在
73+
`single | group | isolated` 三姿态下启动,逐 S 行用**真实的违规插入**断言 enforcement
74+
(S1/S2/S3/S4/S5/S6/S7/S8/S9/S11/S12),并逐姿态捕获物化出的索引键,断言三者
75+
**逐字节相同** —— 「没有任何索引形状读姿态」这句话一旦有两者不同就是假的。相同性
76+
断言配了一条正向断言(对着期望的键形状),这样「三次都什么都没建」不会读成「一致」。
77+
外加 ADR 只要的那一条 transition smoke:在 `single` 下建库、`isolated` 下重新打开,
78+
drift op 为零。
79+
80+
对既有部署的影响:除新增的安装期确认外,本次不改变任何已有物化行为。字段级
81+
`unique: true` 一如既往合法。
Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
---
2+
"@objectstack/service-analytics": minor
3+
---
4+
5+
fix(service-analytics)!: 分析查询的 `where` —— `$not` 变 NULL-safe、`{$not:{}}` 变零行、`$or``{}` 析取项不再被丢 (#5325)
6+
7+
`filter-normalizer.ts``buildNode` 是这个包里**第二份**同缺陷拷贝:第一份
8+
(`read-scope-sql.ts``compileNode`,RLS 读作用域)已由 #5297 修好,而这一份编译的是
9+
**作者自己写的 `where`** —— dashboard widget / dataset 的筛选器。两者是各自独立的函数,
10+
所以那一单合入后这三条仍然在。以 `driver-sql` 同一份 fixture 实测(4 行,行 3、4 的
11+
`stage` 为 NULL,行 3 的 `amount` 为 NULL,行 4 的 `owner` 为 NULL):
12+
13+
| widget 的 `where` | 改前取到的行 | 改后(= driver-memory / formula / #5296 后的 driver-sql) |
14+
|---|---|---|
15+
| `{ $not: { stage: 'won' } }` | `2` | `2,3,4` |
16+
| `{ $not: { stage: { $in: ['won'] } } }` | `2` | `2,3,4` |
17+
| `{ $not: {} }` | **全表** | **零行** |
18+
| `{ $or: [{ stage: 'won' }, {}] }` | `1` | 全表 |
19+
| `{ $not: { $or: [{stage:'won'},{owner:'u1'}] } }` | `2` | `2,4` |
20+
21+
**这是可观察的行为变更,不是内部重构 —— 已有的图表数值会变:**
22+
23+
- **`{$not: {}}` 的 widget 此前画的是整个数据集,现在是零行。** `buildNode({})` 返回
24+
`null`(= 无约束 = TRUE),`$not` 分支的 `if (inner)` 因此为假,整条 `$not` 消失,
25+
WHERE 一个字都不发 —— 一条意思是「什么都不显示」的筛选器显示了全部。`NOT TRUE ≡ FALSE`,
26+
现在它编译成 `1 = 0`
27+
- **`$not` 下 NULL 行的去留变了,所以图上的数字会变。** SQL 是三值逻辑而 `WHERE` 只保留
28+
TRUE,裸 `NOT (stage = ?)``stage` 为 NULL 的行全部丢掉;`driver-memory``formula`
29+
和(#5296 之后的)`driver-sql` 都把它们算进来。同一条 widget filter,在分析查询和普通
30+
`find()` 上给出不同的行集,取决于哪个后端接住它。#5146 已拍板 JS 家族的答案为准,本次
31+
按同一口径把守卫**下推到叶子**(`{col: {$null: false}}` / `{$or: [{col:{$null:true}}, …]}`,
32+
极性逐算子决定)。**受影响的图表数值会上升**(负向筛选现在包含空值行)。
33+
- **`$or` 里的 `{}` 析取项不再被丢。** TRUE 是 AND 的单位元但**吸收** OR,所以
34+
`{$or: [{stage:'won'}, {}]}` 整条为 TRUE;此前它被 `.filter(n => n !== null)` 丢掉,
35+
查询被静默**收紧**成剩余分支。
36+
- **空集合是布尔常量,不再是「没有谓词」。** `{stage: {$in: []}}` 此前编译成空子句
37+
→ 无约束 → 画全表,现在是零行(`1 = 0`);`{$nin: []}` 不排除任何行。
38+
- **两处新的响亮拒收(此前静默放宽):** `$not` / `$or` / `$and`**非对象**操作数
39+
(`{$not: null}` 曾整条消失 → 等于不筛),以及**零个操作符的字段约束** `{a: {}}`
40+
—— 后者按 #5240 的拍板拒收,与 driver-sql / driver-memory / formula 一致;不这么做的话,
41+
「TRUE 吸收 OR」会把 `{$or: [{a: {}}, {b: 2}]}``b = 2` 放宽成全表。
42+
43+
实现落在 normalizer 而不是某个 strategy:守卫在这一层是**结构**(多一个 `$null` 合取项),
44+
`filterNodeToCondition` 交给 ObjectQL 引擎后在**任何驱动上都成立**,包括本身不 NULL-safe
45+
的那些;只加在 raw-SQL 那条路径,等于说「分析查询的 `$not` 是什么意思取决于哪个驱动接住它」。
46+
代价是引擎路径会**双重加守卫**,已实测幂等(`NOT (c IS NOT NULL AND (c IS NOT NULL AND c = v))`
47+
与单层等价),只是 SQL 多一层冗余谓词。
48+
49+
`NormalizedFilterNode` 因此新增布尔常量 kind —— 该联合此前只有 `leaf | and | or | not`,
50+
没有 FALSE 的表示法,这正是 `{$not:{}}` 只能编译成「什么都不发」的根本原因。三个编译器
51+
(`native-sql-strategy.compileFilterNode``objectql-strategy.filterNodeToCondition`
52+
回显给浏览器的 `renderFilterNodeSql`)各自实现它;引擎路径用的是 `{$not: {}}`,即
53+
driver-sql / formula / driver-memory 参考匹配器早已钉住的零行写法(#5134),没有另造第二种。
54+
55+
`$and: []` / `$or: []` 的空组合子**不在本次范围**,仍然 fail-closed 抛错(独立裁定见 #5322),
56+
并已加用例钉在抛错这一侧。
Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
---
2+
"@objectstack/service-analytics": patch
3+
"@objectstack/rest": patch
4+
---
5+
6+
fix(analytics,rest): an analytics filter refusal reaches the caller as `400 INVALID_FILTER`, not `500 ANALYTICS_QUERY_FAILED` (#5352)
7+
8+
Misspell an operator in a dashboard widget's filter and analytics refuses it —
9+
correctly, and loudly, which is the posture #3948 / #5240 / #5325 / #5334 each
10+
argued for one refusal at a time: dropping a predicate the compiler cannot
11+
express does not narrow the query, it **widens** it to rows the author excluded,
12+
and a chart drawn over the whole dataset looks like a working chart.
13+
14+
The refusal never reached the author. It landed as `500 ANALYTICS_QUERY_FAILED`
15+
— read as "the platform is broken" rather than "your filter has a typo", and
16+
counted by ops alerting as a 5xx. The identical mistake on `find()` has answered
17+
`400 INVALID_FILTER` since #3948, so one authoring error had two wire shapes,
18+
chosen by which face happened to catch it.
19+
20+
**One defect, two halves — either alone leaves it unfixed.**
21+
22+
- **Producer** (`filter-normalizer.ts`): seven of its nine refusals were bare
23+
`throw new Error(…)` carrying no `code`/`status`. All nine now go through the
24+
`invalidFilterError` helper #5334 introduced (`INVALID_FILTER` / 400), which
25+
becomes the module's only way to refuse.
26+
- **Consumer** (`rest-server.ts`, `POST /analytics/dataset/query`): the catch
27+
discarded `error.code` / `error.status` and re-derived the classification from
28+
a hardcoded list of message substrings — so a producer that took ADR-0112
29+
seriously was punished for it. It now reads the envelope **first**; the
30+
substring list is demoted to a fallback for the families that still carry no
31+
envelope.
32+
33+
**Observable behaviour change — read this if you alert or retry on status.**
34+
The same request that returned `500 ANALYTICS_QUERY_FAILED` now returns
35+
`400 INVALID_FILTER` (and, for two neighbouring conditions whose producers
36+
already declared an envelope this route was discarding, `400 INVALID_FIELD` for
37+
a measure over a field the object does not have, `404 CUBE_NOT_FOUND` for an
38+
unregistered cube). Monitoring that counted these as server faults will see the
39+
5xx rate drop and a 4xx rate appear; a client that retries on 5xx will stop
40+
retrying a request that could only ever fail the same way. Both are the intended
41+
correction — the condition was always the caller's mistake — but they are
42+
visible, so they are stated rather than buried.
43+
44+
**Which inputs are refused did not change.** This changes the SHAPE of the
45+
error and nothing about the judgement that produced it: no refusal condition
46+
was touched, no input that used to compile now refuses, and no input that used
47+
to refuse now compiles. That claim is pinned input-by-input (refusals *and*
48+
accepted inputs with their compiled trees) in
49+
`filter-refusal-envelope.test.ts`, which is green both before and after the
50+
change — only the envelope assertions move.
51+
52+
The message-substring list survives on purpose. All six of its entries were
53+
re-verified as bare `Error`s (`dataset-compiler.ts`, `native-sql-strategy.ts`,
54+
`dataset-executor.ts`, `read-scope-sql.ts`), so deleting it would regress those
55+
families from `400 DATASET_INVALID` to 500. It is a placeholder for their
56+
enveloping, not a second classification mechanism, and it is now documented as
57+
such: a new refusal should carry a `code`/`status` and be served by the
58+
envelope branch for free. The passthrough is deliberately **4xx-only** and
59+
requires **both** `code` and `status`, so an internal fault can never be
60+
re-labelled as the caller's fault, and this route never invents a code a
61+
producer failed to supply.
Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
---
2+
"@objectstack/service-analytics": patch
3+
---
4+
5+
fix(service-analytics): a `where` written as a `FilterArray` is lowered instead of silently dropped (#5334)
6+
7+
**Observable behaviour change.** An analytics query whose `where` arrived as an
8+
ARRAY had its filter **deleted**: `normalizeAnalyticsFilterTree` answered every
9+
array with `return null`, so no predicate was compiled, no error was raised, and
10+
the widget charted the **entire dataset**. The compiled SQL stayed perfectly
11+
valid — just broader than the author asked for — which is why it was invisible
12+
to every test that asserts a SQL string. The issue's own measurement:
13+
`generateSql({cube:'deals', measures:['total'], dimensions:['id'], where:
14+
[['stage','=','won']]})` emitted `SELECT id AS "id", COUNT(*) AS "total" FROM
15+
"deal" GROUP BY id` with an empty `params`. It now emits the bound `WHERE` and
16+
returns the two won deals.
17+
18+
`FilterArray` (`['stage','=','won']`, `['and', […], […]]`, `[[…], […]]`) is
19+
INPUT-ONLY authoring sugar (#5285), and #5158's ruling C says every door into
20+
the runtime lowers it through the single `parseFilterAST` sink before anything
21+
downstream sees a filter. #5329 closed ObjectQL's six entry points that way and
22+
deleted the four drivers' private array dialects. Analytics is the **fifth
23+
door**: it compiles `where` itself — to SQL (`NativeSQLStrategy`) or to a
24+
`FilterCondition` for the engine (`ObjectQLStrategy`) — so nothing upstream
25+
lowers for it. It now gives the same three answers the engine door gives:
26+
27+
- `[]` — "no filter", not a failed filter: no predicate, no error (unchanged).
28+
- A well-formed `FilterArray`**lowered** through `parseFilterAST`, so both
29+
spellings of one filter select the same rows on both strategies.
30+
- Any other non-empty array — **refused** with `INVALID_FILTER` / 400
31+
(ADR-0112), the envelope the drivers' `filterArrayReachedDriverError` uses.
32+
This is where the undeclared INFIX form (`[condA, 'or', condB]`) lands, and
33+
where a list of `FilterCondition` objects (`[{stage:'won'}]`) lands — neither
34+
is a `FilterArray`, `parseFilterAST` has no lowering for either, and dropping
35+
them is what returned the unfiltered dataset.
36+
37+
Lowering rather than refusing keeps one dashboard's metadata meaning one thing:
38+
the same `where` on a plain `find()` already lowers at the engine door, so
39+
refusing it here would have forked the product by which face read the metadata.

0 commit comments

Comments
 (0)