Merge remote-tracking branch 'origin/main' into claude/issue-5701-regex-retire-icontains

Author: claude

.changeset/action-ctx-session-contract.md

@@ -0,0 +1,66 @@
+---
+"@objectstack/spec": minor
+"@objectstack/runtime": patch
+---
+
+feat(spec): declare the action-body `ctx.session` contract (#5697)
+
+An action body reads `ctx.session` on every dispatch, and until now **nothing
+declared it**. `actionContext` is a bare `any` at both dispatch sites
+(`domains/actions.ts`, `action-execution.ts`), the sandbox seam types
+`ScriptContext.session` as `unknown`, and the one spec-side mention was an
+inline literal on `ActionHandlerContext` carrying a `[k: string]: unknown`
+catch-all. Declared-nowhere, produced-anyway: no schema, no gate, no generated
+reference page, and nothing the liveness ledger could reach.
+
+That is how the surface drifted without anyone noticing. Its `roles` key carries
+`ExecutionContext.positions` — the ADR-0090 D3 vocabulary handed to authors under
+the one spelling that ADR forbids — while the hook side retired its own
+`session.roles` at #5050. One platform, one key name, two opposite answers.
+
+**`ActionSessionSchema` (`@objectstack/spec/ui`) declares that shape as built.**
+
+```ts
+{ userId?: string; organizationId?: string; roles?: string[] }
+```
+
+This release changes **nothing about what the runtime produces** — it is phase 1
+of #5613's contract-first ruling, and declaring current reality is deliberately
+not the same as endorsing it:
+
+- `roles` is declared **deprecated** in its `.describe()` and its JSDoc. The
+  rename to `positions`, with a deprecation window and an ADR-0087 semantic
+  migration, is #5613 phase 2. There is deliberately **no `positions` key yet** —
+  minting one before the migration would ship two live spellings of one value.
+- The schema is **not strict**, matching `HookContextSchema`: this is a runtime
+  shape the platform hands a body, never authored, and closing it would turn a
+  future engine-side enrichment into a parse failure for whoever parses a context
+  they were given.
+
+Three facts the declaration now states, all of them previously discoverable only
+by reading the builder:
+
+- **Absent means the key is absent.** The builder uses conditional spreads, so
+  `'organizationId' in ctx.session` answers `false` — not "present and
+  `undefined`". The hook path's `input.id` on a bulk write is the opposite case
+  (#5668); an `in` test does not port between them.
+- **No identity envelope yields no session at all** — `undefined`, never `{}`, so
+  a body can tell "no caller" from "an anonymous caller" (#3712). One consequence:
+  `roles` never appears on its own.
+- **`organizationId` is the blessed name** for the caller's active org; the
+  v11-removed `session.tenantId` alias (#3280 / #3290) does not come back.
+
+Type-only on the runtime side, no behaviour change: `buildActionSession()` now
+declares `ActionSession | undefined` instead of `any | undefined`, and
+`ActionHandlerContext.session` is the schema's inferred type rather than an
+inline literal with a catch-all. A handler annotated with `ActionHandler` that
+read an undeclared key off `ctx.session` now gets a compile error naming it —
+that key was never produced. `ScriptContext.session` deliberately stays
+`unknown`: it is one seam over both body kinds, and hook and action sessions are
+different objects.
+
+The declaration ships with the gate it needed —
+`packages/runtime/src/action-session-shape-contract.test.ts` executes the real
+producer and asserts a non-strict parse of the built object returns it
+**unchanged**, so a key the builder starts producing without declaring here is
+stripped and the pin goes red.

.changeset/analytics-dataset-refusal-envelope.md

@@ -0,0 +1,73 @@
+---
+"@objectstack/service-analytics": patch
+"@objectstack/rest": patch
+"@objectstack/spec": patch
+---
+
+fix(analytics,rest): five dataset refusals declare `DATASET_INVALID` / 400 themselves, and the route's message-sniffing list shrinks to one entry (#5367)
+
+`POST /analytics/dataset/query` answered `400 DATASET_INVALID` for six error
+families because the route recognised their **prose**, not because the errors
+said anything about themselves. #5352 gave the catch an ADR-0112 envelope branch
+(`error.code` + a 4xx `error.status`, read first) and had to leave a hardcoded
+list of message substrings behind it, since all six producers were still bare
+`throw new Error(…)`:
+
+```
+/not declared in the dataset|not backed by a declared relationship|
+ not supported by the v1 dataset runtime|read-scope-sql|
+ not a selected dimension or measure|is not a subset of the selected dimensions/
+```
+
+That made the HTTP status of six families a property of their wording.
+Rephrasing `dataset-compiler`'s "is not declared in the dataset's `include`" —
+no logic change — moved that refusal from 400 to 500, i.e. re-opened #5352 for a
+different family, and no test and no gate would have gone red. Prime Directive
+#12 permits an accommodation like that only while it is declared, loud, tested
+**and removable on a schedule**; #5366 delivered the first three and nothing
+carried the fourth.
+
+**Five producers now declare their own verdict.** A new
+`dataset-refusal.ts` in `@objectstack/service-analytics` exports
+`datasetInvalidError` — the same shape as that package's existing
+`invalidFilterError` (`INVALID_FILTER` / 400) and `assertDimensionFields`
+(`INVALID_FIELD` / 400) — and five sites throw through it:
+
+- `dataset-compiler.ts` — a measure whose aggregate the v1 runtime cannot lower;
+  a dimension/measure traversing a relationship path the dataset never declared
+  in `include`;
+- `dataset-executor.ts` — an `order` key that is not a selected dimension or
+  measure; a `totals` grouping that is not a subset of the selected dimensions;
+- `native-sql-strategy.ts` — a join outside the dataset's declared allowlist.
+
+Their five entries are gone from the route's list, which is now a single
+`read-scope-sql` test.
+
+**`read-scope-sql` deliberately stays.** Its ten fail-closed refusals are RLS
+read-scope lowering failures whose inputs are an admin-authored policy and a
+compiler-generated join alias — not caller input — so `DATASET_INVALID` ("your
+request is invalid") may well be the wrong verdict and choosing the right one is
+a separate judgement, still tracked by #5367. Deleting the entry before that
+judgement lands would regress those ten from `400 DATASET_INVALID` to 500.
+
+**No outward behaviour change for the five.** They answered
+`400 DATASET_INVALID` before and answer `400 DATASET_INVALID` now, with the same
+message; what changed is the mechanism, from message-matching to the producer's
+own declaration. The one visible difference is for a bare `Error` that merely
+*resembles* one of those messages: it is no longer promoted to a 400. That is the
+point — a phrase is no longer a classification.
+
+`DATASET_INVALID` is registered in `ERROR_CODE_LEDGER` under
+`@objectstack/service-analytics` as well as `@objectstack/rest` (provenance, per
+ADR-0112 D3; the code itself is unchanged and the union does not grow), and the
+constructor types it as `RegisteredErrorCode` so an unregistered code is a
+compile error rather than a body some route rejects at runtime.
+
+Coverage: `dataset-refusal-envelope.test.ts` (service-analytics) pins each of the
+five refusals against its real producer — the refusal SET first, green before and
+after, then the envelope; `analytics-dataset-refusal-envelope.test.ts` (rest)
+drives all five end-to-end through a real `AnalyticsService` with positive
+controls on both the aggregate and raw-SQL paths; and
+`analytics-filter-refusal-envelope.test.ts` pins the deletion in both directions
+— the five messages answer 400 when enveloped and 500 when bare, so re-adding a
+regex entry turns it red.

.changeset/conversion-shadowed-alias-by-value.md

@@ -0,0 +1,51 @@
+---
+"@objectstack/spec": patch
+---
+
+fix(spec): an ADR-0087 rename resolves a shadowed alias by VALUE, instead of always leaving it behind (#4923)
+
+Every ADR-0087 D2 conversion that renames a key (`renameKey` /
+`renameConfigKey`, so `object` → `objectName`, `filters` → `filter`,
+`flow` → `flowName`, the four `notify` aliases, `description` → `subtitle`,
+the datasource driver-config aliases, …) used to do **nothing at all** when the
+canonical key was already present. The retired spelling then stayed in the
+converted metadata forever — the conversion had not finished converting.
+
+That was invisible while flow-node config contracts were `.strip`: the dead key
+was silently dropped at the execute-time parse and the node ran. Once #4001 批 9
+made those contracts strict it stopped being invisible — a stored `subflow`
+carrying `{ flowName, flow }` loads today and would be refused at execute time
+as a guard failure (not routable through a `fault` edge).
+
+**What changes.** A rename that meets both spellings now splits on whether the
+two values actually disagree:
+
+- **Same value** (structural equality, so two separately-authored
+  `{ status: 'stale' }` filters count as one declaration) → the alias carries
+  nothing the canonical key does not, so it is **deleted** and the conversion
+  emits its usual notice. Lossless hygiene, and it makes the transform
+  idempotent in both shape and notices.
+- **Different values** → **both keys are kept** and no notice is emitted. Two
+  spellings holding two different values is genuine author ambiguity, and an
+  upgrade tool that silently picked the canonical one would be editing a
+  configuration the customer never agreed to. The surviving pair is what lets
+  the strict node-config gates refuse with a prescription that **names both
+  keys** and asks for a decision.
+
+`notify`'s nested `source: { object, id }` lift follows the same rule: a part
+that repeats its flat counterpart is redundant and `source` is dropped, while a
+part that disagrees leaves the node **entirely** untouched so `source` reaches
+the strict contract intact. (The `wait` node's loose-key lift is deliberately
+NOT covered — it moves keys between two locations rather than resolving two
+spellings of one slot.)
+
+**What an author sees.** Metadata that named one slot twice with the same value
+loses the retired spelling at load and gains one deprecation notice per removal
+— the same notice a plain rename already emitted, so `objectstack validate`
+output is unchanged in kind. Metadata that named one slot twice with *different*
+values is unchanged by the conversion and is now refused by the strict
+node-config contracts with a message naming both keys; the fix is to decide
+which value is right, put it on the canonical key, and delete the alias. The
+batch-9 prescriptions were reworded accordingly: they no longer say the
+surviving twin is "dead" (true only under the old rule), because a key that now
+reaches that parse holds a value the canonical key does not.

.changeset/dangling-audit-provenance-bucket.md

@@ -0,0 +1,61 @@
+---
+"@objectstack/objectql": minor
+---
+
+feat(objectql): the dangling-reference audit stops skipping `readonly`
+references and files them in their own `provenance` bucket (#4743)
+
+`auditDanglingReferences` used to drop every `readonly` reference field before
+reading a single row. That skip rested on two grounds, and #4556 removed one of
+them: the platform no longer writes a NON-ID into a reference column
+(`sys_metadata_history.recorded_by` stored the sentinel string `'system'`; it
+stores `NULL` now). What the skip still covered afterwards was exactly one
+family — the audit-provenance fields `created_by` / `updated_by` /
+`organization_id` that `applySystemFields` injects, all `readonly: true`.
+
+Those hold **genuine ids, and genuine ids dangle**: delete one user and every
+row they ever created points `created_by` at a row that is gone. "Who did this"
+failing to resolve is precisely the question an audit trail exists to answer,
+so the remaining skip was blindness rather than economy. The audit now probes
+them.
+
+**They do not join `dangling`.** A deleted actor and a broken business foreign
+key are different findings with different remedies (usually nothing to do vs.
+re-seed the target or clear the link), and merging them would bury the second
+under the first. Two new report keys carry the new class, mirroring the
+unknown/absent split the report already makes everywhere else:
+
+| Key | Means |
+|:--|:--|
+| `provenance: DanglingReference[]` | a `readonly` provenance reference that resolves to nothing — same row shape as `dangling` |
+| `provenanceUndetermined: number` | a provenance reference whose target could not be probed at all |
+
+Both are **additive and optional in the type**, exactly like `aborted`: an
+existing consumer keeps compiling and keeps reading `dangling` with its meaning
+unchanged (a link the model *declares* is broken). Every report this module
+produces sets both explicitly.
+
+⚠️ **Expect `provenance` to be large on the first run against an aged
+database.** One deleted user dangles every row they ever touched. That number
+is pre-existing state being reported for the first time — not damage the audit
+caught being done, and not a regression introduced by looking at it.
+
+For the same reason `provenance` **alone does not raise the summary warning**.
+On a database of any age it is non-empty on every healthy run, and a line that
+always fires is the #4747 broken alarm again — it would train its reader
+straight past the run where `dangling` had something in it. The counts ride
+along in the payload whenever the line fires for a real finding, and the
+itemised rows are always in the returned report. `provenanceUndetermined` is
+separate from `undetermined` for the same reason: on a stack that never
+registers `sys_user`, every provenance value probes "cannot tell", and that is
+a fact about which platform tables are mounted, not about the audited data.
+
+Scan order gained a third tier to keep the change from costing the signal it
+sits next to: security surface, then objects carrying a business reference,
+then the provenance-only remainder. Admitting the family means nearly every
+object now has an auditable field, so without the tier a bounded run would
+spend its row budget on tables carrying only provenance and never reach the
+business findings the budget was built for.
+
+Part of #4743 (fact 2). The stale `assertReferencesResolve` comment in
+`engine.ts` (fact 1) is deliberately untouched here.

.changeset/env-settings-option-table-gate.md

@@ -0,0 +1,38 @@
+---
+'@objectstack/service-settings': patch
+---
+
+Settings: an `OS_*` env override is now checked against the specifier's declared `options` table (#5204)
+
+A manifest's `options` table has been enforced on the write path since #5131, but
+`SettingsService.get()` produced an effective value by a second route that never
+consulted it: an `OS_*` override was reshaped by the default's type and returned
+straight from the top of the cascade with `locked: true`. So the providers #5094 and
+#5133 retired from `mail.provider` could walk back in through the one door with no
+gate on it — `OS_MAIL_PROVIDER=sendgrid` reached the mail plugin unchallenged — and a
+plain typo such as `OS_BRANDING_THEME_MODE=drak` was served to every consumer as a
+normal value with normal-looking provenance, each consumer left to improvise.
+
+An override whose value the table does not declare is now **ignored** rather than
+repaired: the value falls through to the next layer of the cascade (a stored
+global/tenant/user value, else the manifest default), and the read API reports that
+layer honestly instead of claiming `source: 'env'` for a value not in force. The
+rejection is logged once at `error`, naming the variable, the rejected value, the legal
+value set and the consequence. The same audit runs at `registerManifest`, so a
+misconfigured deployment learns at boot rather than whenever somebody first opens the
+settings page.
+
+Registration **reports but never refuses**: option tables move, a pin that was legal
+the day it was written must not turn an upgrade into a crash-on-start.
+
+Two behaviour notes for anyone relying on the old shape:
+
+- Keys with no declared option table are untouched — text, boolean, number and
+  password overrides behave exactly as before. The check applies only to
+  `select`/`radio`/`multiselect` specifiers that declare a non-empty table.
+- A **rejected** override no longer pins its key against writes. `setMany` used to
+  refuse on the mere presence of the variable; judged by presence, an ignored value
+  would have left the key configurable by nothing at all — env value discarded, UI
+  refused with `SETTINGS_LOCKED`, and `get()` reporting `locked: false` to a settings
+  page whose save would then fail. An override that *is* in force still locks the key,
+  unchanged.

.changeset/hook-context-session-positions-preserve-audit.md

@@ -0,0 +1,53 @@
+---
+"@objectstack/spec": minor
+---
+
+feat(spec): `HookContext.session` declares `positions` and `preserveAudit` (#5605)
+
+Two keys the engine has been **producing** all along, that consumers have been
+**reading** and the docs have been **teaching**, were missing from the contract:
+`HookContextSchema.session` declared `userId` / `actor` / `organizationId` /
+`accessToken` / `isSystem` / `skipTriggers` / `skipAutomations` and nothing else.
+Both are now declared, per the maintainer ruling on #5605.
+
+This is the mirror of the `session.roles` retirement (#5050). That key was
+declared-never-produced, so it was removed; these two are
+produced-never-declared, so they are added. Same `session` block, opposite
+drift, opposite fix.
+
+**What was broken.** `HookContextSchema` is deliberately not `.strict()` — it is
+the runtime shape the engine hands a handler, and strictness there would make
+every engine-side enrichment a breaking change for anyone parsing a context they
+were given. The cost of that tolerance is that an undeclared key is **stripped
+in silence**:
+
+- `HookContextSchema.parse(ctx)` — the exact call the generated reference page
+  documents as the way to consume a context — returned a session with the
+  caller's `positions` and the import's `preserveAudit` dropped on the floor.
+- A handler typed the way the automation docs teach, `(ctx: HookContext)`, could
+  not read either key: `ctx.session?.positions` was a `TS2339`. The two
+  `kernel/runtime-services` pages that teach
+  `positions: ctx.session?.positions` compiled only because they annotate `ctx`
+  as `any` — copying both the code and the documented annotation did not build.
+
+**`positions`** (`string[]`, optional) is the ADR-0090 D3 placement vocabulary,
+copied verbatim from `ExecutionContext.positions` by ObjectQL's `buildSession()`.
+Its `.describe()` states the boundary the ruling asked for, because the boundary
+is the whole reason this key needed a decision rather than a patch: it is
+**readable context, never an authorization input**. A hook may forward it as the
+sharing service's evaluation context, tailor a message, or log it. A hook must
+not make the access decision itself by testing it — privilege is judged by the
+security service on the execution context (capability grants `permissions`,
+placements `positions`, and the derived posture). A hook re-deciding access from
+this array decides, somewhere with no access to the grant model, something that
+was already decided; that is structurally the mistake the `roles` tombstone
+exists to prevent, one vocabulary later.
+
+**`preserveAudit`** (`boolean`, optional) is the #3493 historical-import flag:
+server-set, opt-in, absent on normal writes, and read by the built-in audit hook
+to keep a caller-supplied `updated_at`/`updated_by` instead of stamping the
+import instant. It has a live consumer, so it could only ever be declared.
+
+Purely additive — both keys are optional, the shape stays non-strict, and no
+existing context, handler or stored metadata changes. Contexts are built per
+operation and never persisted, so there is nothing to migrate.