|
| 1 | +# Unknown-key strictness ledger — the authorable / wire / open triage (#4001) |
| 2 | + |
| 3 | +**Date**: 2026-07-30 · **Issue**: [#4001](https://github.com/objectstack-ai/objectstack/issues/4001) · **Builds on**: #3405 / #3746 (the `ActionParamSchema` template), #1535 (`ObjectSchema.create` unknown-key error), ADR-0089 D3a (view/page strict), ADR-0078 (no-silently-inert-metadata), ADR-0049 (enforce-or-remove), ADR-0054 (ratchet idiom) |
| 4 | + |
| 5 | +Zod's default is `.strip`: a key the schema does not declare is **silently |
| 6 | +discarded** and the instance goes on parsing. On an authorable surface that is |
| 7 | +the worst failure mode — the author (human or AI) gets a success envelope and |
| 8 | +ships metadata that quietly ignores their config. This ledger is #4001 step 1: |
| 9 | +the persisted classification of *which* `z.object` sites that default is wrong |
| 10 | +for, so the `.strict()` ratchet advances on evidence instead of a one-shot |
| 11 | +sweep. **1885 sites was never the target number** — a large share of the spec |
| 12 | +is wire/response shape where strictness would be a forward-compat bug. |
| 13 | + |
| 14 | +## Classification rule |
| 15 | + |
| 16 | +One question decides the class: **who writes this schema's input?** |
| 17 | + |
| 18 | +| Class | Input written by | Unknown-key policy | |
| 19 | +|---|---|---| |
| 20 | +| **authorable** | A human or AI author, into `*.object.ts` / `defineStack` config / Studio / MCP | `.strict()` + fixable error (the ratchet target) | |
| 21 | +| **wire** | Another machine: server responses, connector payloads, runtime envelopes, persisted runtime state | stay tolerant (`.strip` / `.passthrough`); strictness here turns an upstream *addition* into our parse crash | |
| 22 | +| **open** | Deliberately schemaless user data (record bodies, per-node-type `config`, React props) | stay open; a *sibling* contract validates it (e.g. a node executor's `configSchema`, #4027/#4040) | |
| 23 | + |
| 24 | +Mixed files carry both — classify per schema, not per file. A **response-side |
| 25 | +extension of an authoring schema** (e.g. `EffectiveObjectPermissionSchema`) |
| 26 | +must explicitly `.strip()` back, because `.extend()` inherits `.strict()`. |
| 27 | + |
| 28 | +## Standard wiring |
| 29 | + |
| 30 | +`strictUnknownKeyError` in `shared/suggestions.zod.ts` (generalized from the |
| 31 | +#3746 hand-rolled map) is the one factory every strict authoring schema wires: |
| 32 | + |
| 33 | +```ts |
| 34 | +z.object({ ... }, { error: strictUnknownKeyError({ surface, knownKeys, aliases, guidance, history }) }).strict() |
| 35 | +``` |
| 36 | + |
| 37 | +- `aliases` — semantic near-misses edit distance cannot reach (`visibleWhen` → |
| 38 | + `visible`, `from` → `source`, `read` → `allowRead`). |
| 39 | +- `guidance` — exact-key prescriptions: **tombstones for retired keys** (the |
| 40 | + rejection carries the upgrade — AGENTS.md Post-Task Checklist #3) and |
| 41 | + wrong-layer pointers (`apiOperations` is response-side; `objectName` belongs |
| 42 | + on the start node). |
| 43 | +- Key lists live beside the schema and are **drift-guarded by tests** (an |
| 44 | + "accepts every declared key" probe), because the schema body is lazy. |
| 45 | + |
| 46 | +Every ratchet step ships only with the empirical zero-breakage pass: full |
| 47 | +`@objectstack/spec` suite + `tsc`, downstream consumer suites, and |
| 48 | +`objectstack validate` on all three example apps. Inference is not evidence — |
| 49 | +the first application of this gate (below) caught a real spec gap. |
| 50 | + |
| 51 | +## Ratchet state |
| 52 | + |
| 53 | +### Already strict before #4001 (for reference) |
| 54 | + |
| 55 | +| Surface | Since | |
| 56 | +|---|---| |
| 57 | +| `data/object.zod.ts` — `ObjectSchema.create()` unknown-key error + `UNKNOWN_KEY_GUIDANCE` tombstones; strict capabilities / tenancy / CRUD-override blocks | #1535, #2377, #2763 | |
| 58 | +| `ui/view.zod.ts` / `ui/page.zod.ts` — form/page component schemas | ADR-0089 D3a | |
| 59 | +| `ui/dashboard.zod.ts`, `data/field.zod.ts` (blocks), `ai/agent.zod.ts`, `ai/tool.zod.ts` (blocks) | various | |
| 60 | +| `ui/action.zod.ts` — `ActionParamSchema` (the template) | #3405 / #3746 | |
| 61 | + |
| 62 | +### This step (#4001 Tier-A slice) |
| 63 | + |
| 64 | +| Schema | File | Evidence class | |
| 65 | +|---|---|---| |
| 66 | +| `PermissionSetSchema`, `ObjectPermissionSchema`, `FieldPermissionSchema`, `AdminScopeSchema` | `security/permission.zod.ts` | A silently dropped key on the capability container is the ADR-0049 asymmetry itself; retired `contextVariables` (ADR-0105 D11) / `isProfile` (ADR-0090 D2) get tombstones | |
| 67 | +| `FlowSchema`, `FlowNodeSchema`, `FlowEdgeSchema`, `FlowVariableSchema` | `automation/flow.zod.ts` | Most AI-authored surface; cloud#688 / #2419 class. Node `config` stays **open** (executor `configSchema` owns it, #4027/#4040) | |
| 68 | +| `ActionParamSchema` re-homed onto the shared factory | `ui/action.zod.ts` | #3746 template, byte-identical messages | |
| 69 | + |
| 70 | +**Findings log** (what the gate caught the day it was turned on): |
| 71 | +`PermissionSetSchema` could not represent `description` — yet the built-in |
| 72 | +default sets author it and `plugin-security`'s Setup projection reads it |
| 73 | +(`permission-set-projection.ts`). It had been silently stripped at every parse |
| 74 | +(the ADR-0078 §3 inverse-drift class). Fixed contract-first: `description` is |
| 75 | +now a declared key. This is the empirical argument for the ratchet: the |
| 76 | +inference "no metadata in the repo carries unknown keys" was **false**, and |
| 77 | +only the strict gate could prove it. |
| 78 | + |
| 79 | +## File-level triage — the five authorable directories |
| 80 | + |
| 81 | +Site counts are `z.object(` occurrences per file (2026-07-30, this branch). |
| 82 | +Classification is per the rule above; **(p)** marks a provisional call made |
| 83 | +from the file's exports/JSDoc rather than a full read — verify before |
| 84 | +tightening (the #4001 "sharing-rule lesson": candidates, not verdicts). |
| 85 | + |
| 86 | +### `ui/` — 192 sites |
| 87 | + |
| 88 | +| File | Sites | Class | Note / next action | |
| 89 | +|---|---|---|---| |
| 90 | +| `action.zod.ts` | 8 | authorable | param schema strict (#3746); remaining blocks ride later steps | |
| 91 | +| `view.zod.ts` | 51 | authorable | partially strict (ADR-0089); long tail of sub-blocks | |
| 92 | +| `component.zod.ts` | 29 | authorable | **next candidate** — SDUI component defs; check React-prop open slots first (p) | |
| 93 | +| `theme.zod.ts` | 14 | authorable (p) | authored themes | |
| 94 | +| `app.zod.ts` | 11 | authorable | **next verified step** — `AppSchema` + nav-item union; recursive `NavigationItemSchema` needs union-error care | |
| 95 | +| `dashboard.zod.ts` | 11 | authorable | partially strict | |
| 96 | +| `widget.zod.ts` | 9 | authorable (p) | | |
| 97 | +| `page.zod.ts` | 7 | authorable | partially strict (ADR-0089) | |
| 98 | +| `chart.zod.ts` / `i18n.zod.ts` / `responsive.zod.ts` | 6+6+6 | authorable (p) | i18n label shapes are wide-open records by design — verify | |
| 99 | +| `dataset.zod.ts` / `animation.zod.ts` / `dnd.zod.ts` / `keyboard.zod.ts` / `touch.zod.ts` | 4 ea | authorable (p) | interaction configs | |
| 100 | +| `notification.zod.ts` / `offline.zod.ts` / `report.zod.ts` | 3 ea | authorable (p) | | |
| 101 | +| `sharing.zod.ts` | 2 | authorable (p) | public-sharing config | |
| 102 | + |
| 103 | +### `data/` — 163 sites |
| 104 | + |
| 105 | +| File | Sites | Class | Note | |
| 106 | +|---|---|---|---| |
| 107 | +| `object.zod.ts` | 20 | authorable | top-level already guarded (#1535); inner blocks partially strict | |
| 108 | +| `data-engine.zod.ts` | 14 | wire (p) | engine contract shapes | |
| 109 | +| `external-lookup.zod.ts` | 12 | mixed (p) | authored config + wire results | |
| 110 | +| `seed-loader.zod.ts` | 12 | mixed (p) | seed file shapes are authored; loader state is runtime | |
| 111 | +| `field.zod.ts` | 11 | authorable | partially strict | |
| 112 | +| `filter.zod.ts` / `query.zod.ts` | 11+10 | open | query dialect — user data flows through; validated semantically elsewhere | |
| 113 | +| `driver-nosql.zod.ts` / `driver.zod.ts` / `driver-sql.zod.ts` | 10+9+2 | wire | driver capability contracts | |
| 114 | +| `datasource.zod.ts` | 9 | authorable (p) | stack-authored config — **candidate** | |
| 115 | +| `analytics.zod.ts` | 8 | mixed (p) | | |
| 116 | +| `document.zod.ts` | 8 | wire (p) | | |
| 117 | +| `hook.zod.ts` / `hook-body.zod.ts` | 6+2 | authorable (p) | `defineHook` — **candidate** | |
| 118 | +| `mapping.zod.ts` | 3 | authorable (p) | | |
| 119 | +| `external-catalog.zod.ts` | 4 | wire (p) | | |
| 120 | +| `field-value.zod.ts` / `seed.zod.ts` / `validation.zod.ts` | 1 ea | mixed (p) | | |
| 121 | + |
| 122 | +### `automation/` — 80 sites |
| 123 | + |
| 124 | +| File | Sites | Class | Note | |
| 125 | +|---|---|---|---| |
| 126 | +| `flow.zod.ts` | 12 | authorable | **strict as of #4001** (4 schemas; `FlowVersionHistorySchema` is runtime — stays tolerant) | |
| 127 | +| `sync.zod.ts` / `etl.zod.ts` | 12+10 | authorable (p) | authored pipelines — **candidates** | |
| 128 | +| `trigger-registry.zod.ts` | 11 | mixed | descriptors are code-registered (wire-ish); bindings authored | |
| 129 | +| `execution.zod.ts` | 8 | wire | run-state envelopes — never strict | |
| 130 | +| `state-machine.zod.ts` | 7 | authorable (p) | | |
| 131 | +| `control-flow.zod.ts` | 6 | authorable (p) | validated structurally by `validateControlFlow` | |
| 132 | +| `bpmn-interop.zod.ts` | 5 | wire (p) | interop import shapes | |
| 133 | +| `approval.zod.ts` | 4 | authorable | **next candidate** — v17 approval nodes are new authoring surface | |
| 134 | +| `node-executor.zod.ts` | 4 | wire | executor contract | |
| 135 | +| `webhook.zod.ts` | 1 | authorable (p) | spec-only (#3461) | |
| 136 | + |
| 137 | +### `security/` — 20 sites |
| 138 | + |
| 139 | +| File | Sites | Class | Note | |
| 140 | +|---|---|---|---| |
| 141 | +| `explain.zod.ts` | 11 | wire | permission-explain responses — never strict | |
| 142 | +| `permission.zod.ts` | 4 | authorable | **strict as of #4001**; `EffectiveObjectPermissionSchema` explicitly `.strip()`s (wire) | |
| 143 | +| `rls.zod.ts` | 3 | authorable | **next candidate** — a stripped RLS key is a silent policy hole | |
| 144 | +| `sharing.zod.ts` | 2 | authorable | **next candidate** — same class | |
| 145 | + |
| 146 | +### `studio/` — 27 sites |
| 147 | + |
| 148 | +| File | Sites | Class | Note | |
| 149 | +|---|---|---|---| |
| 150 | +| `object-designer.zod.ts` | 12 | authorable (p) | Studio-written JSON — machine-authored but *our* machine; strict protects the builder itself | |
| 151 | +| `plugin.zod.ts` | 8 | mixed (p) | | |
| 152 | +| `flow-builder.zod.ts` | 7 | authorable (p) | independent of `FlowSchema` shapes | |
| 153 | + |
| 154 | +## Other directories (coarse; classify per schema before touching) |
| 155 | + |
| 156 | +| Dir | Sites | Dominant class | Rationale | |
| 157 | +|---|---|---|---| |
| 158 | +| `api/` | 426 | wire | REST/GraphQL request/response contracts — tolerant by design | |
| 159 | +| `system/` | 383 | mixed | manifest/datasource blocks are authored; runtime envelopes are wire | |
| 160 | +| `kernel/` | 351 | wire | plugin/kernel contracts, code-to-code | |
| 161 | +| `cloud/` | 83 | wire | multi-tenant runtime | |
| 162 | +| `ai/` | 75 | mixed | agent/tool/skill definitions authored (partially strict already); model/provider payloads wire | |
| 163 | +| `integration/` | 64 | wire | connector payloads — upstream adds fields freely | |
| 164 | +| `identity/` | 34 | mixed | position/user shapes authored; auth payloads wire | |
| 165 | +| `shared/` | 25 | n/a | utilities and building blocks; strictness decided at the consuming schema | |
| 166 | +| `qa/` | 6 | n/a | test fixtures | |
| 167 | + |
| 168 | +## Next steps (verify-then-enforce, one shape at a time) |
| 169 | + |
| 170 | +1. `ui/app.zod.ts` — `AppSchema` + navigation union (highest-traffic remaining |
| 171 | + authorable type; needs union-error design so the strict error is readable). |
| 172 | +2. `security/rls.zod.ts` + `security/sharing.zod.ts` — small, security-class. |
| 173 | +3. `automation/approval.zod.ts` — new v17 authoring surface, tighten while young. |
| 174 | +4. `data/hook.zod.ts`, `data/datasource.zod.ts` — `defineHook` / stack config. |
| 175 | +5. Promote this ledger to a machine-checked gate (pattern of |
| 176 | + `packages/spec/liveness/` + `check:liveness`) once enough of the surface is |
| 177 | + classified that the table above is enforceable rather than descriptive. |
| 178 | + |
| 179 | +Long tail stays gated on a verification pass per shape — never a one-shot |
| 180 | +"make all ~453 sites strict" (ADR-0054 ratchet; #4001's own recommendation). |
0 commit comments