Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions .changeset/strictness-ledger-gate.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
---
"@objectstack/spec": patch
---

Add a gate that holds the #4001 strictness ledger to the code it describes.

`docs/audits/2026-07-unknown-key-strictness-ledger.md` is the campaign's map:
which `z.object` sites are authorable (the `.strict()` ratchet target), which are
wire, which are deliberately open. Every step reads it to pick the next move, and
nothing kept it honest. It went stale twice in one week — a classification that
verification disproved (`hook.zod.ts`, #4207) and a "next step" that had shipped
months earlier (the warning layer, #4218). A map that drifts is worse than no
map, because it gets followed.

`check:strictness-ledger` enforces the claims in it that are mechanically
checkable:

- **Site counts** — the ledger states its own method (`z.object(` occurrences per
file), so every number is verifiable. A stale count means schemas were added or
removed under a `Class` verdict nobody re-examined.
- **Coverage** — every `*.zod.ts` with sites in a triaged directory must have a
row. Zero-site files (pure enum/token modules) are skipped, and become
reportable the day they grow their first `z.object(`.
- **Section totals**, and that a row claiming "strict as of" names a file that
really contains `.strict()`.

The `Class` column is deliberately not checked — authorable vs wire vs open is a
human judgement, and the campaign's rule is verify-before-tightening. The gate
protects the arithmetic and coverage so that judgement is made against current
code.

First run found 11 drifts, including six moved counts (`ui/app.zod.ts` had gone
11 → 18), two unbalanced section totals, and one genuinely unclassified file
(`automation/io-node-config.zod.ts`). All corrected here.
12 changes: 12 additions & 0 deletions .github/workflows/spec-liveness-check.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,9 @@ on:
# The variant/doc gate reads hand-written pages, so editing one can break it
# without touching packages/spec at all.
- 'content/docs/**'
# Same for the strictness ledger — it is a doc, and editing it can break
# the gate that now holds it to the code.
- 'docs/audits/**'

permissions:
contents: read
Expand Down Expand Up @@ -61,3 +64,12 @@ jobs:
# apps.mdx said the nav tree had "eight item types" while the schema had nine.
- name: Check discriminated-union variants are documented
run: pnpm --filter @objectstack/spec check:variant-docs

# #4001 follow-up. The strictness ledger is the campaign's map — which
# z.object sites are authorable, wire or open. Every step reads it to pick
# the next move, and nothing kept it honest: it went stale twice in one
# week (a classification verification disproved, and a "next step" that had
# shipped months earlier). A map that drifts is worse than no map, because
# it gets followed. This holds its arithmetic and its coverage to the code.
- name: Check the strictness ledger matches the code
run: pnpm --filter @objectstack/spec check:strictness-ledger
69 changes: 56 additions & 13 deletions docs/audits/2026-07-unknown-key-strictness-ledger.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,24 +129,24 @@ Classification is per the rule above; **(p)** marks a provisional call made
from the file's exports/JSDoc rather than a full read — verify before
tightening (the #4001 "sharing-rule lesson": candidates, not verdicts).

### `ui/` — 192 sites
### `ui/` — 197 sites

| File | Sites | Class | Note / next action |
|---|---|---|---|
| `action.zod.ts` | 8 | authorable | param schema strict (#3746); remaining blocks ride later steps |
| `view.zod.ts` | 51 | authorable | partially strict (ADR-0089); long tail of sub-blocks |
| `action.zod.ts` | 9 | authorable | param schema strict (#3746); remaining blocks ride later steps |
| `view.zod.ts` | 50 | authorable | partially strict (ADR-0089); long tail of sub-blocks |
| `component.zod.ts` | 29 | authorable | **next candidate** — SDUI component defs; check React-prop open slots first (p) |
| `theme.zod.ts` | 14 | authorable (p) | authored themes |
| `app.zod.ts` | 11 | authorable | **strict as of #4001 PR B** — `AppSchema` + branding / area / context-selector / contribution, and the nav-item union converted to `z.discriminatedUnion('type', …)` (the union-error question, settled empirically: matched-branch-only errors, exact recursive paths, `toJSONSchema` clean). Per-target `params` stay open. PR A (#4142) tombstoned the seven audit-dead keys first |
| `app.zod.ts` | 18 | authorable | **strict as of #4001 PR B** — `AppSchema` + branding / area / context-selector / contribution, and the nav-item union converted to `z.discriminatedUnion('type', …)` (the union-error question, settled empirically: matched-branch-only errors, exact recursive paths, `toJSONSchema` clean). Per-target `params` stay open. PR A (#4142) tombstoned the seven audit-dead keys first |
| `dashboard.zod.ts` | 11 | authorable | partially strict |
| `widget.zod.ts` | 9 | authorable (p) | |
| `page.zod.ts` | 7 | authorable | partially strict (ADR-0089) |
| `chart.zod.ts` / `i18n.zod.ts` / `responsive.zod.ts` | 6+6+6 | authorable (p) | i18n label shapes are wide-open records by design — verify |
| `dataset.zod.ts` / `animation.zod.ts` / `dnd.zod.ts` / `keyboard.zod.ts` / `touch.zod.ts` | 4 ea | authorable (p) | interaction configs |
| `chart.zod.ts` / `i18n.zod.ts` / `responsive.zod.ts` | 6+6+4 | authorable (p) | i18n label shapes are wide-open records by design — verify |
| `dataset.zod.ts` / `animation.zod.ts` / `dnd.zod.ts` / `keyboard.zod.ts` / `touch.zod.ts` | 4+4+4+4+7 | authorable (p) | interaction configs |
| `notification.zod.ts` / `offline.zod.ts` / `report.zod.ts` | 3 ea | authorable (p) | |
| `sharing.zod.ts` | 2 | authorable (p) | public-sharing config |

### `data/` — 163 sites
### `data/` — 154 sites

| File | Sites | Class | Note |
|---|---|---|---|
Expand All @@ -165,11 +165,11 @@ tightening (the #4001 "sharing-rule lesson": candidates, not verdicts).
| `external-catalog.zod.ts` | 4 | wire (p) | |
| `field-value.zod.ts` / `seed.zod.ts` / `validation.zod.ts` | 1 ea | mixed (p) | |

### `automation/` — 80 sites
### `automation/` — 89 sites

| File | Sites | Class | Note |
|---|---|---|---|
| `flow.zod.ts` | 12 | authorable | **strict as of #4001** (4 schemas; `FlowVersionHistorySchema` is runtime — stays tolerant) |
| `flow.zod.ts` | 11 | authorable | **strict as of #4001** (4 schemas; `FlowVersionHistorySchema` is runtime — stays tolerant) |
| `sync.zod.ts` / `etl.zod.ts` | 12+10 | authorable (p) | authored pipelines — **candidates** |
| `trigger-registry.zod.ts` | 11 | mixed | descriptors are code-registered (wire-ish); bindings authored |
| `execution.zod.ts` | 8 | wire | run-state envelopes — never strict |
Expand All @@ -178,6 +178,8 @@ tightening (the #4001 "sharing-rule lesson": candidates, not verdicts).
| `bpmn-interop.zod.ts` | 5 | wire (p) | interop import shapes |
| `approval.zod.ts` | 4 | authorable | **strict as of #4001 step 3** — all four authoring schemas (node config / approver / escalation / decision-output). The published JSON schema carries `additionalProperties: false` into the Studio form AND `registerFlow()` config validation (#4027/#4040), so an unknown key in an approval node's `config` is rejected at registration too — verified: `z.toJSONSchema` on the strict lazySchema does not throw (#3746 hazard checked) |
| `node-executor.zod.ts` | 4 | wire | executor contract |
| `io-node-config.zod.ts` | 2 | authorable | `NotifyConfigSchema` / `HttpConfigSchema` (#4045) — the sibling contracts that validate the **open** `config` slot on flow `notify` / `http` nodes. Authored per-node, so the open-slot exemption above does not extend to them; candidate once the executors' own drift is verified |
| `builtin-node-config.zod.ts` | 8 | authorable | Same family (#4045): the CRUD quartet, `screen`, `map`. Written from what the executors read rather than from the descriptors' `configSchema` literals, and reconciled bidirectionally by `builtin-node-form-zod-ledger.test.ts` — so unlike most rows here, this one already has a drift check of its own. Same candidacy note as `io-node-config` |
| `webhook.zod.ts` | 1 | authorable (p) | spec-only (#3461) |

### `security/` — 20 sites
Expand Down Expand Up @@ -213,10 +215,7 @@ tightening (the #4001 "sharing-rule lesson": candidates, not verdicts).

## Next steps (verify-then-enforce, one shape at a time)

1. Promote this ledger to a machine-checked gate (pattern of
`packages/spec/liveness/` + `check:liveness`) once enough of the surface is
classified that the table above is enforceable rather than descriptive.
2. Let the warning layer run in the wild for a release, then schedule the v18
1. Let the warning layer run in the wild for a release, then schedule the v18
strict close-out on what it actually reports — which is the whole point of
having built it. Nothing more to do here until there is field data.

Expand Down Expand Up @@ -292,5 +291,49 @@ apps (23 + 6 objects, 20 + 1 pages, 4 + 1 datasets, 3 + 1 dashboards) report
**zero** unknown keys. No finding this time — worth recording precisely because
the app step's `ACCOUNT_APP.defaultOpen` came from exactly this class of check.

## This file is now machine-checked

`pnpm --filter @objectstack/spec check:strictness-ledger` (wired into the Spec
Liveness Check workflow) holds the two claims here that are mechanically
checkable, so this map cannot go stale in silence again:

- **Site counts.** The method is stated above — `z.object(` occurrences per file
— so every number in the triage tables is verifiable. A count that no longer
matches means schemas were added or removed under a `Class` verdict nobody
re-examined. Touching a file forces you back through this ledger.
- **Coverage.** Every `*.zod.ts` in a triaged directory that HAS sites must have
a row. A new one is undeclared surface. Zero-site files (pure enum/token
modules like `data/date-macros.zod.ts`) are skipped — there is nothing to
classify — and become reportable the day they grow their first `z.object(`.
- **Section totals**, and that any row claiming "strict as of" names a file that
really contains `.strict()`.

Deliberately NOT checked: the `Class` column. Authorable vs wire vs open is a
judgement about who writes the input, and this campaign's rule is
verify-before-tightening. The gate protects the arithmetic and the coverage so
that judgement is always made against current code.

**What it found on its first run — 11 drifts, in a file being actively edited
by the campaign that owns it.** Six counts had moved (`ui/app.zod.ts` 11 → 18,
`ui/touch.zod.ts` 4 → 7, `ui/action.zod.ts` 8 → 9, `ui/view.zod.ts` 51 → 50,
`ui/responsive.zod.ts` 6 → 4, `automation/flow.zod.ts` 12 → 11), two section
totals no longer summed, and six files had no row at all. The `app.zod.ts` gap
was **self-inflicted**: the app step (#4165) added seven schemas to that file
and updated the row's prose without touching its count. Five of the six
undeclared files turned out to have zero sites — which is what motivated the
skip rule above — leaving `automation/io-node-config.zod.ts` as the one genuine
omission, now classified.

That is the argument for the gate in one paragraph: the people most familiar
with this ledger, editing it in the same week, still left eleven drifts in it.

**And then it worked for real, before it had even merged.** While the gate sat
in review, `main` landed `automation/builtin-node-config.zod.ts` (#4045/#4228) —
eight new sites, sibling to `io-node-config.zod.ts`. Merging `main` turned the
gate red on a branch whose own diff had not touched a single schema, which is
precisely the intended behaviour: the file arrived, so someone had to classify
it. It is now a row. Every existing count survived that merge unchanged, so the
failure was exactly as narrow as it should have been.

Long tail stays gated on a verification pass per shape — never a one-shot
"make all ~453 sites strict" (ADR-0054 ratchet; #4001's own recommendation).
1 change: 1 addition & 0 deletions packages/spec/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -210,6 +210,7 @@
"check:liveness": "tsx scripts/liveness/check-liveness.mts",
"check:empty-state": "tsx scripts/liveness/check-empty-state.mts",
"check:variant-docs": "tsx scripts/check-variant-docs.mts",
"check:strictness-ledger": "tsx scripts/check-strictness-ledger.mts",
"gen:react-blocks": "tsx scripts/build-react-blocks-contract.ts",
"check:react-blocks": "tsx scripts/build-react-blocks-contract.ts --check",
"check:react-conformance": "tsx scripts/check-react-blocks-conformance.ts",
Expand Down
Loading
Loading