docs: implementation-accuracy audit of the #4212-family affected docs - #4312
Merged
Conversation
os-zhuang
force-pushed
the
claude/docs-accuracy-4212-followup
branch
from
July 31, 2026 05:39
04ec8ce to
998bece
Compare
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
WIP checkpoint — the adversarial verification pass is still running and will
add repairs on top of this commit. Recorded now so the work survives.
An audit agent read each doc, located the backing implementation under
packages/, and applied evidence-backed fixes in place (158 across 18 docs).
Every fix carries file:line evidence in the run's fix log. The heaviest:
- kernel/cluster.mdx (22): documented an `eventBus.emit(...)` API that exists
nowhere in the codebase, a `ctx.cluster` PluginContext property that does
not exist, `defineStack({ cluster })` that stack.zod.ts silently strips,
fabricated `postgres`/`nats` drivers, the wrong env var (real one is
OS_CLUSTER_DRIVER), and an at-least-once delivery promise neither shipped
driver provides.
- protocol/kernel/plugin-spec.mdx (20): `plugin.manifest.ts` as the manifest
filename — grep-empty across the implementation, the real one is
objectstack.plugin.json; `context.plugins.isInstalled()` which does not
exist; compound semver ranges ('>=1.0.0 <2.0.0') that make the resolver's
anchored parse throw; init described as registration order when bootstrap()
runs the dependency-topological order.
- data-modeling/external-datasources.mdx (3): credentialsRef samples used a
`secret:` prefix the shipped SecretBinder cannot dereference (`sys_secret:`),
so copying the sample fails closed at connect.
- deployment/migration-from-objectql.mdx (2): a `^3.1.0` pin that can never
resolve to the current 16.1.0 line.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HrRNgrWaRtggzmrHpbomyh
Ran the repo's docs-accuracy-audit workflow scoped to the drift-check advisory from #4242 (14 docs) plus 4 the retired-surface grep implicated. 158 evidence-backed fixes across 18 docs, each cited to file:line. The largest classes: - kernel/cluster.mdx (22): a documented `eventBus.emit(...)` that exists nowhere in packages/, a `ctx.cluster` PluginContext property that does not exist, `defineStack({ cluster })` that stack.zod.ts silently strips, `postgres`/`nats` listed as shipped drivers when neither package exists, `REDIS_URL` for the real `OS_CLUSTER_DRIVER`, and an at-least-once delivery promise no shipped driver provides. - protocol/kernel/plugin-spec.mdx (20): `plugin.manifest.ts` as the manifest filename — grep-empty across packages/, apps/ and examples/; the real one is `objectstack.plugin.json`. Plus `context.plugins.isInstalled()` (no such member), compound semver ranges (`'>=1.0.0 <2.0.0'`) that make the resolver throw, and init ordering described as registration order when bootstrap() runs a dependency topological sort. - data-modeling/external-datasources.mdx: `credentialsRef: 'secret:…'` — the shipped binder only resolves `sys_secret:<id>`, so the sample as written fails closed at connect. - deployment/migration-from-objectql.mdx: a `^3.1.0` pin that can never resolve to the current 16.1.0 line. VERIFICATION STATUS — 7 of 18 docs carry an adversarial-verifier pass; the other 11 do not (the run hit the session limit mid-phase). The verifier earned its keep on the 7 it reached: 8 repairs, including a fabricated `enforceNetworkAccess` (real: `enforceNetworkRequest`, plugin-permission-enforcer.ts:212) and an over-broad retirement claim that would have contradicted the live app-bundle `onEnable`. The remaining 11 are committed unverified and MUST NOT ship until their verifier pass runs — the PR stays draft until then. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HrRNgrWaRtggzmrHpbomyh
The audit pass removed a "role" occurrence from kernel/events.mdx (now clean) and one from kernel/services-checklist.mdx (4 → 3). check-role-word is a ratchet, so an improvement fails the gate until the baseline records it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HrRNgrWaRtggzmrHpbomyh
os-zhuang
force-pushed
the
claude/docs-accuracy-4212-followup
branch
from
July 31, 2026 05:43
b9d2ea5 to
2383192
Compare
…l/kernel/index) Verifier pass for two more of the 18 audited docs completed: 27 of the audit's fixes confirmed against code, 5 repaired. Checkpoint commit — the remaining verifier passes are still running and will land separately. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HrRNgrWaRtggzmrHpbomyh
Completes the verifier phase for all 18 audited docs (36/36 agents, 0 errors). Across the run: 158 audit fixes, 29 verifier repairs, 91 residual items recorded rather than guessed at. The repairs are why this phase exists — the audit agents were confident and sometimes wrong. Representative catches: - `enforceNetworkAccess` invented for the real `enforceNetworkRequest` (plugin-permission-enforcer.ts:212). - A retirement callout widened to "None of it existed" over a list containing `onEnable`, which would have contradicted the live app-bundle contract AppPlugin.start() dispatches (app-plugin.ts:574). - `cluster.lock.withLock(key, fn)` attributed to service-job's cron scheduler, which actually does `lock.acquire(...)` + manual release (cron-job-adapter.ts:139-147). - A "(planned)" marker dropped from the webhook dispatcher, which filed an already-shipping partitioned dispatcher under "Not started". Gates: check-doc-authoring 215 files clean; check-role-word OK (44 baselined); docs build compiles all pages. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HrRNgrWaRtggzmrHpbomyh
os-zhuang
marked this pull request as ready for review
July 31, 2026 06:24
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Ran the repo's
docs-accuracy-auditworkflow scoped to the drift-check advisory from #4242 (its 14 docs) plus 4 more that a grep for the retired surfaces implicated. 158 evidence-backed fixes across 18 docs, each cited tofile:line, then 29 adversarial-verifier repairs on top.The findings are mostly the "copy this and it breaks" class:
kernel/cluster.mdxeventBus.emit(...)that exists nowhere inpackages/; actx.clusterPluginContext property that does not exist;defineStack({ cluster })thatstack.zod.tssilently strips;postgres/natslisted as shipped drivers when neither package exists;REDIS_URLfor the realOS_CLUSTER_DRIVER; an at-least-once delivery promise no shipped driver providesprotocol/kernel/plugin-spec.mdxplugin.manifest.tsas the manifest filename — grep-empty acrosspackages/,apps/,examples/; the real one isobjectstack.plugin.json. Alsocontext.plugins.isInstalled()(no such member), compound semver ranges ('>=1.0.0 <2.0.0') that make the resolver throw, andinitordering described as registration order whenbootstrap()runs a dependency topological sortconcepts/metadata-lifecycle.mdxprotocol/kernel/index.mdxkernel/services-checklist.mdxdata-modeling/formulas.mdxreleases/v9.mdxprotocol/objectql/state-machine.mdxplugins/index.mdxos devdefaults to a project-anchored SQLite file, not an auto-registered memory driverkernel/services.mdx,releases/implementation-status.mdxdeployment/vercel.mdxprotocol/objectql/query-syntax.mdx,permissions/authentication.mdxkernel/events.mdx,data-modeling/external-datasources.mdxcredentialsRef: 'secret:…'— the shipped binder only resolvessys_secret:<id>, so the sample as written fails closed at connectdeployment/migration-from-objectql.mdx,plugins/packages.mdx^3.1.0pin that can never resolve to the current 16.1.0 lineAdversarial verification — all 18 docs
36/36 agents completed, 0 errors. Every doc carries a verifier pass that re-checked each applied fix against live code and repaired over-corrections. That phase earned its keep — the audit agents were confident and sometimes wrong:
enforceNetworkAccessinvented for the realenforceNetworkRequest(plugin-permission-enforcer.ts:212);onEnable, which would have contradicted the live app-bundle contractAppPlugin.start()dispatches (app-plugin.ts:574);cluster.lock.withLock(key, fn)attributed to service-job's cron scheduler, which actually doeslock.acquire(...)+ manual release (cron-job-adapter.ts:139-147);91 residual items the verifiers flagged as suspected-but-unproven are recorded in the run journal rather than guessed at in prose.
Verification
pnpm --filter @objectstack/docs build— exit 0, all pages compile.node scripts/check-doc-authoring.mjs— 215 files clean.node scripts/check-role-word.mjs— OK. The audit removed aroleoccurrence fromkernel/events.mdxand one fromservices-checklist.mdx(4 → 3); since that check is a ratchet, an improvement fails the gate until recorded, so the baseline is ratcheted down in its own commit.Follow-ups the audit surfaced but did not fix
packages/core/README.md:64,94still documentsctx.eventBus.on/.emitas part ofPluginContext. No such runtime object exists — a code-side stale doc, not fixable here.kernel/cluster.mdx'sleader-electedstrategy resolves only topackages/spec/src/kernel/cluster.zod.tsand its test — no runtime consumer ofclusterScope/leaderStrategy. Another declared-but-unbuilt surface wanting its own ADR-0049 enforce-or-remove pass.ClusterDriverSchemais a closed enum, so a driver registered under an arbitrary name viaregisterClusterDriver('etcd', …)can never be selected —driver: 'custom'resolves only a factory registered under the literal key'custom'.examples/app-showcase/src/system/datasources/showcase-external.datasource.tsstill carries the brokencredentialsRef: 'secret:warehouse/password'form in a commented-out demo — the likely origin of the doc error.