Skip to content

docs: finish implementation-accuracy audit — durable reference grouping + hand-written doc sync (#1880)#3243

Merged
os-zhuang merged 11 commits into
mainfrom
claude/implementation-accuracy-audit-eavdgx
Jul 19, 2026
Merged

docs: finish implementation-accuracy audit — durable reference grouping + hand-written doc sync (#1880)#3243
os-zhuang merged 11 commits into
mainfrom
claude/implementation-accuracy-audit-eavdgx

Conversation

@os-zhuang

@os-zhuang os-zhuang commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Continues the docs implementation-accuracy effort (#1880; follows #1866 / #1904). Wave 1 audit complete — 164/164 hand-written docs.

Summary

  • Wave 1 — hand-written doc audit: 164/164 docs via a multi-agent pipeline (per doc: locate the implementation in packages/ → surgical fix → adversarial verify), all in a dedicated worktree.
    • 335 evidence-backed fixes across 116 docs (the other 48 were already accurate → no change).
    • 14 of those were verifier repairs — the adversarial pass catching and correcting a wrong audit edit before it landed (e.g. a fabricated permissionSet: stack key → the real permissions:; a fabricated projection claim removed).
    • 259 report-only residuals logged — ambiguous / aspirational / frontmatter-locked items the agents correctly did not "fix".
  • Wave 2.1 — durable reference grouping: moved docs(nav): group Protocol Reference modules and clarify top-level nav #1862's per-category sidebar grouping into build-docs.ts (fumadocs ---Section--- separators, drift-safe). gen:docs --check green.
  • Wave 2.2 — generated-reference quality: generator formatting verified solid (no dangling links, valid tables); filled ui/view describe strings. The ~1,180-property describe backlog (mostly self-documenting z.literal discriminants) is tracked continuation.

Representative fixes: removed the retired feed client namespace; password fields are masked-on-read, not hashed; the certified measure flag is ADR-0021-not-yet-implemented; removed softDelete / versioning / fileAttachmentConfig / vectorConfig (all removed in 16.x, #2377); referenceFilterslookupFilters; fabricated definePlugin() / defineObject()ObjectSchema.create; error-handling fabricated SCREAMING_SNAKE codes + WebSocket/Business-Logic sections → real StandardErrorCode values.

Method / quality notes

  • Every fix cites file:line evidence; unverifiable claims were left unchanged per rule (e.g. I reverted one percent 0-1↔0-100 change I couldn't confirm in this repo — the scale lives in the objectui renderer).
  • A few batches had their adversarial-verify phase cut short by usage limits; those docs were audit-only and hand-checked against the spec before commit. The two protocol docs whose first audit was interrupted mid-edit were reverted and cleanly re-audited.
  • Vercel docs build (gen:schema && gen:docs && next build) green on every push.

Flagged follow-ups from the issue — both resolved

CODE / frontmatter follow-ups surfaced by the audit (out of scope for this docs-only PR)

Real implementation gaps the audit found but couldn't fix here:

  • explain: os explain object documents ownership as "own" | "extend" — real values are user | org | none #3244os explain object documents ownership as "own" | "extend" (real: user | org | none). (filed)
  • kernel/contracts/cache-service.mdx frontmatter claims "namespaces, and bulk operations" the contract doesn't have (preserve-frontmatter rule blocked the body-only fix).
  • OS_DATABASE_DRIVER=memory has no branch in serve.ts dispatch (falls through to SQLite in-memory).
  • @objectstack/driver-sql declares peerDep sqlite3 but loads better-sqlite3 at runtime.
  • formula js dialect is registered only as a stub (hasDialect('js') is false).
  • protocol/knowledge.mdx has broken ai-capabilities.mdx cross-references (target removed in the reorg).
  • glossary "TCK (Technology Compatibility Kit)" is grep-empty (aspirational term).

Wave 2.2 describe-string backlog (continuation of #1880)

~1,180 generated-reference properties still lack descriptions — most are self-documenting z.literal discriminants; the genuine gaps concentrate in internal protocol schemas (e.g. data-engine). Left for a follow-up.


Left as a draft for maintainer review — not auto-merging.

🤖 Generated with Claude Code

https://claude.ai/code/session_012urEihGTAsQP2xizAqfCwt

claude added 3 commits July 18, 2026 15:31
gen:docs rewrote each references/{cat}/meta.json as a flat alphabetical
list on every run, silently discarding the per-category grouping that was
added by hand in #1862. Move the grouping into build-docs.ts so it
survives regeneration: a SECTION_GROUPS map drives fumadocs "---Section---"
separators for the 9 large categories (ai, api, automation, cloud, data,
integration, kernel, system, ui); smaller categories keep the flat list.

The mapping degrades gracefully — pages present on disk but unmapped still
appear (under a trailing "More" separator) and names that produce no page
are ignored — so adding or removing a schema never drops it from the
sidebar. `gen:docs --check` stays green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012urEihGTAsQP2xizAqfCwt
…iew config (#1880)

Fill empty zod .describe() strings on the most author-facing view schema
so the generated ui/view reference prose is useful:
- CalendarConfig / GanttConfig field references (startDateField,
  endDateField, titleField, colorField, progressField, dependenciesField)
- the type-specific view config objects (kanban / calendar / gantt / tree)

Regenerated content/docs/references/ui/view.mdx from the updated schema.
Part of the Wave 2.2 describe-string effort; the broader backlog
(~1,180 empty descriptions, concentrated in internal protocol schemas
such as data-engine) remains as tracked continuation of #1880.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012urEihGTAsQP2xizAqfCwt
…Wave 1, batch 1)

First batch of evidence-backed accuracy fixes from the multi-agent doc
audit (one agent per doc → locate implementation in packages/ →
adversarial verify). The api section (11 docs) is fully audited and
adversarially verified — 12 fixes, 0 verifier repairs; the ai/automation
docs here are audit-verified.

Representative fixes: removed the retired `feed` client namespace,
corrected the connect discovery response shape (features → capabilities),
and fixed the aiService.chatWithTools() call signature.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012urEihGTAsQP2xizAqfCwt
@vercel

vercel Bot commented Jul 18, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
spec Ready Ready Preview, Comment Jul 19, 2026 12:53pm

Request Review

…batch 2)

More evidence-backed accuracy fixes from the audit workflow. The adversarial
verify phase was cut short by a session usage limit, so each fix here was
additionally hand-checked against the spec before committing:

- data-modeling/field-type-decision-tree: `password` is masked-on-read, not
  hashed — prefer `secret` for real credentials (#2036).
- data-modeling/analytics: the `certified` measure flag is an ADR-0021 design
  goal, not yet implemented (DatasetMeasureSchema has no such field) — removed
  the example usage and corrected the guidance.
- concepts/metadata-driven: config-key example `referenceFilters` →
  `lookupFilters` (referenceFilters removed in 16.x, #2377 / ADR-0049).
- concepts/metadata-lifecycle: per-org overlay table listed removed
  `role`/`profile` types → `permission`/`position`.
- concepts/index: dropped the auto-GraphQL claim (REST is auto-generated).
- automation/webhooks: sys_webhook filter field `events[]` → `triggers`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012urEihGTAsQP2xizAqfCwt
…lementation (#1880 Wave 1, batch 3)

30 docs audited AND adversarially verified this window; 20 carry
evidence-backed fixes. Highlights:

- field-types: `password` is masked-on-read (ADR-0100), not hashed — real
  credential hashing is owned by better-auth (sys_account.password); removed
  fabricated `fileAttachmentConfig` and `vectorConfig` (both removed in the
  16.x line, #2377).
- objects: removed non-existent `tags` / `recordName` props and the
  `softDelete` / `versioning` config blocks (removed in 16.0, #2377 / ADR-0049,
  now tombstoned); corrected `feeds` / `activities` capability defaults (both
  default `true`).
- validation-rules, getting-started/cli, deployment/*, and others: assorted
  API name, flag, path and example corrections.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012urEihGTAsQP2xizAqfCwt
…ave 1, batch 4)

30 docs audited AND adversarially verified; 18 carry evidence-backed fixes
(getting-started remainder + kernel contracts / runtime-services). Highlights:

- kernel/contracts/data-engine: `@objectstack/spec` → `@objectstack/spec/data`
  subpath import; trimmed the error-code table to codes the engine actually
  emits (with HTTP mappings); removed the unsupported `top` → `limit` OData alias.
- kernel/events, cluster, contracts/*, runtime-services/*: assorted API name,
  type, and import-path corrections.
- getting-started: AI-primitives scope note (NLQ / RAG-orchestration /
  predictive pipelines were removed in v1); quick-reference and
  your-first-project example fixes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012urEihGTAsQP2xizAqfCwt
…1880 Wave 1, batch 5)

Commits the 23 permissions/plugins/protocol docs whose audit completed
cleanly this window. Highlights:
- plugins/*: removed the fabricated `definePlugin()` helper from examples —
  a plugin is a plain default-export object; no such helper exists.
- permissions/explain: corrected the explain `via` value to "direct grant"
  (adversarial-verify repair); permission-metadata, administrator-guide,
  authentication, field-level-security, sharing-rules: scope/flag/API fixes.
- protocol/diagram, protocol/kernel/config-resolution: path/reference fixes.

Some docs' verify pass was cut short by a weekly usage limit, so a subset is
audit-only (hand-scanned: no define*/scale/count regressions; MDX build-safe).
Two docs whose audit itself was interrupted (protocol/kernel/error-handling
and http-protocol) are reverted and deferred for a clean re-audit after the
limit resets.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012urEihGTAsQP2xizAqfCwt
…ave 1, batch 6)

30 docs audited AND adversarially verified (protocol, releases, ui); 28 carry
fixes — 103 fixes total, 8 of them verifier repairs (the adversarial pass
caught and corrected wrong audit edits, e.g. a bad `permissionSet:` stack key
→ `permissions:`, and a fabricated projection "id/audit columns" claim).

Highlights:
- protocol/kernel/*: removed the fabricated `definePlugin()` / `defineObject()`
  helpers (use ObjectSchema.create) and corrected `defineView` imports to
  `@objectstack/spec/ui`; aspirational manifest/lifecycle blocks left in place
  where the page already discloses them as design-intent.
- protocol/knowledge: driver-turso / knowledge-turso are cloud-only (not
  open-core) → replaced with open-core-appropriate driver-mongodb / llamaindex.
- releases/v13,v15 + objectql/* + ui/*: import-path, key-name, and example fixes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012urEihGTAsQP2xizAqfCwt
…-protocol (#1880 Wave 1, batch 7 — final)

Final batch — completes the 164-doc hand-written implementation-accuracy
audit. 13 docs audited AND adversarially verified. Includes a clean
re-audit of the two protocol docs whose first pass was interrupted by a
usage limit (their partial diffs were reverted earlier):

- protocol/kernel/error-handling: replaced fabricated SCREAMING_SNAKE error
  codes with the real lowercase StandardErrorCode values (unauthenticated,
  permission_denied, resource_not_found, delete_restricted,
  rate_limit_exceeded, service_unavailable, …) and removed the fabricated
  "WebSocket-Specific Errors" / "Business Logic Errors" sections — the error
  contract is HTTP/REST, not multi-channel.
- protocol/kernel/http-protocol: corrected discovery routes, response-shape
  names, and fetch examples to the real /api/v1 surface.
- ui/{forms,views,dashboards,pages,setup-app,...}: example, flag and
  field-name corrections.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012urEihGTAsQP2xizAqfCwt
Bring the docs-audit branch up to date with main (29 commits) before merge.

Conflict resolutions:
- content/docs/automation/flows.mdx: took main's newer run-history paragraph
  (sys_automation_run durable history / bounded step log) which superseded this
  branch's audit wording of the same section.
- content/docs/references/automation/meta.json: regenerated; added main's new
  `time-relative-trigger` schema to the automation "Flow & Execution" sidebar
  grouping in build-docs.ts. `gen:docs --check` green (254 files in sync).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012urEihGTAsQP2xizAqfCwt
@github-actions github-actions Bot added documentation Improvements or additions to documentation protocol:ui tooling labels Jul 19, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): @objectstack/spec.

103 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/ai/agents.mdx (via @objectstack/spec)
  • content/docs/ai/skills-reference.mdx (via @objectstack/spec)
  • content/docs/ai/skills.mdx (via @objectstack/spec)
  • content/docs/api/client-sdk.mdx (via @objectstack/spec)
  • content/docs/api/environment-routing.mdx (via @objectstack/spec)
  • content/docs/api/error-catalog.mdx (via @objectstack/spec)
  • content/docs/api/error-handling-client.mdx (via @objectstack/spec)
  • content/docs/api/error-handling-server.mdx (via @objectstack/spec)
  • content/docs/api/index.mdx (via @objectstack/spec)
  • content/docs/automation/approvals.mdx (via packages/spec)
  • content/docs/automation/flows.mdx (via @objectstack/spec)
  • content/docs/automation/hook-bodies.mdx (via packages/spec)
  • content/docs/automation/hooks.mdx (via @objectstack/spec)
  • content/docs/automation/index.mdx (via @objectstack/spec)
  • content/docs/automation/webhooks.mdx (via @objectstack/spec)
  • content/docs/automation/workflows.mdx (via @objectstack/spec)
  • content/docs/concepts/architecture.mdx (via @objectstack/spec)
  • content/docs/concepts/design-principles.mdx (via packages/spec)
  • content/docs/concepts/index.mdx (via @objectstack/spec)
  • content/docs/concepts/metadata-driven.mdx (via @objectstack/spec)
  • content/docs/concepts/metadata-lifecycle.mdx (via packages/spec)
  • content/docs/concepts/north-star.mdx (via packages/spec)
  • content/docs/data-modeling/analytics.mdx (via @objectstack/spec)
  • content/docs/data-modeling/drivers.mdx (via @objectstack/spec)
  • content/docs/data-modeling/external-datasources.mdx (via @objectstack/spec)
  • content/docs/data-modeling/field-types.mdx (via @objectstack/spec)
  • content/docs/data-modeling/fields.mdx (via @objectstack/spec)
  • content/docs/data-modeling/formulas.mdx (via @objectstack/spec)
  • content/docs/data-modeling/index.mdx (via @objectstack/spec)
  • content/docs/data-modeling/objects.mdx (via @objectstack/spec)
  • content/docs/data-modeling/queries.mdx (via @objectstack/spec)
  • content/docs/data-modeling/schema-design.mdx (via @objectstack/spec)
  • content/docs/data-modeling/seed-data.mdx (via @objectstack/spec)
  • content/docs/data-modeling/validation-rules.mdx (via @objectstack/spec)
  • content/docs/data-modeling/validation.mdx (via @objectstack/spec)
  • content/docs/deployment/troubleshooting.mdx (via @objectstack/spec)
  • content/docs/getting-started/build-with-claude-code.mdx (via @objectstack/spec)
  • content/docs/getting-started/cli.mdx (via @objectstack/spec)
  • content/docs/getting-started/common-patterns.mdx (via @objectstack/spec)
  • content/docs/getting-started/examples.mdx (via @objectstack/spec)
  • content/docs/getting-started/quick-reference.mdx (via @objectstack/spec)
  • content/docs/getting-started/quick-start.mdx (via @objectstack/spec)
  • content/docs/getting-started/validating-metadata.mdx (via @objectstack/spec)
  • content/docs/getting-started/your-first-project.mdx (via @objectstack/spec)
  • content/docs/kernel/cluster.mdx (via @objectstack/spec)
  • content/docs/kernel/contracts/auth-service.mdx (via packages/spec)
  • content/docs/kernel/contracts/cache-service.mdx (via packages/spec)
  • content/docs/kernel/contracts/data-engine.mdx (via @objectstack/spec)
  • content/docs/kernel/contracts/index.mdx (via @objectstack/spec)
  • content/docs/kernel/contracts/metadata-service.mdx (via packages/spec)
  • content/docs/kernel/contracts/storage-service.mdx (via packages/spec)
  • content/docs/kernel/index.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/email-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/index.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/queue-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/sharing-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/sms-service.mdx (via packages/spec)
  • content/docs/kernel/runtime-services/storage-service.mdx (via packages/spec)
  • content/docs/kernel/services-checklist.mdx (via @objectstack/spec)
  • content/docs/permissions/authorization.mdx (via @objectstack/spec)
  • content/docs/permissions/permission-sets.mdx (via @objectstack/spec)
  • content/docs/permissions/permissions-matrix.mdx (via @objectstack/spec)
  • content/docs/permissions/positions.mdx (via @objectstack/spec)
  • content/docs/permissions/rls.mdx (via @objectstack/spec)
  • content/docs/permissions/sharing-rules.mdx (via @objectstack/spec)
  • content/docs/plugins/adding-a-metadata-type.mdx (via @objectstack/spec)
  • content/docs/plugins/development.mdx (via @objectstack/spec)
  • content/docs/plugins/index.mdx (via @objectstack/spec)
  • content/docs/plugins/packages.mdx (via @objectstack/spec)
  • content/docs/protocol/backward-compatibility.mdx (via @objectstack/spec)
  • content/docs/protocol/diagram.mdx (via packages/spec)
  • content/docs/protocol/kernel/config-resolution.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/i18n-standard.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/index.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/lifecycle.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/plugin-spec.mdx (via @objectstack/spec)
  • content/docs/protocol/kernel/runtime-capabilities.mdx (via @objectstack/spec)
  • content/docs/protocol/knowledge.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/index.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/query-syntax.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/schema.mdx (via @objectstack/spec)
  • content/docs/protocol/objectql/security.mdx (via packages/spec)
  • content/docs/protocol/objectql/state-machine.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/actions.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/concept.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/index.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/layout-dsl.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/record-alert.mdx (via @objectstack/spec)
  • content/docs/protocol/objectui/widget-contract.mdx (via @objectstack/spec)
  • content/docs/releases/implementation-status.mdx (via @objectstack/spec)
  • content/docs/releases/index.mdx (via @objectstack/spec)
  • content/docs/releases/v12.mdx (via @objectstack/spec)
  • content/docs/releases/v13.mdx (via @objectstack/spec)
  • content/docs/releases/v9.mdx (via @objectstack/spec)
  • content/docs/ui/actions.mdx (via @objectstack/spec)
  • content/docs/ui/create-vs-edit-form.mdx (via @objectstack/spec)
  • content/docs/ui/dashboards.mdx (via @objectstack/spec)
  • content/docs/ui/forms.mdx (via @objectstack/spec)
  • content/docs/ui/index.mdx (via @objectstack/spec)
  • content/docs/ui/public-data-collection.mdx (via @objectstack/spec)
  • content/docs/ui/setup-app.mdx (via @objectstack/spec)
  • content/docs/ui/translations.mdx (via @objectstack/spec)
  • content/docs/ui/views.mdx (via @objectstack/spec)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@os-zhuang
os-zhuang marked this pull request as ready for review July 19, 2026 12:46
The audit reduced discouraged "role" wording in 10 docs; commit the improved
baseline so `check:role-word` locks in the improvement (per the check's own
ratchet-down instruction).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012urEihGTAsQP2xizAqfCwt
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation protocol:ui size/xl tooling

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants