Skip to content

feat: add doc-accuracy reviewer to catch harmful documentation changes - #714

Open
Iheanacho-ai wants to merge 1 commit into
siderolabs:mainfrom
Iheanacho-ai:check-doc-code
Open

feat: add doc-accuracy reviewer to catch harmful documentation changes#714
Iheanacho-ai wants to merge 1 commit into
siderolabs:mainfrom
Iheanacho-ai:check-doc-code

Conversation

@Iheanacho-ai

Copy link
Copy Markdown
Member

What this adds

A new local make target, check-doc-accuracy, that reviews documentation for changes that would harm a reader who follows them — not just typos, but commands that silently lose data, destroy things, or weaken security — before they ship.

Why

We recently had a near-miss where an edit dropped a volume-mount flag (-v $HOME/etcd:/_out/etcd) from a docker run command in the self-hosted Omni guide. The command still ran with no error, but etcd data was no longer persisted — so recreating the container would have wiped a self-hosted customer's entire Omni state. Nothing "broke" on copy-paste; it worked, and silently destroyed data.

A flag validator scoped to talosctl/omnictl would never have caught it — it was a plain docker command, and the command was valid. The only thing that catches that class of bug is a reviewer that reads the snippet and reasons about its blast radius. This tool does exactly that.

How it works

tools/doc-accuracy is a small Go program that drives the claude CLI headless as a read-only documentation reviewer (Edit/Write are disallowed — it reports, it never changes docs). It reviews the .mdx files you changed and, when a snippet or claim concerns Talos, Omni, extensions, or the discovery service, cross-checks it against the upstream source.

It prioritizes harm over mere breakage:

  • silent data loss (a stateful service run without its persistence mount)
  • destructive/irreversible commands (rm -rf, docker volume rm, kubectl delete, git push --force, disk wipes…)
  • removed safeguards (a deleted flag/mount/--dry-run/backup/confirmation)
  • security downgrades (disabling TLS/auth, chmod 777, binding to 0.0.0.0, curl … | sudo sh, leaked secrets)
  • plus ordinary wrong-flag/wrong-value/false-prose mistakes

…applied to every command in a snippet, not only the Sidero CLIs.

Usage

make check-doc-accuracy                                   # files you changed vs HEAD
make check-doc-accuracy DOC=public/omni/....mdx           # one specific file
make check-doc-accuracy DOC_ACCURACY_BASE=origin/main     # PR-style, vs a branch
make check-doc-accuracy-all                               # the whole public/ tree (slow)

@github-project-automation github-project-automation Bot moved this to To Do in Planning Aug 20, 2026
@talos-bot talos-bot moved this from To Do to In Review in Planning Aug 20, 2026
Signed-off-by: Amarachi Iheanacho <amarachi.iheanacho@siderolabs.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants