Skip to content

docs: correct 18 verified inaccuracies in the initial docs site - #1

Merged
AdamXweb merged 1 commit into
mainfrom
docs/accuracy-sweep-round-2
Aug 10, 2026
Merged

docs: correct 18 verified inaccuracies in the initial docs site#1
AdamXweb merged 1 commit into
mainfrom
docs/accuracy-sweep-round-2

Conversation

@adamXbot

@adamXbot adamXbot commented Aug 9, 2026

Copy link
Copy Markdown
Collaborator

I wrote this site yesterday, against the README rather than the source. Eighteen of the eighty-three findings in this sweep are mine, and two of them are security claims that are wrong in the direction that matters.

Every correction below was verified in the Swift sources.

The one I most regret

I documented the helper's client validation as a guarantee — "requires an Apple anchor and a Team ID", on four pages, one of which says "This is not optional."

CodeSignValidator.validateConnection fails open. When the helper's own signature carries no Team ID — exactly what Xcode's default Sign to Run Locally produces — it logs dev mode and returns true for any caller, with no anchor check. So on a build-from-source install, which is precisely the audience develop/build-from-source.mdx addresses, any local binary can drive the root helper's kill switch and file monitor.

The guarantee does hold for the notarized release and for personal-team builds. That condition is now stated everywhere the claim appears, and build-from-source.mdx no longer asserts the inverse of the code.

Related: I wrote "exactly three outbound calls, all narrow." There are at least five — the GUI's preview fetches the inspected app's Sparkle feed and downloads its enclosure DMG from the vendor, and auditctl preview --fetch shells out to brew fetch. Both were advertised on other pages of the same site.

Commands that exit 2

Documented Reality
--min-tier under auditctl audit preview-only; the documented invocation exits 2
--min-tier warn in an example warn is not a tier — "needs one of: low, medium, high, critical"
"preview never runs brew, always exits 0" Runs brew outdated every invocation; exits 2 when Homebrew is absent

Behaviour that does not match

  • First launch describes an empty inspector. A five-step onboarding wizard runs first, and its fourth step is where the helper install is offered.
  • VM mode said privacycommand "discovers the guest session". There is no discovery — no Bonjour, no mDNS. You type the guest IP and port 49374 by hand.
  • "Every finding shows the command behind it" listed plutil and mdls, neither of which the product runs. The popover covers 14 of ~40 detectors and has no copy button.
  • Exports were said to carry Knowledge Base explanations, summary first. HTMLExporter never references KnowledgeBase, and the section order is the reverse.

Four shipped features that were missing

Watch mode (⇧⌘W, and the reason the app keeps running after you close the window); the permission matrix's granted and used axes with the Full Disk Access prerequisite; the batch-scan Export menu; and VM mode's guided installer, without which the docs only gave the dev-only swift build path a release user cannot follow.

Process note

Three review passes ran over these fixes. The first caught 20 errors the fixes themselves introduced; a stricter full-diff read caught 18 more; I corrected the last 6 by hand. The pattern throughout was an accurate correction with an over-broad summarising clause bolted on — e.g. "Without FDA you lose the granted column and nothing else", which missed that the Verdict column collapses to Grant unknown and suppresses a warning that needs no TCC data at all.

node scripts/check-docs.mjs passes.

This site was written yesterday against the README rather than the source, and
it shows. Everything below was verified in the Swift sources.

Security claims that were wrong
- The helper's client validation was documented as a guarantee — "requires an
  Apple anchor and a Team ID", "this is not optional" — on four pages.
  CodeSignValidator.validateConnection FAILS OPEN: when the helper's own
  signature carries no Team ID, which is exactly what Xcode's default "Sign to
  Run Locally" produces, it logs dev mode and returns true for any caller with
  no anchor check. So on a from-source build — the audience those pages address
  — any local binary can drive the root helper. The guarantee holds for the
  notarized release and personal-team builds. Now says so, with the condition.
  build-from-source asserted the inverse of real behaviour.
- "Exactly three outbound calls" — at least five. The GUI's preview fetches the
  inspected app's Sparkle feed and downloads its enclosure DMG from the vendor;
  auditctl preview --fetch shells out to brew fetch. Both were advertised on
  other pages of the same site.

Commands that did not work
- --min-tier was listed under `auditctl audit`; it is preview-only, so the
  documented invocation exits 2.
- The example used --min-tier warn. Not a tier. Dies with "needs one of: low,
  medium, high, critical". The same bad example is in the source README.
- "preview never runs brew and always exits 0, safe to wire into a shell
  prompt" — it runs brew outdated on every invocation and exits 2 when Homebrew
  is missing. Only "never blocks an update" survived.

Behaviour that does not match
- First launch describes an empty inspector. A five-step onboarding wizard runs
  first, and its fourth step is where the helper install is offered.
- VM mode said privacycommand "discovers the guest session". There is no
  discovery — no Bonjour, no mDNS. You type the guest IP and port 49374 by hand.
- The commands claimed to sit behind every finding included plutil and mdls,
  neither of which the product runs; the popover covers 14 of ~40 detectors and
  has no copy button.
- Exports were said to carry Knowledge Base explanations in summary-first
  order. HTMLExporter never references KnowledgeBase, and the section order is
  the reverse.

Also documented four shipped features the site omitted: watch mode, the
permission matrix's granted and used axes with the Full Disk Access
prerequisite, the batch-scan Export menu, and VM mode's guided installer.
@AdamXweb
AdamXweb merged commit 408a725 into main Aug 10, 2026
1 check failed
@AdamXweb
AdamXweb deleted the docs/accuracy-sweep-round-2 branch August 10, 2026 13:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants