Spam: evaluate the ~785 signups imported since the last spam-detection pass

[themightychris]

The 2026-09-17 cutover data refresh merged 787 new people records from laddr (a week of signups). `prune-spam` consults the pre-computed evaluations on the data repo's `spam-detection` branch, and those records have no evaluations yet, so only 2 were pruned. Most of the rest are the usual spam signups and are now on `published`.

To do: on the `spam-detection` branch, run `evaluate-heuristic` then `evaluate-llm` for the delta (see `docs/operations/spam-detection.md`), then re-run `prune-spam --threshold=0.8` against `published` and push. Longer term, fold the evaluators into the refresh pipeline so a re-sync never lands unevaluated records.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LFyA5poHwrhAktrnsKrUiQ

[themightychris]

Done 2026-09-18 — and the backlog was much larger than the ~785 in the title.

What was actually unevaluated: the pipeline had not run since 2026-05-20. 5,231 accounts created May–September had no verdict at all, and a second, older gap turned out to matter more: 8,642 people on published carried a heuristic-v1 spam verdict that had never been LLM-confirmed. The heuristic record has a score but no confidence field, so prune-spam's ≥0.8 rule could never act on it — those profiles have been live since the first prune.

Run (on the data repo's spam-detection branch, merged with tonight's published):

• fetch-slack → 4,897 presence / 297 channels; evaluate-slack-messages → no new messages
• evaluate-heuristic over all 23,072 → 8,661 spam / 2,958 legit / 11,453 uncertain
• evaluate-llm (uncertain) → 2,512 new profiles: 1,819 spam / 689 legit
• evaluate-llm --filter spam → 8,582 heuristic-spam profiles: 8,533 spam / 45 legit / 2 uncertain (heuristic precision 99.5%)
• Two prunes: 1,819 then 8,592 (2 protected by project membership; 1,538 orphaned tag-assignments cascaded)

Result: published went 23,072 → 12,661 people; prod and sandbox hot-reloaded. Evaluations pushed to spam-detection (33,738 Haiku evals cached). Cost ≈ $25 in Haiku.

Gotchas for the runbook: the branch tooling (gitsheets 1.3.2 / @iarna/toml) rejects TOML 1.0 multi-line strings that end in """", which the 2.x importer emits for bios ending in a quote (59 records — patched locally in the scratch clone's parser, no data changed); gitsheets 1.x writes commits straight to git, so it must run in a real clone, not a linked worktree, and the working tree lags HEAD. Follow-ups: (1) make the refresh runbook end with evaluate-llm --filter spam so heuristic-only spam never sits unconfirmed again — or give heuristic records a confidence the prune can use; (2) the signup vector is closed now that the rewrite only creates accounts via GitHub, as long as laddr's registration at the legacy alias stays unlinked.