fix(skills): make broken-link check finish and stop flagging bot blocks

.agents/skills/check_for_broken_links/SKILL.md

@@ -12,24 +12,42 @@ This skill checks the Warp Astro Starlight documentation for broken links by sca
 From the docs repo root:
 
 ```bash
-python3 .warp/skills/check_for_broken_links/check_links.py
+python3 .agents/skills/check_for_broken_links/check_links.py
 ```
 
 ### Options
 
 - `--internal-only`: Only check internal links (fast, no HTTP requests)
 - `--external-only`: Only check external links
 - `--timeout N`: HTTP timeout in seconds (default: 10)
+- `--workers N`: Concurrent external link requests (default: 16)
+- `--strict`: Treat bot-blocked (401/403/429) responses as broken links
 - `--output FILE`: Save results to JSON file
 - `--slack-notify`: Send results to Slack (requires `SLACK_BOT_TOKEN` and `GROWTH_DOCS_SLACK_CHANNEL_ID` env vars)
 - `--slack-channel ID`: Override the default Slack channel
 
 ### Quick internal-only check:
 
 ```bash
-python3 .warp/skills/check_for_broken_links/check_links.py --internal-only
+python3 .agents/skills/check_for_broken_links/check_links.py --internal-only
 ```
 
+### How external links are checked
+
+External URLs are deduplicated and resolved concurrently before the per-file
+pass begins, so a page with hundreds of links (such as `changelog/2026.mdx`)
+doesn't stall the run. A full check of the docs takes roughly a minute.
+
+If you're on a slow or rate-limited network, lower the concurrency and raise the
+timeout:
+
+```bash
+python3 .agents/skills/check_for_broken_links/check_links.py --workers 4 --timeout 20
+```
+
+Requests use a browser-like `User-Agent`, retry once on a timeout or connection
+reset, and fall back to `GET` when `HEAD` fails.
+
 ## Output Format
 
 The script outputs a report like:
@@ -55,6 +73,30 @@ src/content/docs/getting-started/what-is-warp.md:42
   Error: HTTP 404
 ```
 
+### Bot-blocked links
+
+URLs that answer `401`, `403`, or `429` are reported in a separate
+`BOT-BLOCKED` section, grouped by URL, and are **not** counted as broken. These
+statuses mean the server refused an automated client, not that the page is gone.
+Cloudflare interstitials (`sourceforge.net`), OpenAI's bot challenge, and
+login-gated destinations (Slack invite links) all land here and are working
+links in a browser.
+
+```
+### BOT-BLOCKED (2 URLs, not counted as broken)
+
+These returned 401/403/429, which means the server refused an
+automated request. Verify in a browser before changing them.
+
+  https://vimdoc.sourceforge.net/htmldoc/motion.html
+    Error: HTTP 403 (2 occurrence(s))
+    First seen: src/content/docs/terminal/editor/vim.mdx:39
+```
+
+Don't "fix" these without confirming in a browser first — replacing a working
+link because a scraper was blocked makes the docs worse. Use `--strict` if you
+want them to fail the run.
+
 ## Fixing Broken Links
 
 After running the script, fix each broken link based on the error type:
@@ -93,9 +135,14 @@ After running the script, fix each broken link based on the error type:
    - Consider linking to an archived version if appropriate
 
 2. **Timeout/Connection Error**: Temporary issue or site blocking bots
-   - Re-run the check to confirm it's persistent
+   - The checker already retries once, but heavy concurrency can still cause a
+     flake. Re-run with `--workers 4 --timeout 20` to confirm it's persistent
    - Visit the URL manually to verify
 
+3. **HTTP 401/403/429**: Reported under `BOT-BLOCKED`, not as broken
+   - Verify in a browser before touching the link; it's usually working
+   - Only change it if the page is genuinely gone or has moved
+
 ### Adding Redirects
 
 If content moved, you can add a redirect in the appropriate `vercel.json (redirects)`:
@@ -112,7 +159,7 @@ If content moved, you can add a redirect in the appropriate `vercel.json (redire
 
 1. Create a branch: `git checkout -b fix/broken-links`
 2. Fix the broken links identified by the script
-3. Re-run the script to verify all fixes: `python3 .warp/skills/check_for_broken_links/check_links.py`
+3. Re-run the script to verify all fixes: `python3 .agents/skills/check_for_broken_links/check_links.py`
 4. Commit and create a PR
 
 ## Slack Notifications
@@ -132,7 +179,7 @@ You'll be prompted to enter the token securely. The token needs `chat:write` sco
 ### Usage
 
 ```bash
-python3 .warp/skills/check_for_broken_links/check_links.py --internal-only --slack-notify
+python3 .agents/skills/check_for_broken_links/check_links.py --internal-only --slack-notify
 ```
 
 For ambient agent runs, the `SLACK_BOT_TOKEN` secret is automatically injected as an environment variable.
@@ -142,7 +189,7 @@ For ambient agent runs, the `SLACK_BOT_TOKEN` secret is automatically injected a
 To post to a different channel:
 
 ```bash
-python3 .warp/skills/check_for_broken_links/check_links.py --slack-notify --slack-channel YOUR_CHANNEL_ID
+python3 .agents/skills/check_for_broken_links/check_links.py --slack-notify --slack-channel YOUR_CHANNEL_ID
 ```
 
 ## Dependencies