fix(skills): make broken-link check finish and stop flagging bot blocks

[hongyi-chen]

Summary

I ran the check_for_broken_links skill across the docs to audit link health. There are no broken links — 3,306 internal and 1,099 unique external URLs all resolve.

Getting that answer required fixing the checker, which had two defects that made it unusable. This PR fixes those so the next run is actually trustworthy. No content pages are changed.

The external check never finished

Every external URL was fetched serially, with a time.sleep(0.05) between files. changelog/2026.mdx alone has 696 links, mostly GitHub URLs that rate-limit, so a single file consumed the entire run. It hung at file 57 of 354 on two separate attempts, once for 15 minutes without advancing.

External URLs are now deduplicated and resolved up front across a thread pool, leaving the per-file pass as pure cache reads. A full check now completes in ~60s. Sessions are thread-local, since requests.Session isn't thread-safe.

Concurrency is tunable via --workers (default 16) for slow or rate-limited networks:

python3 .agents/skills/check_for_broken_links/check_links.py --workers 4 --timeout 20

Bot blocks were reported as broken links

The report was entirely false positives. All 26 flagged links returned 401/403/429 — servers refusing an automated client — with zero genuine 404s:

• vimdoc.sourceforge.net, zsh.sourceforge.io — Cloudflare Just a moment... interstitial
• help.openai.com, platform.openai.com — OpenAI bot challenge
• go.warp.dev/join-preview — resolves correctly to the community Slack invite; Slack serves a sign-in page to non-browsers

I confirmed each of these loads normally with browser headers. Reporting them as broken is actively harmful: it invites "fixing" working links, and it buries the real 404s.

These statuses now go to a separate BOT-BLOCKED section, grouped by URL rather than listed per occurrence, and don't fail the run. --strict restores the old behavior.

### BOT-BLOCKED (11 URLs, not counted as broken)

These returned 401/403/429, which means the server refused an
automated request. Verify in a browser before changing them.

  https://vimdoc.sourceforge.net/htmldoc/motion.html
    Error: HTTP 403 (2 occurrence(s))
    First seen: code/code-editor/code-editor-vim-keybindings.mdx:37

Other fixes

• Browser-like User-Agent — on its own this resolves platform.openai.com and tiktok.com, which previously 403'd.
• Retry once on timeouts and connection resets. These were genuinely flaky: code.claude.com/docs timed out under concurrency but returns 200 on every manual attempt.
• Fall back to GET whenever HEAD fails, not only on 405. Many servers mishandle HEAD with other 4xx/5xx codes.
• Corrected the documented script paths from .warp/skills/... to .agents/skills/..., where the script actually lives. Every command in the skill doc was wrong, so copy-pasting them failed.

Verification

• --internal-only: 3,306 links, 0 broken (unchanged from before this PR)
• Full run: 1,513 external checks, 0 broken, 26 bot-blocked across 11 URLs, exit 0
• --strict: correctly reclassifies bot blocks as broken and exits 1
• Slack message formatting verified for both the pass and fail paths

Note for a follow-up

Six sibling skills carry the same stale .warp/skills paths and are untouched here to keep this scoped: create_pr, draft_conceptual, draft_faq, draft_quickstart, draft_reference, draft_troubleshooting.

Separately, warpdotdev/gitbook still has the pre-migration copy of this skill and several links this repo already fixed (blob/master/LICENSE → LICENSE-AGPL, docs.warp.dev/warp/... prefixes). That repo is archived and read-only, so no action is possible there.

Conversation: https://app.warp.dev/conversation/dc5ca31f-c34a-4ec3-9e17-17904f23b9a0
Run: https://oz.warp.dev/runs/019fb91e-69b7-7b0f-a11c-008367c9986c

This PR was generated with Oz.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
docs	Ready	Preview	Aug 1, 2026 12:02am

[oz-for-oss]

@hongyi-chen

I'm starting a first review of this pull request.

You can view the conversation on Warp.

I completed the review and no human review was requested for this pull request.

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

[oz-for-oss]

Overview

This PR updates the broken-link checker to deduplicate and concurrently resolve external URLs, use browser-like request headers, retry transient network failures, classify 401/403/429 responses separately as bot-blocked unless --strict is used, and correct the skill documentation paths.

Concerns

No blocking correctness, security, or error-handling concerns found in the annotated diff.

Verdict

Found: 0 critical, 0 important, 0 suggestions

Approve

Comment /oz-review on this pull request to retrigger a review (up to 3 times on the same pull request).

Powered by Oz

[advait-m]