Point Zeek links in docs at new locations and exclude them from link checker - #7233
Open
philrz wants to merge 1 commit into
Open
Point Zeek links in docs at new locations and exclude them from link checker#7233philrz wants to merge 1 commit into
philrz wants to merge 1 commit into
Conversation
nwt
approved these changes
Aug 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
For the past few days our automated link checkers have been consistently getting HTTP 429/403 errors in the spots where we still link to the Zeek docs. I dug into this and updated to newer links to avoid redirects, but this alone did not address the problem. I've concluded that for now we should just exclude them from the link checker config, and I've opened zeek/zeek#5819 with the Zeek team should they be interested in collaborating on a fix that would allow link checkers to still work.
Details
After updating to the newer links didn't improve things, I also learned about the Read the Docs guidance that suggests adding a self-identifying User Agent might increase our chances of success. It didn't help, but I've kept that config intact here since I figure it doesn't hurt to be a better citizen.
The "Cloudflare challenge" called out in the comment next to the link exclusion boils down to only permitting clients that are capable of executing JavaScript. In my testing this was verified via GitHub Actions runners, a residential IP, and an unrelated third-party fetcher, all of which get HTTP 429/403 while a browser loads the same URLs fine. There's no request rate low enough to avoid it, as even a single cold request from a runner is challenged. The same Read the Docs guidance links off to the topic of being able to apply to be a "Cloudflare verified bot", but as this problem is not affecting us widely and I'm not sure of the likelihood of being whitelisted if we follow that process, for now I'm game to just add the link checker exclusions, see what comes of the issue with the Zeek team, and check links manually from time to time.