Resolve the gutendex 403: WAF-aware live-smoke, baseUrl for self-hosting, docs#19
Merged
Merged
Conversation
The descriptive bot UA is still blocked from datacenter IPs (live-smoke runs 1-2 both hit 403); try a browser-like default. config.userAgent still overrides. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018hoN6wKWxcs8WbUNCM9rtt
gutendex.com's Cloudflare blocks datacenter IPs at the fingerprint level — verified across live-smoke runs 1-3 that neither the descriptive bot UA nor a browser UA gets through, so the failure carries no signal about API drift (the suite's purpose) and failed every weekly run. - testkit: liveSmoke gains opt-in tolerateUpstreamBlock — skip with a warning on HTTP 403 only; 404/5xx/schema/empty still fail - gutendex: opt in; revert the (proven ineffective) browser UA back to the honest descriptive UA, keep explicit Accept: application/json Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018hoN6wKWxcs8WbUNCM9rtt
Upstream docs frame gutendex.com as a test instance ("You should run
your own server, but you can test queries at gutendex.com") with no
rate-limit contract or official mirrors, and its Cloudflare front
blocks datacenter IPs — the doc-sanctioned path for production/CI
traffic is self-hosting. The provider hardcoded gutendex.com; baseUrl
opens the escape hatch.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018hoN6wKWxcs8WbUNCM9rtt
…duction Root README footnote + provider README hosting note: the upstream docs frame gutendex.com as a test instance and its Cloudflare front blocks datacenter IPs; desktop/local use works out of the box, production and server-side traffic should self-host and pass baseUrl. Documents the graceful-degradation behavior when blocked. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018hoN6wKWxcs8WbUNCM9rtt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes out the long-standing gutendex 403 that failed every live-smoke run (including the pre-existing failure on main's first scheduled run).
Diagnosis
gutendex.com's Cloudflare front blocks datacenter IPs (GitHub Actions runners) at the fingerprint level — verified empirically across live-smoke runs 1–3 that neither the existing descriptive bot UA nor a browser-like UA gets through. The upstream docs settle the question: gutendex.com is the maintainer's test instance ("You should run your own server, but you can test queries at gutendex.com") with no rate-limit contract, API keys, or official mirrors. The block is intended behavior, not a fault to bypass.Changes
@refkit/provider-testkit—liveSmokegains an opt-intolerateUpstreamBlock: skip with a warning on HTTP 403 only; 404s, 5xx, schema changes, and empty results still fail. The weekly drift check stops crying wolf on a signal that carries no drift information.@refkit/provider-gutendexbaseUrlconfig — the doc-sanctioned path for production/server-side traffic is self-hosting, and the provider previously hardcodedgutendex.com.gutendex({ baseUrl: 'https://your-instance' })opens the escape hatch (with URL-join handling and tests).tolerateUpstreamBlock; requests send explicitAccept: application/json; the browser-UA experiment is reverted to the honest descriptive UA with a comment recording that both UAs were verified blocked.baseUrl, and a blocked source degrades gracefully (failedinmeta.providers, other sources still return).Verification
provider-testkit+provider-gutendexpatches)🤖 Generated with Claude Code
https://claude.ai/code/session_018hoN6wKWxcs8WbUNCM9rtt
Generated by Claude Code