CurlImpersonateHttpClient does not raise ProxyError when a proxy refuses the CONNECT tunnel, so the
session is not rotated.
The bundled libcurl went 8.15.0 -> 8.21.0 and now reports CURLE_COULDNT_CONNECT with
CONNECT tunnel failed, response 400 instead of CURLE_RECV_ERROR, which curl_cffi no longer maps to
ProxyError. _is_proxy_error matches neither the type nor any needle in ROTATE_PROXY_ERRORS. The
version is capped in pyproject.toml (#2107).
Fix: recognize the CONNECT tunnel failed message in CurlImpersonateHttpClient._is_proxy_error.
Failing tests: test_crawl_with_proxy_disabled[curl], test_send_request_with_proxy_disabled[curl].
Acceptance
The curl-cffi<0.16.0 cap removed and the tests above passing with the latest version.
✍️ Drafted by Claude Code
CurlImpersonateHttpClientdoes not raiseProxyErrorwhen a proxy refuses the CONNECT tunnel, so thesession is not rotated.
The bundled libcurl went 8.15.0 -> 8.21.0 and now reports
CURLE_COULDNT_CONNECTwithCONNECT tunnel failed, response 400instead ofCURLE_RECV_ERROR, whichcurl_cffino longer maps toProxyError._is_proxy_errormatches neither the type nor any needle inROTATE_PROXY_ERRORS. Theversion is capped in
pyproject.toml(#2107).Fix: recognize the
CONNECT tunnel failedmessage inCurlImpersonateHttpClient._is_proxy_error.Failing tests:
test_crawl_with_proxy_disabled[curl],test_send_request_with_proxy_disabled[curl].Acceptance
The
curl-cffi<0.16.0cap removed and the tests above passing with the latest version.✍️ Drafted by Claude Code