Skip to content

fix: add TCP keepalive to default httpx transport to prevent NAT hangs #3368

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Oxygen56 wants to merge 1 commit into
base: main
Choose a base branch
from
+30 −0
Open

fix: add TCP keepalive to default httpx transport to prevent NAT hangs #3368

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions src/openai/_base_client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
from typing_extensions import Unpack, Literal, override, get_origin

import anyio
import socket
import httpx
import distro
import pydantic
Expand Down Expand Up @@ -831,11 +832,39 @@ def _idempotency_key(self) -> str:
return f"stainless-python-retry-{uuid.uuid4()}"


def _build_keepalive_socket_options() -> list[tuple[int, int, int | bool]]:
"""Build socket options for TCP keepalive.

Enables SO_KEEPALIVE and sets platform-appropriate TCP keepalive
parameters to prevent NAT gateways from silently dropping idle
connections during long-running non-streaming requests.
"""
opts: list[tuple[int, int, int | bool]] = [(socket.SOL_SOCKET, socket.SO_KEEPALIVE, 1)]

if hasattr(socket, "TCP_KEEPIDLE"):
# Linux: seconds before sending the first keepalive probe
opts.append((socket.IPPROTO_TCP, socket.TCP_KEEPIDLE, 60))
elif hasattr(socket, "TCP_KEEPALIVE"):
# macOS: seconds before sending the first keepalive probe
opts.append((socket.IPPROTO_TCP, socket.TCP_KEEPALIVE, 60))

if hasattr(socket, "TCP_KEEPINTVL"):
# Seconds between subsequent keepalive probes
opts.append((socket.IPPROTO_TCP, socket.TCP_KEEPINTVL, 60))

if hasattr(socket, "TCP_KEEPCNT"):
# Number of unacknowledged probes before declaring the connection dead
opts.append((socket.IPPROTO_TCP, socket.TCP_KEEPCNT, 5))

return opts


class _DefaultHttpxClient(httpx.Client):
def __init__(self, **kwargs: Any) -> None:
kwargs.setdefault("timeout", DEFAULT_TIMEOUT)
kwargs.setdefault("limits", DEFAULT_CONNECTION_LIMITS)
kwargs.setdefault("follow_redirects", True)
kwargs.setdefault("transport", httpx.HTTPTransport(socket_options=_build_keepalive_socket_options()))
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot Jun 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Raise the httpx floor before using socket_options

This package still declares httpx>=0.23.0, <1 in pyproject.toml, but socket_options was only added to HTTPTransport/AsyncHTTPTransport in httpx 0.25.0. In environments that satisfy the current dependency with httpx 0.23.x or 0.24.x, constructing the default OpenAI client will raise TypeError: ... unexpected keyword argument 'socket_options' before any request is made. Please either guard this argument for older httpx versions or bump the minimum dependency.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot Jun 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve httpx client options when adding keepalive

Creating and passing a concrete transport here causes httpx.Client to use that transport as-is, so the limits, verify, cert, trust_env, http1, and http2 kwargs set on DefaultHttpxClient(...) are no longer applied to the default non-proxy transport. This regresses both the SDK's DEFAULT_CONNECTION_LIMITS and documented customization such as DefaultHttpxClient(http2=True) unless callers manually build their own transport. Please build the transport with the effective kwargs or use an approach that does not bypass httpx's transport initialization.

Useful? React with 👍 / 👎.

super().__init__(**kwargs)


Expand Down Expand Up @@ -1423,6 +1452,7 @@ def __init__(self, **kwargs: Any) -> None:
kwargs.setdefault("timeout", DEFAULT_TIMEOUT)
kwargs.setdefault("limits", DEFAULT_CONNECTION_LIMITS)
kwargs.setdefault("follow_redirects", True)
kwargs.setdefault("transport", httpx.AsyncHTTPTransport(socket_options=_build_keepalive_socket_options()))
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot Jun 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve async httpx client options when adding keepalive

The async default client has the same issue: supplying a prebuilt AsyncHTTPTransport means httpx.AsyncClient does not apply its limits, TLS, env, or HTTP/2 options to the default transport. Users relying on DefaultAsyncHttpxClient(http2=True) or the SDK's default connection limits silently get httpx transport defaults instead. Please pass the effective options into the transport or avoid replacing httpx's normal transport construction.

Useful? React with 👍 / 👎.

super().__init__(**kwargs)


Expand Down