Claude: support custom models in gateway discovery - #375
Conversation
c731346 to
19590dc
Compare
110431c to
19590dc
Compare
| return headers | ||
|
|
||
|
|
||
| class _ModelAliases: |
There was a problem hiding this comment.
_ModelAlias -> _AnthropicModelAliases
| if not isinstance(model, dict) or not isinstance(model.get("id"), str): | ||
| continue | ||
| model_id = model["id"] | ||
| if "claude" in model_id.lower() or "anthropic" in model_id.lower(): |
There was a problem hiding this comment.
is in the right check or is it startsWith?
There was a problem hiding this comment.
Yes, in is correct since claude code checks if "claude" or "anthropic" exist anywhere in the name.
|
|
||
| def rewrite_path(self, path: str) -> str: | ||
| parsed = urlsplit(path) | ||
| if parsed.path != "/v1/models": |
There was a problem hiding this comment.
can you make this anthropic/v1/models? we shouldnt be doing this for codex
| transform_models=self.command == "GET" | ||
| and urlsplit(self.path).path == "/v1/models", |
There was a problem hiding this comment.
can you split this into a variable instead?
should_transform_model_names = self.command == "GET" and urlsplit(self.path).path == "/v1/models"
| transform_models=self.command == "GET" | ||
| and urlsplit(self.path).path == "/v1/models", |
| self, | ||
| resp: httpx.Response, | ||
| *, | ||
| transform_models: bool = False, |
There was a problem hiding this comment.
no default value, force call site to specify
| if key.lower() not in _HOP_BY_HOP and not ( | ||
| transform_models and key.lower() == "content-encoding" | ||
| ): |
There was a problem hiding this comment.
can you rename this into a clearer variable? like what does the below signify?
not (
transform_models and key.lower() == "content-encoding"
)
| for chunk in resp.iter_raw(): | ||
| response_chunks = ( | ||
| [self.model_aliases.advertise_models(resp.read())] | ||
| if transform_models and 200 <= resp.status_code < 300 |
There was a problem hiding this comment.
make 200 and 300 constants
b21bcf3 to
dabbd5a
Compare
19590dc to
86edc01
Compare
There was a problem hiding this comment.
nit could you make 127.0.0.1 a const and put it in a generic file? i'll need to use it as well
| self._original_by_alias: dict[str, str] = {} | ||
| self._lock = threading.Lock() | ||
|
|
||
| def advertise_models(self, body: bytes) -> bytes: |
There was a problem hiding this comment.
advertise_models are like the models the user sees right? maybe UserDisplayedModels
| drop_stale_content_encoding = ( | ||
| transform_model_names and header_name == "content-encoding" | ||
| ) |
There was a problem hiding this comment.
what does this mean? why can't we have content-encoding?
| @@ -395,11 +498,17 @@ def start_proxy( | |||
| # to the gateway instead of a fresh handshake per request. Don't follow | |||
| # redirects — a proxy relays 3xx verbatim. | |||
| client = httpx.Client(base_url=upstream_base, timeout=_UPSTREAM_TIMEOUT, follow_redirects=False) | |||
There was a problem hiding this comment.
hmmm is upstream_base then changing? because it used to be /anthropic. will this break relayed auth?
There was a problem hiding this comment.
ic, is that why you had /v1/models as the orig path to match one? maybe we should leave it as it was before then and just rename to anthropic_gateway_proxy since it looks like this thing is pretty anthropic specific
| body = self.anthropic_model_aliases.rewrite_body(self.path, body) | ||
| url = self.anthropic_model_aliases.rewrite_path(self.path).lstrip("/") |
There was a problem hiding this comment.
will url here be diff btw relayed auth and the models?
2 participants
Summary
Claude Code filters gateway-discovered models by ID, which hides custom model-service FQNs that do not contain
claudeoranthropic.This change updates the relayed gateway proxy to:
anthropic-aigw-ID while preserving the originaldisplay_nameTesting
uv run ruff check .uv run pytest tests/test_gateway_proxy.py(28 passed)uv run pytest(1944 passed, 37 skipped, 1 environment-dependent failure: the Claude user-agent e2e timed out before reaching its local capture server)🥞 Stacked PR
Use this link to review incremental changes.