Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -8014,3 +8014,7 @@ Original filing (2026-04-18): the session emitted `SessionStart hook (completed)
833. **DONE — Native Ollama provider support via `OLLAMA_HOST` env var** — PR #3213 showed the routing gap: local model names like `qwen3:8b` don't match any provider prefix and `OPENAI_BASE_URL` was the only workaround. Ollama users had to set two env vars (`OPENAI_BASE_URL` + `OPENAI_API_KEY`) and unset Anthropic vars. Fix: (1) `OLLAMA_HOST` env var takes priority over all other routing; when set, all models route to the local OpenAI-compatible endpoint; (2) no API key required — placeholder token used for Authorization header; (3) model names with colons/dots bypass strict `provider/model` syntax validation when `OLLAMA_HOST` is set; (4) `detect_provider_kind()` checks `OLLAMA_HOST` first in the routing cascade; (5) `ProviderClient` dispatch uses `from_ollama_env()` when `OLLAMA_HOST` is set; (6) `OLLAMA_CONFIG` constant added to `openai_compat.rs`; (7) `from_ollama_env()` method constructs client from env with no auth requirement; (8) updated `USAGE.md` and `docs/local-openai-compatible-providers.md` to recommend `OLLAMA_HOST` as the preferred env var; (9) added `test_ollama_host_bypasses_model_validation` unit test. Supersedes PR #3213 (which had a duplicate `if let` bug in `mod.rs`). [SCOPE: claw-code]

**Verification.** `cargo fmt --manifest-path rust/Cargo.toml --all -- --check`; `cargo test --manifest-path rust/Cargo.toml -p api -p rusty-claude-cli --bin claw`; `cargo test --manifest-path rust/Cargo.toml -p rusty-claude-cli --test output_format_contract`; dogfooded against mock OpenAI-compatible server with `OLLAMA_HOST=http://127.0.0.1:11434` — `claw --model "qwen3:8b" prompt "Reply exactly: HELLO_WORLD_123"` returned expected output; `claw --output-format json --model "qwen3:8b" prompt "test"` returned parseable JSON; `claw --output-format json doctor` returned `status: warn` (expected, no Anthropic creds); `claw --output-format json --model "qwen3:8b" status` returned `status: ok, model: qwen3:8b`; empty model still rejected with typed error.

834. **DONE — Local-model syntax bypass only covered Ollama-shaped tags, not bare LM Studio/llama.cpp aliases** — follow-up to #833/PR #3213. `is_local_openai_model_syntax()` waived strict `provider/model` syntax under `OPENAI_BASE_URL` only when the model name contained a `.` or `:` (Ollama-tag-shaped). A local server registering a model under a bare alias with none of those characters — for example an LM Studio custom name or a llama.cpp `--alias` value like `mistral` — still failed CLI-parse-time validation unless the user discovered the `local/` prefix workaround. Fix: (1) `is_local_openai_compatible_base_url()` in `openai_compat.rs`, already used to waive the API-key requirement for loopback/private-network base URLs, is now `pub` and re-exported from the `api` crate root; (2) `is_local_openai_model_syntax()` in `rusty-claude-cli`'s `main.rs` calls it against the configured `OPENAI_BASE_URL` and accepts any non-empty model name when the host is local/private, matching the existing `OLLAMA_HOST` bypass; (3) remote/hosted `OPENAI_BASE_URL` gateways keep the stricter dot/colon-tag heuristic so a mistyped bare name doesn't silently route to a paid hosted API; (4) `local/` remains the explicit escape hatch for slash-containing IDs regardless of host; (5) added an LM Studio section and updated routing notes in `docs/local-openai-compatible-providers.md`. [SCOPE: claw-code]

**Verification.** `scripts/fmt.sh --check`; `cargo clippy --manifest-path rust/Cargo.toml --workspace --all-targets -- -D warnings`; `cargo test --manifest-path rust/Cargo.toml --workspace`; new tests `local_openai_base_url_bypasses_bare_model_names`, `private_network_openai_base_url_bypasses_bare_model_names`, and `remote_openai_base_url_still_requires_provider_model_or_tag_syntax` in `rusty-claude-cli`'s `alias_resolution_tests` module; `is_local_openai_compatible_base_url_is_reachable_via_crate_root` in `api`'s `client.rs` confirming the re-export.
16 changes: 14 additions & 2 deletions docs/local-openai-compatible-providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

This guide covers two common offline/local workflows:

1. running Claw against an OpenAI-compatible local model server such as Ollama, llama.cpp, or vLLM; and
1. running Claw against an OpenAI-compatible local model server such as Ollama, LM Studio, llama.cpp, or vLLM; and
2. installing local skills from disk so Claw can discover them without network access.

## Claw is not Claude-only
Expand All @@ -25,7 +25,8 @@ Routing notes:

- Use the `openai/` prefix for OpenAI-compatible gateways when you need prefix routing to win over ambient Anthropic credentials, for example `--model "openai/gpt-4.1-mini"` with OpenRouter.
- For local servers, prefer the exact model ID reported by the server (`qwen3:latest`, `llama3.2`, etc.). If your local gateway exposes slash-containing IDs, prefix the exact slug with `local/` so Claw routes through OpenAI-compatible transport while sending the rest verbatim, for example `--model "local/Qwen/Qwen2.5-Coder-7B-Instruct"`.
- If you have multiple provider keys in your environment, `OPENAI_BASE_URL` plus local-looking tags such as `llama3.2` or `qwen2.5-coder:7b` selects the local OpenAI-compatible route; use `local/` for slash-containing local IDs.
- When `OPENAI_BASE_URL` points at a loopback or private-network address (`localhost`, `127.0.0.1`, `10.x`, `192.168.x`, `172.16-31.x`), Claw accepts **any** non-empty model name — bare aliases like `mistral` work with no dot, colon, or slash required. This covers LM Studio and llama.cpp server setups that register models under a plain `--alias`, not just Ollama-style tagged names (`llama3.2`, `qwen2.5-coder:7b`). `local/` is still needed only to preserve slash-containing IDs verbatim (e.g. Hugging Face repo IDs for vLLM/mlx-lm).
- Against a remote/hosted OpenAI-compatible gateway (not local/private), Claw keeps the stricter `provider/model` or dot/colon-tag requirement so a mistyped bare name doesn't silently route to a paid hosted API.
- Tool workflows need model/server support for OpenAI-compatible tool calls. Plain prompt smoke tests can pass even when slash/tool workflows still fail because the server returns an incompatible tool-call shape.

## Raw `/v1/chat/completions` smoke test
Expand Down Expand Up @@ -81,6 +82,17 @@ export OPENAI_API_KEY="local-dev-token"
claw --model "qwen2.5-coder" prompt "Reply exactly HELLO_WORLD_123"
```

## LM Studio

Start LM Studio's local server (Developer tab → Start Server, default port `1234`), or headless via `lms server start`. LM Studio commonly serves models under a bare alias with no dot or colon (for example `mistral-7b-instruct` or a custom name set in the model settings).

```bash
export OPENAI_BASE_URL="http://127.0.0.1:1234/v1"
claw --model "mistral-7b-instruct" prompt "Reply exactly HELLO_WORLD_123"
```

`OPENAI_API_KEY` can stay unset for LM Studio's default authless local server. Use the exact model identifier LM Studio reports (check `curl $OPENAI_BASE_URL/models`) if the bare alias above doesn't match what you loaded.

## vLLM or another OpenAI-compatible server

Start vLLM with an OpenAI-compatible API server:
Expand Down
17 changes: 17 additions & 0 deletions rust/crates/api/src/client.rs
Original file line number Diff line number Diff line change
Expand Up @@ -262,4 +262,21 @@ mod tests {
other => panic!("Expected ProviderClient::OpenAi for local model, got: {other:?}"),
}
}

#[test]
fn is_local_openai_compatible_base_url_is_reachable_via_crate_root() {
// Regression: this helper must stay `pub` and re-exported from the
// crate root so `rusty-claude-cli`'s model-syntax validation can
// reuse the same local/private-address detection used here to
// waive the API-key requirement.
assert!(crate::is_local_openai_compatible_base_url(
"http://127.0.0.1:1234/v1"
));
assert!(crate::is_local_openai_compatible_base_url(
"http://192.168.1.50:8000/v1"
));
assert!(!crate::is_local_openai_compatible_base_url(
"https://openrouter.ai/api/v1"
));
}
}
6 changes: 3 additions & 3 deletions rust/crates/api/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,9 @@ pub use prompt_cache::{
pub use providers::anthropic::{AnthropicClient, AnthropicClient as ApiClient, AuthSource};
pub use providers::openai_compat::{
build_chat_completion_request, check_request_body_size, estimate_request_body_size,
flatten_tool_result_content, is_reasoning_model, model_rejects_is_error_field,
model_requires_reasoning_content_in_history, translate_message, OpenAiCompatClient,
OpenAiCompatConfig,
flatten_tool_result_content, is_local_openai_compatible_base_url, is_reasoning_model,
model_rejects_is_error_field, model_requires_reasoning_content_in_history, translate_message,
OpenAiCompatClient, OpenAiCompatConfig,
};
pub use providers::{
detect_provider_kind, max_tokens_for_model, max_tokens_for_model_with_override,
Expand Down
7 changes: 6 additions & 1 deletion rust/crates/api/src/providers/openai_compat.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1027,7 +1027,12 @@ fn url_host(url: &str) -> &str {
host_port.split(':').next().unwrap_or("")
}

fn is_local_openai_compatible_base_url(url: &str) -> bool {
/// Returns true when `url`'s host is loopback or a private-network address
/// (`localhost`, `127.0.0.1`, `10.x`, `192.168.x`, `172.16-31.x`). Used to
/// waive auth requirements for local OpenAI-compatible servers (Ollama, LM
/// Studio, llama.cpp, vLLM) that don't expect a real API key.
#[must_use]
pub fn is_local_openai_compatible_base_url(url: &str) -> bool {
let host = url_host(url.trim());
if host.eq_ignore_ascii_case("localhost") || host == "::1" {
return true;
Expand Down
46 changes: 45 additions & 1 deletion rust/crates/rusty-claude-cli/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2986,7 +2986,18 @@ fn is_local_openai_model_syntax(model: &str) -> bool {
if let Some(rest) = model.strip_prefix("local/") {
return !rest.is_empty() && rest.split('/').all(|segment| !segment.is_empty());
}
std::env::var_os("OPENAI_BASE_URL").is_some() && (model.contains(':') || model.contains('.'))
let Ok(base_url) = std::env::var("OPENAI_BASE_URL") else {
return false;
};
// A local/private OPENAI_BASE_URL (Ollama, LM Studio, llama.cpp, vLLM, ...)
// is trusted to accept any non-empty model name the server actually
// serves, matching the OLLAMA_HOST bypass above. Remote gateways keep the
// stricter dot/colon-tag heuristic so a bare typo doesn't silently route
// to a paid hosted API.
if api::is_local_openai_compatible_base_url(&base_url) {
return !model.is_empty();
}
model.contains(':') || model.contains('.')
}

fn config_alias_for_current_dir(alias: &str) -> Option<String> {
Expand Down Expand Up @@ -19828,4 +19839,37 @@ mod alias_resolution_tests {
// Empty model still rejected
assert!(validate_model_syntax("").is_err());
}

#[test]
fn local_openai_base_url_bypasses_bare_model_names() {
let _guard = ollama_env_lock();
let _ollama_env = EnvVarGuard::unset("OLLAMA_HOST");
// LM Studio's default port; a bare alias with no dot/colon/slash,
// as LM Studio and llama.cpp commonly register models under.
let _openai_env = EnvVarGuard::set("OPENAI_BASE_URL", "http://127.0.0.1:1234/v1");
assert!(validate_model_syntax("mistral").is_ok());
assert!(validate_model_syntax("local-7b").is_ok());
// Empty model is still rejected even when the base URL is local.
assert!(validate_model_syntax("").is_err());
}

#[test]
fn private_network_openai_base_url_bypasses_bare_model_names() {
let _guard = ollama_env_lock();
let _ollama_env = EnvVarGuard::unset("OLLAMA_HOST");
let _openai_env = EnvVarGuard::set("OPENAI_BASE_URL", "http://192.168.1.50:8000/v1");
assert!(validate_model_syntax("mistral").is_ok());
}

#[test]
fn remote_openai_base_url_still_requires_provider_model_or_tag_syntax() {
let _guard = ollama_env_lock();
let _ollama_env = EnvVarGuard::unset("OLLAMA_HOST");
let _openai_env = EnvVarGuard::set("OPENAI_BASE_URL", "https://openrouter.ai/api/v1");
// A bare name with no dot/colon/slash against a remote gateway must
// keep failing so a typo doesn't silently route to a paid hosted API.
assert!(validate_model_syntax("mistral").is_err());
// The existing dot/colon-tag heuristic still applies for remote hosts.
assert!(validate_model_syntax("qwen2.5-coder:7b").is_ok());
}
}