diff --git a/docs/DASHBOARD.md b/docs/DASHBOARD.md index 644965e..0d14701 100644 --- a/docs/DASHBOARD.md +++ b/docs/DASHBOARD.md @@ -110,7 +110,11 @@ Overview keeps status and routing in one health-first area: - **Hosts & Routing** presents execution-host health, the primary-host policy, per-activity routes, escalation paths, and routed host models. A configured route is assignment intent, not evidence of which inference provider served a particular session. -- **Providers** presents inference-provider bindings and their configuration provenance. +- **Providers** presents inference-provider bindings and their configuration provenance. A + registered provider is eligible configuration, not evidence that a request selected or used it. + Direct Ruflo agents must explicitly select OpenRouter or Ollama together with a provider-native + model, and the Ruflo/MCP process must inherit the required credential environment. Served-provider + and served-model claims come from **Usage → Scorecard** evidence instead. - **Runtime** presents operational services, processes, and MCP readiness. - **Intelligence** presents memory, learning, and quality-improvement signals machine-wide: an always-visible rollup folded across every project on this machine where memory or intelligence has @@ -174,6 +178,13 @@ the serving provider, so a historical row may honestly show **Not recorded**. Th evidence, not a claim that Claude Code was served by Anthropic or by any provider inferred from its model. +This distinction also applies to Ruflo's project-scoped `agents.providers` registry. Ruflo 3.38.8+ +can execute an explicitly spawned `--provider openrouter --model z-ai/glm-5.2` agent through +OpenRouter, but registration by `ak host pick --provider openrouter:z-ai/glm-5.2` does not retarget +every direct agent. `RUFLO_PROVIDER=openrouter` is a process-wide override; explicit per-agent +provider and model selection is the reproducible path. Restart a long-lived Ruflo/MCP process after +adding `OPENROUTER_API_KEY`, because it inherits environment variables only when it starts. + Usage transcript masking happens on the server. Redaction is marked, there is no reveal or export control, and the original masked value never reaches the browser. See [ADR-0009](adr/0009-usage-scorecard-local-transcript-analytics.md) for the full evidence and pricing diff --git a/docs/PROVIDERS.md b/docs/PROVIDERS.md index 11ff5c9..f8733e5 100644 --- a/docs/PROVIDERS.md +++ b/docs/PROVIDERS.md @@ -74,6 +74,12 @@ however the endpoint is served. `local-openai` is not an AQE provider type — ` credentials, fragments, or secret-bearing query parameters. See [ADR-0028](adr/0028-local-openai-compatible-providers.md). +A binding declares a compatible relationship; it does not add a new execution branch inside an +upstream tool. In particular, Ruflo's direct `agent_execute` path currently dispatches persisted +provider configuration through its explicit Ollama and OpenRouter branches. A `local-openai` +binding remains valid for Codex/OpenCode configuration without implying that Ruflo can select the +literal provider id `local-openai` for direct execution. + ## External host adapters (experimental) Want `ak` to manage a host CLI it doesn't ship in-tree — driving local models through something @@ -161,24 +167,71 @@ for you: ℹ codex CLI detected — run `ak host pick` to let ruflo use both claude and codex ``` -## Two configs, one front door +## Native configs, one front door -A question that trips people up: **do ruflo and agentic-qe read the same config?** No. -They are two independent routing subsystems, each with its **own config store and its own -escalation machinery**. There is no shared file they both read. What unifies them is `ak`: +A question that trips people up: **do Ruflo and agentic-qe read the same config?** No. +They are independent routing subsystems, each with its **own config store**. There is no shared +file they both read. What unifies them is `ak`: it takes your intent once (in `kit.json`) and writes **each tool's own native config** — converging, proving, and able to undo. `ak` is a facilitator, not a config layer the tools depend on. -| | **agentic-qe** (`HybridRouter`) | **ruflo** (provider router) | +| | **agentic-qe** (`HybridRouter`) | **Ruflo direct agent execution** | |---|---|---| -| **Config store** | `.agentic-qe/llm-config.json` (per project) + env | `~/.agentic-flow/router.config.json` (or `--router-config`) + env | -| **Precedence** (highest wins) | explicit override → env (`AQE_LLM_*`, API keys) → disk file → built-in defaults | file (`defaultProvider`, `fallbackChain`) + CLI overrides | +| **Config store** | `.agentic-qe/llm-config.json` (per project) + env | `claude-flow.config.json` or `.claude-flow/config.json` (per project), `CLAUDE_FLOW_CONFIG`, and env | +| **Precedence** (highest wins) | explicit override → env (`AQE_LLM_*`, API keys) → disk file → built-in defaults | per-agent provider → `RUFLO_PROVIDER` → credential env → persisted `agents.providers` → inference/default | | **Change the provider** | `AQE_LLM_PROVIDER=` (env) — a provider whose API key is in the env is auto-enabled | `ruflo providers configure -p -m ` | -| **Change the model** | per-provider `models` in the chain; per-activity `agentOverrides` (aqe ≥ 3.13.1) | `models.{default,fast,advanced}` per provider in the file | -| **Escalation / fallback** | ordered `fallbackChain` + circuit breaker + retry/backoff | `fallbackChain` + `routing.mode` (cost/quality/perf/rule-based) + circuit breaker | +| **Change the model** | per-provider `models` in the chain; per-activity `agentOverrides` (aqe ≥ 3.13.1) | agent model first; then the selected persisted provider's `model` | +| **Escalation / fallback** | ordered `fallbackChain` + circuit breaker + retry/backoff | provider selection for this path; not the enhanced router fallback chain | | **The `ak` way** | `--aqe-provider` / `--aqe-fallback` / `--route` | `--provider :` | +Ruflo also bundles agentic-flow's enhanced model router, whose separate store is +`~/.agentic-flow/router.config.json` (or `--router-config`). `ak --provider` does not write that +router's `defaultProvider`, `fallbackChain`, or routing modes; it maps specifically to +`ruflo providers configure` and the project-scoped `agents.providers` registry. + +Ruflo 3.38.8 fixed direct agent execution so it can consume that persisted registry +([upstream #2962](https://github.com/ruvnet/ruflo/issues/2962)). Registration and selection remain +different operations: `--provider` registers an eligible provider/model; an explicit per-agent +provider or `RUFLO_PROVIDER` still outranks it. OpenRouter registered by `ak` also needs +`OPENROUTER_API_KEY` in the environment. For a fresh keyless Ollama entry with no endpoint env, +`ak` supplies `http://127.0.0.1:11434`; it preserves an existing Ruflo `baseUrl`, and an explicit +`endpoint` on the `kit.json` model entry wins. `OLLAMA_API_KEY` keeps Ruflo's cloud behavior, while +`OLLAMA_BASE_URL` remains the environment override. `ak` surfaces a degraded warning when the +installed Ruflo is older than 3.38.8. + +For a direct Ruflo agent using an OpenRouter-vended model, the complete user path is: + +```bash +# Export this where the long-lived Ruflo/MCP process will inherit it, then restart that process. +export OPENROUTER_API_KEY=... + +# Persist eligible provider/model intent. `ak sync` reapplies it later. +ak host pick --provider 'openrouter:z-ai/glm-5.2' + +# Select the provider and model for the direct agent that will execute. +ruflo agent spawn --type coder --provider openrouter --model z-ai/glm-5.2 +``` + +Execute the returned agent id through Ruflo's `agent_execute` MCP tool or a Ruflo workflow. Spawning +an agent, even with `--task`, is registration rather than execution. Treat the response's served +model/provider evidence as the proof that routing occurred. Omitting the provider/model flags does +not mean the `ak`-registered model becomes every agent's default. `RUFLO_PROVIDER=openrouter` can +force the provider for the whole Ruflo process, but explicit per-agent selection is narrower and +reproducible. + +`ak run` is a separate host-execution path: it invokes Claude, Codex, or OpenCode adapters and does +not dispatch through Ruflo's direct-provider registry. To use an OpenRouter model in an `ak run` +pipeline, route an OpenCode activity to its provider-qualified model, for example: + +```bash +ak run feature "implement the change" \ + --route 'implementation:opencode:openrouter/z-ai/glm-5.2' +``` + +That path requires OpenCode's OpenRouter authentication/configuration; the Ruflo provider entry is +not a substitute for the host adapter's own credentials. + Two axes cut across both (see the intro): **hosts** (which agent CLI runs the ruflo loop — `ENABLE_CLAUDE_CODE` / `ENABLE_CODEX`) are separate from **providers** (which LLM the routers use). Level 4 below is the full `ak`-way ↔ raw-tool-way map for every knob in this table. @@ -395,13 +448,14 @@ is the tool's own native config, and you can set it by hand — or let `ak` and coexist. `ak` merges-not-clobbers and backs up first, mirroring how rUv itself layers config (`mergeWithDefaults(config, defaults)` — sensible defaults, override with your partial). -The two config stores each knob below lives in — and their precedence — are summarized in -[Two configs, one front door](#two-configs-one-front-door) above. +The native config stores each knob below lives in — and their precedence — are summarized in +[Native configs, one front door](#native-configs-one-front-door) above. | You want to… | `ak` way | The raw ruflo/aqe way it maps to | | ------------------------------------ | --------------------------------- | --------------------------------------------------- | | Enable claude/codex hosts | `ak host pick` | `ENABLE_CLAUDE_CODE` / `ENABLE_CODEX` env + managed bridge/guidance | -| Register a ruflo LLM provider | `--provider openai:gpt-5.6` | `ruflo providers configure -p openai -m gpt-5.6` | +| Register a Ruflo LLM provider | `--provider ollama:qwen3.6:27b` | `ruflo providers configure -p ollama -m qwen3.6:27b -e http://127.0.0.1:11434` | +| Select a direct Ruflo provider | per-agent/raw setting | agent `--provider` or `RUFLO_PROVIDER=ollama` / `openrouter` | | Set which LLM runs QE | `--aqe-provider gemini` | `AQE_LLM_PROVIDER=gemini` (env) | | Order QE's fallback chain | `--aqe-fallback '…'` | edit `.agentic-qe/llm-config.json` / `aqe llm-router config` | | Cap QE spend | (kit.json `maxBudgetUsd`) | `AQE_MAX_BUDGET_USD` / `--max-budget-usd` | diff --git a/docs/adr/0028-local-openai-compatible-providers.md b/docs/adr/0028-local-openai-compatible-providers.md index 1b6d635..65bd6f1 100644 --- a/docs/adr/0028-local-openai-compatible-providers.md +++ b/docs/adr/0028-local-openai-compatible-providers.md @@ -2,11 +2,13 @@ - **Status:** Accepted - **Date:** 2026-08-11 -- **Updated:** 2026-08-14 +- **Updated:** 2026-08-25 - **Update note:** Accepted with corrections after review of PR #131: the quoted Hermes `api_mode: openai` value is annotated as invalid rather than reproduced as valid (F-30), and the AQE-projection asymmetry between `ollama` and `local-openai` is now stated explicitly as intentional (F-29). Implemented with in-tree projections `['ruflo', 'codex', 'opencode']`. + Reconciled after ruvnet/ruflo#2962: projection is configuration eligibility, not proof that + Ruflo's direct `agent_execute` dispatcher implements an arbitrary provider id. - **Deciders:** agentic-kit maintainers - **Related:** [ADR-0011](0011-local-model-provenance-zero-cost-and-transcript-fidelity.md), [ADR-0016](0016-capability-driven-integration-adapters.md), @@ -128,12 +130,20 @@ deliberate: because AQE does not. `ak status`'s provider surface reflects this distinction; surfacing it clearly is a sibling work package's scope, not this ADR's. +`'ruflo'` has the same bounded meaning as every projection in this domain: the relationship can be +represented on Ruflo's provider-configuration surface. It does **not** claim that Ruflo's direct +agent executor dispatches an arbitrary provider name. Ruflo 3.38.8 fixed persisted provider +selection for its implemented Ollama and OpenRouter branches (ruvnet/ruflo#2962); it did not add a +generic `local-openai` branch. Therefore a `local-openai` binding is not direct-agent-execution +evidence, and agentic-kit must not present registration as successful selection or execution. + ## Consequences - A machine serving models from MLX, LM Studio, llama.cpp, vLLM, or anything else speaking OpenAI-compatible HTTP on loopback can be described to ak without a new ADR per vendor, and without inventing a provider id the user did not choose. -- Every host that can be pointed at an OpenAI-compatible base URL gains a nameable local provider. +- Every host with a concrete OpenAI-compatible projection can gain a nameable local provider; a + registry projection alone does not make another tool's direct executor support that provider id. - The generic row deliberately supports **less** than `ollama`: no catalogue, no runtime probe, no digest. Surfaces that show local-model detail for Ollama will show less for `local-openai`, and that gap is the honest reading of the evidence, not a defect to paper over. diff --git a/docs/ddd/integration-management.md b/docs/ddd/integration-management.md index 63b2e1f..885e0ac 100644 --- a/docs/ddd/integration-management.md +++ b/docs/ddd/integration-management.md @@ -41,7 +41,9 @@ participate in other workflows. An inference provider describes credentials, transports, projections, observability sources, and bounded capabilities such as discovery, quota, pricing, and cache accounting. Provider identity -does not imply a host or routing capability. +does not imply a host or routing capability. A projection declares that configuration can be +represented on a native surface; it does not manufacture an execution branch in that tool. +Registration, selection, and observed execution remain separate facts. ### Provider binding diff --git a/src/commands/setup.mjs b/src/commands/setup.mjs index ed6d6ba..8409bb5 100644 --- a/src/commands/setup.mjs +++ b/src/commands/setup.mjs @@ -415,11 +415,13 @@ export async function run_project({ flags, cfg, trustDisclosed = false }) { const rmcp = await ensureRufloMcpInCodex(cfg, root); if (rmcp.changed) saveKitConfig(cfg); // persist reverse MCP ownership if (rmcp.changed || !rmcp.ok) (rmcp.ok ? ok : warn)(`ruflo→codex MCP: ${rmcp.detail}`); - const prov = await applyProviders(cfg, root); - if (prov.changed) (prov.ok ? ok : warn)(`providers: ${prov.detail}`); } else if (await have('codex')) { info('codex CLI detected — enable dual-host with: ak host pick'); } + // Provider routing is independent of the enabled execution-host set. Apply + // persisted Ruflo providers for Claude-only setups too (#128 / ruflo#2962). + const prov = await applyProviders(cfg, root); + if (prov.changed || !prov.ok || prov.status === 'degraded') reportOutcome('providers', prov); // 10. statusline footer — LAST, after ruflo + aqe have settled the helper. // A still-missing footer is a WARN (not silent info): it means the AQE / diff --git a/src/commands/status.mjs b/src/commands/status.mjs index ba2c88c..e785624 100644 --- a/src/commands/status.mjs +++ b/src/commands/status.mjs @@ -17,13 +17,13 @@ import { listDaemons, staleDaemons } from '../lib/daemons.mjs'; import { scanRvf } from '../lib/rvf.mjs'; import { registry, syncBlocks, blocksForTarget, retiredForTarget, guidanceTargets } from '../lib/blocks.mjs'; import { loadKitConfig } from '../lib/config.mjs'; -import { driftReport, selfDrift } from '../lib/versions.mjs'; +import { driftReport, selfDrift, installedVersion, cmpVersions } from '../lib/versions.mjs'; import { upstreamCveCounterFabricated, fixStatusline, helperStampStale } from '../lib/statusline.mjs'; import { drift as ruvnetBrainDrift, nightlyAgentPresent as rbNightlyPresent, NIGHTLY_LABEL as RB_NIGHTLY_LABEL } from '../lib/ruvnet-brain.mjs'; import { coherence as adbCoherence } from '../lib/agentdb.mjs'; import { readJson } from '../lib/settings.mjs'; import { have } from '../lib/exec.mjs'; -import { HOSTS, settingsTarget, isDefault, managedEnv, MANAGED_ENV_KEYS, hostInstallState, hostAuthState, bothHostsEnabled, aqeRouterFile, aqeSupportsAgentOverrides, credentialGaps, collectIntegrationFacts } from '../lib/providers.mjs'; +import { HOSTS, settingsTarget, isDefault, managedEnv, MANAGED_ENV_KEYS, hostInstallState, hostAuthState, bothHostsEnabled, aqeRouterFile, aqeSupportsAgentOverrides, credentialGaps, collectIntegrationFacts, MIN_RUFLO_PERSISTED_PROVIDER_VERSION } from '../lib/providers.mjs'; import { hostsWithLifecycle, isBuiltinHost, lifecycleExecutionEnabled } from '../lib/adapters/lifecycle-registry.mjs'; import { PROVIDER_REGISTRY } from '../lib/adapters/index.mjs'; import { configuredPolicyToAgentOverrides, agentOverridesDrift, routingSummary, divergedRoutes } from '../lib/routing.mjs'; @@ -686,6 +686,39 @@ export async function collect({ pkgRoot, cwd = process.cwd() }) { } } } + // A kit.json provider/model entry is registration intent. Ruflo >=3.38.8 + // can honor explicit OpenRouter/Ollama provider+model selection, but the + // registry does not retarget every agent and it is not execution evidence. + // Keep that distinction in the status rows the dashboard consumes. + const rufloModels = cfg.providers?.models ?? []; + if (rufloModels.length) { + const intent = rufloModels + .filter((entry) => entry?.id) + .map((entry) => `${entry.id}${entry.model ? `:${entry.model}` : ''}`) + .join(', '); + const rufloVersion = installedVersion('ruflo'); + const affected = !!rufloVersion + && cmpVersions(rufloVersion, MIN_RUFLO_PERSISTED_PROVIDER_VERSION) < 0; + const missingOpenRouterKey = rufloModels.some((entry) => entry?.id === 'openrouter') + && !integrationFacts.providers?.openrouter?.credentialPresent; + const directIds = new Set(['ollama', 'openrouter']); + const registryOnly = [...new Set(rufloModels + .map((entry) => entry?.id) + .filter((id) => id && !directIds.has(id)))]; + if (affected) { + rows.push(row('providers', 'warn', + `ruflo provider intent: ${intent} — ruflo ${rufloVersion} cannot honor persisted provider/model execution; needs >=${MIN_RUFLO_PERSISTED_PROVIDER_VERSION}`)); + } else if (missingOpenRouterKey) { + rows.push(row('providers', 'warn', + `ruflo provider intent: ${intent} — direct agents must select provider + model; openrouter needs OPENROUTER_API_KEY in the Ruflo/MCP process`)); + } else { + const unsupported = registryOnly.length + ? `; no direct-agent execution branch for ${registryOnly.join(', ')}` + : ''; + rows.push(row('providers', 'info', + `ruflo provider intent: ${intent} — direct agents must select provider + model; Usage proves served execution${unsupported}`)); + } + } // ADR-0028 F-29: local-openai is a local ($0) provider deliberately NOT // projected to 'aqe' (unlike ollama, which is) — surface that asymmetry // plainly so it reads as a fact, not a bug. Registry-driven (billing + diff --git a/src/commands/x/host.mjs b/src/commands/x/host.mjs index 096b0c4..962af7b 100644 --- a/src/commands/x/host.mjs +++ b/src/commands/x/host.mjs @@ -113,7 +113,7 @@ Options (pick, all optional — omit for interactive): (metered providers work too, e.g. add 'openrouter:z-ai/glm-5.2' — GLM via OpenRouter, needs OPENROUTER_API_KEY in the env) - --provider register ruflo API providers (e.g. openai:gpt-5.6) + --provider register ruflo providers (e.g. ollama:qwen3.6:27b) --route 'act:host[:model]' override one activity's routing (repeatable), e.g. --route 'implementation:claude:claude-opus-5' activities: specification, architecture, design, @@ -148,9 +148,11 @@ Examples: const stamp = (source) => (entries) => entries.map((e) => ({ ...e, source })); /** Parse 'claude-code:m1,m2; openai:gpt-5.6' → [{provider, models:[…]}, …]. */ -const parseFallback = (str) => str.split(';').map((s) => s.trim()).filter(Boolean).map((tok) => { - const [provider, models] = tok.split(':'); - return { provider: provider.trim().toLowerCase(), models: (models ?? '').split(',').map((m) => m.trim()).filter(Boolean) }; +export const parseFallback = (str) => str.split(';').map((s) => s.trim()).filter(Boolean).map((tok) => { + const delimiter = tok.indexOf(':'); + const provider = delimiter < 0 ? tok : tok.slice(0, delimiter); + const models = delimiter < 0 ? '' : tok.slice(delimiter + 1); + return { provider: provider.trim().toLowerCase(), models: models.split(',').map((m) => m.trim()).filter(Boolean) }; }); export async function run({ flags, positionals, pkgRoot }) { @@ -255,12 +257,12 @@ async function status({ flags, cwd }) { } const cm = cfg.providers.models ?? []; - console.log(bold('\nruflo LLM API providers') + dim(' (ruflo router; keys read from env)')); + console.log(bold('\nruflo LLM providers') + dim(' (registered intent; direct agents select provider + model)')); for (const p of API_PROVIDERS) { const cfgEntry = cm.find((m) => m.id === p.id); const key = p.keyEnv.length ? (providers[p.id]?.credentialPresent ? 'key present' : 'no key') : 'local'; - const conf = cfgEntry ? `configured${cfgEntry.model ? ` (${cfgEntry.model})` : ''}` : dim('not configured'); + const conf = cfgEntry ? `registered${cfgEntry.model ? ` (${cfgEntry.model})` : ''}` : dim('not registered'); console.log(` ${p.id.padEnd(10)} ${key.padEnd(12)} ${conf}`); } @@ -390,8 +392,11 @@ async function off({ cwd, pkgRoot }) { return ret.ok ? 0 : 1; } -const parseModels = (csv) => csv.split(',').map((s) => s.trim()).filter(Boolean).map((tok) => { - const [id, model] = tok.split(':'); +export const parseModels = (csv) => csv.split(',').map((s) => s.trim()).filter(Boolean).map((tok) => { + const delimiter = tok.indexOf(':'); + if (delimiter < 0) return { id: tok }; + const id = tok.slice(0, delimiter); + const model = tok.slice(delimiter + 1); return model ? { id, model } : { id }; }); @@ -508,7 +513,7 @@ async function pick({ flags, cwd, pkgRoot }) { aqeFallback = fAns ? stamp('user')(parseFallback(fAns)) : (suggestion ? stamp('suggested')(parseFallback(suggestion.toLowerCase())) : []); - const provAns = (await rl.question('ruflo API-key providers to register (e.g. openai:gpt-5.6, blank to skip): ')).trim(); + const provAns = (await rl.question('ruflo providers to register (e.g. ollama:qwen3.6:27b, blank to skip): ')).trim(); if (provAns) models = parseModels(provAns); rl.close(); } @@ -707,7 +712,7 @@ async function pick({ flags, cwd, pkgRoot }) { if (rmcp.changed) saveKitConfig(cfg); // persist reverse MCP ownership if (rmcp.changed || !rmcp.ok) (rmcp.ok ? ok : warn)(`ruflo→codex MCP: ${rmcp.detail}`); const prov = await applyProviders(cfg, cwd); - (prov.ok ? (prov.changed ? ok : info) : warn)(`ruflo providers: ${prov.detail}`); + (prov.status === 'degraded' ? warn : prov.ok ? (prov.changed ? ok : info) : warn)(`ruflo providers: ${prov.detail}`); ok('saved to kit.json — reapplied on every `ak sync`; undo with `ak host off`'); if (seed.seeded) ok(`per-activity routing seeded — ${seed.count} activities (dual-host defaults; tune with --route or edit kit.json)`); printActivityRoutingTable(cfg); diff --git a/src/lib/config.mjs b/src/lib/config.mjs index 1d7424c..4bd4a9e 100644 --- a/src/lib/config.mjs +++ b/src/lib/config.mjs @@ -40,7 +40,7 @@ const DEFAULTS = { providers: { aqeProvider: null, // AQE_LLM_PROVIDER (claude-code|openai|gemini|…); null = aqe default aqeFallback: [], // [{ provider, models:[...] }] — ordered aqe fallback chain (.agentic-qe/llm-config.json) - models: [], // [{ id:'openai', model:'gpt-5.6' }] — ruflo API-key providers + models: [], // [{ id:'ollama', model:'qwen3.6:27b', endpoint?:'http://127.0.0.1:11434' }] — ruflo providers maxBudgetUsd: null, // → AQE_MAX_BUDGET_USD when set }, statusline: { codex: null }, // {preset,lastProjection}: explicit ownership of Codex [tui] keys diff --git a/src/lib/dashboard/page.mjs b/src/lib/dashboard/page.mjs index 67299a0..460daa3 100644 --- a/src/lib/dashboard/page.mjs +++ b/src/lib/dashboard/page.mjs @@ -334,6 +334,11 @@ export function renderPage({ name, version }) {

Inference providers

Provider bindings, availability, provenance, and configuration health.

+
ℹA registered provider is eligible configuration, + not proof that it was selected or served a request. Direct Ruflo agents must select + OpenRouter or Ollama together with a provider-native model; long-lived Ruflo/MCP + processes must inherit the required credential environment. See Usage → Scorecard + for independently observed provider and model evidence.
diff --git a/src/lib/providers.mjs b/src/lib/providers.mjs index 0fabec2..92f5f66 100644 --- a/src/lib/providers.mjs +++ b/src/lib/providers.mjs @@ -5,8 +5,9 @@ // - ruflo ADR-034 "Optional MCP Backends" (ACCEPTED): Claude Code / Gemini / OpenAI // Codex backends are enabled via env vars ENABLE_CLAUDE_CODE / ENABLE_CODEX / // ENABLE_GEMINI_MCP. -// - `ruflo providers configure -p -m ` persists API-key providers -// (anthropic/openai/google/ollama) to ruflo's config. +// - `ruflo providers configure -p -m [-e ]` persists +// provider records to ruflo's project config. Ruflo >=3.38.8's direct +// agent_execute path consumes persisted Ollama/OpenRouter entries (#2962). // - agentic-qe LLM selector `AQE_LLM_PROVIDER=` (ADR-123, // dist/shared/llm/router/config-store.js) force-selects ANY provider in // ALL_PROVIDER_TYPES — claude-code (subscription), claude/openai/gemini/ @@ -18,7 +19,7 @@ // Two independent axes: // host axis — which agent CLI executes a managed worker (claude, codex, // opencode). -// provider axis — which LLM the *routers* use: ruflo's API-key providers +// provider axis — which LLM the *routers* use: ruflo's persisted providers // (`ruflo providers configure`) and aqe's `AQE_LLM_PROVIDER`. // Independent of the host axis; keys live in the env, never kit.json. import fs from 'node:fs'; @@ -34,7 +35,7 @@ import { HOST_ADAPTERS } from './hosts.mjs'; import { HOST_REGISTRY, PROVIDER_REGISTRY, normalizeIntegrationFacts, defaultHostMap, } from './adapters/index.mjs'; -import { CURRENT_INTEGRATIONS_VERSION } from './adapters/config.mjs'; +import { CURRENT_INTEGRATIONS_VERSION, validateEndpoint } from './adapters/config.mjs'; import { opencodeMcpStatus } from './opencode.mjs'; import { rufloCodexMcpStatus } from './mcp.mjs'; import { @@ -61,6 +62,7 @@ const apiProvider = (id, keyEnv) => { export const API_PROVIDERS = [ apiProvider('anthropic', ['ANTHROPIC_API_KEY']), apiProvider('openai', ['OPENAI_API_KEY']), + apiProvider('openrouter', ['OPENROUTER_API_KEY']), { id: 'google', keyEnv: ['GOOGLE_API_KEY', 'GEMINI_API_KEY'] }, apiProvider('ollama', []), // local; presence = reachable daemon (not checked here) ]; @@ -735,8 +737,8 @@ export function applyHosts(cfg, cwd = process.cwd()) { return { ok: true, changed, detail: `hosts=${on} (${scope}${changed ? ', written' : ', in sync'})` }; } -// id/model reach a real subprocess argv (`ruflo providers configure -p -// -m `). exec.mjs's shell:false + resolved-argv fix is the real +// id/model/endpoint reach a real subprocess argv (`ruflo providers configure +// -p -m -e `). exec.mjs's shell:false + resolved-argv fix is the real // injection defense (no shell ever parses these), but kit.json is user-edited // and `--provider` is a CLI flag with no upstream allowlist — this grammar is // defense-in-depth so a malformed value fails fast and visibly here rather @@ -744,28 +746,86 @@ export function applyHosts(cfg, cwd = process.cwd()) { // API_PROVIDERS (a narrower, unrelated list — the api-key-only providers this // module can check env keys for): ruflo's own provider set is broader // (openrouter, azure-openai, bedrock, cognitum, …) and ruflo validates the id -// itself; this only rejects shapes no real provider/model id has. -export const PROVIDER_TOKEN_RE = /^[A-Za-z0-9._-]{1,128}$/; +// itself; this only rejects shapes no real provider/model id has. Provider ids +// and model ids deliberately use different grammars: tagged Ollama models and +// vendor-qualified OpenRouter slugs contain ':' and '/' respectively. +export const PROVIDER_ID_RE = /^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$/; +export const PROVIDER_MODEL_RE = /^[A-Za-z0-9][A-Za-z0-9._:/-]{0,255}$/; +export const MIN_RUFLO_PERSISTED_PROVIDER_VERSION = '3.38.8'; +export const DEFAULT_OLLAMA_ENDPOINT = 'http://127.0.0.1:11434'; + +/** Read the same cwd-scoped provider entry Ruflo's ConfigFileManager will use. + * This is read-only preservation logic: when a user already set a custom + * baseUrl directly with Ruflo, ak omits `-e` so Ruflo's upsert keeps it. */ +export function persistedRufloProvider(cwd, providerId, { env = process.env } = {}) { + const projectFiles = [ + path.resolve(cwd, 'claude-flow.config.json'), + path.resolve(cwd, '.claude-flow', 'config.json'), + ]; + const envFile = env.CLAUDE_FLOW_CONFIG ? path.resolve(cwd, env.CLAUDE_FLOW_CONFIG) : null; + const file = [...projectFiles, envFile].find((candidate) => candidate && fs.existsSync(candidate)); + if (!file) return null; + const providers = readJson(file)?.agents?.providers; + if (!Array.isArray(providers)) return null; + return providers.find((entry) => typeof entry?.name === 'string' + && entry.name.toLowerCase() === providerId.toLowerCase()) ?? null; +} -/** Register configured API-key providers with ruflo (keys read from env, never - * passed here). Idempotent — ruflo upserts. Returns {ok, detail}. */ -export async function applyProviders(cfg, cwd = process.cwd()) { +/** Register configured providers with Ruflo (keys read from env, never passed + * here). Idempotent — Ruflo upserts. A fresh Ollama entry receives its standard + * loopback endpoint; a pre-existing custom Ruflo endpoint is preserved. */ +export async function applyProviders(cfg, cwd = process.cwd(), { + haveFn = have, + runner = run, + versionFn = installedVersion, + env = process.env, +} = {}) { const models = cfg.providers?.models ?? []; - if (models.length === 0) return { ok: true, changed: false, detail: 'no API-key providers configured' }; - if (!(await have('ruflo'))) return { ok: false, detail: 'ruflo not on PATH' }; + if (models.length === 0) return { ok: true, changed: false, status: 'ok', detail: 'no providers configured' }; + if (!(await haveFn('ruflo'))) return { ok: false, changed: false, status: 'failed', detail: 'ruflo not on PATH' }; + const rufloVersion = versionFn('ruflo'); + const providerSelectionSupported = !rufloVersion + || cmpVersions(rufloVersion, MIN_RUFLO_PERSISTED_PROVIDER_VERSION) >= 0; const done = []; + let attempted = 0; for (const m of models) { if (!m?.id) continue; - if (!PROVIDER_TOKEN_RE.test(m.id) || (m.model && !PROVIDER_TOKEN_RE.test(m.model))) { + if (typeof m.id !== 'string' || !PROVIDER_ID_RE.test(m.id) + || (m.model && (typeof m.model !== 'string' || !PROVIDER_MODEL_RE.test(m.model)))) { done.push(`${m.id}(invalid)`); continue; } const args = ['providers', 'configure', '-p', m.id]; if (m.model) args.push('-m', m.model); - const r = await run('ruflo', args, { cwd, timeout: 60_000 }); + const existing = persistedRufloProvider(cwd, m.id, { env }); + let endpoint = m.endpoint; + if (endpoint === undefined && m.id.toLowerCase() === 'ollama' && !existing?.baseUrl + && !env.OLLAMA_BASE_URL && !env.OLLAMA_API_KEY) { + endpoint = DEFAULT_OLLAMA_ENDPOINT; + } + if (endpoint !== undefined) { + const validation = typeof endpoint === 'string' + ? validateEndpoint(endpoint) + : { ok: false, reason: 'invalid-url' }; + if (!validation.ok) { + done.push(`${m.id}(invalid endpoint: ${validation.reason})`); + continue; + } + args.push('-e', validation.normalized); + } + attempted += 1; + const r = await runner('ruflo', args, { cwd, timeout: 60_000 }); done.push(`${m.id}${r.code === 0 ? '' : '(failed)'}`); } - return { ok: done.every((d) => !d.includes('failed') && !d.includes('invalid')), changed: true, detail: `configured: ${done.join(', ')}` }; + const ok = done.every((d) => !d.includes('failed') && !d.includes('invalid')); + const compatibility = providerSelectionSupported ? '' + : `; ruflo ${rufloVersion} registers providers but agent_execute needs >=${MIN_RUFLO_PERSISTED_PROVIDER_VERSION} to select persisted config`; + return { + ok, + changed: attempted > 0, + status: !ok ? 'failed' : providerSelectionSupported ? 'ok' : 'degraded', + detail: `registered: ${done.join(', ')}${compatibility}`, + }; } /** Reversible teardown: strip every managed env key from the target file. */ diff --git a/tests/kit/provider-cli.test.mjs b/tests/kit/provider-cli.test.mjs index 4ca1161..c1d2afc 100644 --- a/tests/kit/provider-cli.test.mjs +++ b/tests/kit/provider-cli.test.mjs @@ -13,6 +13,7 @@ import fs from 'node:fs'; import os from 'node:os'; import path from 'node:path'; import { DUAL_ROLE_TIP, JUDGE_BIAS_TIP } from '../../src/lib/providers.mjs'; +import { parseFallback, parseModels } from '../../src/commands/x/host.mjs'; import { defaultHostMap } from '../../src/lib/adapters/index.mjs'; // Tripwire (#137): a spawned `ak x host pick` whose cwd falls back to the test @@ -33,6 +34,20 @@ after(() => { const BIN = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../../bin/agentic-kit.mjs'); +test('provider CLI parsing splits only the provider delimiter', () => { + assert.deepEqual(parseModels('ollama:qwen3.6:27b, openrouter:z-ai/glm-5.2'), [ + { id: 'ollama', model: 'qwen3.6:27b' }, + { id: 'openrouter', model: 'z-ai/glm-5.2' }, + ]); +}); + +test('AQE fallback parsing also preserves provider-native model delimiters', () => { + assert.deepEqual(parseFallback('ollama:qwen3.6:27b; openrouter:z-ai/glm-5.2'), [ + { provider: 'ollama', models: ['qwen3.6:27b'] }, + { provider: 'openrouter', models: ['z-ai/glm-5.2'] }, + ]); +}); + function sandbox({ hosts }) { const home = fs.mkdtempSync(path.join(os.tmpdir(), 'kit-prov-cli-home-')); const cfgDir = path.join(home, '.config', 'agentic-kit'); diff --git a/tests/kit/provider-credentials.test.mjs b/tests/kit/provider-credentials.test.mjs index 696d37b..78a5adc 100644 --- a/tests/kit/provider-credentials.test.mjs +++ b/tests/kit/provider-credentials.test.mjs @@ -13,7 +13,7 @@ import path from 'node:path'; import { loadKitConfig } from '../../src/lib/config.mjs'; import { AQE_CONSTRUCTIBLE_PROVIDERS } from '../../src/lib/routing.mjs'; import { - AQE_PROVIDER_TYPES, AQE_PROVIDER_CREDENTIALS, aqeProviderCredential, + API_PROVIDERS, AQE_PROVIDER_TYPES, AQE_PROVIDER_CREDENTIALS, aqeProviderCredential, credentialGaps, detectAqeProviders, fallbackSource, applyAqeRouter, aqeRouterFile, collectIntegrationFacts, } from '../../src/lib/providers.mjs'; @@ -99,9 +99,9 @@ test('every credential descriptor declares exactly one credential mechanism', () }); test('openrouter — the one credentialed provider on the reported machine — has a descriptor', () => { - // It was absent from API_PROVIDERS entirely, so the only live key on the box - // was invisible to every ak surface while keyless `openai` displayed as configured. assert.ok(AQE_PROVIDER_CREDENTIALS.openrouter?.keyEnv?.includes('OPENROUTER_API_KEY')); + assert.ok(API_PROVIDERS.some((provider) => provider.id === 'openrouter'), + 'Ruflo provider status must not hide the upstream direct-execution provider'); }); // ── aqeProviderCredential ─────────────────────────────────────────────────── diff --git a/tests/kit/provider-dashboard-copy.test.mjs b/tests/kit/provider-dashboard-copy.test.mjs new file mode 100644 index 0000000..1cb4ab7 --- /dev/null +++ b/tests/kit/provider-dashboard-copy.test.mjs @@ -0,0 +1,10 @@ +import { test } from 'node:test'; +import assert from 'node:assert/strict'; +import { renderPage } from '../../src/lib/dashboard/page.mjs'; + +test('Providers distinguishes registration, selection, and observed execution', () => { + const html = renderPage({ name: 'agentic-kit', version: 'test' }); + assert.match(html, /A registered provider is eligible configuration/); + assert.match(html, /Direct Ruflo agents must select/); + assert.match(html, /independently observed provider and model evidence/); +}); diff --git a/tests/kit/providers.test.mjs b/tests/kit/providers.test.mjs index fd6059a..4a34358 100644 --- a/tests/kit/providers.test.mjs +++ b/tests/kit/providers.test.mjs @@ -10,7 +10,7 @@ import { HOSTS, installHost, applyAqeRouter, undoAqeRouter, aqeRouterFile, AQE_PROVIDER_TYPES, bothHostsEnabled, DUAL_ROLE_TIP, suggestedFallbackFor, AQE_FALLBACK_CODEX_SUGGESTION, - PROVIDER_TOKEN_RE, seedActivityRoutesIfMultiHost, + PROVIDER_ID_RE, PROVIDER_MODEL_RE, applyProviders, seedActivityRoutesIfMultiHost, } from '../../src/lib/providers.mjs'; import * as paths from '../../src/lib/paths.mjs'; import { managedHostIds, routableHostIds, HOST_REGISTRY } from '../../src/lib/adapters/index.mjs'; @@ -21,17 +21,146 @@ import { managedHostIds, routableHostIds, HOST_REGISTRY } from '../../src/lib/ad // that rejects a hostile/malformed kit.json entry before it ever reaches // run('ruflo', …) — pinned directly since applyProviders itself needs a real // `ruflo` binary on PATH to exercise end-to-end. -test('PROVIDER_TOKEN_RE accepts real-shaped provider/model ids', () => { - for (const ok of ['openai', 'claude-opus-5', 'gpt-5.6', 'azure-openai', 'z-ai/glm-5.2'.split('/')[1]]) { - assert.ok(PROVIDER_TOKEN_RE.test(ok), `expected to accept ${JSON.stringify(ok)}`); +test('provider id/model grammars accept real Ruflo shapes without conflating the two', () => { + for (const ok of ['openai', 'openrouter', 'azure-openai']) { + assert.ok(PROVIDER_ID_RE.test(ok), `expected provider id ${JSON.stringify(ok)}`); + } + for (const ok of ['claude-opus-5', 'gpt-5.6', 'qwen3.6:27b', 'z-ai/glm-5.2']) { + assert.ok(PROVIDER_MODEL_RE.test(ok), `expected model id ${JSON.stringify(ok)}`); } }); -test('PROVIDER_TOKEN_RE rejects shell metacharacters and whitespace', () => { +test('provider id/model grammars reject shell metacharacters and whitespace', () => { for (const bad of [ 'gpt-5.6 & calc.exe', 'openai; rm -rf /', 'model`whoami`', 'a|b', 'a$(b)', - 'has space', '', 'x'.repeat(129), + 'has space', '', ]) { - assert.ok(!PROVIDER_TOKEN_RE.test(bad), `expected to reject ${JSON.stringify(bad)}`); + assert.ok(!PROVIDER_ID_RE.test(bad), `provider id accepted ${JSON.stringify(bad)}`); + assert.ok(!PROVIDER_MODEL_RE.test(bad), `model id accepted ${JSON.stringify(bad)}`); + } + assert.ok(!PROVIDER_ID_RE.test('x'.repeat(129)), 'provider id length is bounded'); + assert.ok(!PROVIDER_MODEL_RE.test('x'.repeat(257)), 'model id length is bounded'); + assert.ok(!PROVIDER_ID_RE.test('z-ai/glm-5.2'), 'provider ids cannot contain model separators'); +}); + +test('applyProviders preserves tagged and vendor-qualified model ids', async () => { + const cwd = tmpProject(); + const calls = []; + try { + const cfg = defaultCfg(); + cfg.providers.models = [ + { id: 'ollama', model: 'qwen3.6:27b' }, + { id: 'openrouter', model: 'z-ai/glm-5.2' }, + ]; + const result = await applyProviders(cfg, cwd, { + haveFn: async () => true, + runner: async (bin, args, opts) => { calls.push({ bin, args, opts }); return { code: 0 }; }, + versionFn: () => '3.38.20', + }); + assert.equal(result.ok, true); + assert.equal(result.status, 'ok'); + assert.deepEqual(calls.map(({ args }) => args), [ + ['providers', 'configure', '-p', 'ollama', '-m', 'qwen3.6:27b', '-e', 'http://127.0.0.1:11434'], + ['providers', 'configure', '-p', 'openrouter', '-m', 'z-ai/glm-5.2'], + ]); + } finally { + rm(cwd); + } +}); + +test('applyProviders preserves an existing Ruflo endpoint unless kit.json owns one', async () => { + const cwd = tmpProject(); + const calls = []; + try { + fs.writeFileSync(path.join(cwd, 'claude-flow.config.json'), JSON.stringify({ + agents: { providers: [{ name: 'ollama', enabled: true, baseUrl: 'http://localhost:22434' }] }, + })); + const cfg = defaultCfg(); + cfg.providers.models = [{ id: 'ollama', model: 'qwen3.6:27b' }]; + await applyProviders(cfg, cwd, { + haveFn: async () => true, + runner: async (_bin, args) => { calls.push(args); return { code: 0 }; }, + versionFn: () => '3.38.20', + }); + assert.deepEqual(calls[0], ['providers', 'configure', '-p', 'ollama', '-m', 'qwen3.6:27b']); + + cfg.providers.models[0].endpoint = 'http://127.0.0.1:33434'; + await applyProviders(cfg, cwd, { + haveFn: async () => true, + runner: async (_bin, args) => { calls.push(args); return { code: 0 }; }, + versionFn: () => '3.38.20', + }); + assert.deepEqual(calls[1], [ + 'providers', 'configure', '-p', 'ollama', '-m', 'qwen3.6:27b', '-e', 'http://127.0.0.1:33434', + ]); + } finally { + rm(cwd); + } +}); + +test('applyProviders does not replace Ollama Cloud or env endpoint selection with loopback', async () => { + const cwd = tmpProject(); + const calls = []; + try { + const cfg = defaultCfg(); + cfg.providers.models = [{ id: 'ollama', model: 'gpt-oss:120b-cloud' }]; + for (const env of [ + { OLLAMA_API_KEY: 'present' }, + { OLLAMA_BASE_URL: 'http://127.0.0.1:22434' }, + ]) { + await applyProviders(cfg, cwd, { + env, + haveFn: async () => true, + runner: async (_bin, args) => { calls.push(args); return { code: 0 }; }, + versionFn: () => '3.38.20', + }); + } + assert.deepEqual(calls, [ + ['providers', 'configure', '-p', 'ollama', '-m', 'gpt-oss:120b-cloud'], + ['providers', 'configure', '-p', 'ollama', '-m', 'gpt-oss:120b-cloud'], + ]); + } finally { + rm(cwd); + } +}); + +test('applyProviders rejects unsafe endpoints before invoking Ruflo', async () => { + const cwd = tmpProject(); + let called = false; + try { + const cfg = defaultCfg(); + cfg.providers.models = [{ + id: 'ollama', model: 'qwen3.6:27b', endpoint: 'http://example.com:11434', + }]; + const result = await applyProviders(cfg, cwd, { + haveFn: async () => true, + runner: async () => { called = true; return { code: 0 }; }, + versionFn: () => '3.38.20', + }); + assert.equal(called, false); + assert.equal(result.ok, false); + assert.equal(result.status, 'failed'); + assert.match(result.detail, /ollama\(invalid endpoint: remote-http\)/); + } finally { + rm(cwd); + } +}); + +test('applyProviders warns when persisted provider selection predates Ruflo 3.38.8', async () => { + const cwd = tmpProject(); + try { + const cfg = defaultCfg(); + cfg.providers.models = [{ id: 'openrouter', model: 'z-ai/glm-5.2' }]; + const result = await applyProviders(cfg, cwd, { + haveFn: async () => true, + runner: async () => ({ code: 0 }), + versionFn: () => '3.38.7', + }); + assert.equal(result.ok, true, 'registration still succeeds'); + assert.equal(result.status, 'degraded'); + assert.match(result.detail, /ruflo 3\.38\.7/); + assert.match(result.detail, />=3\.38\.8/); + } finally { + rm(cwd); } }); diff --git a/tests/kit/status-command.test.mjs b/tests/kit/status-command.test.mjs index f0c5093..7026cca 100644 --- a/tests/kit/status-command.test.mjs +++ b/tests/kit/status-command.test.mjs @@ -74,6 +74,18 @@ test('every row carries the documented shape', async () => { } }); +test('ruflo provider intent never claims registration alone is routed execution', async () => { + seedHome(offlineKitConfig({ + providers: { models: [{ id: 'openrouter', model: 'z-ai/glm-5.2' }] }, + })); + const providerRows = rowsFor(await collect(), 'providers'); + const intent = providerRows.find((r) => r.message.startsWith('ruflo provider intent:')); + assert.ok(intent, `expected a ruflo intent row: ${providerRows.map((r) => r.message)}`); + assert.match(intent.message, /openrouter:z-ai\/glm-5\.2/); + assert.match(intent.message, /direct agents must select provider \+ model/); + assert.doesNotMatch(intent.message, /routable|executed successfully/); +}); + test('a missing global ruflo is a FAIL with a fix; a present one is ok', async () => { seedHome(); const withRuflo = await collect();