Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 92 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -213,6 +213,97 @@ parses the gateway's `x-routeplane-*` response headers: `provider`, `trace_id`,
`request_id`, `cache`, `guardrails`, `hedged`, `shed`, `budget_remaining`,
`budget_warning`, `compliance_warning`, `pii_masked`, `idempotent_replayed`.

## Prompt management

Managed prompt templates are fetched, rendered, and run through the ordinary
chat pipeline — so residency routing, guardrails, caching, and budgets all still
apply to a prompt completion.

```python
# The stored version, no render, no upstream call.
version = client.prompts.get("welcome-v2")

# Render only. `missing="empty"` substitutes nothing for an unsupplied variable
# instead of failing; `cohort` is the sticky key for an A/B-tested prompt.
rendered = client.prompts.render(
"welcome-v2", variables={"name": "Rohit"}, missing="empty", cohort="user-7"
)

# Render and run it. Any extra keyword becomes a chat-request override and beats
# the version's stored defaults.
completion = client.prompts.complete(
"welcome-v2",
variables={"name": "Rohit"},
model="gpt-4o-mini",
temperature=0.2,
)
```

| Method | Endpoint |
| --- | --- |
| `prompts.get(ref)` | `GET /v1/prompts/{ref}` |
| `prompts.render(ref, …)` | `POST /v1/prompts/{ref}/render` |
| `prompts.complete(ref, …)` | `POST /v1/prompts/{ref}/completions` |

Requires the `PromptRegistry` entitlement — otherwise 403 `feature_not_entitled`,
or `feature_not_released` if it is entitled but still behind a rollout holdback.

## Agentic security (MCP gateway)

The MCP gateway is a **default-deny policy boundary** for agent tool calls. A
grant for one server never authorizes the same tool name on another, tool
arguments are checked against an SSRF egress guard, and tool results are
inspected on the return leg before they re-enter the model's context.

A deny is the system working, so policy verdicts come back as values rather than
exceptions — `authorize_tool_call`, `inspect_result`, `sampling_evaluate`, and
`run_step` return a typed `Decision` or `RunStep` for both outcomes. Everything
else raises on a non-2xx as usual.

```python
mcp = client.mcp_security

# Account one iteration against the run's ceiling / budget / kill switch.
step = mcp.run_step(run_id="run-001", agent_id="support-agent", cost_micro_usd=1200)
if not step.should_continue:
raise SystemExit(f"halted after {step.iterations}: {step.reason}")

# Authorize the specific (server, tool) call before the agent makes it.
decision = mcp.authorize_tool_call(
agent_id="support-agent",
server="filesystem",
tool="fetch_document",
arguments={"url": "https://docs.example.test/policy.pdf"},
)
if not decision.allowed:
print(decision.reason, decision.status_code) # 429 also has retry_after_ms

# Screen what came back before it reaches the model.
verdict = mcp.inspect_result(content=tool_result)
```

| Method | Endpoint |
| --- | --- |
| `mcp_security.authorize_tool_call(…)` | `POST /v1/mcp/tool-call/authorize` |
| `mcp_security.inspect_result(…)` | `POST /v1/mcp/tool-result/inspect` |
| `mcp_security.run_step(…)` | `POST /v1/mcp/run/step` |
| `mcp_security.sampling_evaluate(…)` | `POST /v1/mcp/sampling/evaluate` |
| `mcp_security.list_runs()` | `GET /v1/mcp/runs` |
| `mcp_security.security_events()` | `GET /v1/mcp/security/events` |
| `mcp_security.hitl.approve(…)` / `.deny(…)` | `POST /v1/mcp/hitl/{approve,deny}` |
| `mcp_security.hitl.status(…)` / `.pending()` | `GET /v1/mcp/hitl/status/{id}`, `/pending` |
| `mcp_security.receipts.issue(…)` / `.verify(…)` | `POST /v1/mcp/receipt/{issue,verify}` |
| `mcp_security.anomaly.status(…)` / `.clear(…)` | `GET /v1/mcp/anomaly/status/{id}`, `POST /clear` |

Requires the `AgenticSecurity` entitlement. A tenant without it is not told the
surface exists: these routes answer **404**, not 403. So an
`httpx.HTTPStatusError` for 404 here usually means *not entitled* rather than
*wrong path*.

`agent_id` is optional wherever a gateway key is bound to an agent identity —
the binding supplies it, and a value that *disagrees* with the binding is denied
rather than trusted.

## Examples

Runnable scripts live in [`examples/`](examples):
Expand All @@ -224,6 +315,7 @@ Runnable scripts live in [`examples/`](examples):
| [`streaming_with_meta.py`](examples/streaming_with_meta.py) | Streaming with the gateway's decision metadata |
| [`metadata.py`](examples/metadata.py) | `create_with_meta` — completion plus typed `RouteplaneMeta` |
| [`resources.py`](examples/resources.py) | Non-OpenAI surfaces — status, logs, FinOps, prompts, cache |
| [`agentic_security.py`](examples/agentic_security.py) | Mediating an agent tool loop through the MCP gateway |
| [`langchain_integration.py`](examples/langchain_integration.py) | LangChain (`ChatOpenAI`) |
| [`llamaindex_integration.py`](examples/llamaindex_integration.py) | LlamaIndex (`llama-index-llms-openai`) |
| [`crewai_integration.py`](examples/crewai_integration.py) | CrewAI (`LLM`) |
Expand Down
62 changes: 62 additions & 0 deletions examples/agentic_security.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
"""Mediating an agent's tool loop through the MCP gateway.

Every route used here needs the ``AgenticSecurity`` entitlement. Without it the
gateway answers 404 rather than 403 — it does not reveal that the surface
exists — so an ``httpx.HTTPStatusError`` for 404 means "not entitled".

pip install routeplane
python examples/agentic_security.py
"""

from routeplane import Routeplane

rp = Routeplane(api_key="rp_live_...")
mcp = rp.mcp_security

RUN_ID = "run-2026-07-27-001"
AGENT_ID = "support-agent"

# 1. Account the iteration first. The run carries an iteration ceiling, a cost
# budget, and a kill switch; a "stop" means halt the loop, not retry it.
step = mcp.run_step(run_id=RUN_ID, agent_id=AGENT_ID, cost_micro_usd=1200)
if not step.should_continue:
raise SystemExit(f"run halted after {step.iterations} iterations: {step.reason}")

# 2. Authorize the specific tool call. Default-deny: the agent needs a grant for
# this exact (server, tool) pair, and every URL in the arguments has to clear
# the SSRF egress guard.
decision = mcp.authorize_tool_call(
agent_id=AGENT_ID,
server="filesystem",
tool="fetch_document",
arguments={"url": "https://docs.example.test/policy.pdf"},
run_id=RUN_ID,
)
if not decision.allowed:
if decision.status_code == 429:
raise SystemExit(f"quota exhausted, retry in {decision.retry_after_ms}ms")
raise SystemExit(f"tool call refused: {decision.reason}")

tool_result = "...whatever the MCP server returned..."

# 3. Screen the result before it re-enters the model's context. This is where an
# indirect prompt injection or a leaked secret gets caught.
verdict = mcp.inspect_result(content=tool_result)
if not verdict.allowed:
raise SystemExit(f"result withheld: {verdict.reason}")

# 4. Bind the whole action into a signed, chained receipt. Arguments are reduced
# to a values-free shape and the result to a digest — neither is stored.
receipt = mcp.receipts.issue(
run_id=RUN_ID,
agent_id=AGENT_ID,
server="filesystem",
tool="fetch_document",
decision="allowed",
result=tool_result,
)
print(f"receipt: {receipt}")

# Operator views: what the gateway has been refusing, and what it has been running.
print(f"recent denials: {mcp.security_events()}")
print(f"recent runs: {mcp.list_runs()}")
3 changes: 3 additions & 0 deletions src/routeplane/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
from .client import Routeplane
from .headers import headers
from .meta import RouteplaneMeta
from .resources import Decision, RunStep

__all__ = [
"Routeplane",
Expand All @@ -23,5 +24,7 @@
"AsyncRouteplaneStream",
"headers",
"RouteplaneMeta",
"Decision",
"RunStep",
"__version__",
]
4 changes: 3 additions & 1 deletion src/routeplane/resources/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
from .feedback import FeedbackResource
from .finops import FinopsResource
from .logs import LogsResource
from .mcp import McpResource
from .mcp import Decision, McpResource, RunStep
from .models import ModelsResource
from .prompts import PromptsResource
from .providers import ProvidersResource
Expand All @@ -22,10 +22,12 @@
"BaseResource",
"AnalyticsResource",
"CacheResource",
"Decision",
"FeedbackResource",
"FinopsResource",
"LogsResource",
"McpResource",
"RunStep",
"ModelsResource",
"PromptsResource",
"ProvidersResource",
Expand Down
16 changes: 14 additions & 2 deletions src/routeplane/resources/_base.py
Original file line number Diff line number Diff line change
Expand Up @@ -82,10 +82,12 @@ def _get(
*,
params: Optional[Mapping[str, Any]] = None,
headers: Optional[Mapping[str, str]] = None,
expect: tuple[int, ...] = (),
) -> httpx.Response:
merged = {**self._auth_headers, **dict(headers or {})}
response = self._client.get(self._url(path), params=params, headers=merged)
response.raise_for_status()
if response.status_code not in expect:
response.raise_for_status()
return response

def _post(
Expand All @@ -94,10 +96,20 @@ def _post(
*,
json: Any = None,
headers: Optional[Mapping[str, str]] = None,
expect: tuple[int, ...] = (),
) -> httpx.Response:
"""POST and raise on error.

``expect`` lists additional status codes to return instead of raising —
for endpoints where a non-2xx is a normal answer rather than a failure.
The MCP policy gates are the case that needs it: a default-deny verdict
arrives as a structured ``422`` body, so raising would turn the single
most common agentic-security outcome into an exception.
"""
merged = {**self._auth_headers, **dict(headers or {})}
response = self._client.post(self._url(path), json=json, headers=merged)
response.raise_for_status()
if response.status_code not in expect:
response.raise_for_status()
return response

def _delete(
Expand Down
Loading
Loading