Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions en/docs/ai-gateway/next/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ A gateway for managing and securing AI traffic, including Large Language Model (
## Quick Start

- [LLM Quick Start Guide](quick-start-guide.md) — Set up the gateway and route traffic to LLM providers like OpenAI
- [MCP Quick Start Guide](mcp-proxy/quick-start-guide.md) — Set up the gateway and route traffic to MCP servers
- [MCP Quick Start Guide](mcp-proxy/create-an-mcp-proxy.md) — Set up the gateway and route traffic to MCP servers

## Key Concepts

Expand Down Expand Up @@ -49,7 +49,7 @@ An LLM Provider represents a connection to an AI backend service such as OpenAI,

Once configured, the LLM Provider allows traffic to flow through the gateway to the AI backend.

To connect the gateway to AWS Bedrock, see [Configure an AWS Bedrock LLM Provider](llm-proxy/configure-aws-bedrock-provider.md). The guide covers both Bedrock bearer API keys and AWS Signature Version 4 (SigV4) authentication.
To connect the gateway to AWS Bedrock, see [Configure an AWS Bedrock LLM Provider](connect-llm-providers/configure-aws-bedrock-provider.md). The guide covers both Bedrock bearer API keys and AWS Signature Version 4 (SigV4) authentication.

### LLM Proxy

Expand Down
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
---
title: "Configure an AWS Bedrock Large Language Model provider"
description: "Connect API Platform AI Gateway to AWS Bedrock using a bearer API key or AWS Signature Version 4 authentication, then invoke a model through the gateway."
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/configure-aws-bedrock-provider/
md_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/configure-aws-bedrock-provider.md
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/connect-llm-providers/configure-aws-bedrock-provider/
md_url: https://wso2.com/api-platform/docs/ai-gateway/connect-llm-providers/configure-aws-bedrock-provider.md
tags:
- ai-gateway
- llm
- aws-bedrock
- authentication
author: WSO2 API Platform Documentation Team
last_updated: 2026-08-04
last_updated: 2026-08-11
content_type: "guide"
---

Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
---
title: "LLM Provider Templates"
description: "Reference for LLM Provider Templates in API Platform AI Gateway, covering built-in templates for OpenAI, Anthropic, Gemini, and more."
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/llm-templates/
md_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/llm-templates.md
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/connect-llm-providers/llm-templates/
md_url: https://wso2.com/api-platform/docs/ai-gateway/connect-llm-providers/llm-templates.md
tags:
- ai-gateway
- llm
- reference
author: WSO2 API Platform Documentation Team
last_updated: 2026-08-07
last_updated: 2026-08-11
content_type: "reference"
---

Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
---
title: "Secure the management API"
description: "Configure Basic Auth or JWT/IDP authentication and role-based authorization for the AI Gateway Controller REST API."
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/gateway-controller-management-api/authentication/
md_url: https://wso2.com/api-platform/docs/ai-gateway/gateway-controller-management-api/authentication.md
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/control-access/secure-the-management-api/
md_url: https://wso2.com/api-platform/docs/ai-gateway/control-access/secure-the-management-api.md
tags:
- ai-gateway
- security
- authentication
author: WSO2 API Platform Documentation Team
last_updated: 2026-08-10
last_updated: 2026-08-11
content_type: "how-to"
---

Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
---
title: "AWS Bedrock Guardrail"
description: "Validate LLM request and response content against AWS Bedrock Guardrails for content filtering, topic detection, and PII masking."
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/guardrails/aws-bedrock-guardrail/
md_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/guardrails/aws-bedrock-guardrail.md
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/aws-bedrock-guardrail/
md_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/aws-bedrock-guardrail.md
tags:
- ai-gateway
- guardrails
- aws
author: WSO2 API Platform Documentation Team
last_updated: 2026-08-07
last_updated: 2026-08-11
content_type: "reference"
---

Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
---
title: "Azure Content Safety"
description: "Validate LLM request and response content against Azure Content Safety API, detecting hate speech, sexual content, and violence."
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/guardrails/azure-content-safety/
md_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/guardrails/azure-content-safety.md
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/azure-content-safety/
md_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/azure-content-safety.md
tags:
- ai-gateway
- guardrails
- azure
author: WSO2 API Platform Documentation Team
last_updated: 2026-08-07
last_updated: 2026-08-11
content_type: "reference"
---

Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
---
title: "Content Length Guardrail"
description: "Enforce minimum and maximum byte length limits on LLM request and response payloads using the Content Length guardrail."
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/guardrails/content-length/
md_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/guardrails/content-length.md
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/content-length/
md_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/content-length.md
tags:
- ai-gateway
- guardrails
- policies
author: WSO2 API Platform Documentation Team
last_updated: 2026-08-07
last_updated: 2026-08-11
content_type: "reference"
---

Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
title: "Guardrail execution order"
description: "How guardrails execute across phases in the AI Gateway dual-hop model: the LLM Proxy chain runs before the LLM Provider chain on request, and in reverse on response."
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/guardrails/execution-order/
md_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/guardrails/execution-order.md
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/execution-order/
md_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/execution-order.md
tags:
- ai-gateway
- guardrails
Expand All @@ -18,7 +18,7 @@ content_type: "concept"

The AI Gateway routes every request through two policy chains in sequence: the **LLM Proxy chain** and the **LLM Provider chain**. Each chain runs the same four execution phases, but the order in which the two chains execute differs between the request and response paths.

This document covers the dual-hop execution model. For foundational concepts — within-chain policy ordering, streaming mode, and short-circuit behavior — see [Policy execution order](../../../../../api-gateway/next/policies/policy-execution-order.md). The same rules apply to each chain individually.
This document covers the dual-hop execution model. For foundational concepts — within-chain policy ordering, streaming mode, and short-circuit behavior — see [Policy execution order](../../../api-gateway/next/policies/policy-execution-order.md). The same rules apply to each chain individually.

## The two chains

Expand Down Expand Up @@ -80,7 +80,7 @@ Client

The following diagram shows how requests pass through the LLM Proxy chain and LLM Provider chain in sequence, and how responses return through both chains in reverse order:

![Dual-hop guardrail execution diagram showing request flowing through LLM Proxy chain then LLM Provider chain in forward order across request phases, and response flowing in reverse order through LLM Provider chain then LLM Proxy chain back to the client](../../../../assets/img/api-gateway/gateway-policy-execution-order-llm-provider-proxy.png)
![Diagram of dual-hop guardrail execution: a request traverses the LLM Proxy chain then the LLM Provider chain, and the response returns in reverse order](../../../assets/img/api-gateway/gateway-policy-execution-order-llm-provider-proxy.png)
<!-- image source: https://docs.google.com/drawings/d/1Hvnv_89Hd0T5JQr1tg0DLIQTHWDn2szgj13NDhjyccM/edit?usp=sharing -->

This mirrors the request wrapping at the chain level: the LLM Proxy wraps the LLM Provider on the way in, so on the way back the inner chain (Provider) unwinds first, then the outer chain (Proxy).
Expand All @@ -99,10 +99,10 @@ When an LLM returns a streaming response (such as SSE from a chat completion end

The following diagram shows how streaming body chunks flow through the policy chain at each hop:

![Streaming policy chain execution order diagram showing request body chunks flowing through the policy chain via OnRequestBodyChunk in forward order, and response chunks flowing in reverse order via OnResponseBodyChunk](../../../../assets/img/api-gateway/gateway-policy-execution-order-streaming.png)
![Diagram of streaming policy chain execution order, with request body chunks flowing forward through the chain and response chunks flowing in reverse](../../../assets/img/api-gateway/gateway-policy-execution-order-streaming.png)
<!-- image source: https://docs.google.com/drawings/d/1HyceTR0htslHoBm3xintrMkZo0jqKfHhnhAcctGgUDs/edit?usp=sharing -->

## Related topics

- [Guardrails overview](overview.md)
- [Policy execution order](../../../../../api-gateway/next/policies/policy-execution-order.md) — within-chain ordering, streaming, and short-circuit rules that apply to each chain individually
- [Policy execution order](../../../api-gateway/next/policies/policy-execution-order.md) — within-chain ordering, streaming, and short-circuit rules that apply to each chain individually
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
---
title: "JSON Schema Guardrail"
description: "Validate LLM request and response body content against a JSON Schema definition to enforce structured data formats."
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/guardrails/json-schema/
md_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/guardrails/json-schema.md
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/json-schema/
md_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/json-schema.md
tags:
- ai-gateway
- guardrails
- policies
author: WSO2 API Platform Documentation Team
last_updated: 2026-08-07
last_updated: 2026-08-11
content_type: "reference"
---

Expand Down
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
---
title: "Guardrails overview"
description: "Overview of AI Gateway guardrails: LLM-aware policies for content filtering, safety, and compliance that run in the LLM Proxy request and response pipeline."
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/guardrails/overview/
md_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/guardrails/overview.md
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/overview/
md_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/overview.md
tags:
- ai-gateway
- guardrails
- llm-proxy
author: WSO2 API Platform Documentation Team
last_updated: 2026-08-10
last_updated: 2026-08-11
content_type: "concept"
---

# Guardrails overview

Guardrails are policies that run in the LLM Proxy's request and response pipeline to validate, filter, or transform content before it reaches an LLM or is returned to the client.

Guardrails use the same underlying policy engine as [API Gateway policies](../../../../../api-gateway/next/policies/overview.md). Each guardrail declares which execution phases it participates in, and the engine calls the appropriate hook for each phase.
Guardrails use the same underlying policy engine as [API Gateway policies](../../../api-gateway/next/policies/overview.md). Each guardrail declares which execution phases it participates in, and the engine calls the appropriate hook for each phase.
Comment thread
coderabbitai[bot] marked this conversation as resolved.

## What guardrails do

Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
---
title: "PII Masking Regex Guardrail"
description: "Mask or redact personally identifiable information from LLM request and response bodies using configurable regex patterns."
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/guardrails/pii-masking-regex/
md_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/guardrails/pii-masking-regex.md
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/pii-masking-regex/
md_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/pii-masking-regex.md
tags:
- ai-gateway
- guardrails
- security
author: WSO2 API Platform Documentation Team
last_updated: 2026-08-07
last_updated: 2026-08-11
content_type: "reference"
---

Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
---
title: "Prompt Decorator"
description: "Prepend or append custom content to LLM prompts, injecting system messages or standard instructions before requests reach the model."
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/prompt-management/prompt-decorator/
md_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/prompt-management/prompt-decorator.md
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/prompt-decorator/
md_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/prompt-decorator.md
tags:
- ai-gateway
- llm
- prompt-management
author: WSO2 API Platform Documentation Team
last_updated: 2026-08-07
last_updated: 2026-08-11
content_type: "reference"
---

Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
---
title: "Prompt Template"
description: "Replace 'template://' URI patterns in LLM request payloads with predefined, parameterized prompt templates."
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/prompt-management/prompt-template/
md_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/prompt-management/prompt-template.md
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/prompt-template/
md_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/prompt-template.md
tags:
- ai-gateway
- llm
- prompt-management
author: WSO2 API Platform Documentation Team
last_updated: 2026-08-07
last_updated: 2026-08-11
content_type: "reference"
---

Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
---
title: "Regex Guardrail"
description: "Validate LLM request and response content against regular expression patterns to enforce formats or detect prohibited content."
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/guardrails/regex/
md_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/guardrails/regex.md
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/regex/
md_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/regex.md
tags:
- ai-gateway
- guardrails
- policies
author: WSO2 API Platform Documentation Team
last_updated: 2026-08-07
last_updated: 2026-08-11
content_type: "reference"
---

Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
---
title: "Semantic Prompt Guardrail"
description: "Block semantically similar prompts using embedding-based similarity matching against configured allow and deny phrase lists."
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/guardrails/semantic-prompt-guard/
md_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/guardrails/semantic-prompt-guard.md
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/semantic-prompt-guard/
md_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/semantic-prompt-guard.md
tags:
- ai-gateway
- guardrails
- llm
author: WSO2 API Platform Documentation Team
last_updated: 2026-08-07
last_updated: 2026-08-11
content_type: "reference"
---

Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
---
title: "Sentence Count Guardrail"
description: "Enforce minimum and maximum sentence count limits on LLM request and response content using the Sentence Count guardrail."
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/guardrails/sentence-count/
md_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/guardrails/sentence-count.md
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/sentence-count/
md_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/sentence-count.md
tags:
- ai-gateway
- guardrails
- policies
author: WSO2 API Platform Documentation Team
last_updated: 2026-08-07
last_updated: 2026-08-11
content_type: "reference"
---

Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
---
title: "URL Guardrail"
description: "Validate URLs found in LLM request and response content by checking reachability through DNS resolution or HTTP HEAD requests."
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/guardrails/url/
md_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/guardrails/url.md
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/url/
md_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/url.md
tags:
- ai-gateway
- guardrails
- policies
author: WSO2 API Platform Documentation Team
last_updated: 2026-08-07
last_updated: 2026-08-11
content_type: "reference"
---

Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
---
title: "Word Count Guardrail"
description: "Enforce minimum and maximum word count limits on LLM request and response content using the Word Count guardrail."
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/guardrails/word-count/
md_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/guardrails/word-count.md
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/word-count/
md_url: https://wso2.com/api-platform/docs/ai-gateway/control-content/word-count.md
tags:
- ai-gateway
- guardrails
- policies
author: WSO2 API Platform Documentation Team
last_updated: 2026-08-07
last_updated: 2026-08-11
content_type: "reference"
---

Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
---
title: "Model Round Robin"
description: "Distribute LLM requests evenly across multiple AI models in a cyclic round-robin pattern with automatic failure suspension."
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/load-balancing/model-round-robin/
md_url: https://wso2.com/api-platform/docs/ai-gateway/llm-proxy/load-balancing/model-round-robin.md
canonical_url: https://wso2.com/api-platform/docs/ai-gateway/control-cost-and-traffic/model-round-robin/
md_url: https://wso2.com/api-platform/docs/ai-gateway/control-cost-and-traffic/model-round-robin.md
tags:
- ai-gateway
- llm
- load-balancing
author: WSO2 API Platform Documentation Team
last_updated: 2026-07-30
last_updated: 2026-08-11
content_type: "reference"
---

Expand Down Expand Up @@ -85,7 +85,7 @@ operationPolicies:
suspendDuration: 60
```

The entry without `provider` uses the proxy's primary provider. The other entries route to providers declared under `spec.additionalProviders`. See [Multi-Provider Routing for LLM Proxies](../multi-provider-routing.md) for provider, authentication, transformer, and alias configuration.
The entry without `provider` uses the proxy's primary provider. The other entries route to providers declared under `spec.additionalProviders`. See [Multi-Provider Routing for LLM Proxies](../expose-llms/multi-provider-routing.md) for provider, authentication, transformer, and alias configuration.

### Example 1: Basic Round Robin with Payload-based Model

Expand Down
Loading