Skip to content

docs(skills): align plugin skills with APISIX 3.18.0 behavior #61

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
kayx23 merged 4 commits into from
Aug 25, 2026
Merged

docs(skills): align plugin skills with APISIX 3.18.0 behavior #61

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
b2530e1
docs(skills): align plugin skills with APISIX 3.18.0 behavior
kayx23 Aug 25, 2026
b473d35
docs(skills): keep skill version at 1.0.0 with the a6 CLI
kayx23 Aug 25, 2026
d4fc801
docs(skills): write complete search-length skill descriptions
kayx23 Aug 25, 2026
2c69571
docs(skills): correct APISIX 3.18 behavior details
kayx23 Aug 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 43 additions & 18 deletions skills/a6-plugin-ai-content-moderation/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
---
name: a6-plugin-ai-content-moderation
description: >-
Skill for configuring APISIX AI content moderation plugins via the a6 CLI.
Covers both ai-aws-content-moderation (AWS Comprehend, request-only) and
ai-aliyun-content-moderation (Aliyun, request + response with streaming),
toxicity thresholds, category filtering, and integration with ai-proxy.
Skill for configuring APISIX AWS and Aliyun AI content moderation via the
a6 CLI. Covers request and response checks, streaming, deny_code, and
ai-proxy.
version: "1.0.0"
author: Apache APISIX Contributors
license: Apache-2.0
Expand All @@ -29,15 +28,21 @@ in LLM requests and responses:

| Plugin | Provider | Request | Response | Streaming |
|--------|----------|---------|----------|-----------|
| `ai-aws-content-moderation` | AWS Comprehend | ✅ | ❌ | ❌ |
| `ai-aws-content-moderation` | AWS Comprehend | ✅ | ✅ | ✅ |
| `ai-aliyun-content-moderation` | Aliyun Moderation Plus | ✅ | ✅ | ✅ |

Both must be used alongside `ai-proxy` or `ai-proxy-multi`.
Both must be used alongside `ai-proxy` or `ai-proxy-multi` so the plugin can
moderate decoded AI content. AWS response and streaming moderation, `fail_mode`,
role selection, and the default `deny_code` of `200` apply from APISIX 3.18.0.
For field tables and protocol details, see
https://docs.api7.ai/hub/ai-aws-content-moderation and
https://docs.api7.ai/hub/ai-aliyun-content-moderation.

## When to Use

- Block toxic, hateful, or sexual content before it reaches the LLM
- Filter harmful LLM responses before they reach clients (Aliyun only)
- Moderate harmful LLM responses. Non-streaming responses can be denied before
delivery; streaming checks cannot retract chunks already sent
- Enforce content policies with configurable thresholds
- Comply with content safety regulations

Expand All @@ -46,19 +51,21 @@ Both must be used alongside `ai-proxy` or `ai-proxy-multi`.
```
ai-prompt-template (priority 1071)
ai-prompt-decorator (priority 1070)
ai-aws-content-moderation (priority 1050) ← runs BEFORE ai-proxy
ai-proxy (priority 1040)
ai-aws-content-moderation (priority 1031) ← runs AFTER ai-proxy
ai-aliyun-content-moderation (priority 1029) ← runs AFTER ai-proxy
```

The AWS plugin blocks requests before they reach the LLM. The Aliyun plugin
runs after `ai-proxy` sets context and can check both requests and responses.
A larger priority number runs earlier. Both moderation plugins run after
`ai-proxy` or `ai-proxy-multi` so they can read the decoded AI request.
They still block a flagged request before the upstream LLM is called.

---

## Plugin 1: ai-aws-content-moderation

Uses the AWS Comprehend `detectToxicContent` API to score request content.
Uses the AWS Comprehend `DetectToxicContent` API to score request and response
content.

### Configuration Reference

Expand All @@ -69,6 +76,18 @@ Uses the AWS Comprehend `detectToxicContent` API to score request content.
| `comprehend.region` | string | **Yes** | — | AWS region (e.g. `us-east-1`) |
| `comprehend.endpoint` | string | No | Auto | Custom Comprehend endpoint |
| `comprehend.ssl_verify` | boolean | No | `true` | Verify SSL certificate |
| `check_request` | boolean | No | `true` | Enable request moderation |
| `check_response` | boolean | No | `false` | Enable response moderation |
| `request_check_roles` | array | No | `user`, `tool`, `system`, `assistant` | Roles to moderate on the request |
| `request_check_mode` | string | No | `all` | `all` or `last` (latest consecutive block). `system` is always checked when selected |
| `request_check_length_limit` | integer | No | `1000` | Maximum bytes per Comprehend request text segment |
| `response_check_length_limit` | integer | No | `1000` | Maximum bytes per Comprehend response text segment |
| `stream_check_mode` | string | No | `final_packet` | `realtime` or `final_packet` when `check_response` is true |
| `stream_check_cache_size` | integer | No | `128` | Maximum characters per moderation batch in `realtime` mode |
| `stream_check_interval` | number | No | `3` | Seconds between moderation batches in `realtime` mode |
| `fail_mode` | string | No | `skip` | `skip`, `warn`, or `error` for non-AI / unrecognized traffic |
| `deny_code` | number | No | `200` | HTTP status for a denied request before headers are sent |
| `deny_message` | string | No | — | Custom denial message |
| `moderation_categories` | object | No | — | Per-category thresholds (0-1) |
| `moderation_threshold` | number | No | `0.5` | Overall toxicity threshold (0-1) |

Expand Down Expand Up @@ -125,7 +144,9 @@ a6 route create -f - <<'EOF'
EOF
```

Toxic requests are rejected with HTTP 400:
By default, a flagged request is denied with HTTP 200 and a provider-compatible
refusal so AI SDKs can parse the body. Set `deny_code: 400` when clients must
treat moderation as an HTTP error.

```
request body exceeds HATE_SPEECH threshold
Expand Down Expand Up @@ -172,6 +193,9 @@ response moderation, and real-time streaming moderation.
| `request_check_length_limit` | number | No | `2000` | Max chars per request chunk |
| `response_check_service` | string | No | `llm_response_moderation` | Aliyun service for response checks |
| `response_check_length_limit` | number | No | `5000` | Max chars per response chunk |
| `request_check_mode` | string | No | `last` | `last` (latest consecutive selected turns) or `all` |
| `request_check_roles` | array | No | `["user"]` | `user`, `tool`, or `system`. Assistant history cannot be selected |
| `fail_mode` | string | No | `skip` | `skip`, `warn`, or `error` for non-AI / unrecognized traffic |
| `risk_level_bar` | string | No | `high` | Threshold: `none`, `low`, `medium`, `high`, `max` |
| `deny_code` | number | No | `200` | HTTP status code for rejected content |
| `deny_message` | string | No | — | Custom rejection message |
Expand All @@ -193,8 +217,8 @@ Setting `risk_level_bar: "low"` blocks everything rated `low` or above.

| Mode | Behavior |
|------|----------|
| `final_packet` | Buffers entire response, checks at end |
| `realtime` | Checks content in batches during streaming, can interrupt mid-response |
| `final_packet` | Checks the assembled response at the end and annotates the final stream packet with `risk_level`; it cannot retract earlier chunks |
| `realtime` | Checks content in batches during streaming and can replace the remainder of the stream after a violation; it cannot retract earlier chunks |

### Step-by-Step: Aliyun Request + Response Moderation

Expand Down Expand Up @@ -260,7 +284,7 @@ EOF
### Pattern A: Request-only filtering (AWS)

```
Client → [AWS Comprehend blocks toxic] → ai-proxy → LLM → Response → Client
Client → ai-proxy [sets context] → [AWS Comprehend blocks toxic] → LLM → Response → Client
```

```yaml
Expand Down Expand Up @@ -347,9 +371,10 @@ routes:

| Symptom | Cause | Fix |
|---------|-------|-----|
| "no ai instance picked" | Aliyun plugin used without ai-proxy | Always configure ai-proxy or ai-proxy-multi on the same route |
| AWS plugin not blocking | Threshold too permissive | Lower `moderation_threshold` or per-category thresholds |
| "no ai instance picked" | Moderation plugin used without ai-proxy | Always configure ai-proxy or ai-proxy-multi on the same route |
| AWS denial is HTTP 200 | Default `deny_code` is `200` | Set `deny_code: 400` if clients expect an HTTP error |
| AWS plugin not blocking | Threshold too permissive, or `request_check_roles` omitted the role | Lower thresholds; AWS defaults to all roles with `request_check_mode: all` |
| Aliyun response moderation inactive | `check_response` defaults to `false` | Explicitly set `check_response: true` |
| "Specified signature is not matched" | Wrong Aliyun credentials | Verify `access_key_id` and `access_key_secret` |
| High latency | Double moderation (both plugins) | Use one moderation provider per route, not both |
| Streaming interrupted mid-response | Aliyun realtime mode detected violation | Expected behavior; adjust `risk_level_bar` or use `final_packet` mode |
| Streaming interrupted mid-response | A moderation plugin in `realtime` mode detected a violation | Expected behavior; adjust the moderation threshold or use `final_packet` mode |
101 changes: 83 additions & 18 deletions skills/a6-plugin-limit-count/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,11 +1,8 @@
---
name: a6-plugin-limit-count
description: >-
Skill for configuring the Apache APISIX limit-count plugin via the a6 CLI.
Covers fixed-window rate limiting, count/time_window configuration, key types,
Redis and Redis-cluster policies for distributed limiting, group-based shared
quotas, consumer-level vs route-level limiting, response headers, and common
operational patterns.
Skill for configuring the APISIX limit-count plugin via the a6 CLI. Covers
fixed and sliding windows, Redis Sentinel, delayed sync, and shared quotas.
version: "1.0.0"
author: Apache APISIX Contributors
license: Apache-2.0
Expand All @@ -24,11 +21,16 @@ metadata:

## Overview

The `limit-count` plugin rate-limits requests using a fixed-window counter
algorithm. Define a maximum number of requests (`count`) within a time interval
(`time_window`). Supports per-IP, per-consumer, per-header, or custom variable
keys. For distributed APISIX deployments, use Redis or Redis-cluster as the
shared counter backend.
The `limit-count` plugin rate-limits requests using a counter in a time window.
Define a maximum number of requests (`count`) within an interval (`time_window`).
The default `window_type` is `fixed`. Set `window_type: sliding` to smooth bursts
at window boundaries. Supports per-IP, per-consumer, per-header, or custom
variable keys. For distributed APISIX deployments, share counters through Redis,
Redis Cluster, or Redis Sentinel (`policy: redis-sentinel`).

Redis Sentinel, sliding windows, and delayed Redis synchronization (`sync_interval`)
are available from APISIX 3.18.0. Field tables and examples:
https://docs.api7.ai/hub/limit-count

## When to Use

Expand All @@ -50,7 +52,9 @@ shared counter backend.
| `rejected_code` | integer | No | `503` | HTTP status on rejection (200–599) |
| `rejected_msg` | string | No | — | Custom rejection message body |
| `group` | string | No | — | Share counters across routes with same group ID |
| `policy` | string | No | `"local"` | Storage: `"local"`, `"redis"`, or `"redis-cluster"` |
| `policy` | string | No | `"local"` | Storage: `"local"`, `"redis"`, `"redis-cluster"`, or `"redis-sentinel"` |
| `window_type` | string | No | `"fixed"` | `"fixed"` or `"sliding"` (APISIX 3.18.0+) |
| `sync_interval` | number | No | `-1` | Redis sync interval in seconds. `-1` syncs every request. Min `0.1` when enabled; must be smaller than a numeric `time_window` |
| `show_limit_quota_header` | boolean | No | `true` | Include X-RateLimit-* headers in responses |
| `allow_degradation` | boolean | No | `false` | Allow requests when plugin fails |

Expand Down Expand Up @@ -78,6 +82,22 @@ shared counter backend.
| `redis_timeout` | integer | No | `1000` | Timeout in milliseconds |
| `redis_cluster_ssl` | boolean | No | `false` | Enable TLS |

### Redis Sentinel Fields (when `policy: "redis-sentinel"`, APISIX 3.18.0+)

| Field | Type | Required | Default | Description |
|-------|------|----------|---------|-------------|
| `redis_sentinels` | array[object] | **Yes** | — | Sentinel nodes: `{ "host": "...", "port": 26379 }` |
| `redis_master_name` | string | **Yes** | — | Sentinel-monitored master name |
| `redis_role` | string | No | `"master"` | `"master"` or `"slave"` |
| `redis_username` | string | No | — | Redis ACL username |
| `redis_password` | string | No | — | Redis password |
| `redis_database` | integer | No | `0` | Redis database index |
| `sentinel_username` | string | No | — | Redis Sentinel ACL username |
| `sentinel_password` | string | No | — | Redis Sentinel password |
| `redis_connect_timeout` | integer | No | `1000` | Connection timeout in milliseconds |
| `redis_read_timeout` | integer | No | `1000` | Read timeout in milliseconds |
| `redis_keepalive_timeout` | integer | No | `60000` | Keepalive timeout in milliseconds |

## Key Types

| `key_type` | `key` Format | Example | Description |
Expand Down Expand Up @@ -234,6 +254,51 @@ All routes with `"group": "api-v1"` share the same 1000 req/hour counter.

Use Redis when running multiple APISIX nodes to share counters.

### Redis Sentinel

```json
{
"plugins": {
"limit-count": {
"count": 1000,
"time_window": 60,
"key": "remote_addr",
"policy": "redis-sentinel",
"redis_master_name": "mymaster",
"redis_sentinels": [
{ "host": "192.168.1.10", "port": 26379 },
{ "host": "192.168.1.11", "port": 26379 }
],
"rejected_code": 429
}
}
}
```

### Sliding window and delayed Redis sync

```json
{
"plugins": {
"limit-count": {
"count": 1000,
"time_window": 60,
"window_type": "sliding",
"policy": "redis",
"redis_host": "redis.example.com",
"sync_interval": 1,
"rejected_code": 429
}
}
}
```

`sync_interval` also works with `redis-cluster` and `redis-sentinel`. A numeric
`time_window` must be greater than `sync_interval`, or APISIX rejects the plugin
configuration. If a variable-based `time_window` resolves to a value less than
or equal to `sync_interval` at request time, APISIX falls back to per-request
synchronization.

### Redis cluster

```json
Expand Down Expand Up @@ -261,20 +326,20 @@ Use Redis when running multiple APISIX nodes to share counters.

| Symptom | Cause | Fix |
|---------|-------|-----|
| Limits not shared across APISIX nodes | Using `policy: "local"` (default) | Switch to `"redis"` or `"redis-cluster"` |
| Limits not shared across APISIX nodes | Using `policy: "local"` (default) | Switch to `"redis"`, `"redis-cluster"`, or `"redis-sentinel"` |
| Group config rejected | Mismatched configs in same group | Ensure all routes in group have identical limit-count config |
| Unexpected counter reset | Fixed-window boundary | Normal behavior — counters reset at fixed intervals |
| Unexpected counter reset | Fixed-window boundary | Normal for `window_type: fixed`; use `"sliding"` to smooth bursts |
| Key empty, all clients share one counter | Variable doesn't exist | Verify key variable name; falls back to `remote_addr` |
| Rate limit headers missing | `show_limit_quota_header: false` | Set to `true` (default) |
| 503 instead of 429 | Default `rejected_code` is 503 | Set `rejected_code: 429` explicitly |

## Fixed-Window Algorithm Note

`limit-count` uses a fixed-window algorithm. Counters reset at exact intervals.
This means a burst at the boundary of two windows can temporarily exceed the
intended rate (e.g., 100 req/min allows 200 requests if 100 come at t=59s and
100 at t=61s). For smoother rate limiting, combine with `limit-req` (leaky
bucket).
`limit-count` defaults to a fixed-window algorithm. Counters reset at exact
intervals, so a burst at the boundary of two windows can temporarily exceed the
intended rate (for example, 100 req/min allows 200 requests if 100 come at
t=59s and 100 at t=61s). Set `window_type: sliding` to weight the previous
window, or combine with `limit-req` (leaky bucket).

## Config Sync Example

Expand Down
Loading
Loading