Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 39 additions & 1 deletion docs/realtime/guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -244,7 +244,11 @@ Bare `RealtimeAgent` handoffs are auto-wrapped, and `realtime_handoff(...)` lets

### Guardrails

Realtime agents support output guardrails on agent responses and input guardrails on function-tool calls. Output guardrails run on debounced transcript accumulation rather than on every partial token, and they emit `guardrail_tripped` instead of raising an exception.
Realtime agents support output guardrails on agent responses and input guardrails on the user's
transcribed audio. (Function-tool calls have their own, separate tool input guardrails, which are a
distinct feature from the transcript input guardrails described here.) Output guardrails run on
debounced transcript accumulation rather than on every partial token, and they emit
`guardrail_tripped` instead of raising an exception.

```python
from agents.guardrail import GuardrailFunctionOutput, OutputGuardrail
Expand All @@ -266,6 +270,40 @@ agent = RealtimeAgent(

When a realtime output guardrail trips, the session interrupts the active response, forces `response.cancel`, emits `guardrail_tripped`, and sends a follow-up user message that names the triggered guardrail so the model can produce a replacement response. Your audio player should still listen for `audio_interrupted` and stop local playback immediately, because guardrails run on debounced transcript text and some audio may already be buffered when the tripwire fires.

Realtime agents also support **input guardrails** that run on the user's transcribed audio. Configure
them via `RealtimeAgent.input_guardrails` or `RealtimeRunConfig["input_guardrails"]`; the two lists
are combined and de-duplicated per turn. They run once on the completed user transcript (the
`input_audio_transcription_completed` event), and when one trips the session emits an
`input_guardrail_tripped` event, forces `response.cancel`, and sends a follow-up user message that
names the triggered guardrail.

!!! note
Unlike the regular non-realtime runner, `RealtimeSession` runs all input guardrails in parallel. Since the Realtime API automatically starts generating responses concurrently when the user finishes speaking, sequential (blocking) input guardrails (`run_in_parallel=False`) cannot prevent the model from starting a response and are unsupported. Configuring an input guardrail with `run_in_parallel=False` in `RealtimeSession` will raise a `ValueError`.


```python
from agents.guardrail import GuardrailFunctionOutput, InputGuardrail


def no_jailbreak(context, agent, user_input):
return GuardrailFunctionOutput(
tripwire_triggered="jailbreak" in user_input.lower(),
output_info=None,
)


agent = RealtimeAgent(
name="Assistant",
instructions="...",
input_guardrails=[InputGuardrail(guardrail_function=no_jailbreak)],
)
```

Two limitations are worth noting. Input guardrails only run on transcribed audio, so text sent
through `session.send_message()` is not checked. And because guardrails run in a background task,
the forced cancel reliably interrupts a response that is already in flight, but a response created
in the narrow window after the guardrail resolves may not be cancelled.

## SIP and telephony

The Python SDK includes a first-class SIP attach flow via [`OpenAIRealtimeSIPModel`][agents.realtime.openai_realtime.OpenAIRealtimeSIPModel].
Expand Down
1 change: 1 addition & 0 deletions docs/ref/realtime/events.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@

### Guardrail Events
::: agents.realtime.events.RealtimeGuardrailTripped
::: agents.realtime.events.RealtimeInputGuardrailTripped

### History Events
::: agents.realtime.events.RealtimeHistoryAdded
Expand Down
4 changes: 4 additions & 0 deletions examples/realtime/app/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,10 @@ async def _serialize_event(self, event: RealtimeSessionEvent) -> dict[str, Any]:
base_event["guardrail_results"] = [
{"name": result.guardrail.name} for result in event.guardrail_results
]
elif event.type == "input_guardrail_tripped":
base_event["guardrail_results"] = [
{"name": result.guardrail.name} for result in event.guardrail_results
]
elif event.type == "raw_model_event":
base_event["raw_model_event"] = {
"type": event.data.type,
Expand Down
2 changes: 2 additions & 0 deletions src/agents/realtime/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
RealtimeHandoffEvent,
RealtimeHistoryAdded,
RealtimeHistoryUpdated,
RealtimeInputGuardrailTripped,
RealtimeRawModelEvent,
RealtimeSessionEvent,
RealtimeToolApprovalRequired,
Expand Down Expand Up @@ -132,6 +133,7 @@
"RealtimeHandoffEvent",
"RealtimeHistoryAdded",
"RealtimeHistoryUpdated",
"RealtimeInputGuardrailTripped",
"RealtimeRawModelEvent",
"RealtimeSessionEvent",
"RealtimeToolApprovalRequired",
Expand Down
9 changes: 8 additions & 1 deletion src/agents/realtime/agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
from agents.prompts import Prompt

from ..agent import AgentBase
from ..guardrail import OutputGuardrail
from ..guardrail import InputGuardrail, OutputGuardrail
from ..handoffs import Handoff
from ..lifecycle import AgentHooksBase, RunHooksBase
from ..logger import logger
Expand Down Expand Up @@ -79,6 +79,13 @@ class RealtimeAgent(AgentBase, Generic[TContext]):
"""A class that receives callbacks on various lifecycle events for this agent.
"""

input_guardrails: list[InputGuardrail[TContext]] = field(default_factory=list)
"""A list of checks that run on the user's transcribed audio input. They run once on the
completed user transcript and, when tripped, force a cancel of the in-progress response. This
reliably interrupts a response that is already in flight, but a response created after the
guardrail resolves may not be interrupted. Text input sent via `send_message` is not checked.
"""

def __post_init__(self) -> None:
if not isinstance(self.name, str):
raise TypeError(f"RealtimeAgent name must be a string, got {type(self.name).__name__}")
Expand Down
5 changes: 4 additions & 1 deletion src/agents/realtime/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@

from agents.prompts import Prompt

from ..guardrail import OutputGuardrail
from ..guardrail import InputGuardrail, OutputGuardrail
from ..handoffs import Handoff
from ..model_settings import ToolChoice
from ..run_config import ToolErrorFormatter
Expand Down Expand Up @@ -281,6 +281,9 @@ class RealtimeRunConfig(TypedDict):
tool_error_formatter: NotRequired[ToolErrorFormatter]
"""Optional callback that formats tool error messages returned to the model."""

input_guardrails: NotRequired[list[InputGuardrail[Any]]]
"""List of input guardrails to run on the user's transcribed audio input."""

# TODO (rm) Add history audio storage config


Expand Down
26 changes: 25 additions & 1 deletion src/agents/realtime/events.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
from dataclasses import dataclass
from typing import Any, Literal, TypeAlias

from ..guardrail import OutputGuardrailResult
from ..guardrail import InputGuardrailResult, OutputGuardrailResult
from ..run_context import RunContextWrapper
from ..tool import Tool
from .agent import RealtimeAgent
Expand Down Expand Up @@ -243,6 +243,29 @@ class RealtimeGuardrailTripped:
type: Literal["guardrail_tripped"] = "guardrail_tripped"


@dataclass
class RealtimeInputGuardrailTripped:
"""An input guardrail has been tripped on the user's transcribed input.

When a guardrail trips, the session forces a cancel of the in-progress response. This
reliably interrupts a response that is already in flight. Because guardrails run in a
background task, a response that is created in the narrow window after the guardrail
resolves but before the cancel can take effect may not be interrupted.
"""

guardrail_results: list[InputGuardrailResult]
"""The tripped input guardrail result(s). The session cancels on the first tripwire, so this
normally contains a single result."""

message: str
"""The user transcript that triggered the guardrail."""

info: RealtimeEventInfo
"""Common info for all events, such as the context."""

type: Literal["input_guardrail_tripped"] = "input_guardrail_tripped"


@dataclass
class RealtimeInputAudioTimeoutTriggered:
"""Called when the model detects a period of inactivity/silence from the user."""
Expand All @@ -268,6 +291,7 @@ class RealtimeInputAudioTimeoutTriggered:
| RealtimeHistoryUpdated
| RealtimeHistoryAdded
| RealtimeGuardrailTripped
| RealtimeInputGuardrailTripped
| RealtimeInputAudioTimeoutTriggered
)
"""An event emitted by the realtime session."""
Loading