Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions src/data/nav/aitransport.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,10 @@ export default {
name: 'Core SDK',
link: '/docs/ai-transport/getting-started/core-sdk',
},
{
name: 'OpenAI',
link: '/docs/ai-transport/getting-started/openai',
},
{
name: 'Vercel AI SDK',
link: '/docs/ai-transport/getting-started/vercel-ai-sdk',
Expand Down Expand Up @@ -110,6 +114,10 @@ export default {
{
name: 'Frameworks',
pages: [
{
name: 'OpenAI',
link: '/docs/ai-transport/frameworks/openai',
},
{
name: 'Vercel AI SDK UI',
link: '/docs/ai-transport/frameworks/vercel-ai-sdk-ui',
Expand Down
5 changes: 3 additions & 2 deletions src/pages/docs/ai-transport/concepts/codecs.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
title: "Codecs"
meta_description: "Understand codecs in AI Transport: the translation layer between your AI framework's events and Ably's messages."
meta_keywords: "AI Transport, codec, TInput, TOutput, UserMessage, Regenerate, ToolResult, custom codec, Vercel"
intro: "A codec translates between your AI framework's events and Ably's messages. The same connection carries a Vercel UIMessage, an OpenAI completion, or a custom domain event without changing the SDK."
intro: "A codec translates between your AI framework's events and Ably's messages. The same connection carries a Vercel UIMessage, an OpenAI Responses event, or a custom domain event without changing the SDK."
---

A codec is the translation layer between domain-specific message models and Ably's native message primitives. It defines how events in the application's domain (text deltas, tool calls, finish signals, or whatever the domain model requires) are encoded into Ably messages, and how incoming Ably messages are decoded back into domain events.
Expand Down Expand Up @@ -76,11 +76,12 @@ const myCodec = {
```
</Code>

Most users don't write a custom codec; the bundled [Vercel codec](/docs/ai-transport/api/javascript/vercel/codec) covers the Vercel AI SDK end-to-end. See the [Codec API reference](/docs/ai-transport/api/javascript/core/codec) for the full interface.
Most users don't write a custom codec. The SDK bundles a [Vercel codec](/docs/ai-transport/api/javascript/vercel/codec) for the Vercel AI SDK and a [ResponsesCodec](/docs/ai-transport/frameworks/openai) for the OpenAI Responses API, each covering its framework end-to-end. See the [Codec API reference](/docs/ai-transport/api/javascript/core/codec) for the full interface.

## Read next <a id="next"/>

- [Codec API reference](/docs/ai-transport/api/javascript/core/codec): the full `Codec`, `Encoder`, `Decoder` contracts and well-known variant types.
- [Vercel codec](/docs/ai-transport/api/javascript/vercel/codec): the pre-built codec from `createUIMessageCodec()` for the Vercel AI SDK.
- [OpenAI Responses](/docs/ai-transport/frameworks/openai): the `ResponsesCodec` for the OpenAI Responses API.
- [Codec architecture](/docs/ai-transport/internals/codec-architecture): the wire-level encoder/decoder pipeline.
- [Connections](/docs/ai-transport/concepts/connections): how the codec is bound into every connection.
23 changes: 23 additions & 0 deletions src/pages/docs/ai-transport/features/tool-calling.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,29 @@ if (pending) {

The result is addressed to the suspended assistant message by `codecMessageId`. Reusing the original `runId` keeps the resume on the same run instead of starting a fresh one.

## OpenAI codec <a id="openai"/>

The examples above use the Vercel codec, but the OpenAI Responses codec supports the same tool surface: server-executed function calls, client-executed tools, tool failures, and human approvals. The suspend and resume mechanics live in the transport, so they are identical for both codecs. The wire types and the factory payload field names differ.

The OpenAI codec expresses tool state against the Responses types, so a tool call is a `function_call` item and its result is a `function_call_output` item. The client factories take snake_case payloads keyed by `call_id`, and you address each to the assistant message that holds the call:

<Code>
```javascript
import { ResponsesCodec } from '@ably/ai-transport/openai';

// A client-run tool succeeded.
await view.send(ResponsesCodec.createToolResult(codecMessageId, { call_id, output }), { runId });

// A client-run tool failed. The message becomes the output the model sees next turn.
await view.send(ResponsesCodec.createToolResultError(codecMessageId, { call_id, message }), { runId });

// A user approved or denied a gated tool. A denial resolves entirely on the client.
await view.send(ResponsesCodec.createToolApprovalResponse(codecMessageId, { call_id, approved, reason }), { runId });
```
</Code>

The Responses `function_call_output` item has no field for an approval decision or an error, so the codec holds that render-only state on `OpenAIMessage.toolCallStates`, a map keyed by `call_id`. `toResponsesInput` never reads it, so it cannot reach the model. See [OpenAI Responses](/docs/ai-transport/frameworks/openai) for the agentic loop and the approval-request output.

## History persistence <a id="history"/>

Tool invocations and results are part of the channel's message history. When a client reconnects or a late joiner loads the conversation, tool activity is replayed along with text messages. The view reconstructs tool state so the UI shows the correct status: pending, complete, or failed.
Expand Down
135 changes: 135 additions & 0 deletions src/pages/docs/ai-transport/frameworks/openai.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,135 @@
---
title: "OpenAI Responses"
meta_description: "How Ably AI Transport integrates with the OpenAI Responses API. The ResponsesCodec encodes the Responses event stream onto an Ably channel, and toResponsesInput feeds the conversation back to the model."
meta_keywords: "AI Transport, OpenAI, Responses API, ResponsesCodec, toResponsesInput, function calling, client-side tools, tool approval, reasoning, durable sessions, server"
intro: "The OpenAI Responses API streams model output as typed events over HTTP. AI Transport's ResponsesCodec encodes that stream onto an Ably channel, so the same server code feeds durable sessions instead of an ephemeral HTTP response."
---

The [OpenAI Responses API](https://platform.openai.com/docs/api-reference/responses) is OpenAI's interface for calling a model and getting back a stream of typed events: output text, reasoning, refusals, function-call arguments, and lifecycle events. AI Transport writes that stream to an Ably channel, so reconnects, multi-device, and bidirectional control come for free.

The `ResponsesCodec` from `@ably/ai-transport/openai` passes each raw Responses event through to the channel and reassembles the conversation on the client. There is no OpenAI-specific transport, so you use the generic [`createAgentSession`](/docs/ai-transport/api/javascript/core/agent-session) from `@ably/ai-transport` and pass it the codec.

To build a working app with this codec, see [Get started with OpenAI](/docs/ai-transport/getting-started/openai).

## What the OpenAI Responses API brings <a id="openai-brings"/>

The `openai` npm package owns the model call and the typed event model:

| Capability | Description |
| --- | --- |
| Responses streaming | `client.responses.create({ stream: true })` returns a stream of `ResponseStreamEvent`s: output-text deltas, reasoning, refusals, function-call arguments, and item lifecycle events. |
| Server-side tools | You advertise function tools on the request. The model emits function calls, you run them, and you feed the outputs back on the next Responses API call. |
| Reasoning models | Reasoning items carry the model's summarised thinking, and `encrypted_content` for the no-store, zero-data-retention round trip. |
| Round-trippable items | Each output item the codec stores is also valid Responses API input, so the conversation feeds the next turn without conversion. |
| Typed SDK | The official SDK owns auth, the HTTP call, streaming, and the Responses type definitions. |

## What AI Transport adds <a id="ait-adds"/>

AI Transport adds to the Responses API, writing the model stream to a durable session that outlives any single connection:

| Capability | Description |
| --- | --- |
| Durable sessions | Tokens flow through an Ably channel that outlives any single connection. A client reconnects and resumes from where it left off. |
| Multi-device sync | Every device subscribed to the session sees the same conversation in realtime. |
| Bidirectional control | Cancel, steer, and interrupt the agent from any client. No separate control channel. |
| Active run tracking | `view.runs()` exposes which clients have Runs streaming and which Runs are in progress. |
| Conversation branching | Edit and regenerate create forks in the conversation tree, not destructive replacements. |
| History and replay | Load the full conversation on reconnect, page refresh, or new device join. |
| Token compaction | Reconnecting clients receive accumulated responses, not a replay of every token. |

These are the same capability bullets used on the [Vercel AI SDK Core](/docs/ai-transport/frameworks/vercel-ai-sdk-core) page; the session surface is the same whichever model layer you integrate against.

## Where they connect <a id="how-they-connect"/>

On the server, AI Transport swaps the HTTP response sink for `Run.pipe()`. Without it, the OpenAI SDK gives you no helper for forwarding a Responses stream over HTTP, so you serialise each event into a Server-Sent Events response by hand:

<Code>
```javascript
const encoder = new TextEncoder();
return new Response(
new ReadableStream({
async start(controller) {
for await (const event of stream) {
controller.enqueue(encoder.encode(`data: ${JSON.stringify(event)}\n\n`));
}
controller.close();
},
}),
{ headers: { 'Content-Type': 'text/event-stream' } },
);
```
</Code>

With AI Transport, a raw `ResponseStreamEvent` is already valid codec output and `run.pipe` takes the SDK's async iterable directly, so the same stream reaches a durable session in two lines, and every subscribed client resumes, syncs, and cancels it:

<Code>
```javascript
const { reason } = await run.pipe(stream);
await run.end({ reason });
```
</Code>

The integration has three pieces:

1. The `ResponsesCodec` from `@ably/ai-transport/openai` encodes each `ResponseStreamEvent` as Ably messages and reassembles them into `OpenAIMessage`s on the client. It streams assistant text, refusals, reasoning, and function-call arguments, and repairs a client that joins a stream mid-way. The codec passes the raw events through, so the wire tracks OpenAI's own event model.
2. `createAgentSession({ client, channelName, codec: ResponsesCodec })` from `@ably/ai-transport` constructs the agent session bound to the channel from the invocation.
3. `run.pipe()` reads the model's event stream, encodes each event, and publishes the resulting Ably messages. `run.abortSignal` wires cancellation through from the client to the Responses API request.

To feed the next turn, `toResponsesInput` flattens the conversation read from `run.view` back into the Responses `input` array. Each stored `OpenAIMessage` already holds valid Responses input items, so the flatten needs no conversion:

<Code>
```javascript
import { toResponsesInput } from '@ably/ai-transport/openai';

while (run.view.hasOlder()) {
await run.view.loadOlder();
}
const input = toResponsesInput(run.view.getMessages().map(({ message }) => message));
```
</Code>

## Run server-side tools <a id="tools"/>

A Responses stream never carries a function call's *output*. OpenAI surfaces tool output only as model input on the next turn, so the codec adds one event of its own, `function_call_output`, for the agent to publish after it runs a tool.

Server-executed tools do not suspend the Run. The agent runs an agentic loop: call the Responses API, and if the model emits function calls, run them, append the model's output items and the tool outputs to the input, and call it again. The loop continues until the model produces a reply with no tool calls. Each unit of work publishes under its own `run.pipe`, so a Run that calls one tool produces three messages: the turn that emitted the calls, the tool outputs, and the final text turn.

For reasoning models, the loop must re-append the whole turn's output items, including the reasoning items that preceded a function call, since reasoning models expect that reasoning to travel with the call on the next request. The [runnable demo](https://github.com/ably/ably-ai-transport-js) implements the full loop.

## Run client-side tools and approvals <a id="client-tools"/>

The codec also carries the client-driven half of tool calling: a client executes a tool in the browser and publishes the result, reports a tool failure, or answers a human approval prompt. The suspend and resume mechanics belong to the transport, so they work the same way they do for the Vercel codec. The agent calls `run.suspend()` to wait for a client, the client publishes its input on the same `runId`, and a continuation resumes the Run.

`ResponsesCodec` exposes the full well-known factory set. You address each client input to the assistant message that holds the `function_call`, and key each payload by the OpenAI snake_case `call_id`:

<Code>
```javascript
import { ResponsesCodec } from '@ably/ai-transport/openai';

// A client-run tool succeeded.
await view.send(ResponsesCodec.createToolResult(codecMessageId, { call_id, output }), { runId });

// A client-run tool failed. The message becomes the output the model sees next turn.
await view.send(ResponsesCodec.createToolResultError(codecMessageId, { call_id, message }), { runId });

// A user approved or denied a gated tool. A denial resolves entirely on the client.
await view.send(ResponsesCodec.createToolApprovalResponse(codecMessageId, { call_id, approved, reason }), { runId });
```
</Code>

To gate a tool on a human decision, the agent publishes the codec's own `tool-approval-request` output, carrying the `call_id`, the tool name, and the arguments so a client can render the prompt without the streamed `function_call`.

The Responses `function_call_output` item has no field for an approval decision or an error, so the codec holds that render-only state out of band on `OpenAIMessage.toolCallStates`, a map keyed by `call_id`. `toResponsesInput` never reads that state, so it cannot reach the model.

## Scope and trade-offs <a id="scope"/>

The `ResponsesCodec` covers the shapes the Responses API streams: assistant text, refusals, reasoning (summary and raw), and server-executed function calls. It also covers the client-driven tool surface: client-executed tools, tool failures, and human approvals. Hosted tools (web and file search, code interpreter, image generation, MCP, custom tools) are not yet supported.

The codec transmits the raw Responses events rather than a normalised abstraction. The wire therefore tracks OpenAI's own event model, which keeps stored items round-trippable to the Responses API but ties a conversation to the Responses shape. To integrate a different model provider behind one abstraction, use [Vercel AI SDK Core](/docs/ai-transport/frameworks/vercel-ai-sdk-core) instead.

## Read next <a id="next"/>

- [Get started with OpenAI](/docs/ai-transport/getting-started/openai): build a working app.
- [Codecs](/docs/ai-transport/concepts/codecs): how a codec translates a framework's events into Ably messages.
- [Tool calling](/docs/ai-transport/features/tool-calling): the tool-calling model across the SDK.
- [Agent session API reference](/docs/ai-transport/api/javascript/core/agent-session): the full API surface for the agent session.
Loading