Add structured output support to Responses API

[dmedovich]

Fixes #905

Adds support for Responses API structured output by mapping text.format
to the existing Chat Completions response_format request sent to the backend.

The issue specifically calls out /engines/v1/responses and structured output
handling where the backend supports it.

Approach

Responses requests use:

{
  "text": {
    "format": {
      "type": "json_schema",
      "name": "answer_prompt",
      "strict": true,
      "schema": {}
    }
  }
}

The existing backend path expects the Chat Completions shape instead, so this PR
translates that into:

{
  "response_format": {
    "type": "json_schema",
    "json_schema": {
      "name": "answer_prompt",
      "strict": true,
      "schema": {}
    }
  }
}

json_object is mapped the same way to response_format.type = json_object.
The original text config is kept on the returned Response object.

Also added /engines/v1/responses and /engines/{backend}/v1/responses, matching
the existing engine-qualified Chat Completions routes.

Behavior

store: false is now honored for both normal and streaming responses: the
response is returned, but not saved for later lookup.

Unsupported Responses-only fields now return 400 instead of being silently
ignored. background: false, truncation: "disabled", and explicit null
hosted fields are treated as no-ops.

Trade-offs to flag

This stays on top of the existing Responses compatibility layer. It does not
replace streaming with native OpenAI Responses typed events.

I'm happy to work on that larger native Responses API follow-up too if we want
to take it in that direction.

tested

• ✅ git diff --check
• ✅ go test ./pkg/responses ./pkg/routing
• ✅ go test -race ./pkg/responses ./pkg/routing ./pkg/inference/scheduling
• ✅ make validate-all
• ✅ make e2e

make e2e includes a new structured output case through /responses.

[sourcery-ai]

Hey - I've left some high level feedback:

• The new engines-based Responses routes are registered both in NewHTTPHandler and in routing.NewRouter, which makes the path structure harder to reason about; consider centralizing the route definitions in one place (likely the router) and keeping the handler unaware of /engines/... prefixes.
• The compatibility error messages (e.g., in validateUnsupportedRequestFields and unsupported tool types) are quite verbose and repeated; you might want to factor out shared helpers/constants for these strings to keep them consistent and easier to maintain.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- The new engines-based Responses routes are registered both in `NewHTTPHandler` and in `routing.NewRouter`, which makes the path structure harder to reason about; consider centralizing the route definitions in one place (likely the router) and keeping the handler unaware of `/engines/...` prefixes.
- The compatibility error messages (e.g., in `validateUnsupportedRequestFields` and unsupported tool types) are quite verbose and repeated; you might want to factor out shared helpers/constants for these strings to keep them consistent and easier to maintain.

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[gemini-code-assist]

Code Review

This pull request introduces support for structured outputs (JSON schema and JSON object formats) and conditional storage in the OpenAI Responses API compatibility layer. It updates the request and response models, implements request transformation and validation for unsupported fields, registers new backend-qualified routing paths, and adds comprehensive end-to-end and unit tests to verify these additions. No review comments were provided, so I have no feedback to address.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[dmedovich]

sourcery-ai
with the current nested ServeMux setup, routing.NewRouter still needs to decide which top-level paths are delegated to the Responses handler, while responses.NewHTTPHandler needs to register the concrete method/path patterns on its own mux. So I kept the existing boundary and avoided a larger router refactor in this PR

[ericcurtin]

Build failed, needs a fix