[api][integrations] Support models' native structured output (foundation + OpenAI)#843
Open
weiqingy wants to merge 1 commit into
Open
[api][integrations] Support models' native structured output (foundation + OpenAI)#843weiqingy wants to merge 1 commit into
weiqingy wants to merge 1 commit into
Conversation
…ion + OpenAI) Add the foundation for using a model provider's native structured-output capability at the chat-model connection layer, plus the OpenAI implementation, in both Java and Python. Previously output_schema was honored only by prompt-engineering the request and parsing the response text. The request's output schema is carried to the connection through a reserved key in the existing modelParams/kwargs map, so the abstract chat() signature is unchanged. Each connection declares a boolean native-structured-output capability (default false). A connection applies the native API only when a schema is present, no tools are bound on the call, the schema is a POJO (Java) / BaseModel (Python) rather than a RowTypeInfo, and the setup is same-language. The reserved key is always removed before the SDK call so it cannot leak into a provider request. The prompt-engineered path is retained as the fallback and is unaffected: in the ReAct loop tools are always bound, so the native path stays dormant there. OpenAI applies response_format json_schema with strict validation. Other connections only strip the reserved key; their native paths and the ReActAgent final-output wiring follow in later changes.
github-actions
Bot
added
doc-not-needed
wenjin272
added
fixVersion/0.4.0
and removed
fixVersion/0.3.0
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Linked issue: #280
Purpose of change
Today
output_schemais honored only by prompt-engineering and parsing the response text; no integration uses a provider's native structured-output API. This PR adds the foundation for native structured output at the chat-model connection layer, plus the OpenAI implementation, in Java and Python. It's the first in a small stack under #280 (Azure/Ollama, Anthropic, DashScope follow; ReActAgent final-output wiring is a separate follow-up).How it works:
__structured_output_schema__) in the existingmodelParams/kwargsmap, so the abstractchat()signature is unchanged.supportsNativeStructuredOutput()/supports_native_structured_output), defaultfalse.BaseModel(notRowTypeInfo), and the setup is same-language. The key is always stripped before the SDK call so it can't leak into a request.OpenAI applies
response_formatjson_schema strict. Other connections only strip the reserved key for now. The same-language guard avoids marshaling a schema object across the Pemja bridge, where native structured output can't work anyway (a JavaClassis not a PythonBaseModel).Tests
Unit tests with the SDK mocked (no network): native applied with schema and no tools (Java + Python); not applied when tools are bound or for
RowTypeInfo; the reserved key never reaches a provider SDK; the same-language threading guard; and existing ReActAgent prompt-path tests remain green.API
Yes — additive only.
BaseChatModelConnectiongains a public reserved-key constant and aprotectedcapability method (defaultfalse); no existing signatures change.Documentation
doc-neededdoc-not-neededdoc-included