docs(examples): add runnable structured output example - #4418
Conversation
Restores the example deferred out of OpenHands#4207, updated for the merged API. Shows a response_schema on TerminalTool (forced per-command rationale) and on FinishTool (typed final answer), and reads results back with parse_response / parse_last_response. Referenced by the structured output guide in OpenHands/docs#695. Co-authored-by: Vasco Schiavo <115561717+VascoSch92@users.noreply.github.com>
|
🚦 CI is currently failing on this PR's latest commit. Please fix the failing checks before OpenHands reviews it - this is re-checked automatically once you push a new commit. (A maintainer can also request This is an automated check - no AI was used to generate this comment. |
|
🤖 OpenHands is reviewing this PR. Head commit: This comment was posted by an AI agent (OpenHands). |
all-hands-bot
left a comment
There was a problem hiding this comment.
This review was created by an AI agent (OpenHands) on behalf of the repository maintainers.
Summary
This PR adds examples/01_standalone_sdk/56_structured_output.py, a runnable example demonstrating the response_schema structured-output feature landed in #4207. The example shows both augmenting an existing tool (TerminalTool) with forced-annotation fields and binding a typed ProjectFacts model to FinishTool for the final answer.
Verification
I verified every API call in the example against the current main codebase and ran end-to-end checks:
register_tool("FinishTool", FinishTool)— Correct and necessary.FinishToolis inBUILT_IN_TOOL_CLASSES(used byinclude_default_tools) but is not registered in the tool resolver registry (_REG) by default, so it must be registered explicitly to resolveTool(name="FinishTool", ...)viaresolve_tool. This matches the pattern intests/sdk/tool/test_response_schema.py.Tool(name=TerminalTool.name, params={"response_schema": CommandRationale})— Correct.CommandRationalefields (purpose,expected_outcome) do not collide withTerminalActionfields (command,is_input,timeout,reset) and are not in_RESERVED_RESPONSE_FIELDS. Verified resolution andparse_response()round-trip.include_default_tools=["ThinkTool"]— Correct. This skips the auto-injected defaultFinishToolso the schema-bound one fromtools=is used instead. Verifiedtools_mapcontains bothfinish(with schema) andthink.agent.tools_map["finish"]— Correct. The resolved tool's.nameattribute is"finish"(from_camel_to_snake("FinishTool").removesuffix("_tool")), not"FinishTool".finish_tool.parse_last_response(events)andterminal_tool.parse_response(event.action)— Both verified end-to-end with synthetic events.parse_last_responsecorrectly walks reversed events, re-splits arguments, and returns a typedProjectFacts.conversation.conversation_stats.get_combined_metrics().accumulated_cost— Matches the pattern in other examples (e.g.46_agent_settings.py).EXAMPLE_COST:marker — Present, satisfying the example CI gate.ruff check/ruff format— Clean.
Example CI Discovery
The file lives under examples/01_standalone_sdk/ (in _TARGET_DIRECTORIES), is not in _EXCLUDED_EXAMPLES, and will be discovered and run by tests/examples/test_examples.py.
Risk Assessment
Low risk. This is a documentation-only change (new example file, no modifications to SDK, tools, or tests). All API usage has been verified against the current codebase. No security concerns — the example reads credentials from environment variables and performs no privileged operations.
No Material Findings
No correctness, security, or maintainability issues were found. The example is well-structured, follows the flat script-like pattern used by other examples, and accurately documents the reserved-field constraints in its inline comment.
|
@luciobaiocchi did you run the example? Could you post the output? |
|
Yes I ran it with AGENT COMMENT: Every terminal call carries the annotation fields, and the final answer comes back as a typed One honest note from the run: mid-way the model also tried to put |
|
🤖 OpenHands is reviewing this PR. Head commit: This comment was posted by an AI agent (OpenHands). |
all-hands-bot
left a comment
There was a problem hiding this comment.
This review was created by an AI agent (OpenHands) on behalf of the repository maintainers.
Summary
This PR adds examples/01_standalone_sdk/56_structured_output.py, a runnable example demonstrating the response_schema structured-output feature from #4207. The example demonstrates two patterns: (1) augmenting TerminalTool with forced per-call annotation fields (purpose, expected_outcome) via a CommandRationale schema, and (2) binding a typed ProjectFacts model to FinishTool for the final answer, recovered via parse_last_response().
The head SHA (c6ee477) is a merge of main into the branch since the last review (b46f97f); the example file itself is unchanged, but I re-verified all API usage against the merged codebase.
Verification
I verified every API call against the current workspace (head SHA c6ee477) and ran all pre-commit hooks:
register_tool("FinishTool", FinishTool)(line 48) — Correct and necessary.FinishToolis listed inBUILT_IN_TOOL_CLASSES(used byinclude_default_tools) but is not auto-registered in the tool resolver registry (_REG). Confirmed vialist_registered_tools()returning empty before the call. The registration name"FinishTool"matches theTool(name="FinishTool", ...)spec on line 66.Tool(name=TerminalTool.name, params={"response_schema": CommandRationale})(line 64) — Correct.CommandRationalefields (purpose,expected_outcome) do not collide withTerminalActionfields (command,is_input,timeout,reset) and are not in_RESERVED_RESPONSE_FIELDS(kind,security_risk,structured_output,summary).include_default_tools=["ThinkTool"](line 69) — Correct and necessary. This skips the auto-injected defaultFinishToolso the schema-bound one fromtools=is the solefinishtool, avoiding a duplicate-name error. The comment on line 68 explains this.agent.tools_map["finish"](line 84) — Correct. The resolvedFinishToolinstance has.name == "finish"(from_camel_to_snake("FinishTool").removesuffix("_tool")), not"FinishTool".agent.tools_map[TerminalTool.name](line 83) — Correct.TerminalTool.name == "terminal".terminal_tool.parse_response(event.action)(line 96) andfinish_tool.parse_last_response(events)(line 103) — Both match the method signatures intool.py(lines 558, 580).parse_last_responsewalks reversed events, re-splits arguments via_split_response_arguments, and returns a typedProjectFacts | None.conversation.conversation_stats.get_combined_metrics().accumulated_cost(line 111) — Matches the pattern used across 20+ existing examples.EXAMPLE_COST:marker (line 112) — Present, satisfying the example CI gate.
Pre-commit hooks all pass: ruff check, ruff format, pycodestyle, pyright, check-import-rules, check-tool-registration.
Example CI Discovery
The file lives under examples/01_standalone_sdk/ (already in _TARGET_DIRECTORIES in tests/examples/test_examples.py), is not in _EXCLUDED_EXAMPLES, and will be auto-discovered via sorted(directory.glob("*.py")). No changes to the test file are needed.
Risk Assessment
Low risk. Documentation-only change (new example file, no modifications to SDK, tools, or tests). No security concerns — the example reads credentials from environment variables and performs no privileged operations. All API usage verified against the merged main codebase.
No Material Findings
No correctness, security, or maintainability issues were found. The example is well-structured, follows the top-level code-flow convention used by other linear walkthrough examples, accurately documents the reserved-field constraints in its inline comment, and correctly demonstrates both halves of the response_schema feature.
HUMAN:
Added example code for #4207
Referenced by the structured output guide in OpenHands/docs#695.
AGENT:
Why
#4207 landed
response_schemabut shipped without a runnable example — the onewritten for it was deferred out of the PR to unblock the docs-example CI gate.
The docs guide for the feature (OpenHands/docs#695) needs it: the other SDK
guides all embed a synced
examples/...block, and @VascoSch92 asked for thesame treatment there.
Summary
Adds
examples/01_standalone_sdk/56_structured_output.py, restored from thedeferred version and updated for the merged API. It demonstrates both halves of
the feature on one run:
TerminalToolwith a schema — every command must carry apurposeandan
expected_outcomeon top of the tool's owncommandfield, showing thata schema can force per-call annotations on an existing tool with no
subclassing.
FinishToolwith a schema — the final answer comes back as a typedProjectFactsobject, read viaparse_last_response().It also shows reading a specific call with
parse_response(event.action)whilewalking the event stream.
Issue Number
Relates to #2566, follows up #4207. Docs counterpart: OpenHands/docs#695
How to Test
The run should print a rationale for each terminal command and a typed final
answer.
ruff check/ruff formatare clean and the file's imports and APIcalls were verified against current
main.Video/Screenshots
Type
Notes
The example is numbered
56_(55 was the highest in use). The docs guidealready references this path;
sync_code_blocks.pyonly warns when the file ismissing, so OpenHands/docs#695 is not blocked by merge order — once this lands,
the sync workflow fills in the code block.