Python: preserve Responses replay metadata across AG-UI continuations

[Eduard van Valkenburg (eavanvalkenburg)]

Motivation & Context

AG-UI supports both stateless snapshot replay and provider-managed conversation continuation. These modes must not leak into each other: stateless runs should replay complete message history without restoring a provider response ID, while service-session runs should send only the incremental turn and restore private provider continuation state.

A cross-provider multi-turn matrix also found that real Responses output_text.logprobs were parsed into response-level metadata but were not retained on assistant content for direct replay. This loses provider-supplied data and prevents conformant replay when the receiving endpoint requires that field.

Description & Review Guide

• What are the major changes?

  • Persist and restore provider service-session IDs only when AG-UI service-session mode is active.
  • Reject use_service_session=True with an explicit store=False using a clear configuration error.
  • Preserve real assistant output_text.logprobs on parsed content and mirror them during direct Responses replay without fabricating empty values.
  • Add live two-turn AG-UI integration matrices for OpenAI Responses, OpenAI Chat Completions, FoundryChatClient, Foundry Prompt Agents, and Foundry Hosted Agents across stateless, conversation, and previous-response modes where applicable.
  • Keep the known Hosted Agent stateless replay incompatibility as a strict expected failure with a TODO to remove it when the upstream service accepts standard output-text replay without logprobs.

• What is the impact of these changes?

  • Stateless AG-UI history no longer accidentally combines full replay with a stored provider response ID.
  • Invalid disabled-storage service-session configurations fail before a provider request.
  • Provider-supplied logprobs survive direct assistant-message replay; messages without logprobs remain unchanged.
  • Multi-turn continuation behavior is continuously exercised across the supported OpenAI and Foundry client surfaces.

• What do you want reviewers to focus on?

  • The boundary between stateless snapshot state and provider-owned continuation state.
  • Selective logprobs preservation without synthetic defaults.
  • Matrix coverage and cleanup of temporary provider conversations and Prompt Agents.
  • The strict xfail scope for the known Hosted Agent stateless replay issue.

Related Issue

Fixes #7905

Contribution Checklist

• The code builds clean without any errors or warnings
• All unit tests pass, and I have added new tests where possible
• The PR follows the Contribution Guidelines
• This PR is linked to an issue and there is no other open PR for this issue (see Related Issue above).
• This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

[github-actions]

Python Test Coverage Report •

File	Stmts	Miss	Cover	Missing
packages/ag-ui/agent_framework_ag_ui
_agent_run.py	1486	148	90%	161, 221, 240, 287, 305, 321–322, 444, 497–498, 505, 614, 618, 620, 637, 664–665, 719, 748, 762, 766, 770, 773, 778, 783, 795, 802–808, 819, 833, 840, 850, 865, 868, 900, 935, 959, 972, 1021, 1039, 1041, 1067, 1069, 1071, 1074–1078, 1102, 1133, 1169, 1224–1225, 1267, 1281, 1326–1332, 1341, 1349, 1377, 1399, 1403, 1406, 1408, 1468–1469, 1475–1481, 1584, 1588–1589, 1592, 1597–1598, 1601, 1607–1608, 1612–1614, 1632–1634, 1659–1660, 1683–1684, 1697, 1739–1741, 1752–1754, 1756, 1770–1772, 1900, 1908, 1938, 1942, 1987, 2155–2156, 2195, 2253, 2362, 2367, 2439, 2441, 2448, 2464, 2469, 2526, 2531, 2535, 2584, 2635, 2710, 2856, 2894, 2902, 2904, 2907, 2913, 2968, 2971, 2981–2982, 2989, 3036
packages/foundry/agent_framework_foundry
_agent.py	260	19	92%	130, 133, 253, 258, 267, 378, 455, 491–492, 516, 519, 528–529, 756–757, 760, 881, 886, 890
_chat_client.py	255	22	91%	112, 114–116, 120–121, 125, 221, 268, 277, 342, 347, 398, 610, 614, 708, 715–718, 765, 847
packages/openai/agent_framework_openai
_chat_client.py	1422	114	91%	361, 374, 729–735, 744–747, 754–758, 766, 812–817, 821–824, 826–828, 835–837, 840, 897, 905, 928, 1145, 1204, 1206, 1208, 1210, 1276, 1290, 1370, 1380, 1385, 1428, 1540–1541, 1556, 1835, 1947, 1952–1953, 2036, 2046, 2073, 2079, 2089, 2095, 2100, 2106, 2111–2112, 2192, 2236, 2239–2242, 2256, 2266–2267, 2279, 2386, 2403, 2406, 2433–2435, 2474, 2491, 2494, 2556, 2563, 2600–2601, 2642, 2680–2681, 2699–2700, 2872–2873, 2891, 2926, 3006–3009, 3196, 3211, 3300–3302, 3312–3313, 3319, 3334, 3467–3468
TOTAL	48261	4483	90%

Python Unit Test Overview

Tests	Skipped	Failures	Errors	Time
9766	36 💤	0 ❌	0 🔥	2m 42s ⏱️

[Copilot]

Pull request overview

Preserves provider continuation boundaries and OpenAI Responses replay metadata across AG-UI turns.

Changes:

• Limits provider session-ID persistence to service-session mode.
• Preserves assistant logprobs for non-streaming replay.
• Adds unit and live provider-matrix coverage.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
python/packages/ag-ui/agent_framework_ag_ui/_agent_run.py	Controls continuation-state persistence and validation.
python/packages/ag-ui/tests/ag_ui/test_run.py	Tests disabled-storage rejection.
python/packages/openai/agent_framework_openai/_chat_client.py	Preserves output-text logprobs.
python/packages/openai/tests/openai/test_openai_chat_client.py	Tests logprobs parsing and replay.
python/packages/openai/tests/openai/test_agui_provider_matrix.py	Adds OpenAI continuation matrices.
python/packages/foundry/tests/foundry/test_agui_provider_matrix.py	Adds Foundry continuation matrices.
python/packages/foundry_hosting/tests/test_responses.py	Tests stateless replay isolation.

---

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

MAF Automated Review — Iteration 1

Result: Findings reported
Scope: full PR (3 commit(s)): a076d28d8702, 477444d0c321, 74a1786cb2b0
Model: gpt-5.6-sol

Overview

The change cleanly separates the standard Responses continuation ID from stateless AG-UI replay and adds focused validation for unsupported storage configurations. It also preserves provider-supplied logprobs for direct non-streaming Responses replay without fabricating values. However, the stateless boundary remains incomplete for providers that keep their continuation identifier in AgentSession.state, allowing Foundry Hosted Agent sessions to be restored despite use_service_session=False.

Reviewed the supplied pull-request change set across correctness, security/reliability, architecture, and failure behavior.
1 verified finding remained after source verification (1 medium) across 1 file. Details are attached to the affected lines below.

Affected areas: python/packages/ag-ui/agent_framework_ag_ui/_agent_run.py