.NET: Workflow Outputs Overhaul: Support Tagging, Filtering Agent Outputs#6045
Open
lokitoth wants to merge 8 commits into
Open
.NET: Workflow Outputs Overhaul: Support Tagging, Filtering Agent Outputs#6045lokitoth wants to merge 8 commits into
lokitoth wants to merge 8 commits into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR overhauls .NET Workflows output handling to support tagging (notably an Intermediate tag) and filtering of executor outputs, bringing behavior closer to Python’s intermediate_output concept. It also expands the orchestration builder API surface (Sequential/Concurrent/Handoff/GroupChat/Magentic) and gates the new runner + Workflow-as-Agent forwarding behavior behind Futures.EnableAgentResponseOutputTaggingAndFiltering.
Changes:
- Introduces
OutputTag, tags onWorkflowOutputEvent, and tag-awareWorkflowBuilder.WithOutputFrom(...)/WithIntermediateOutputFrom(...). - Updates in-proc runner + Workflow-as-Agent forwarding semantics under a Futures opt-in.
- Adds first-class fluent orchestration builders (Sequential/Concurrent) and centralizes common builder logic in
OrchestrationBuilderBase<TBuilder>; adds/reshapes unit tests accordingly.
Reviewed changes
Copilot reviewed 40 out of 40 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| dotnet/src/Microsoft.Agents.AI.Workflows/AgentResponseEvent.cs | Adds tag-aware constructors for terminal/intermediate tagging of agent response events. |
| dotnet/src/Microsoft.Agents.AI.Workflows/AgentResponseUpdateEvent.cs | Adds tag-aware constructors for terminal/intermediate tagging of streaming update events. |
| dotnet/src/Microsoft.Agents.AI.Workflows/AgentWorkflowBuilder.cs | Refactors builder helpers to delegate to new Sequential/Concurrent builder types; adds Create*BuilderWith factories. |
| dotnet/src/Microsoft.Agents.AI.Workflows/Checkpointing/WorkflowInfo.cs | Changes output executor representation to Dictionary<string, HashSet<OutputTag>> with back-compat converter. |
| dotnet/src/Microsoft.Agents.AI.Workflows/Checkpointing/WorkflowInfoOutputExecutorsConverter.cs | Adds legacy array + new map JSON shape support for output executors. |
| dotnet/src/Microsoft.Agents.AI.Workflows/ConcurrentWorkflowBuilder.cs | New fluent builder for concurrent orchestrations with default output designations. |
| dotnet/src/Microsoft.Agents.AI.Workflows/Execution/OutputFilter.cs | Updates output-filter lookup to use keyed tag sets via the workflow’s new output executor map. |
| dotnet/src/Microsoft.Agents.AI.Workflows/Futures.cs | Adds process-wide feature flag to opt into tag-aware filtering for AgentResponse(Update). |
| dotnet/src/Microsoft.Agents.AI.Workflows/GroupChatWorkflowBuilder.cs | Integrates orchestration base + applies Python-aligned default output designations. |
| dotnet/src/Microsoft.Agents.AI.Workflows/HandoffWorkflowBuilder.cs | Integrates orchestration base + applies default output designations + ensures end executor binding. |
| dotnet/src/Microsoft.Agents.AI.Workflows/InProc/InProcessRunnerContext.cs | Moves AgentResponse(Update) into the output filter pipeline when Futures is enabled; applies tags. |
| dotnet/src/Microsoft.Agents.AI.Workflows/MagenticWorkflowBuilder.cs | Integrates orchestration base + applies default output designations for team members vs orchestrator. |
| dotnet/src/Microsoft.Agents.AI.Workflows/OrchestrationBuilderBase.cs | New shared fluent base for name/description + memoized output designation logic. |
| dotnet/src/Microsoft.Agents.AI.Workflows/OutputTag.cs | Introduces OutputTag value type with an internal constructor and an Intermediate singleton. |
| dotnet/src/Microsoft.Agents.AI.Workflows/OutputTagJsonConverter.cs | Adds JSON converter to serialize/deserialize OutputTag as a string. |
| dotnet/src/Microsoft.Agents.AI.Workflows/SequentialWorkflowBuilder.cs | New fluent builder for sequential pipelines with default output designations. |
| dotnet/src/Microsoft.Agents.AI.Workflows/Workflow.cs | Updates workflow output executor storage from HashSet<string> to tag map; updates protocol description logic. |
| dotnet/src/Microsoft.Agents.AI.Workflows/WorkflowBuilder.cs | Updates builder’s internal output executor tracking to a tag map and adds tag-aware registration overloads. |
| dotnet/src/Microsoft.Agents.AI.Workflows/WorkflowBuilderExtensions.cs | Adds WithIntermediateOutputFrom extension to register executors as intermediate output sources. |
| dotnet/src/Microsoft.Agents.AI.Workflows/WorkflowOutputEvent.cs | Adds tags to WorkflowOutputEvent and constructors for no-tags/single-tag/multi-tag cases. |
| dotnet/src/Microsoft.Agents.AI.Workflows/WorkflowOutputEventExtensions.cs | Adds IsIntermediate() helper based on tag membership. |
| dotnet/src/Microsoft.Agents.AI.Workflows/WorkflowSession.cs | Adjusts Workflow-as-Agent forwarding semantics under Futures and intermediate-vs-terminal gating for generic outputs. |
| dotnet/src/Microsoft.Agents.AI.Workflows/WorkflowsJsonUtilities.cs | Updates source-gen JSON contracts (but currently drops WorkflowInfo contract used elsewhere). |
| dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests/AgentWorkflowBuilderTests.cs | Refactors tests to target the static AgentWorkflowBuilder surface after builder extraction. |
| dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests/BackwardsCompatibility/JsonCheckpointSerializationTests.cs | Adds back-compat tests for legacy output executor JSON shape and tagging constructors. |
| dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests/ConcurrentWorkflowBuilderTests.cs | Adds tests for concurrent builder behavior, default/explicit designations, and AsAIAgent forwarding. |
| dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests/Futures/Futures.AgentResponseOutputFilteringAndTaggingTests.cs | Adds runner-level test matrix covering Futures-on/off behavior across payload/designation combinations. |
| dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests/Futures/FuturesScope.cs | Adds scoped helper to flip Futures flag during tests. |
| dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests/Futures/FuturesSerialCollection.cs | Adds xUnit collection to serialize tests that mutate the process-wide Futures flag. |
| dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests/GroupChatWorkflowBuilderTests.cs | Adds tests for group chat builder output designation defaults + validation. |
| dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests/HandoffWorkflowBuilderTests.cs | Adds tests for handoff builder output designation defaults + validation. |
| dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests/InputWaiterTests.cs | Removes embedded OutputFilterTests after extraction into its own file. |
| dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests/JsonSerializationTests.cs | Updates output executor assertions to account for tag sets per executor id. |
| dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests/MagenticWorkflowBuilderTests.cs | Adds tests for Magentic builder designation defaults + validation. |
| dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests/OrchestrationTestHelpers.cs | Adds shared test helpers and test agents used across orchestration builder tests. |
| dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests/OutputFilterTests.cs | Extracts and extends OutputFilter unit tests to validate tag behavior. |
| dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests/OutputTagTests.cs | Adds tests covering OutputTag equality/hashcode/json + constructor visibility. |
| dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests/SequentialWorkflowBuilderTests.cs | Adds tests for sequential builder behavior, default/explicit designations, and AsAIAgent forwarding. |
| dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests/WorkflowBuilderTests.cs | Renames/reshapes builder smoke tests and adds tag-aware WithOutputFrom/WithIntermediateOutputFrom tests. |
| dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests/WorkflowHostSmokeTests.cs | Adds Futures-gated smoke tests for Workflow-as-Agent intermediate and terminal forwarding semantics. |
Comments suppressed due to low confidence (1)
dotnet/src/Microsoft.Agents.AI.Workflows/WorkflowsJsonUtilities.cs:88
WorkflowsJsonUtilities.JsonContextno longer includes a source-generated contract forWorkflowInfo, butWorkflowBuilderstill referencesWorkflowsJsonUtilities.JsonContext.Default.WorkflowInfowhen serializing the workflow definition. This will fail to compile unlessWorkflowInfois re-added to the[JsonSerializable]list (and any other referenced TypeInfo properties).
[JsonSerializable(typeof(ExecutorIdentity))]
[JsonSerializable(typeof(RunnerStateData))]
// Workflow Output Types
[JsonSerializable(typeof(OutputTag))]
// Workflow-as-Agent
[JsonSerializable(typeof(WorkflowChatHistoryProvider.StoreState))]
[JsonSerializable(typeof(WorkflowSession.SessionState))]
Comment on lines
+23
to
+28
| // Reuse the well-known singleton where possible so callers can do reference | ||
| // comparisons on the common case without paying the extra allocation cost. | ||
| if (string.Equals(value, OutputTag.Intermediate.Value, StringComparison.Ordinal)) | ||
| { | ||
| return OutputTag.Intermediate; | ||
| } |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Automated Code Review
Reviewers: 4 | Confidence: 85%
✓ Correctness
No actionable issues found in this dimension.
✓ Security Reliability
No actionable issues found in this dimension.
✓ Test Coverage
No actionable issues found in this dimension.
✗ Design Approach
I found one design issue in the JSON contract setup: the PR adds new checkpoint/output-tag serialization behavior, but the updated source-generated context appears to drop
WorkflowInfofromWorkflowsJsonUtilities.DefaultOptions. That conflicts with the library’s documented AOT/reflection-disabled serialization path and would break checkpoint/workflow-definition marshalling in those environments. That conflicts with the existing public contract, which documents that flag as the control for surfacing outgoing workflow outputs in agent responses, andAgentResponseEventis itself aWorkflowOutputEvent.
Flagged Issues
-
WorkflowHostSmokeTests.cs:861and the similar terminal-case assertion codify behavior that contradicts the currentAsAIAgentAPI contract:includeWorkflowOutputsInResponseis documented as the switch for transforming outgoing workflow outputs intoAgentResponseUpdates, andAgentResponseEventis itself aWorkflowOutputEvent. If Futures-on is intended to bypass that flag, the public contract/docs must change in the same PR; otherwise the tests should keep the event gated byincludeWorkflowOutputsInResponse.
Automated review by lokitoth's agents
Phase 1 of the .NET Workflows outputs overhaul (see working/implementation-plan.md). Pure moves/renames in dotnet/tests/Microsoft.Agents.AI.Workflows.UnitTests; no production code changes, no new test cases. The split keeps each orchestration mode in its own source file so the upcoming tag-aware and orchestration-default test additions land on clean diffs. Renames: * WorkflowBuilderSmokeTests.cs -> WorkflowBuilderTests.cs (with class rename to match). The scope is no longer "smoke"-only once subsequent phases add tag-aware builder tests. * InputWaiterAndOutputFilterTests.cs -> InputWaiterTests.cs + OutputFilterTests.cs. The file already declared the two test classes separately; this split simply gives each its own file so the output-filter cases have a dedicated home for tag-aware additions. Split of AgentWorkflowBuilderTests.cs: * AgentWorkflowBuilderTests.cs is now the outer `public static partial class AgentWorkflowBuilderTests` holding the shared test helpers (DoubleEchoAgent + session + WithBarrier variant, WorkflowRunResult, RunWorkflow* methods) bumped from `private` to `internal` so the new top-level GroupChatWorkflowBuilderTests in the same assembly can reach them. * AgentWorkflowBuilder.SequentialTests.cs (nested SequentialTests): BuildSequential_InvalidArguments_Throws, BuildSequential_AgentsRunInOrderAsync. * AgentWorkflowBuilder.ConcurrentTests.cs (nested ConcurrentTests): BuildConcurrent_InvalidArguments_Throws, BuildConcurrent_AgentsRunInParallelAsync. Sequential and Concurrent are kept as nested classes because they're modes of the same `AgentWorkflowBuilder` static factory and do not produce dedicated builder types. New file: * GroupChatWorkflowBuilderTests.cs (top-level): the existing BuildGroupChat_* and GroupChatManager_* cases moved out of the old AgentWorkflowBuilderTests file. They exercise the `GroupChatWorkflowBuilder` type (returned by `AgentWorkflowBuilder.CreateGroupChatBuilderWith`), so a dedicated top-level test class - matching the convention reserved by the plan for HandoffWorkflowBuilderTests / MagenticWorkflowBuilderTests - is the right home. Cross-class helper references qualify with `AgentWorkflowBuilderTests.DoubleEchoAgent` and `AgentWorkflowBuilderTests.RunWorkflowAsync`. The outer partial class is `static` (and nested classes carry the instance test methods) because the outer holds only static helpers; this satisfies CA1052 without suppressions and is invisible to xUnit discovery, which finds tests on the nested classes as `AgentWorkflowBuilderTests.SequentialTests.*` etc. Validation: `dotnet build` clean on both target frameworks; all 547 tests in Microsoft.Agents.AI.Workflows.UnitTests pass on net10.0. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Phase 2 of the .NET Workflows outputs overhaul. Additive code change
only - no observable runtime behavior change. The runner still uses the
legacy bypass for AgentResponse / AgentResponseUpdate payloads, and the
new `Futures.EnableAgentResponseOutputTaggingAndFiltering` flag defaults
to false. Phase 3 will wire the flag into the runner; this commit only
introduces the types and the builder API.
New public surface:
* `OutputTag` (readonly struct): wraps a string Value with ordinal
equality (IEquatable, GetHashCode, == / !=) so it can participate as a
HashSet element. Internal ctor closes the set. One public singleton:
`OutputTag.Intermediate`. Terminal / regular outputs carry no tag
(empty Tags set). JSON-serialized as a bare string via
[JsonConverter(typeof(OutputTagJsonConverter))], with the converter
rehydrating to the well-known singleton on read.
* `Futures` (static class): hosts opt-in pre-GA behavior switches.
First flag is `EnableAgentResponseOutputTaggingAndFiltering`; XML doc
captures the v2.0.0 obsoletion / v3.0.0 removal lifecycle.
* `WorkflowOutputEvent.Tags`: `HashSet<OutputTag>` exposed directly
(concrete collection, matches the JSON-serialization convention used
for `WorkflowInfo.OutputExecutorIds`). Never null; empty for legacy /
terminal events. New ctors take a single `OutputTag` or
`IEnumerable<OutputTag>?`; the existing (data, executorId) ctor
remains and produces an untagged event. `HasTag(OutputTag)` helper.
`AgentResponseEvent` and `AgentResponseUpdateEvent` gain matching
tag-accepting ctors forwarding to the base.
* `WorkflowOutputEventExtensions.IsIntermediate(this WorkflowOutputEvent)`:
extension method returning `evt.HasTag(OutputTag.Intermediate)`. The
preferred way to ask "is this an intermediate output?" without
reaching into the Tags set.
* `WorkflowBuilder.WithOutputFrom(IEnumerable<ExecutorBinding>, OutputTag)`
and `WorkflowBuilder.WithOutputFrom(ExecutorBinding, OutputTag)`:
forward-looking tagged overloads. The IEnumerable form is the primary
tagged surface; the single-executor form is a convenience for the
common one-executor case. Currently usable for the
`OutputTag.Intermediate` singleton; will become the primary surface
once the `OutputTag` constructor is opened to user-defined tags in
a future release. Callers in this release should prefer the
intent-specific `WithIntermediateOutputFrom` extension for the
intermediate case. Tags accumulate across repeated calls; same tag
repeated dedupes via the HashSet.
* `WorkflowBuilderExtensions.WithIntermediateOutputFrom(this WorkflowBuilder, IEnumerable<ExecutorBinding>)`:
helper that forwards to `WithOutputFrom(executors, OutputTag.Intermediate)`.
Takes an IEnumerable (matching the tagged WithOutputFrom shape) -
callers pass collection literals: `builder.WithIntermediateOutputFrom([a, b])`.
XML doc remarks call out the Futures-flag interaction and the
AIAgent-payload forwarding contract.
Internal shape changes:
* `WorkflowBuilder._outputExecutors`: HashSet<string> -> Dictionary<
string, HashSet<OutputTag>>. The value set is empty for executors
designated only via the untagged WithOutputFrom; contains Intermediate
(and possibly future tags) otherwise.
* `Workflow.OutputExecutors`: HashSet<string> -> Dictionary<string,
HashSet<OutputTag>>.
* `OutputFilter.CanOutput`: `Contains(id)` -> `ContainsKey(id)`.
* `WorkflowInfo.OutputExecutorIds`: HashSet<string> -> Dictionary<
string, HashSet<OutputTag>>, with a custom JsonConverter that reads
both the new map shape (`{id: ["intermediate", ...]}`) and the legacy
array shape (`[id1, id2]`, where each id is treated as an untagged
output). Always writes the map shape. IsMatch updated to compare
per-id tag sets.
Tests landing in this commit (per the test-with-feature principle):
* `OutputTagTests.cs` (6 tests): KnownValues, EqualityIsOrdinalOnValue,
DefaultStructValueIsDistinct (default(OutputTag) does not collide
with the Intermediate singleton in a HashSet),
GetHashCodeMatchesEquals, JsonConverter_RoundtripsValueAsString,
ConstructorIsInternal (reflection-based assertion that the (string)
ctor is `internal`).
* `WorkflowBuilderTests.cs` adds 7 new tests pinning the builder
API contract: RegistersWithEmptyTagSet, AddsIntermediateTag,
MultipleExecutorsAllUntagged, ThenIntermediate_AccumulatesTags,
RepeatedDedupes, OnlyRegistersWithoutPriorWithOutputFrom,
TracksExecutorBinding.
* `BackwardsCompatibility/JsonCheckpointSerializationTests.cs`
(new folder + file, 5 tests): event-level ctor contract tests
(single-tag, no-tag, multi-tag — the last with a custom tag);
IsIntermediate() asserted; load-bearing JSON BC tests for
`WorkflowInfo.OutputExecutorIds` -
`WorkflowOutputExecutorsReadsLegacyArrayShape` (legacy ids map to
empty tag sets) and `WorkflowOutputExecutorsWritesMapShape`.
The plan's three JSON round-trip tests for `WorkflowOutputEvent.Tags`
were dropped: `WorkflowEvent` is not currently a serialized checkpoint
shape (see the comment in WorkflowsJsonUtilities.cs about events not
being persisted), so there is no real back-compat surface to pin
through JSON. They are substituted with in-process ctor/property
round-trip tests that exercise the `Tags` / `HasTag` / `IsIntermediate`
contract.
Validation: full `Microsoft.Agents.AI.Workflows.UnitTests` suite runs
green on net10.0 (565 passing, 0 failing). Core library builds clean
on net472, netstandard2.0, net8.0, net9.0, and net10.0. Test project
builds clean on net472 + net10.0.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…utures flag
`InProcessRunnerContext.YieldOutputAsync` historically special-cased AgentResponse and
AgentResponseUpdate payloads: it built the typed event subclass and emitted it directly,
bypassing the output filter. Rewrites the method so that:
- When `Futures.EnableAgentResponseOutputTaggingAndFiltering` is `false` (the current
default), AgentResponse(Update) keep the legacy bypass — emitted as
AgentResponseEvent / AgentResponseUpdateEvent with no tags. Existing callers see no
behavior change.
- When the flag is `true`, AIAgent payloads flow through the output filter just like
every other payload type: undesignated sources are dropped, and the emitted event
carries the source's tag set (empty for terminal `WithOutputFrom`, `{Intermediate}`
for `WithIntermediateOutputFrom`, the set union when both designations apply).
Non-AIAgent (POCO) outputs also now carry the source's tag set on the emitted
WorkflowOutputEvent unconditionally — additive, since no existing assertion inspected
Tags. Subclass events (`AgentResponseEvent` / `AgentResponseUpdateEvent`) continue to
be emitted under both modes so `switch (evt) { case AgentResponseEvent: ... }`
consumer code keeps matching.
Adds `OutputFilter.TryGetTags` as the tag-aware lookup used by the runner.
`OutputFilter.CanOutput` is kept (still used by the existing sync tests in
`OutputFilterTests.cs`).
Tests
-----
- `Futures/Futures.AgentResponseOutputFilteringAndTaggingTests.cs` (new): the F1–F13
matrix from the plan, covering every combination of `(flag on/off) × (designation)
× (payload shape)`. Uses a `FuturesScope` IDisposable + a `FuturesSerial` xUnit
collection (DisableParallelization = true) to keep the process-global flag from
leaking across parallel tests.
- `OutputFilterTests.cs`: four new `Test_OutputFilter_…` cases for the `TryGetTags`
surface (empty-tag-set for terminal designation, `{Intermediate}` for intermediate
designation, union for accumulated designation, `false` for unregistered).
582/582 unit tests pass on net10.0 (565 baseline + 17 new).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Aligns the .NET orchestration builders with Python's output / intermediate-output distinction. Each builder either applies a Python-aligned default designation set or replays the user's explicit `WithOutputFrom` / `WithIntermediateOutputFrom` calls, never both. Static `AgentWorkflowBuilder.BuildSequential` / `BuildConcurrent` apply defaults unconditionally (no user-facing fluent surface to take control through): - Sequential: terminal `end` + every agent designated intermediate. - Concurrent: terminal `end` + every agent and per-agent accumulator designated intermediate. The three fluent instance builders memoize agent-typed designation calls in a `Dictionary<AIAgent, HashSet<OutputTag>>` (empty set = terminal-only, non-empty = intermediate tag(s)) so repeated calls dedupe naturally. They replay the entries at `Build()` time, suppressing defaults when any call has been made: - `HandoffWorkflowBuilder` / `HandoffWorkflowBuilderCore<TBuilder>` (also picked up by the obsolete `HandoffsWorkflowBuilder` via inheritance). Default: terminal `HandoffEnd` + every handoff agent intermediate. (Bug fix: legacy code relied on `WithOutputFrom(end)` to bind `HandoffEnd`. The new explicit-designation path bypasses that, so `Build()` now calls `BindExecutor(end)` unconditionally to keep validation happy.) - `GroupChatWorkflowBuilder` — default: terminal host + every participant intermediate. - `MagenticWorkflowBuilder` — default: terminal orchestrator + every team member intermediate. Designating a non-participant agent throws `InvalidOperationException`. The bare `WorkflowBuilder` default is unchanged — only the orchestration-style builders gain implicit defaults, matching the plan's non-goal. Tests ----- - `AgentWorkflowBuilder.SequentialTests` / `.ConcurrentTests`: one default-spec assertion each. - `GroupChatWorkflowBuilderTests`: defaults-match-spec, explicit-replaces-defaults, non-participant throws. - `HandoffWorkflowBuilderTests` (new file): same three. - `MagenticWorkflowBuilderTests` (new file): same three. 593/593 unit tests pass on net10.0 (582 baseline + 11 new). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…nder Futures-on Aligns the .NET Workflow-as-Agent surface with Python `as_agent`. Under `Futures.EnableAgentResponseOutputTaggingAndFiltering = true`, `WorkflowSession.InvokeStageAsync` now forwards `AgentResponseEvent` unconditionally — joining `AgentResponseUpdateEvent` in ignoring the host's `includeWorkflowOutputsInResponse` switch. That switch keeps governing the generic `WorkflowOutputEvent` path for non-AIAgent payloads, where it is further short-circuited by an `IsIntermediate()` check (tagged intermediate outputs always surface). Under Futures-off the legacy asymmetry is preserved: `AgentResponseUpdateEvent` always forwarded, `AgentResponseEvent` gated by `includeWorkflowOutputsInResponse`. Back-compat: with `Futures.EnableAgentResponseOutputTaggingAndFiltering` left at its default `false`, observable behavior is identical to before. `Futures` documentation gains a remark explaining the `Workflow.AsAIAgent()` interaction in both flag states. Runner fix ---------- `InProcessRunnerContext.YieldOutputAsync` now skips `Executor.CanOutput` for AgentResponse-shaped payloads under both Futures branches. `AIAgentHostExecutor` doesn't declare AgentResponse(Update) in its `Yields` set, so the historical legacy bypass had silently skipped the check; Phase 3's Futures-on path was running it and would reject AIAgent payloads. AIAgent-shaped payloads are now always a valid output shape, matching the legacy bypass semantics. Phase 4 follow-on ----------------- Switched the three orchestration-builder designation-replay loops to iterate `Dictionary.Keys` with a value lookup instead of constructing/destructuring `KeyValuePair<,>`. Cleaner shape and avoids the netstandard2.0 / net472 `KeyValuePair<,>.Deconstruct` unavailability that surfaced when this branch multi-TFM-built. Tests ----- `WorkflowHostSmokeTests.IntermediateForwarding` (new nested class, 6 tests): - intermediate AgentResponse forwarded past the include-outputs gate (Futures on) - terminal AgentResponse forwarded unconditionally (Futures on) - terminal AgentResponse gated by include flag (Futures off, legacy) - undesignated AIAgent executor emits no AgentResponseEvent under Futures-on - legacy bypass still emits AgentResponseEvent under Futures-off - intermediate tag is observable via `update.RawRepresentation` The class joins the `FuturesSerial` xUnit collection so the process-global flag is serialized against other Futures-toggling tests. 599/599 unit tests pass on net10.0 (593 baseline + 6 new). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…trationBuilderBase
Promotes the Sequential and Concurrent orchestration shapes to first-class fluent
builder classes, matching Handoff / GroupChat / Magentic. Users can call
`WithOutputFrom(agents)` / `WithIntermediateOutputFrom(agents)` to control which
agents are designated output / intermediate sources; when no designation call is
made, the Python-aligned defaults apply (terminal aggregator output + every agent
intermediate; Concurrent also tags per-agent accumulators).
`AgentWorkflowBuilder.BuildSequential(...)` and `BuildConcurrent(...)` are kept
and now delegate to the new builders; observable behavior unchanged. Five static
factories now mirror each other:
- `AgentWorkflowBuilder.CreateSequentialBuilderWith(params IEnumerable<AIAgent>)`
- `AgentWorkflowBuilder.CreateConcurrentBuilderWith(params IEnumerable<AIAgent>)`
- `AgentWorkflowBuilder.CreateHandoffBuilderWith(AIAgent)` (already existed)
- `AgentWorkflowBuilder.CreateGroupChatBuilderWith(Func<...>)` (already existed)
- `AgentWorkflowBuilder.CreateMagenticBuilderWith(AIAgent)` (new)
OrchestrationBuilderBase
------------------------
New abstract `OrchestrationBuilderBase<TBuilder>` unifies the shared fluent
surface across all five orchestration builders: `WithName`, `WithDescription`,
`WithOutputFrom`, `WithIntermediateOutputFrom`, and the
`ApplyOutputDesignations(builder, agentMap, kind, applyDefaults)` helper that
either replays the user's designations or invokes the orchestration-specific
defaults.
Removes ~150 LOC of duplicated designation-management code from the four
non-Handoff builders, plus the equivalent from `HandoffWorkflowBuilderCore`.
Tests
-----
- New `SequentialWorkflowBuilderTests.cs` / `ConcurrentWorkflowBuilderTests.cs`
(replace the old `AgentWorkflowBuilder.{Sequential,Concurrent}Tests.cs`
nested-class files). Method names normalized to
`Test_<BuilderType>_<Scenario>[Async]`.
- Shared helpers (`DoubleEchoAgent`, `DoubleEchoAgentWithBarrier`,
`WorkflowRunResult`, `RunWorkflow*`) moved from the old
`AgentWorkflowBuilderTests` partial class into a new
`OrchestrationTestHelpers` static class in `OrchestrationTestHelpers.cs`.
Downstream test files (Group Chat, Handoff, Sequential, Concurrent) updated
to qualify with `OrchestrationTestHelpers.*`.
- A new `AgentWorkflowBuilderTests.cs` covers the static surface directly:
`BuildSequential` / `BuildConcurrent` invariants and aggregator wiring, plus
null-rejection + round-trip checks for every `Create*BuilderWith` factory.
- New AsAgent intermediate-suppression tests on a nested `AsAgentForwarding`
class for each of Sequential and Concurrent: build with only the terminal
agent designated via `WithOutputFrom`, run via `AsAIAgent(...)`, assert via
`AgentResponseUpdate.AuthorName` that intermediate agents do not surface.
Both join the `FuturesSerial` collection.
- New `Test_<Builder>_WithDescriptionPropagatesToWorkflow` smoke tests on
Sequential and Concurrent (newly available via the base class).
625/625 unit tests pass on net10.0.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
lokitoth
force-pushed
the
dev/dotnet_workflow/orchestration_output_overhaul
branch
from
2bb1bc3 to
583f74a
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation and Context
Brings .NET Workflows' output handling to parity with Python's
intermediate_outputconcept, and grows the orchestration-builder surface to first-class fluent types for all five shapes (Sequential, Concurrent, Handoff, GroupChat, Magentic). Lets workflow authors distinguish intermediate outputs (progress, partial results) from terminal outputs at the executor designation level, and letsWorkflow.AsAIAgent(...)forward intermediates out to callers automatically — matching Python'sas_agentbehavior.New behavior is gated behind a process-wide
Futures.EnableAgentResponseOutputTaggingAndFilteringswitch that ships opt-in (defaultfalse); existing consumers see no behavior change until they explicitly enable it.Description
Six-commit stack, each independently buildable and green. Test count goes 565 → 625; clean across
net472/netstandard2.0/net8.0/net9.0/net10.0.test:test reshuffles — pure rename/split moves in preparation; no cases added or removed.feat:OutputTag + Futures + tag-awareWorkflowBuilderAPI — newOutputTagstruct (ChatRole-shaped, singleIntermediatesingleton, internal ctor),Futuresstatic class for opt-in pre-GA behavior,WorkflowOutputEvent.TagsasHashSet<OutputTag>, threeWorkflowBuilder.WithOutputFromoverloads plus aWithIntermediateOutputFromextension. JSON:Workflow.OutputExecutorsnow serializes as a map; the converter accepts the legacystring[]shape on read.feat:Futures-gated runner change —InProcessRunnerContext.YieldOutputAsyncrewritten so AgentResponse(Update) payloads no longer special-case the filter when the flag is on. Untagged terminals carry emptyTags; intermediate-designated executors carry{Intermediate}.feat:orchestration builders —Handoff/GroupChat/Magenticbuilders gainWithOutputFrom/WithIntermediateOutputFrom(agent-typed, memoized) with Python-aligned defaults when no designation is made.feat:Workflow-as-Agent forwarding —WorkflowSession.InvokeStageAsyncupdated so that under Futures-on,AgentResponseEventis forwarded unconditionally (matchingAgentResponseUpdateEvent's today-behavior).Futuresdocumentation gains a remark about theAsAIAgentinteraction.feat:SequentialWorkflowBuilder/ConcurrentWorkflowBuilder+OrchestrationBuilderBase<TBuilder>— promotes Sequential and Concurrent to first-class fluent types; introduces a generic abstract base that unifiesWithName/WithDescription/WithOutputFrom/WithIntermediateOutputFromacross all five orchestration builders (~150 LOC of duplicated code removed). StaticAgentWorkflowBuilder.BuildSequential/BuildConcurrentkeep working; newCreate*BuilderWithfactories cover the full set.Default designations (when no explicit call is made): terminal aggregator is
Output, every participating agent isIntermediate. The bareWorkflowBuilderdefault is unchanged.Lifecycle:
Futures.EnableAgentResponseOutputTaggingAndFilteringis documented as a one-release migration helper —[Obsolete]in v2.0.0 when the new behavior becomes the default, removed in v3.0.0.Contribution Checklist
[ ] Is this a breaking change? No — new behavior is opt-in via theFuturesflag; defaultfalsepreserves current behavior. The full suite also passes with the flag forced ON globally.