Add a routing context factory for request-scoped policy state

[joshuajyue]

Follow-up to #7662.

Summary

Adds one protected virtual method to RoutingChatClient:

protected virtual RoutingContext CreateContext(
    IEnumerable<ChatMessage> messages,
    ChatOptions? options) => new(messages, options);

GetResponseAsync and GetStreamingResponseAsync call it instead of constructing a RoutingContext directly, so a derived class can return its own subclass and have it flow through selection and, for FailoverChatClient, every routing update for that request.

OrderedFailoverChatClient moves onto it, which removes its ConcurrentDictionary<RoutingContext, int>.

Why

Request-scoped policy state already exists — OrderedFailoverChatClient keeps a next-client index across attempts. The question is only where it lives.

Today it lives in a side table keyed by the context, because there is nowhere else to put it. One RoutingContext is already created per request and handed to every SelectClientAsync and OnRoutingUpdateAsync call for that request, so it has exactly the right lifetime, and it is already a non-sealed public class with a public constructor. The only thing missing is that the base class hardcodes its construction, so a subclass instance can never reach the selector.

	side table	context state
Capability	same	same
Lifetime	manual: every exit path must remove the entry	automatic: collected with the context
Leak if a request ends without a terminal update	yes	not possible
Concurrency	needs ConcurrentDictionary; every access hashes and may contend	a field on a per-request object; no synchronization
Access cost	hash lookup per selection and per update	field read
Typing	dictionary value type, unpacked at each use	typed property
Disposal	must clear the dictionary	nothing to clear

The capability row is the unchanged one: a side table works. RoutingContext does not override Equals/GetHashCode, so reference-keying is sound. CreateContext is not about adding functionality.

The leak is the difference: state must survive from a nonterminal update to the following selection, so it cannot be removed eagerly, and a request can end without a terminal update — an abandoned streaming enumerator, or selection failing after state was already stored. Today the derived class is responsible for cleaning up on those paths. Context-owned state hands that responsibility to the GC.

RoutingContext still clones the caller's options in its constructor, and a subclass has to call that constructor, so the caller's instance stays protected either way. A derived context can also be a natural home for per-attempt options a selector wants to shape, without adding anything to the routing API.

Changes

• RoutingChatClient — adds CreateContext; both invocation methods call it, with a null guard matching the existing SelectClientAsync check.
• FailoverChatClient — no new API. It inherits CreateContext, and because it declares both invocation methods sealed override, the factory is guaranteed to run for its derived types.
• OrderedFailoverChatClient — moves its next-client index onto a private context subclass; removes the ConcurrentDictionary, the per-call lookups, both cleanup paths, and the _requestStates.Clear() in Dispose.
• Tests — CreateContext coverage at both the RoutingChatClient and FailoverChatClient levels, including a null return; OrderedFailover_AbandonedStreamDoesNotAffectLaterRequests replaces the test that used reflection to inspect the removed dictionary.

Notes

• A derived class that overrides CreateContext casts the context back to its own type in SelectClientAsync. OrderedFailoverChatClient throws ArgumentException naming CreateContext if the type does not match, rather than surfacing a raw InvalidCastException.
• Additive and experimental: the virtual has a working default, so existing subclasses are unaffected.
• A generic RoutingChatClient<TContext> would remove the downcast, at the cost of a type parameter across the whole hierarchy.

Validation

• Build clean on all target frameworks, 0 warnings.
• Microsoft.Extensions.AI.Tests — 762 passed.
• Microsoft.Extensions.AI.Abstractions.Tests — 1646 passed.
• API manifests updated by hand and verified against MakeApiBaselines.ps1 output.

Microsoft Reviewers: Open in CodeFlow

[jozkee]

Nice, thanks.

[Copilot]

Pull request overview

Adds an extensibility point to chat routing so derived routing clients can create a per-request RoutingContext (including custom subclasses) and have it flow consistently through selection and (for failover) routing updates, enabling request-scoped policy state without side tables.

Changes:

• Add protected virtual RoutingContext CreateContext(IEnumerable<ChatMessage>, ChatOptions?) to RoutingChatClient and use it in both non-streaming and streaming entry points (with a null-return guard).
• Update FailoverChatClient to use CreateContext in its sealed overrides and document the request-scoped-state pattern.
• Refactor OrderedFailoverChatClient to store its “next client index” on a private RoutingContext subclass, removing the per-request ConcurrentDictionary and related cleanup paths; update tests and API manifests accordingly.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
test/Libraries/Microsoft.Extensions.AI.Tests/ChatRouting/OrderedFailoverChatClientTests.cs	Replaces reflection-based state-leak test with an abandoned-stream scenario to validate state scoping.
test/Libraries/Microsoft.Extensions.AI.Tests/ChatRouting/FailoverChatClientTests.cs	Adds coverage ensuring custom contexts flow through selection and updates; adds null-context guard coverage.
test/Libraries/Microsoft.Extensions.AI.Abstractions.Tests/ChatRouting/RoutingChatClientTests.cs	Adds coverage ensuring custom contexts flow to selection; adds null-context guard coverage.
src/Libraries/Microsoft.Extensions.AI/Microsoft.Extensions.AI.json	Updates API manifest to include OrderedFailoverChatClient.CreateContext override.
src/Libraries/Microsoft.Extensions.AI/ChatRouting/OrderedFailoverChatClient.cs	Moves ordered failover request state onto a context subclass; removes side-table state.
src/Libraries/Microsoft.Extensions.AI/ChatRouting/FailoverChatClient.cs	Switches to CreateContext for request context creation and documents request-scoped context state.
src/Libraries/Microsoft.Extensions.AI.Abstractions/Microsoft.Extensions.AI.Abstractions.json	Updates API manifest to include RoutingChatClient.CreateContext.
src/Libraries/Microsoft.Extensions.AI.Abstractions/ChatRouting/RoutingChatClient.cs	Introduces CreateContext factory and uses it in both invocation paths with a null guard.

[jozkee]

if an unexpected RoutingContext instance is ever passed here

that would require src changes, which will trigger this assertion during testing.