test only(agents): inherit statefulness into member agents' tool workers

[ling-senpeng13]

The bug

A stateful swarm/team shares one session, and the server domain-routes the tasks of everything inside it — taskToDomain maps the member agents' tool tasks to the session domain. But AgentRuntime._register_workers read agent.stateful on the immediate agent only.

When a tool hangs off a swarm member — and members aren't themselves stateful — its worker registered with domain=None while its tasks were queued in the domain queue. Nothing was polling there:

ref        : call_W975OtGeIHXtzDxGvI4Q6Zgu_0__1
taskDefName: swarm_tool
domain     : '7203a6c8df354e15b6a30393afe9c72f'
pollCount  : 0
startTime  : 0

The enclosing FORK's JOIN never satisfied and the workflow ran to TIMED_OUT.

The fix

Propagate statefulness downward so worker registration matches how the server routes:

effective_stateful = bool(getattr(agent, "stateful", False)) or inherited_stateful

_register_workers takes inherited_stateful and passes it to both recursion sites (sub-agents, and nested agent_tool agents).

ConfigSerializer._serialize_agent had the same gap — it dropped agent_stateful when recursing into sub-agents, so member tools were never marked stateful in the config sent to the server. Fixed the same way, so both sides agree. This one wasn't strictly required to make the test pass (the server derives routing from the swarm), but leaving the two halves disagreeing is how this bug happened in the first place.

Also: an e2e helper that was hiding it

_find_tasks_by_type matched only taskDefName. Compiler-generated system tasks carry their identity in referenceTaskName — handoff_check is emitted as an INLINE task, so its taskDefName is literally "INLINE":

ref=e2e_s14_stateful_swarm_handoff_check__1  taskDefName='INLINE'  status=COMPLETED

assert handoff_tasks therefore failed with 20 handoff_check tasks present and COMPLETED. Now matches either field.

Verification

Against a conductor-oss server built from main, isolating each change:

Configuration	Result
stock SDK	FAILED — workflow RUNNING, 304s
runtime fix only	FAILED — workflow COMPLETED, fails handoff_check assert
runtime + test fix	PASSED — 118s
+ serializer fix	PASSED — whole suite14 green, 6/6
full mainline e2e	136 passed, 1 failed, 7 skipped, 1 xfailed

The middle row is the useful one: it isolates the two bugs and shows the runtime fix alone is enough to get the workflow completing.

The one remaining failure is test_get_after_delete_returns_none — an unrelated scheduler 404-vs-None contract mismatch, present before this change.

test_non_stateful_no_domain is the guard against over-propagation, and it still passes — statefulness isn't leaking into agents that never declared it. That's why I ran the whole file rather than the one test.

Draft, because two things want a second opinion

1. Is stateful intended to be inherited? I've assumed yes — a member of a stateful composite shares its session, so its tools are stateful in every sense that matters to routing. If the intended semantics are "only the declaring agent", then the fix belongs on the server side instead (don't domain-route tools that aren't marked stateful), and this PR is wrong.
2. The serializer half changes what's sent to the server, so it can affect compilation. Suite14 is green, but I haven't reasoned through every other topology that flag reaches.

Notes

• Found while investigating why Stateful swarm handoff hangs on main — #1356 stopped registering transfer task defs the SDK still expects conductor#1363 was listed as a known failure in that repo's CI. The root cause recorded there was wrong — it blamed missing *_transfer_to_* task defs; those register fine and the handoff itself succeeds. Corrected in conductor-oss/conductor@63c0715b7.
• Timing correction for anyone reading that issue: a single attempt is ~304s (the SDK's 300s wait), not ~908s. The larger figure was three attempts under conftest's unconditional flaky(reruns=2).

[ling-senpeng13]

Follow-up that resolves open question 1 in the description ("is stateful intended to be inherited?"). I'd flagged it as needing a maintainer's call. It doesn't — the codebase already answers it.

_has_stateful_tools (runtime/runtime.py:142) is the SDK's canonical statefulness predicate, and it is already tree-recursive:

def _has_stateful_tools(agent: Any) -> bool:
    """Return True if the agent is stateful or any @tool has stateful=True."""
    if getattr(agent, "stateful", False):
        return True
    ...
    for sub in getattr(agent, "agents", []):
        if _has_stateful_tools(sub):        # <-- recurses into members
            return True
    return False

That predicate is what decides whether a domain exists at all:

run_id = uuid.uuid4().hex if _has_stateful_tools(agent) else None   # runtime.py:2491, 3550, 3952

and run_id is the task domain (result.py:267: "run_id: Domain UUID for stateful agents; None for stateless").

So the two halves were using different definitions of the same word:

Decision	Predicate	Tree-aware?
Does this run get a domain?	_has_stateful_tools(agent)	yes — recurses into members
Which workers poll that domain?	getattr(agent, "stateful", False) in _register_workers	no — immediate agent only

A stateful swarm therefore always minted a domain and tagged its members' tool tasks, while the members' tool workers were registered without one. Hence pollCount: 0.

That makes this a consistency fix rather than a semantic change: it makes worker registration agree with the definition the SDK already uses one function away, not with a guess about intent.

Two things follow:

1. The stateful-is-inherited semantics are pre-existing, so this PR isn't introducing them.
2. Same applies to the ConfigSerializer half — it was the third place computing statefulness, and the only one of the three that also got it non-recursively.

Worth considering as a separate cleanup (not in this PR): all three sites should probably call one shared helper rather than each re-deriving it. Three independent derivations of one concept is how they drifted apart in the first place.

Still genuinely open, and the reason I'd keep this draft for a moment: open question 2 — the serializer change alters what's sent to the server, and I've only verified suite14 plus the full mainline e2e (136 passed / 1 failed, the failure unrelated). Someone who knows the compiler should sanity-check the topologies that flag reaches.

[ling-senpeng13]

Why neither CI caught this

Worth recording, because the fix is small but the reason it went unnoticed is structural.

This repo's agent-e2e.yml pins the server:

env:
  CONDUCTOR_SERVER_VERSION: "3.32.0-rc.8"   # pinned server release — bump deliberately

3.32.0-rc.8 predates the server change that makes this bug fatal. The SDK has always failed to inherit statefulness into member agents' tool workers, but it only matters once the server domain-routes those member tool tasks — a post-rc.8 behaviour. On rc.8 the SDK's domain=None and the server's routing agree by accident, so test_stateful_swarm_handoff_completes genuinely passes and CI here is honestly green.

Meanwhile conductor-oss/conductor builds the server from main, so it does hit this — and its CI is green for the opposite reason: the test is suppressed there via a {"run": false} known-failures entry, meaning it never executes at all.

	Server under test	Suppression	Why green
python-sdk (here)	pinned 3.32.0-rc.8	none — nothing is xfail-ed or skipped	server predates the change that exposes the bug
conductor-oss/conductor	built from main	run: false xfail	test never runs

Neither side is misreporting. But between the two, the bug had nowhere to surface: the repo that owns the code tests against a server too old to show it, and the repo with a current server had stopped running the test.

The part I'd raise separately

This repo's e2e never runs against a current server. rc.8 is the pin; the current line is well past it. That's a standing blind spot for exactly this class of defect — server/SDK contract drift — and it will hide the next one the same way. Two options, neither in scope for this PR:

• Bump CONDUCTOR_SERVER_VERSION deliberately, as the comment already invites, and fix whatever that surfaces.
• Or keep the pinned lane as the PR gate and add a scheduled run against the latest server, so drift shows up on a cron rather than in a consumer's CI.

The second is the shape most projects settle on: pinned for reproducible PR signal, latest on a schedule for early warning. Happy to open an issue for it if that's useful.

Verification note for this PR

Because of the above, I validated against a server built from conductor-oss main rather than rc.8 — i.e. the configuration that actually exposes the bug. Full mainline e2e there: 136 passed, 1 failed, 7 skipped, 1 xfailed, the single failure being an unrelated scheduler 404-vs-None contract mismatch. A per-test diff against the pre-fix baseline over the 145 tests present in both runs showed 0 regressions and 1 improvement (this test).

[codecov]

Codecov Report

❌ Patch coverage is 85.71429% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
src/conductor/ai/agents/runtime/runtime.py	66.66%	1 Missing ⚠️

Files with missing lines	Coverage Δ
src/conductor/ai/agents/config_serializer.py	64.22% <100.00%> (+0.15%)	⬆️
src/conductor/ai/agents/runtime/runtime.py	50.50% <66.66%> (+0.02%)	⬆️

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.