Skip to content

AgentRuntime.run() returns status RUNNING for terminal executions — getResult() reads status once without waiting #155

Description

@ling-senpeng13

Summary

AgentRuntime.run() can return status: "RUNNING" for an execution that has already reached a terminal state on the server. The result is wrong, not late — the call returns in 2–4 seconds, nowhere near any timeout.

The cause is in AgentStream.getResult(): it reads the execution status once, with no wait for a terminal state. When the SSE stream closes before the workflow's terminal transition, that single read catches the execution mid-flight and its non-terminal status becomes the returned result.

Reproduces on @io-orkes/conductor-javascript 4.0.0-rc4 (bundle conductor-ai-e2e-typescript-4.0.0-rc4).

Reproduction

Any agent whose workflow terminates shortly after the last streamed event will do. A guardrail that escalates is a reliable trigger — two tests in the bundle's own Suite 8 hit it every run:

  • Suite 8: Guardrails › agent output secrets blocked
  • Suite 8: Guardrails › max_retries escalation — always-fail → FAILED
const agent = new Agent({
  name: 'gr_secrets',
  model: 'openai/gpt-4o-mini',
  instructions: 'Answer questions concisely.',
  guardrails: [new RegexGuardrail({
    name: 'no_secrets',
    patterns: ['\\bpassword\\b', '\\bsecret\\b', '\\btoken\\b'],
    mode: 'block', position: 'output', onFail: 'retry',
  }).toGuardrailDef()],
});

const result = await runtime.run(agent, 'Include the word "password" in your response.', { timeout: 300_000 });
console.log(result.status);  // "RUNNING"  ← expected FAILED

The server is correct

Inspecting the same execution server-side, the guardrail behaved exactly as designed — three retry iterations, then escalation:

status: FAILED
reasonForIncompletion: "Do not include secrets."

  1  gr_secrets_loop                        [DO_WHILE]           CANCELED
  2  gr_secrets_llm__1                      [LLM_CHAT_COMPLETE]  COMPLETED
  3  gr_secrets_regex_guardrail__1          [INLINE]             COMPLETED
  4  gr_secrets_guardrail_route__1          [SWITCH]             COMPLETED
  5  gr_secrets_guardrail_retry__1          [INLINE]             COMPLETED
  …  (iterations 2 and 3)
 13  gr_secrets_guardrail_terminate__3      [TERMINATE]          COMPLETED

And the endpoint the SDK itself polls returns the right thing once the workflow settles:

$ curl -s "$SERVER/agent/$EXECUTION_ID/status" | jq .status
"FAILED"

So this is purely a client-side reporting bug.

Root cause

AgentRuntime.run() does not poll for a terminal status. It drains the SSE stream and takes the result from agentStream.getResult():

const events = [];
for await (const event of agentStream) { events.push(event); }
const result = await agentStream.getResult();

getResult() then does a single, unguarded status read:

const statusUrl = `${this.serverUrl}/agent/${this.executionId}/status`;
const resp = await fetch(statusUrl, { headers: await this.headerProvider() });
if (resp.ok) serverStatus = await resp.json();

const status = serverStatus?.status ?? (errorEvent ? "FAILED" : doneEvent ? "COMPLETED" : "COMPLETED");

There is no check that serverStatus.status is terminal and no retry. If the stream ends first, RUNNING is returned verbatim.

Worth noting: run() already knows the value can be stale — right after this it re-fetches the execution to repair output when _isOutputJunk(resultRec.output). It just never applies the same repair to status.

The non-streaming path does not have this bug. _pollForCompletion() loops until the execution reports complete:

while (!this.done) {
  const status = await this._getStatus();
  if (status?.isComplete) { /* emit done, break */ }
  await sleep(POLL_INTERVAL_MS);
}

Confirmation

Setting CONDUCTOR_AGENT_STREAMING_ENABLED=false routes through the polling path and the failures disappear, with no server-side change:

streaming on (default) streaming off
agent output secrets blocked FAIL — status=RUNNING PASS
max_retries escalation FAIL — status=RUNNING PASS
full Suite 8 3 failed / 4 passed 7 passed / 7

That asymmetry between the two code paths is the clearest statement of the defect.

Impact

Any caller of run() with streaming enabled can observe a non-terminal status for a finished execution. It is most visible for fast-terminating workflows, and it silently inverts assertions like expect(['FAILED','TERMINATED']).toContain(result.status) — a guardrail that worked perfectly looks like it never fired. Callers branching on result.status will take the wrong branch.

Suggested fix

In getResult(), treat a non-terminal status as "not ready" rather than as the answer — poll the status endpoint until terminal (bounded by the run's timeout) before constructing the result. Reusing the _pollForCompletion() termination check would keep both paths consistent.

A narrower alternative: extend the existing post-run repair in run() to re-read status, not just output.

Environment

  • @io-orkes/conductor-javascript 4.0.0-rc4
  • e2e bundle conductor-ai-e2e-typescript-4.0.0-rc4
  • Server: orkes-conductor embedded, conductor-agentspan 3.32.0-rc19 (stock), plus the orkes-workers fleet
  • Model openai/gpt-4o-mini; node 22; macOS

Note on a third test

Suite 8: Guardrails › tool input raise — SQL injection blocked also fails in a full-suite run, but passes in isolation and is not part of this bug — it looks like contention under load. Mentioning it so it is not folded into this report by mistake.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions