Skip to content

fix(templates): read long stream-json lines and bound Claude Code shutdown - #542

Open
michaelxu2288 wants to merge 1 commit into
scaleapi:mainfrom
michaelxu2288:fix/claude-code-template-stream-limits
Open

michaelxu2288 wants to merge 1 commit into
scaleapi:mainfrom
michaelxu2288:fix/claude-code-template-stream-limits

Conversation

@michaelxu2288

@michaelxu2288 michaelxu2288 commented Oct 1, 2026 •

Copy link
Copy Markdown

Problem

The Claude Code scaffolds (sync-claude-code, default-claude-code, temporal-claude-code) and their tutorial copies (00_sync/060, 10_async/00_base/130, 10_async/10_temporal/140) share two bugs in how they read the CLI.

1. Lines over 64 KiB abort the turn. stdout is read through asyncio's StreamReader with its default 64 KiB limit (async for chunk in proc.stdout is readline() underneath). Claude Code writes one JSON line per event, so a tool_result that echoes a large file read is a single line far past 64 KiB. readline() then raises ValueError: Separator is found, but chunk is longer than limit, and the turn fails before its result event. The same class of bug was already fixed for agentex agents run (SUBPROCESS_STREAM_LIMIT in cli_utils.py).

2. Cancellation can hang. Cleanup sends SIGTERM and then awaits proc.wait() with no bound. A CLI that ignores or delays SIGTERM blocks request cancellation (sync/async ACP) or Temporal activity cancellation indefinitely.

Fix

In each of the six files:

  • read stdout with limit=STDOUT_LINE_LIMIT (8 MiB);
  • after SIGTERM, wait at most TERMINATE_TIMEOUT_SECONDS (5 s), then SIGKILL;
  • document both in _spawn_claude's docstring.

Nothing else changes. The Codex templates read with read(4096) and use kill(), so neither bug applies to them. The Gemini CLI templates in #516 got the same treatment.

Verification

I put a fake claude on PATH and drove _spawn_claude() from the rendered sync and Temporal templates.

case main this branch
100,000-character stream-json line ValueError: Separator is found, but chunk is longer than limit line read whole
CLI ignores SIGTERM, consumer calls aclose() still hanging after 20 s returns after 5.0 s
  • tests/lib/cli/test_init_templates.py: 26 passed.
  • Each tutorial's tests/test_agent_offline.py: 7, 5 and 5 passed.
  • ruff check is clean on the tutorial files, and all three compile.

RetriggerConfidence Score: 4/5

The PR appears safe to merge, though a subprocess test would help keep both fixes working.

Fix All in CursorFindings

  1. P2 Subprocess fixes lack tests ▶
Fix with agent prompt
### Issue 1
src/agentex/lib/cli/templates/sync-claude-code/project/acp.py.j2:131-137
The new five-second shutdown path and the larger line limit have no test that runs a subprocess. The template test only checks Python syntax, and the tutorial tests replace `_spawn_claude` with fake lines. Please add a subprocess test with a long JSON line and a child that ignores SIGTERM, so a later change cannot silently bring back either bug.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

This PR updates the Claude Code scaffolds and tutorial copies to read stream-json lines up to 8 MiB and to bound subprocess shutdown. It addresses large tool-result lines and cleanup when a caller stops reading.

  • Raises the stdout line limit for Claude Code subprocesses.
  • Waits five seconds after SIGTERM, then sends SIGKILL if the CLI is still running.

Reviews (1) · Last reviewed commit: "fix(templates): read long stream-json li..."

…tdown

The Claude Code scaffolds (sync, async and Temporal) and their tutorial
copies read `claude -p --output-format stream-json` stdout through
asyncio's StreamReader with the default 64 KiB line limit. Claude Code
writes one JSON line per event, so a tool_result that echoes a large
file read is a single line well past 64 KiB; readline() then raises
"Separator is found, but chunk is longer than limit" and the turn aborts.
Read with an 8 MiB limit, the same value `agentex agents run` uses for
its child processes.

Their cleanup also sent SIGTERM and then awaited proc.wait() with no
bound, so a CLI that ignores or delays SIGTERM hangs request or activity
cancellation. Wait at most 5 s, then SIGKILL.

Verified with a fake `claude` on PATH, against the rendered sync and
Temporal templates:
- a 100,000-character line: main raises the ValueError, this branch reads
  it whole;
- a CLI that ignores SIGTERM: main's aclose() is still hanging after
  20 s, this branch returns after 5 s.
tests/lib/cli/test_init_templates.py passes, and the three tutorials'
test_agent_offline.py suites pass (7, 5 and 5 tests).
Comment on lines 131 to +137
proc.terminate()
except ProcessLookupError:
pass
await proc.wait()
try:
await asyncio.wait_for(proc.wait(), timeout=TERMINATE_TIMEOUT_SECONDS)
except TimeoutError:
try:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Subprocess fixes lack tests

The new five-second shutdown path and the larger line limit have no test that runs a subprocess. The template test only checks Python syntax, and the tutorial tests replace _spawn_claude with fake lines. Please add a subprocess test with a long JSON line and a child that ignores SIGTERM, so a later change cannot silently bring back either bug.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agentex/lib/cli/templates/sync-claude-code/project/acp.py.j2
Line: 131-137

Comment:
**Subprocess fixes lack tests**

The new five-second shutdown path and the larger line limit have no test that runs a subprocess. The template test only checks Python syntax, and the tutorial tests replace `_spawn_claude` with fake lines. Please add a subprocess test with a long JSON line and a child that ignores SIGTERM, so a later change cannot silently bring back either bug.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Fix in Cursor Fix in Claude Code Fix in Codex

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant