feat(serve): NDJSON streaming for Ollama /api/chat + /api/generate — true drop-in (PMAT-928)

[noahgift]

Summary

PMAT-923 (#2216) wired Ollama /api/chat + /api/generate into the 5 real apr serve routers but always returned a single coalesced (done:true) body — the adapter forced stream:false. Ollama clients default to stream:true and expect newline-delimited JSON: a sequence of {...,message:{role,content:<token>},done:false} chunks then a terminal {...,done:true,...stats}. So a default Ollama client streaming from apr serve saw a non-streaming body.

This PR makes an Ollama request with stream != false (Ollama's wire default) respond with a chunked application/x-ndjson body — one done:false object per token then a terminal done:true object with stats. stream:false keeps the existing single coalesced object (no regression).

Streaming is real (reuses the /v1 token stream)

The /v1/chat/completions handler on the APR-CPU router already does true per-token SSE streaming via spawn_cpu_streaming_task → generate_with_cache_streaming → an mpsc channel of token IDs. The Ollama NDJSON path reuses that same incremental stream: a new spawn_cpu_token_text_stream sends each token's decoded text through the channel, and ollama_ndjson_stream reshapes each token into an NDJSON line. Only the wire framing differs (NDJSON lines vs SSE data: events) — it is NOT a re-decode of a coalesced batch result.

Wired into all real apr serve routers (the #2216 lesson — apr serve does NOT mount realizar's create_router):

• APR-CPU (build_apr_cpu_router): true per-token NDJSON.
• GPU-fallback (build_gpu_router), WGPU, SafeTensors x2: batch backends (generate_with_cache), so they honor stream:true with correct NDJSON framing (one content chunk + terminal done:true) over their coalesced result via reshape_openai_to_ollama_ndjson — honest about granularity, correct wire shape.

Falsifier

crates/apr-cli/tests/ollama_ndjson_streaming.rs drives the REAL APR-CPU serve router via build_demo_streaming_apr_cpu_router_for_test (a scripted token sequence flows through the IDENTICAL mpsc + NDJSON reshape pipeline the transformer uses — only the token source is faked, never the wire framing):

• stream:true → asserts application/x-ndjson AND multiple newline-delimited objects with intermediate done:false token chunks + a final done:true (NOT one object); token chunks reassemble to the full generation.
• stream:false → exactly one coalesced object (application/json, done:true).
• absent stream → streams (Ollama default).

RED on the old coalesced code (single application/json object even with stream:true); GREEN on the fix. Mutation-verified: forcing the stream branch off internally collapses the body back to one coalesced object → the multi-line falsifiers flip RED (verified locally).

Contract

contracts/apr-serve-openai-compat-v1.yaml v1.14.0 → v1.15.0 adds OBLIG-OLLAMA-NDJSON-STREAMING + 3 single-line falsifier refs. pv validate + pv lint contracts/ pass (0 errors).

Tests

• cargo test -p apr-cli --lib serve::ollama — 14 pass (incl. 7 new NDJSON unit tests)
• cargo test -p apr-cli --test ollama_ndjson_streaming — 4 pass (new falsifier)
• cargo test -p apr-cli --test ollama_api_serve_compat — 3 pass (PMAT-923, no regression)
• cargo test -p aprender-serve --lib — 15450 pass
• cargo clippy -p apr-cli --all-targets --features inference clean on all touched files

🤖 Generated with Claude Code