Skip to content

Keep unfinished speech turns fail closed - #221

Merged
RobVanProd merged 4 commits into
mainfrom
codex/speech-terminal-reliability
Aug 11, 2026
Merged

Keep unfinished speech turns fail closed#221
RobVanProd merged 4 commits into
mainfrom
codex/speech-terminal-reliability

Conversation

@RobVanProd

@RobVanProd RobVanProd commented Aug 6, 2026

Copy link
Copy Markdown
Owner

Outcome

Keeps interrupted, stalled, expired, and incomplete microphone turns fail closed. Only a valid VAD trailing-silence terminal can reach Whisper; cancellation clears host PCM and runs zero STT/model/TTS.

Firmware

  • Measures endpoint timing from captured PCM, not missing wall time.
  • Uses nonblocking socket writes with retained terminal retry/cancel semantics.
  • Cancels on mic stalls, chunk discontinuity, backpressure, expiry, or terminal failure.
  • Adds phase, terminal, and write-deferral telemetry.

Host

  • Adds a 14.5 s absolute capture lease and 4 s inactivity lease.
  • Handles utterance_cancel idempotently and rejects stale/mismatched ends.
  • Clears partial PCM on cancellation, expiry, supersession, and disconnect.

Verification

  • pio test -e native_logic: 313/313 pass
  • python -m unittest discover -s bridge -p test_*.py: 584/584 pass
  • python -m py_compile bridge/lan_service.py: pass
  • pio run -e stackchan_release_full: pass
  • private stackchan_release_forensics build: pass, SHA-256 f1a6319fb1d6703c48f2d248c962ce4a267a7eed1db83b894e587b0933f370c2

Physical deployment and completed/incomplete utterance evidence remain pending and will be added without treating simulated evidence as physical.


Physical evidence, and a further firmware fix (2026-08-11)

The candidate image was still losing turns

Deployed f1a6319f (the image built from 52b06de1) to the reference robot and drove real wake turns.
A completed capture was rejected by the host:

audio_bytes declared 94400, received 92800; chunks declared 59, received 58
audio_end_counts_match: false   transcript_present: false
→ audio_without_utterance, payload_bytes: 1600, audio_protocol_errors: 1

The shortfall is exactly one 800-sample int16 chunk, and that chunk arrived after the terminal.

Root cause

BridgeSocketWriter holds text and binary in two independent single-slot buffers, and
drainPending() takes queued text before queued binary. submitPcmBytes counts a chunk as soon as it
lands in the binary slot, so an utterance_end queued in that window declares a chunk still sitting in
the writer and reaches the host ahead of it.

Two consequences that were invisible before:

  • The write-deferral telemetry could not see this. There was no backpressure — a text frame simply
    jumped a binary one — so bridge_network_write_deferrals stayed at 0 through every failure.
  • Firmware counted turns the host rejected. queueText() returning true marked the terminal
    delivered and incremented turnsCompleted, which is why wake_cue_captures_completed read 1 with no
    transcript. Firmware and host disagreed about the same utterance.

servicePendingTerminal() also had no firmware caller at all — it was reachable only from tests,
because the terminal had always been queued on the first attempt.

Fix (0205825e)

Hold an End terminal while the writer still owes binary bytes, queued or partially written, and retry
once the audio is gone. Cancel keeps pre-empting on purpose: cancelling is meant to beat the remaining
chunks out, and the host discards partial PCM for them. The existing terminalRetryMs bound still
applies, so audio that never drains fails the turn closed rather than leaving an unterminated upload.
Deferring makes the retry path live, so it now runs from the capture drain loop and the main service loop.

Native coverage asserts the wire order directly. With the hold defeated,
end_never_overtakes_queued_pcm and end_held_by_audio_fails_closed_on_timeout both fail while
cancel_still_preempts_queued_pcm passes either way.

Physical evidence on the exact image

Installed d631da56c004cc9c2263cbc8c01cf7a27fd0dfca5861c5461f2292681386d129, built from clean source
0205825e, OTA-confirmed on app0 with rollback armed, reset_reason software, boot_count 1.

utterance chunks received / declared bytes counts match transcript
23:19:46Z 149 / 149 238,400 yes yes
23:20:00Z 99 / 99 158,400 yes yes
23:20:44Z 217 / 217 347,200 yes yes

Zero audio_count_mismatch, zero orphaned audio_without_utterance. The 217-chunk capture is 10.85 s of
audio, well beyond anything that previously survived.

bridge_uplink_errors 0   queue_failures 0   terminal_timeouts 0   terminal_retries 0
wake_capture_discontinuities 0   wake_capture_timeouts 0   network_write_failures 0
mww_uplink_submit_failed 0   captures_completed 5   playback_starts 3   motion_enabled false

bridge_uplink_errors: 0 meets the zero-delta requirement carried from F2. Every response completed the
full response_started → response_end_deferred → reply_pending → response_end_after_playback_complete
sequence, and conversation v2 chained a second turn from one wake word before closing with episode
distillation.

Limits of this evidence

  • terminalAudioDeferrals is not exported to /debug in this image, so it cannot be shown that the
    new hold was exercised. The claim supported is narrower: the failure mode did not recur across three
    utterances including a 10.85 s capture, and terminal delivery was never harmed. Adding the export
    requires a rebuild, so it is deliberately not committed here — that would break the binding between
    this branch and the qualified image.
  • One earlier turn logged a completed capture on the robot with no corresponding host record at all. It
    occurred only while the supply was sagging and has not recurred in five subsequent captures. It remains
    unexplained rather than attributed.
  • One stale-state defect was found and is not addressed here: the host dashboard latches
    robot.connected and mode indefinitely after the robot disappears. Fixed separately in Report a silent robot as gone instead of latching its last state #222.

Correction. An earlier revision of this description also claimed the robot latches
bridge_state: error after a Wi-Fi recovery. That claim is withdrawn: it was not reproduced.
bridge_state is the BridgeClient conversational state machine
(Offline/Connecting/Ready/Listening/Thinking/Responding/Error), and a later observation showed it
advancing to listening on the following turn without intervention. The error reading was a
transient state, not a latch.

Operating note

An initial run produced no replies at all. The cause was power, not speech: VBUS sagged to 4,489 mV
against a 4,600 mV VINDPM target, so the PMIC stopped charging and the battery supplemented the system at
228 mA. In that state the microphone path degraded badly — 14 speech chunks out of 142 captured, level
below the noise floor. On wall power (4,810 mV, VINDPM released) the same firmware gave 134 speech chunks
and three spoken replies. Speech qualification should confirm character_energy_charging before drawing
conclusions about capture behaviour.

RobVanProd and others added 4 commits August 5, 2026 21:01
`bridge/transcript_diagnostics.py` arrived in a0f56b7 as a module-level
import of `lan_service.py`, but no packaging manifest learned about it. The
packaged desktop brain therefore extracted an incomplete bridge and died with
ModuleNotFoundError before argument parsing, which is what turned
DesktopBrainSupervisorTest red: every file-existence assertion passed and the
`--help` exit-code assertion did not.

The same omission reached the release package and its verifier, so a published
`stackchan_release_full` brain would have failed to start for the same reason.

Add the module to the desktop resource copy, the runtime extraction list, the
release packager, and the release verifier, and assert its presence in the
desktop test so a future module cannot go unpackaged silently.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The contract fixture still pinned the pre-PR #216 boundary: 130 chunks of
50 ms, a 6,500 ms capture ceiling. Firmware has carried the reviewed 240-chunk,
12,000 ms ceiling since #216, so the fixture failed the native-tests job on a
value it was supposed to be protecting. A gate that rejects the reviewed
constant is not guarding anything.

Move the pin to 240 chunks / 12,000 ms and derive both messages from the
variables so the two can no longer drift apart. Also assert the PR #217
ordering the fixture never covered: the capture ceiling must stay strictly
below the 15,000 ms wake-gate privacy guard, which is the inequality that
closed the F2 equal-threshold race.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Observed on the reference robot running the exact PR #221 candidate image
f1a6319f: a real wake turn was rejected by the host with audio_count_mismatch
(declared 59 chunks / 94,400 bytes, received 58 / 92,800), followed by an
orphaned 1,600-byte frame logged as audio_without_utterance. The difference is
exactly one 800-sample int16 chunk.

BridgeSocketWriter keeps text and binary in two independent single-slot buffers
and drainPending() takes queued text before queued binary. submitPcmBytes counts
a chunk as soon as it lands in the binary slot, so an utterance_end queued in the
same window declares a chunk that is still sitting in the writer and reaches the
host ahead of it. The write-deferral telemetry could not see this: there was no
backpressure, just a text frame jumping a binary one, which is why
bridge_network_write_deferrals stayed at 0 through every failure.

Worse, queueText() returned true, so the uplink marked the terminal delivered
and incremented turnsCompleted. Firmware counted the turn a success while the
host was rejecting it -- wake_cue_captures_completed and the host transcript
disagreed about the same utterance.

Hold an End terminal while the writer still owes binary bytes, queued or
partially written, and retry it once the audio is gone. Cancel terminals keep
pre-empting on purpose: cancelling is meant to beat the remaining chunks out and
the host discards partial PCM for them. The existing terminalRetryMs bound still
applies, so audio that never drains fails the turn closed instead of leaving an
unterminated upload open.

servicePendingTerminal() had no firmware caller at all -- it was reachable only
from tests, because the terminal had always been queued on the first attempt.
Deferring makes the retry path live, so drive it from the capture drain loop and
from the main service loop.

Native coverage asserts the wire order directly: with the hold defeated,
end_never_overtakes_queued_pcm and end_held_by_audio_fails_closed_on_timeout
both fail, while cancel_still_preempts_queued_pcm passes either way.

pio test -e native_logic: 316/316.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@RobVanProd
RobVanProd marked this pull request as ready for review August 11, 2026 23:25
@RobVanProd
RobVanProd changed the base branch from codex/aliveness-repository-truth to main August 11, 2026 23:32
@RobVanProd RobVanProd closed this Aug 11, 2026
@RobVanProd RobVanProd reopened this Aug 11, 2026
@RobVanProd
RobVanProd merged commit 8c8d454 into main Aug 11, 2026
11 checks passed
@RobVanProd

Copy link
Copy Markdown
Owner Author

Extended physical evidence on the same image (2026-08-11, post-merge)

Continued use of installed d631da56c004cc9c2263cbc8c01cf7a27fd0dfca5861c5461f2292681386d129 on app0 has grown the evidence from 3 utterances to 10, all still exact:

time (UTC) chunks received / declared counts match transcript
23:19:46 149 / 149 yes yes
23:20:00 99 / 99 yes yes
23:20:44 217 / 217 yes yes
23:33:02 57 / 57 yes yes
23:33:14 59 / 59 yes yes
23:33:24 65 / 65 yes yes
23:33:38 117 / 117 yes yes
23:33:48 69 / 69 yes yes
23:33:59 61 / 61 yes yes
23:34:10 60 / 60 yes yes

10 of 10 exact. Zero audio_count_mismatch. Zero orphaned audio_without_utterance. Longest capture 217 chunks = 10.85 s.

Robot counters after 66 minutes on this image: bridge_uplink_errors: 0, wake_cue_captures_completed: 12, bridge_downlink_playback_starts: 10, boot_count: 1, reset_reason: software — no reset since the OTA — and motion, servo rail, and torque all still off.

Conversation v2 sustained a seven-turn exchange

Session 5 ran 23:32:52 → 23:34:19 on a single wake word, chaining seven turns through utterance_committed → reply_window_open → listening without re-waking, then closing cleanly into episode distillation:

23:32:52 wake              session=5 turns=0
23:32:56 utterance_committed             turns=1
...
23:34:05 utterance_committed             turns=7
23:34:19 session_closed
23:34:20 episode_distillation

That is the back-and-forth conversation this lane exists to make possible, running on the fixed image.

Still not claimed

terminalAudioDeferrals remains unexported in this image, so this still cannot show the hold engaging — only that the failure did not recur across ten utterances with terminal delivery never harmed. #227 exports the counter for the next flash.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant