Harden Pi compaction, turn lifecycle, and session durability

[sambitcreate]

Summary

• complete Pi semantic compaction across primary and child turns, including adaptive thresholds, mid-loop checkpoints, bounded overflow recovery, and strict split-turn/map-reduce summary validation
• make Pi journals crash-recoverable and privacy-safe with transactional generation envelopes, corrupt/torn journal recovery, exact-owner deletion, orphan cleanup, and renderer-safe chat projection
• preserve provider-authored Pi protocol and usage accounting while validating stored provenance and keeping diagnostics private
• serialize effectful foreground tools, forward cancellation, restore exact schedule state, and reconcile exact terminal assistant output
• document the AgentHarness boundary and update the Pi implementation plans and coverage

Validation

• npm run type-check
• npm run test:compaction (95/95)
• focused provenance and schedule-service suites (19/19)
• npm run test (full suite, including 41 native worktree-remover tests and computer-use broker tests)

Review process

Three independent fresh-context reviewers audited compaction correctness, durability/privacy, and completeness. Their validated findings were fixed in the final three commits.

[pullfrog]

Caution

The context-overflow compact-and-retry and the transient-provider retry are both broken in production: the failed assistant attempt can never be isolated because transaction markers sit on the branch tail. See the inline comment for the exact mechanism and a fix.

Reviewed changes

• Crash-recoverable Pi journals — transaction envelopes, torn-final-line repair, corrupt-journal quarantine, exact-owner deletion, and startup orphan reconciliation.
• Pi provenance + renderer privacy — canonical provider protocol persisted under ChatMessage.pi and stripped by chatForRenderer before any renderer-facing return.
• Compaction hardening — pre-prompt pressure checks, adaptive thresholds, bounded map-reduce, and structural summary validation.
• Turn lifecycle + retry — sequential foreground tools, crash-recovery turn envelope, exact terminal projection reconciliation, and abortable retry backoff.
• Schedule + usage — AbortSignal forwarding through schedule mutations with restore semantics, plus compaction-source usage accounting and cacheWrite1h retention.

  ^{｜ Fix all ➔ ｜ Fix 👍s ➔ ｜ View workflow run ｜ Using DeepSeek Pro ｜ 𝕏}

[pullfrog]

✅ No new issues found.

Reviewed changes

• Unwrapped journal transactions for retries — abandonRetryableAssistant now scans backward past AIDEN_PI_TRANSACTION/AIDEN_CHAT_MESSAGE_MARKER markers to isolate the failed assistant attempt, then rolls back the entire committed envelope (via the begin marker's parentId), restoring the context-overflow compact-and-retry and transient-provider retry paths that the transaction wrapping had silently broken.
• Reproduced the production append path in tests — the overflow and transient-provider tests now append the failed assistant through appendPiMessages and assert the branch is left free of transaction markers and the failed error message after rollback.

  ^{｜ View workflow run ｜ Using DeepSeek Pro ｜ 𝕏}