Track OpenClaw Telegram latency + self-healing auth health ops overlay

[marcusglee11]

Summary

Backfill tracking issue for an OpenClaw runtime/ops change that was made during Telegram alert/latency cleanup.

This issue exists because quick runtime/config/cron/ops overlay changes should not remain as untracked local patches. They need an auditable record with intent, touched surfaces, evidence, rollback, and upgrade revalidation.

Current status

Implemented locally; monitoring/revalidation required.

Change set

Telegram latency mitigation

• Set OpenClaw config:
  • channels.telegram.pollingStallThresholdMs = 60000
• Restarted OpenClaw gateway to apply.
• Verified config reads back 60000.
• Sent Telegram latency probe:
  • Telegram message id: 380

Self-healing auth health gate

Installed local ops script:

• ~/.openclaw/evidence/viability_recovery_v1/probes/self_healing_auth_gate.sh

Installed host crontab entry:

# OPENCLAW_SELF_HEALING_AUTH_GATE_V1
7 * * * * /bin/bash "$HOME/.openclaw/evidence/viability_recovery_v1/probes/self_healing_auth_gate.sh" >> "$HOME/.openclaw/evidence/viability_recovery_v1/probes/host_cron_self_healing_auth_gate.log" 2>&1

Behavior:

1. Runs openclaw models status --check.
2. If unhealthy/warning, identifies expired OAuth auth profiles.
3. Backs up auth profile store.
4. Removes only expired OAuth profiles.
5. Re-runs health check.
6. Sends Telegram only for fixed or still blocked outcomes.
7. Dedupes/cooldown-limits repeated unhealthy alerts.

Evidence

Config backup before Telegram threshold change:

• ~/.openclaw/evidence/viability_recovery_v1/reports/openclaw.config.bak.before_telegram_polling_threshold.20260428T054415Z.json

Managed ops overlay:

• ~/.openclaw/evidence/viability_recovery_v1/reports/managed_ops_overlay_telegram_health_20260428T054415Z.md

Memory note:

• memory/2026-04-28-telegram-health-overlay.md

Manual/self-healing verification:

• ~/.openclaw/evidence/viability_recovery_v1/reports/self_healing_auth_gate_20260428T053743Z.md
• Latest scheduled run observed:
  • ~/.openclaw/evidence/viability_recovery_v1/reports/self_healing_auth_gate_20260428T060701Z.md
  • initial_exit_code=0
  • post_exit_code=0
  • action=none
  • send_reason=none

Current channel verification:

• openclaw config get channels.telegram.pollingStallThresholdMs -> 60000
• openclaw channels status --deep --probe -> Telegram enabled/configured/running/connected, mode polling, bot @COOLifeOSBot, token env works.

Important follow-up finding

The 60s watchdog threshold helped make the setting explicit and lowered the intended stall ceiling, but logs still show Telegram getUpdates stalls after the change, including examples around:

• 2026-04-28T15:53:38+10:00 active getUpdates stuck for ~86s
• 2026-04-28T16:17:50+10:00 active getUpdates stuck for ~76s
• 2026-04-28T16:20:42+10:00 active getUpdates stuck for ~107s
• 2026-04-28T16:28:55+10:00 active getUpdates stuck for ~113s

So this is not fully closed as a latency fix. It is an ops overlay plus initial mitigation. Next technical investigation should determine why the polling watchdog fires later than 60000ms and/or whether webhook mode is required.

Rollback

1. Restore config from:
   • ~/.openclaw/evidence/viability_recovery_v1/reports/openclaw.config.bak.before_telegram_polling_threshold.20260428T054415Z.json
2. Remove host crontab line containing:
   • self_healing_auth_gate.sh
3. Restore prior crontab if required from:
   • ~/.openclaw/evidence/viability_recovery_v1/reports/crontab.bak.before_self_healing_auth_gate.20260428T053839Z
4. Leave evidence files in place unless intentionally cleaning local evidence.

Upgrade revalidation checklist

After OpenClaw runtime upgrade:

• Confirm channels.telegram.pollingStallThresholdMs still exists and remains 60000.
• Confirm openclaw models status --check exit semantics remain: 0 healthy, 1 unhealthy, 2 warning.
• Run self_healing_auth_gate.sh manually and confirm a report is produced.
• Confirm host crontab still contains OPENCLAW_SELF_HEALING_AUTH_GATE_V1.
• Confirm Telegram inbound/outbound round trip.
• Review Telegram logs for Polling stall detected after the threshold change.

Proposed durable policy

For LifeOS/OpenClaw operations:

• Any runtime/config/cron/ops overlay change gets a GitHub issue.
• Emergency changes may execute first, but must be backfilled immediately.
• Issue must include: intent, exact files/config touched, commands/evidence, rollback, upgrade revalidation, status.

[marcusglee11]

EA dispatch attempt blocked (2026-04-28)

COO attempted to action this via Codex EA.

Result: blocked before repo mutation. No PR created; no files changed.

Evidence from EA run:

• Cheap Codex model attempt failed first:
  • gpt-5-mini unsupported on the current ChatGPT-backed Codex account.
• Respawned with supported Codex default.
• Branch/worktree creation failed because repo git metadata is read-only:
  • git fetch origin main --prune -> cannot open '.git/FETCH_HEAD': Read-only file system
  • git worktree add -b fix/issue-67-openclaw-ops-overlay ... origin/main -> .git/refs/...lock: Read-only file system
• EA could not post this comment itself because its GitHub auth path reported invalid token inside the worker context.
• EA could not send Telegram completion because OpenClaw CLI plugin runtime deps path was read-only inside the Codex sandbox:
  • ~/.openclaw/plugin-runtime-deps/... EROFS: read-only file system
• Repo checkout remained clean on branch:
  • fix/codex-only-ea-dispatch

Coordinator assessment:

This issue remains open and unbuilt into the repo. Next action is to fix the EA execution substrate/write path for LifeOS worktrees/refs, then rerun the #67 implementation task.