feat: derive watchdog thresholds from the pod and make its guards structural

[ppat]

What this changes

The memory watchdog shipped in observe mode with hand-picked constants and
name-matched safety guards. This makes both derived and structural, fixes three
defects found by running it against a live workspace, and — more importantly —
records what the live runs showed about whether the mechanism addresses the
failure this pod actually has.

1. Thresholds and ceilings derive from the pod

Tier thresholds and RLIMIT_DATA ceilings were absolute bytes sized for an
8 GiB pod. On the 4 GiB workspace the watchdog detected its own miscalibration
and warned — the right failure, but it left the mechanism usable at one pod size.

Both now come from memory.max. One critical reserve C = memory.max / 10,
clamped to [384 MiB, 1 GiB]; the ladder is L4 = C, L3 = 1.5C, L2 = 2.5C,
L1 = 4C. Ceilings are sixteenths of memory.max, clamped to [512 MiB, 4 GiB].

memory.max	L4	L3	L2	L1
4 GiB	0.40	0.60	1.00	1.60
8 GiB	0.80	1.20	2.00	3.20
16 GiB	1.00	1.50	2.50	4.00

At 8 GiB this reproduces the hand-tuned numbers it replaces (0.75/1.25/2.00/3.00),
which is the only calibration point that existed. The cap answers the original
objection to percentages — a fraction alone would hand 16 GiB a 4 GiB "critical"
reserve. Below roughly 3 GiB the ladder cannot fit inside the pod at all, so
enforce mode refuses itself and logs why rather than shedding the editor forever.

2. The never-signal guards are structural

They were substring matches over the joined command line and had over-matched
twice — */claude* on a scratchpad path containing /claude, and
*memory-watchdog* on a test harness whose own directory path contained it,
which silently protected every process in that harness and left two full runs
green while asserting nothing.

Now: comm, argv[0]'s basename, and whole path segments of argv elements —
the same kind of rule positive selection already used. The watchdog's own
identity is its pid, ancestors and descendants, not a name. Every guard records
which rule claimed a process, and the tests assert each rule is individually
reachable, because a guard nothing can trigger is untested rather than correct.

3. Frequency is treated as part of correctness

An editor that dies every fifteen minutes gets the watchdog switched off, and a
watchdog that is switched off protects nothing. Each rung now fires at most once
per excursion below L1 and only recovery re-arms it; the fixed 180 s cooldown is
replaced by a settle window that no longer blocks escalation (it used to demote
L3 to L1 for three minutes after an L2 shed — exactly when escalation mattered).
A shed below L4 must free at least 128 MiB or it is declined and logged. Observe
mode publishes the shed rate enforce mode would have produced, so the question
"would this have been tolerable?" is answerable before enabling anything.

4. Three defects found on the live pod

• A stale pidfile disarmed the watchdog permanently. OOM kill → SIGKILL → no
  EXIT trap → pidfile survives on NFS → every restart logged "another instance is
  already running" and exited. The pod was unwatched from the first kill onward.
  Liveness now decides, not the existence of a file.
• Ceilings were computed against the wrong quantity. RLIMIT_DATA accounts
  data_vm, not RSS. Live at rest: extension host 1004 MB data / 497 MB
  resident; file watcher 622 MB / 66 MB. The derived file-watcher ceiling of
  512 MB was below what an idle file watcher already held. A ceiling is now a
  growth allowance above observed usage.
• The poll interval could not see the event it exists for. See below.

Live results (4 GiB test workspace)

Full evidence, reproducible, on that workspace at ~/watchdog-live-evidence/
(2026-08-17-README.md indexes it). The script used there is byte-identical to
this branch.

Run	Setup	Outcome
Idle	enforce, real VS Code tree	Ceilings set on the three real processes, each above its observed VmData; ptyHost untouched; hard limits untouched; L0 throughout; zero sheds
A	runaway at the production rate (~91 MB/s), enforce	OOMKilled in 43 s. The watchdog never left L0 and logged no action at all
B	same runaway, soft RLIMIT_DATA 1.5 GiB on it	RangeError at 960 MB after 10.5 s, process died alone, container never reached its limit
C	same runaway, enforce, with the rate-keyed interval	Sampled every 1 s, climbed L0 → L3 → L4 as designed — and still could not help: no-candidates tier=L3 tree=0 eligible=0. OOMKilled anyway

Run A is why the interval is now keyed on measured dU/dt rather than on tier:
four samples cannot satisfy a three-sample debounce and climb three rungs. Run C
confirms the fix and, in the same breath, shows its limit — seeing the event only
helps if the growth is inside the tree the watchdog manages.

Run C's no-candidates line is the new observability doing its job: an acting
tier that can do nothing now says so every cycle, with the census. Without it,
run C would have looked exactly like run A's silence — which is how the second
guard bug hid.

What this does not fix, stated plainly

The kernel's own OOM records for this workspace name a Claude Code session as the
victim in every kill in the retention window, with one process reaching 7.3 GiB
of anonymous RSS in an 8 GiB cgroup. That window is biased — VS Code was being
deliberately kept away from that workspace — so it is not an exoneration of the
editor. But two things follow regardless:

• The editor tree this watchdog manages is roughly 0.7 GiB. Shedding all of it
  buys seconds against a runaway of that size. The ladder is a brake.
• The lever that does convert the failure into a survivable one is the same
  RLIMIT_DATA cap, pointed at the runaway itself. It is preventive, so it never
  races the spike. Verified for both node and Bun: the allocation fails inside
  the offending process with an ordinary catchable error.

Extending ceilings beyond the editor tree is therefore the obvious next step, and
it is deliberately not in this PR because it decides what may happen to the
operator's own long-running sessions.

Also measured, and a dead end worth recording: BUN_JSC_forceRAMSize and
BUN_JSC_gcMaxHeapSize made no difference to Bun 1.3.14's allocation ceiling
(608 / 640 / 608 / 608 MB across the variants). Bun reports totalmem 121.6 GB
inside the cgroup, and node sizes its heap at 2240 MB from the same illusion, but
the Bun env-var route does not correct it.

Verification

• script-memory-watchdog-test.sh: 169 assertions, run by the watchdog job in
  .github/workflows/test.yaml. New coverage: the derivation across pod sizes
  including both degenerate ends; the historical over-match paths asserted not
  protected, paired with proof they are otherwise reachable; every guard rule
  asserted individually reachable; rung-per-excursion and settle-window
  behaviour; ceilings never below observed usage; interval chosen by rate; stale
  pidfile takeover, with the mutation that must flip it.
• pre-commit run --all-files passes (hadolint SC3037/DL3066 on the Dockerfile
  is pre-existing and untouched).

Disruption

The test workspace was OOM-killed several times producing runs A–C, which killed
the VS Code session attached to it each time. It has been restored to the
released observe-mode watchdog. Nothing ran against any other workspace.

[homelab-workflows-bot]

🎉 This PR is included in version 2.26.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀