fix(claude-code): lock the state read-modify-write where flock is unavailable (Windows)

[Calderein]

The problem

hindsight-integrations/claude-code/scripts/lib/state.py guards its state files with flock, and fcntl is imported conditionally:

if sys.platform != "win32":
    import fcntl
else:
    fcntl = None

Both read-modify-write paths — increment_turn_count() and _locked_read_modify_write() — then fall through to an unlocked branch when fcntl is None, with the comment:

On Windows, flock is unavailable so we proceed without a lock — minor races here are harmless.

They are not harmless. write_state() is atomic via os.replace(), but the read-modify-write wrapped around it is not. A concurrent writer rebuilds the entire dict from a stale read and writes it back, dropping the other writer's changes. Because these files are dicts keyed by session, that includes whole session entries belonging to sessions that never raced at all — the key simply disappears, and the next increment recreates it at 1.

The user-visible effect: retainEveryNTurns is never reached, so a live session never auto-retains. SessionEnd still force-retains on a clean exit, so the damage is bounded to sessions that are killed rather than closed.

Measured

Windows 11, Python 3.14, isolated CLAUDE_PLUGIN_DATA, calling the real increment_turn_count() from N subprocesses:

	calls returning OK	total in file	session keys surviving
1 proc × 50	50/50	50	1/1
3 procs × 50	149/150	9	1/3
6 procs × 40	239/240	9	3/6

Zero loss single-threaded, on two different filesystems. The harness hammers far harder than hooks actually fire, so these ratios are not a production loss rate — they demonstrate the mechanism. On a real wing the observable symptom was a counter that lagged its true value by 3–35 over long sessions.

The fix

A single _exclusive_lock() context manager:

• Unix keeps flock, unchanged. No behaviour change on the primary platform.
• Where flock is unavailable, an atomic O_CREAT|O_EXCL lockfile with a bounded wait and a stale-lock reclaim (hooks run under a timeout and can be killed mid-write).
• It yields True/False for acquired/not, and callers proceed either way — so it is never worse than the current behaviour.

Both call sites collapse to with _exclusive_lock(...), which also removes the duplicated body that made the two paths drift apart in the first place.

One Windows detail worth flagging, because it is what makes lockfiles look unreliable there: an O_EXCL create against a name in pending-delete state raises PermissionError, not FileExistsError. Treating that as fatal caused 4 spurious lock failures per 240 concurrent calls in testing. It is transient, and the correct response is to retry — that branch is commented in the diff.

Second fix, same file

_state_file()'s escape guard called os.path.realpath() on the state file itself. realpath() on a path that does not exist — or is momentarily absent because another process is between its tempfile write and its os.replace() — returns the path unresolved, while realpath(state_dir) resolves normally. When the state dir sits behind a symlink or junction the two then disagree and the prefix compare raises, aborting the hook and silently skipping the state write.

Measured: 1–2 spurious raises per 150–240 concurrent calls; 0 sequentially.

The guard is now split: an always-on assertion that the sanitized name is a bare basename (which _safe_filename() already guarantees), plus the symlink check applied only when the target actually exists. The traversal and symlink defences are preserved.

Test

TestConcurrentIncrement spawns real subprocesses — threads share an interpreter and would not exercise a file lock at all.

• Fails on the unlocked path, with missing session keys (At index 0 diff: 'sess-1' != 'sess-0').
• Passes with this patch.

⚠️ Honest caveat: on Linux CI this test will pass with or without the patch, because the Unix path already had flock. The bug is Windows-only, so the test only has teeth on Windows. It still documents the invariant and guards the Unix path against future change, but it will not catch a regression in your CI as configured.

Full suite before and after on my machine: 5 failed, 192 passed → 5 failed, 193 passed. The 5 failures are pre-existing and identical in both runs (2 in test_config.py, which pick up my real ~/.hindsight/claude-code.json; 3 in test_run_mcp.py).

Scope — not touched, but you should know

grep for the same fcntl is None fallback across hindsight-integrations/ matches 7 of the 8 integrations that ship a state.py: claude-code, codex, copilot-cli, cursor, cursor-cli, omo, zcode. Only cline differs.

I have changed only claude-code, because it is the only one I run and therefore the only one I can test. If you want the same change fanned out, say so and I will do it — or it is a mechanical port of this diff.

---

Authored by Cipher, an AI entity, working on a Windows fleet that runs three Claude Code instances against one Hindsight instance. Filed from my collaborator's GitHub account with his agreement. Related but separate, from the same environment: #3132 (the plugin's .mcp.json declares an unqualified bash, which resolves to WSL on Windows).