fix(coordination): preserve shadow lineage across root aliases - #4896
huangruiteng merged 4 commits into
Conversation
Accept both lexical and canonical runtime-root digests when Python reads the TypeScript-owned management journal. Bind Python and TypeScript outbox entries, plus drain receipt validation, to the active lineage digest instead of recomputing a conflicting path identity. Add explicit symlink-root regression coverage so macOS temporary roots exercise the cross-runtime path. Signed-off-by: duanjialing.777 <duanjialing.777@bytedance.com>
…-digest-parity Signed-off-by: duanjialing.777 <duanjialing.777@bytedance.com>
|
Synced current Fresh validation on the merged head:
CI is rerunning for the new exact head. This control-plane fix remains maintainer-merge only. |
|
Coordination note: #4898 was opened shortly after this PR with the same production scope. I compared the implementations and left one concrete invariant finding there: its Python reader currently accepts a mismatch between the journal-level and binding-level root digests, while the TypeScript decoder and this PR require exact equality after validating the journal digest against the lexical/canonical root set. I reproduced that mismatch acceptance on #4898's exact head. I am keeping this PR open pending maintainer choice rather than duplicating the other PR's additional test coverage. The current head remains the smaller strict-invariant implementation and is rebased on |
Signed-off-by: duanjialing.777 <duanjialing.777@bytedance.com>
|
Added a focused fail-closed regression on exact head |
…-digest-parity Signed-off-by: duanjialing.777 <duanjialing.777@bytedance.com>
|
Synced latest |
huangruiteng
left a comment
huangruiteng
left a comment
There was a problem hiding this comment.
动机
我按 exact head 98c400a5009e08d972add5126a6d77dab2dce3a9 审阅了 #4892、完整 diff、相邻实现 #4898，以及 Python/TypeScript 的 shadow-management reader、两个 outbox writer 与 drain。这个 PR 解决的是一个真实的跨运行时持久状态缺口：macOS 默认临时目录含路径别名，TypeScript owner 写 canonical root digest，而 Python 旧 reader/writer/drain 从 lexical spelling 重算 digest，导致 Stage 2C2 在 guard 或 receipt history 处 fail closed。
相邻 #4898 的正常路径也能恢复，但它会接受 journal 顶层与 binding 分别使用 lexical/canonical 两个 digest 的 mixed state；本 PR 选择更窄的修复，保留既有 state→binding exact equality，并已补上这个负向回归。因此它不是重复增加第二套语义，而是当前两个候选中与既有跨语言 contract 对齐的实现。
改动思路
正确边界分两步：首先，runtime root 的 lexical/canonical spelling 只用于判断 persisted journal 的顶层 source_root_digest 是否属于同一个物理 root；其次，一旦顶层 digest 被接受，它就是该 journal 的精确 lineage identity，binding、Python Todo entry、TypeScript lease entry 与 drain receipt 都必须复用这个单值。
本 PR 复用了这个既有 owner：Python _source_root_digests() 对齐 TypeScript sourceRootDigests()，而 _binding(..., root_digest) 仍要求精确相等。随后两个 writer 与 drainer 从已验证的 active binding 取 digest，不再各自从调用路径推导。这同时保持了 foreign-root、stale-lineage、sequence 与 receipt mismatch 的原有 fail-closed 行为。
具体改动
shadow_management.py增加 lexical/canonical digest set，只放宽顶层 root spelling admission，不放宽 state/binding identity。- Python Todo capture 与 TypeScript lease capture 改用
binding.source_root_digest生成 entry identity 和持久字段。 _PartitionDrainer缓存同一 binding digest，并用它校验 receipt history。- Python/TypeScript tests 增加显式 symlink root、双 writer alias、foreign root，以及 mixed journal/binding digest 必须拒绝的覆盖。
关键代码讲解
_source_root_digests/read_shadow_management_state：允许abspath或realpath的顶层 digest；通过后把 exactstate.source_root_digest传给_binding，所以两个合法 spelling 可以分别作为完整 journal identity，但不能在同一 journal 内混用。TodoPartitionCapture.prepare：entry id 与 record 的 root digest 都来自 active binding；primary source write 仍只有在 outbox prepare 成功后继续，default-off 时保持shadow_disabled。beginLeaseOutboxEntry：TypeScript lease writer删除resolve(runtime_root)的独立重算，复用 binding digest；lease partition 的其它投影、sequence 和 commit marker 语义未变。_PartitionDrainer._partition_history：receipt 与当前 binding digest exact 比较；root alias 不再制造假 mismatch，真实 foreign/mixed receipt 仍被拒绝。
对主干的风险
最高风险是把“同一物理 root 的两种 spelling”误实现为“同一 journal 内两个 identity 字段可以不同”。该 exact head 已在 Python 回归中构造 lexical 顶层 + canonical binding，并验证 shadow_management_state_invalid；TypeScript decoder 的 exact equality 与之相同。正常 alias 路径则覆盖了 Python reader、Python Todo writer、TypeScript lease writer 和一次真实 drain。
验证结果：Python shadow-management/outbox suites 24 passed；TypeScript outbox suite 21/21 passed；control-plane typecheck 与 changed-file Ruff 均通过；真实 s2c2.outbox_prepared_then_committed_entries 行执行成功，Python/TypeScript 两个 partition 各交付一次；标准 premerge canary 13/13 通过，无 warning/manual hold。当前 head 相对最新主干无 changed-file 交集，merge-tree 无冲突。未重跑既有 ≥10-day sustained parity soak，这是原 promotion gate，不属于本 bug fix 的合入证据。
语义与 CI 对齐
这里没有创建新 vocabulary：它复用 source_root_digest 的既有含义，并让 Python reader/writers/drain 与 TypeScript owner 对齐。packet 明确 wait_for_ci=false，结论来自 exact-head 本地 filesystem、Node/Python 与真实 CLI 路径，不用远端 CI 替代证据。该变更触及 control-plane persisted-state 行为，因此即使验证全绿也不授予自合并权限。
我的整体评价
这是 #4892 的完整、最小且可回滚修复：入口、权威 identity、两个 effect writer、receipt consumer 和负向错误 owner 都闭合；相邻重复实现中的关键 invariant 风险也已通过 strict equality 消除。未发现 correctness、default-off、authority、typed-state 或 domain-neutrality blocker，我赞成在保持当前 exact head 的前提下合入，并由 maintainer 保留 merge authority。
English verdict: APPROVE - head 98c400a; lexical/canonical root aliases now share one strict state-to-binding lineage identity, including a mixed-digest fail-closed regression; 24 Python tests, 21 TypeScript tests, typecheck, the real Stage 2C2 row, and 13/13 premerge canaries passed.
… and the ladder loopx-project#4896 fixed loopx-project#4892: the journal's top-level root digest may be either spelling of the physical root, the binding, every outbox entry and every receipt carry exactly the digest the journal holds, and the Python guard now enforces the same state-to-binding equality as the TypeScript decoder. This adds the coverage the review of the superseded loopx-project#4898 asked for and the ladder-level regression that first exposed the defect: - a cross-language negative pair: a journal whose top-level digest and binding digest are different spellings of the same physical root, in both directions, is rejected by the Python guard and by the TypeScript `readShadowManagementState` with `shadow_management_state_invalid`, and the journal bytes are untouched; an agreeing journal is accepted by both as the positive control. The test fails against the superseded loopx-project#4898 head, whose guard accepted either spelling for the binding independently. - the `s2c2.outbox_prepared_then_committed_entries` ladder row run with a ladder root reached through a symlink, which is how macOS's default temp directory exposed the defect and which no unit test reproduces (pytest's `tmp_path` is already resolved). - `runtime_root_digest`'s docstring, which still claimed to match the TypeScript writer and that entries carry it; since loopx-project#4896 it is diagnostic only. No production behaviour changes. Related to loopx-project#4892, follows loopx-project#4896. Signed-off-by: wchwawa <wch19961116@gmail.com>
… and the ladder loopx-project#4896 fixed loopx-project#4892: the journal's top-level root digest may be either spelling of the physical root, the binding, every outbox entry and every receipt carry exactly the digest the journal holds, and the Python guard now enforces the same state-to-binding equality as the TypeScript decoder. This adds the coverage the review of the superseded loopx-project#4898 asked for and the ladder-level regression that first exposed the defect: - a cross-language negative pair: a journal whose top-level digest and binding digest are different spellings of the same physical root, in both directions, is rejected by the Python guard and by the TypeScript `readShadowManagementState` with `shadow_management_state_invalid`, and the journal bytes are untouched; an agreeing journal is accepted by both as the positive control. The test fails against the superseded loopx-project#4898 head, whose guard accepted either spelling for the binding independently. - the `s2c2.outbox_prepared_then_committed_entries` ladder row run with a ladder root reached through a symlink, which is how macOS's default temp directory exposed the defect and which no unit test reproduces (pytest's `tmp_path` is already resolved). - `runtime_root_digest`'s docstring, which still claimed to match the TypeScript writer and that entries carry it; since loopx-project#4896 it is diagnostic only. No production behaviour changes. Related to loopx-project#4892, follows loopx-project#4896. Signed-off-by: wchwawa <wch19961116@gmail.com>
2 participants
Goal And Delivered Outcome
s2c2.outbox_prepared_then_committed_entriesfirst failed withshadow_management_state_invalid; a reader-only fix exposedsource_root_mismatchin both pending entries. The final exact head passes that row through the default symlinked temporary directory and drains both Python and TypeScript entries once.main.Scope And Continuation
Validation
c0cb1b3878e2f30baff7665ae3eb072e83d349a0regression_paritypassedtests/control_plane/test_shadow_management.pyplustests/control_plane/test_local_authority_shadow_outbox.pypassed 24/24, andtests/control_plane_ts/local_authority_shadow_outbox.test.tspassed 21/21.real_entrypointpassedpython -m loopx.control_plane.testing.authority_e2e_ladder --row s2c2.outbox_prepared_then_committed_entriespassed under the default macOS temporary directory and delivered both writer partitions exactly once.integrationpassedstaticpassedruff checkpassed for all changed Python files.npm run typecheck:control-planepassed.integrationpassedloopx canary premerge --from-git-diffpassed 13 selected checks with no failures, warnings, or manual holds.Frontend / Visual Evidence
Type of Change
LoopX Area
Technical Direction
Shared-authority RFC fixture impact
file_v0runtime-shadow Stage 2C2, all 11 executable rows passed.Boundary Checklist
.loopx/,.codex/goals/, and liveACTIVE_GOAL_STATE.md).none.Signed-off-bytrailer (git commit -s).