From db60ccae320c877d03c6992da80360fd60850437 Mon Sep 17 00:00:00 2001 From: Lily Shen <115414357+lilyshen0722@users.noreply.github.com> Date: Tue, 25 Aug 2026 00:25:21 -0700 Subject: [PATCH 1/3] =?UTF-8?q?docs(ax):=20entry=2046=20=E2=80=94=20mergeS?= =?UTF-8?q?tateStatus=20UNKNOWN=20describes=20the=20cache,=20not=20the=20P?= =?UTF-8?q?R?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit @sprint-review measured `mergeStateStatus: UNKNOWN` across eleven merged PRs and one open one and concluded it carries no lifecycle information. Re-measured it and the mechanism is sharper: the first query SCHEDULES the mergeability computation and returns UNKNOWN in the same response; the second returns the result. #942 read UNKNOWN then BLOCKED on two identical consecutive commands, and ten open PRs flipped from UNKNOWN to a real value between two list reads. The operational bite is specific to this sprint: a cold-cache read returns UNKNOWN for exactly the untouched PRs a conflict matrix is assessing, and the bias runs the wrong way — the staler the PR, the colder its cache and the more likely it has actually gone DIRTY. #809 first-read this session came back DIRTY/CONFLICTING. Co-Authored-By: Claude Opus 5 --- _external/clawdbot | 2 +- docs/development/agent-experience-audit.md | 62 ++++++++++++++++++++++ 2 files changed, 63 insertions(+), 1 deletion(-) diff --git a/_external/clawdbot b/_external/clawdbot index 5d88a3f1b..70bd82b80 160000 --- a/_external/clawdbot +++ b/_external/clawdbot @@ -1 +1 @@ -Subproject commit 5d88a3f1bf8aa525479bb2be2083a552a79781ee +Subproject commit 70bd82b80f181b1349a4c7107d78c438e41fc532 diff --git a/docs/development/agent-experience-audit.md b/docs/development/agent-experience-audit.md index c12c72a19..21181c6be 100644 --- a/docs/development/agent-experience-audit.md +++ b/docs/development/agent-experience-audit.md @@ -2573,3 +2573,65 @@ agent path since it was written. - Companion rule, on the method that missed it: reviewer-checklist rule 17 — a mutation proves a term matters to the suite, not that the suite's shape is real. + +## 46. `mergeStateStatus: UNKNOWN` describes GitHub's cache, not the PR, and the query that reads it is what fills the cache (2026-08-25, sprint-review + pod-architect) + +> Numbering: 39, 40, 43, 44 and 45 are reserved by open PRs. Renumber this +> entry, not those, if they land in another order. + +@sprint-review measured the field across a night's PRs and found `UNKNOWN` on +all eleven merged ones *and* on the one still open, concluding it "carries no +lifecycle information at all." That is right about what it cannot do and +understates what it is. Measured again on 2026-08-25: + +| query | #942 | what changed between reads | +|---|---|---| +| `gh pr view 942 --json mergeStateStatus,mergeable` | `UNKNOWN` / `UNKNOWN` | — | +| the same command, immediately again | `BLOCKED` / `MERGEABLE` | nothing | + +Nothing about the PR changed. Nothing about the command changed. **The first +read schedules the mergeability computation and returns `UNKNOWN` in the same +response; the second read returns the result.** The natural experiment on the +open list is the same shape and larger: ten PRs read `UNKNOWN` on the first +`gh pr list`, and all ten read a real value on the second — including seven +that were never queried individually in between. + +So `UNKNOWN` is not a state of the pull request. It is a state of the cache, +and asking is what changes it. + +**Why this is worse than an ambiguous value.** The four rows in entry 12's +table are *correct and insufficient* — each says something true about the +object and needs a second surface to finish the sentence. `UNKNOWN` says +nothing about the object at all. There is no second surface to go find, and +therefore no missing-query feeling to prompt the search; the honest reading is +**re-query, do not interpret.** + +**The operational bite, which is specific to how this pod works.** Every +conflict matrix built this sprint reads `mergeStateStatus`. A cold-cache read +returns `UNKNOWN` for exactly the population you are assessing — the PRs +nobody has touched recently — and `UNKNOWN` in a matrix cell reads as *not +determined* rather than as *not asked*. The two rendered identically all night. +Note the direction of the bias: the staler the PR, the colder its cache, and +the stale ones are the ones most likely to have actually gone `DIRTY`. #809, +first-read this session, came back `DIRTY` / `CONFLICTING` — a real conflict +that a single cold read would have reported as unknown and a matrix would have +shown as blank. + +**What to do.** + +- **Read it twice, and use the second value.** One call is a cache-warming + call whose return value is not an answer. +- **Never put `UNKNOWN` in a results table.** It is not a finding; it is the + absence of one, and a table is exactly where that distinction dies. +- `state` / `mergedAt` are the lifecycle fields, and they are computed + eagerly. Use those for "is it open", never `mergeStateStatus` + (@sprint-review's correction, which stands). +- The merged-PR reading has a *different* cause with the same value — + mergeability is never recomputed after merge, so `UNKNOWN` is terminal + there rather than transient. One value, two mechanisms, no way to tell them + apart from the field. + +*(Companion: entry 45 — a stale PR ref answers with a conflict, not an error. +Same family. In both, the instrument returns a well-formed, plausible value +about a thing it did not actually look at, and the tell is never in the +output.)* From fc59a68f6ce929a3d6a39a96f2a69424dc7eed29 Mon Sep 17 00:00:00 2001 From: Lily Shen <115414357+lilyshen0722@users.noreply.github.com> Date: Tue, 25 Aug 2026 00:47:41 -0700 Subject: [PATCH 2/3] =?UTF-8?q?docs(ax):=20entry=2046=20=E2=80=94=20three?= =?UTF-8?q?=20corrections=20from=20sprint-review's=20wider=20sample?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit sprint-review re-measured across 15 PRs / ~23 observations and found three things wrong with the entry as drafted: 1. "Read it twice, use the second value" is the bug it warns about. #809 needs three reads; ~a fifth of cold PRs do. A fixed count records UNKNOWN in the table the next bullet bans it from. Now count-free: loop until the value is not UNKNOWN. 2. The mechanism was asserted on two observations. "The read schedules the computation" and "a computation was already running" are indistinguishable from that sample, and #1215 — warm on the first read after its own recent update — argues something other than a query fills this cache. The entry now states the symptom, marks the mechanism unresolved, and notes that every operational rule below survives either one. 3. The staleness bias was half right. The base rate holds (stale PRs are both coldest and likeliest to be DIRTY), but colder does not mean slower to resolve: 4 of 5 three-read PRs came back CLEAN. Written as a base rate now, not a cache property, so nobody builds a heuristic on the wrong mechanism. Also folds in the fleet-wide consequence sprint-review measured and left to this entry rather than opening a competing PR: a seat that reads mergeStateStatus once and branches on it mis-reads 14 of 15 cold PRs. Co-Authored-By: Claude Opus 5 --- docs/development/agent-experience-audit.md | 54 +++++++++++++++------- 1 file changed, 38 insertions(+), 16 deletions(-) diff --git a/docs/development/agent-experience-audit.md b/docs/development/agent-experience-audit.md index 21181c6be..e657cebab 100644 --- a/docs/development/agent-experience-audit.md +++ b/docs/development/agent-experience-audit.md @@ -2574,7 +2574,7 @@ agent path since it was written. a mutation proves a term matters to the suite, not that the suite's shape is real. -## 46. `mergeStateStatus: UNKNOWN` describes GitHub's cache, not the PR, and the query that reads it is what fills the cache (2026-08-25, sprint-review + pod-architect) +## 46. `mergeStateStatus: UNKNOWN` describes GitHub's cache, not the PR, and no fixed number of reads settles it (2026-08-25, sprint-review + pod-architect) > Numbering: 39, 40, 43, 44 and 45 are reserved by open PRs. Renumber this > entry, not those, if they land in another order. @@ -2589,15 +2589,22 @@ understates what it is. Measured again on 2026-08-25: | `gh pr view 942 --json mergeStateStatus,mergeable` | `UNKNOWN` / `UNKNOWN` | — | | the same command, immediately again | `BLOCKED` / `MERGEABLE` | nothing | -Nothing about the PR changed. Nothing about the command changed. **The first -read schedules the mergeability computation and returns `UNKNOWN` in the same -response; the second read returns the result.** The natural experiment on the -open list is the same shape and larger: ten PRs read `UNKNOWN` on the first -`gh pr list`, and all ten read a real value on the second — including seven -that were never queried individually in between. +Nothing about the PR changed. Nothing about the command changed. The natural +experiment on the open list is the same shape and larger: ten PRs read +`UNKNOWN` on the first `gh pr list`, and all ten read a real value on the +second — including seven that were never queried individually in between. -So `UNKNOWN` is not a state of the pull request. It is a state of the cache, -and asking is what changes it. +So `UNKNOWN` is not a state of the pull request. It is a state of a cache. + +**That much is measured. The mechanism is not, and this entry deliberately +stops short of it.** The tempting reading — *the read schedules the +computation* — is one hypothesis; *a computation was already running and took +more than one round-trip* is another, and two observations of the same PR +cannot separate them (@sprint-review's correction). One datum argues against +the tempting reading outright: #1215 returned a real value on the *first* read, +having been updated minutes earlier, so something other than a query warms this +cache. Everything operational below survives either mechanism, which is why it +is written from the symptom. **Why this is worse than an ambiguous value.** The four rows in entry 12's table are *correct and insufficient* — each says something true about the @@ -2611,16 +2618,31 @@ conflict matrix built this sprint reads `mergeStateStatus`. A cold-cache read returns `UNKNOWN` for exactly the population you are assessing — the PRs nobody has touched recently — and `UNKNOWN` in a matrix cell reads as *not determined* rather than as *not asked*. The two rendered identically all night. -Note the direction of the bias: the staler the PR, the colder its cache, and -the stale ones are the ones most likely to have actually gone `DIRTY`. #809, -first-read this session, came back `DIRTY` / `CONFLICTING` — a real conflict -that a single cold read would have reported as unknown and a matrix would have -shown as blank. +Note the direction of the bias — and note carefully which half of it is real. +The base-rate half holds: stale PRs are both the coldest reads *and* the ones +most likely to have actually gone `DIRTY`, so the cells a cold matrix blanks +are disproportionately the cells that mattered. #809 is the case: read cold, it +came back `DIRTY` / `CONFLICTING` — a real conflict a single read reports as +unknown and a matrix shows as blank. + +The half that does **not** hold is any claim that colder reads take longer to +resolve. Of @sprint-review's five three-read PRs, four came back `CLEAN` and +one `DIRTY`; read latency did not track the value. This is a property of which +PRs you tend to be asking about, not a property of the cache. Build a heuristic +on the second reading and you will have built it on a mechanism that isn't +there. + +The consequence is fleet-wide, not local to one matrix: **any seat that reads +`mergeStateStatus` once and branches on it mis-reads 14 of 15 cold PRs.** **What to do.** -- **Read it twice, and use the second value.** One call is a cache-warming - call whose return value is not an answer. +- **Re-query until the value is not `UNKNOWN`. Do not fix a number of + reads.** An earlier draft of this bullet said "read it twice, use the second + value", which reintroduces the exact bug the entry is about: it records + `UNKNOWN` for #809, which needed three. Across ~23 observations, cold PRs + took one to three reads, roughly a fifth of them three — and #1215 resolved + on the first. `UNKNOWN` is not a value; loop until you have one. - **Never put `UNKNOWN` in a results table.** It is not a finding; it is the absence of one, and a table is exactly where that distinction dies. - `state` / `mergedAt` are the lifecycle fields, and they are computed From cf41598d0311f7cb5de3053d19b9fbf4375d0364 Mon Sep 17 00:00:00 2001 From: Lily Shen <115414357+lilyshen0722@users.noreply.github.com> Date: Tue, 25 Aug 2026 01:08:16 -0700 Subject: [PATCH 3/3] docs(ax): entry 46 drops two counts their author could not reproduce MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit @sprint-review retracted both numbers they had given me: "roughly a fifth of ~23 observations" needing three reads, and the "four of five three-read PRs came back CLEAN" split. Neither reproduces, and every PR in the sample is warm now, so the population that would settle them no longer exists. What survives is individually-named and still checkable — #1168 and #1206 needed three reads out of a batch of fifteen, #809 needed three, #1215 resolved on the first — plus the qualitative finding that read count did not track CLEAN/DIRTY. Stating no rate is the deliberate choice, not a gap. This entry's own rule is "loop until the value is not UNKNOWN", and a frequency is exactly what tempts the next reader to budget a fixed number of reads — which is the bug the bullet exists to prevent. A number that cannot be re-derived is worse than no number in a document whose subject is instruments that answer confidently without having looked. Co-Authored-By: Claude Opus 5 --- docs/development/agent-experience-audit.md | 22 ++++++++++++++++------ 1 file changed, 16 insertions(+), 6 deletions(-) diff --git a/docs/development/agent-experience-audit.md b/docs/development/agent-experience-audit.md index e657cebab..1ce0e6835 100644 --- a/docs/development/agent-experience-audit.md +++ b/docs/development/agent-experience-audit.md @@ -2626,9 +2626,11 @@ came back `DIRTY` / `CONFLICTING` — a real conflict a single read reports as unknown and a matrix shows as blank. The half that does **not** hold is any claim that colder reads take longer to -resolve. Of @sprint-review's five three-read PRs, four came back `CLEAN` and -one `DIRTY`; read latency did not track the value. This is a property of which -PRs you tend to be asking about, not a property of the cache. Build a heuristic +resolve. Of the PRs that needed three reads, most came back `CLEAN`: read +latency did not track the value. (A "four of five" split was reported and then +retracted as unreproducible — the qualitative finding is what survived, and it +is the whole of what this bullet needs.) This is a property of which PRs you +tend to be asking about, not a property of the cache. Build a heuristic on the second reading and you will have built it on a mechanism that isn't there. @@ -2640,9 +2642,17 @@ The consequence is fleet-wide, not local to one matrix: **any seat that reads - **Re-query until the value is not `UNKNOWN`. Do not fix a number of reads.** An earlier draft of this bullet said "read it twice, use the second value", which reintroduces the exact bug the entry is about: it records - `UNKNOWN` for #809, which needed three. Across ~23 observations, cold PRs - took one to three reads, roughly a fifth of them three — and #1215 resolved - on the first. `UNKNOWN` is not a value; loop until you have one. + `UNKNOWN` for #809, which needed three. Cold PRs took one to three reads — + #1168 and #1206 needed three out of a batch of fifteen, #809 needed three, + #1215 resolved on the first. `UNKNOWN` is not a value; loop until you have + one. + + No rate is stated here on purpose. @sprint-review reported "roughly a fifth + of ~23 observations" and later could not reproduce it, and by then every PR + in the sample was warm — so the population that would settle it no longer + exists. The individually-named cases above are the part that survives, and a + reader needs none of the rest: the rule is "loop", and a frequency would only + tempt someone to budget a fixed number of reads again. - **Never put `UNKNOWN` in a results table.** It is not a finding; it is the absence of one, and a table is exactly where that distinction dies. - `state` / `mergedAt` are the lifecycle fields, and they are computed