Skip to content

Fix Codex weekly reset publication - #3177

Merged
steipete merged 3 commits into
steipete:mainfrom
Zihao-Qi:codex/fix-codex-weekly-reset-refresh
Aug 25, 2026
Merged

Fix Codex weekly reset publication#3177
steipete merged 3 commits into
steipete:mainfrom
Zihao-Qi:codex/fix-codex-weekly-reset-refresh

Conversation

@Zihao-Qi

@Zihao-Qi Zihao-Qi commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Summary

  • keep suspicious Codex weekly readings at or below 1% private instead of publishing the result of an immediate retry
  • persist an account-scoped reset candidate and publish it only after a separate exact OAuth refresh at least 60 seconds later, including the reported 1 available -> 1 available case with an advanced reset boundary
  • discard stale or incompatible candidates and prevent small corrections such as 6% -> 5% -> 5% from triggering reset confetti

Why

Codex can report a fresh post-reset weekly value while the reset credit remains available. The old guard treated that as unproven indefinitely, so both the usage value and its “Updated” timestamp could remain frozen. An immediate second request is not independent evidence, so this change carries a private candidate into a later refresh cycle instead.

Safety

  • only exact OAuth observations are eligible for delayed corroboration
  • candidates require matching account, plan, future reset-boundary evidence, and unchanged positive reset-credit inventory
  • candidates expire after 30 minutes and are discarded on source, identity, plan, boundary, credit, or timestamp mismatch
  • consumed reset-credit evidence remains an immediate confirmation path
  • the public snapshot remains unchanged until corroboration; no timestamp or usage value is synthesized
  • the candidate is stored with the account snapshot, so relaunching does not lose the evidence needed to recover a previously frozen installation

Testing

  • swift test --filter CodexWeeklyResetConfirmationTests — 26 passed
  • swift test --filter CodexWeeklyResetPublicationTests — 16 passed
  • swift test --filter UsageStoreCodexWeeklyResetCelebrationTests — 9 passed
  • swift test --filter '^CodexBarTests\.CodexAccountScopedRefreshTests/' — 120 passed
  • make check — formatting and lint clean
  • make test — 929 selections in 78 groups passed with zero failures, retries, or timeouts

No screenshot is included because this changes provider publication state rather than menu layout. The rollover cases are verified at the deterministic state/model seams, including persisted relaunch recovery.

Scope: this fixes the frozen Codex weekly snapshot in #3179. It does not claim to fix the separate menu-presentation issue in #3168.

Closes #3179

@clawsweeper

clawsweeper Bot commented Aug 24, 2026

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

@Zihao-Qi
Zihao-Qi marked this pull request as ready for review August 24, 2026 18:36
@clawsweeper clawsweeper Bot added merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. P2 Normal priority bug or improvement with limited blast radius. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels Aug 24, 2026
@clawsweeper

clawsweeper Bot commented Aug 24, 2026

Copy link
Copy Markdown

Codex review: needs real behavior proof before merge. Reviewed August 25, 2026, 12:28 AM ET / 04:28 UTC.

ClawSweeper review

What this changes

The PR delays publication of suspicious low Codex weekly readings until a later matching OAuth refresh, persisting per-account evidence and reset-celebration safeguards.

Merge readiness

Blocked until real behavior proof from a real setup is added - 4 items remain

The PR is still necessary, but a persisted candidate can outlive its promised 30-minute limit when provider timestamps stall; it also lacks real after-fix refresh proof.

Priority: P2
Reviewed head: 653628ce584b54e7be2f11b7ace477432ac668ea

Review scores

Measure Result What it means
Overall readiness 🦪 silver shellfish (2/6) The branch has substantial model coverage, but one persisted-expiry defect and missing real behavior proof block merge readiness.
Proof confidence 🦪 silver shellfish (2/6) Needs real behavior proof before merge: The contributor reports automated tests, but no after-fix real Codex refresh trace or redacted runtime artifact is attached; add one while redacting account details and tokens. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Patch quality 🦐 gold shrimp (3/6) 1 actionable review finding remain.

Verification

Check Result Evidence
Real behavior Needs proof Needs real behavior proof before merge: The contributor reports automated tests, but no after-fix real Codex refresh trace or redacted runtime artifact is attached; add one while redacting account details and tokens. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
Evidence reviewed 4 items Current-main comparison: The current-main-to-PR diff adds delayed-candidate admission, persistence, and tests; main does not contain this implementation.
Candidate expiry defect: A candidate is retained whenever the next source timestamp is not newer, bypassing its 30-minute age check.
Adjacent behavior provenance: The current zero-credit confirmation behavior originated in the existing reset-confirmation path that this branch extends.
Findings 1 actionable finding [P2] Expire candidates even when provider timestamps stall
Security None None.

Live Verification

Command: swift test --filter CodexWeeklyResetPublicationTests

Result: FAIL (failed) — execution before step 1 run: sh -lc pnpm install --ignore-scripts --frozen-lockfile failed: ! Corepack is about to download https://registry.npmjs.org/pnpm/-/pnpm-11.23.0.tgz

sh -lc pnpm install --ignore-scripts --frozen-lockfile failed: ! Corepack is about to download https://registry.npmjs.org/pnpm/-/pnpm-11.23.0.tgz

Assertions:

  • FAIL expect_output: CodexWeeklyResetPublicationTests

How this fits together

CodexBar fetches Codex usage and stores a published snapshot for the menu-bar UI. This change decides whether a low weekly reading is safe to publish and whether a reset event is emitted.

flowchart LR
A[OAuth usage refresh] --> B[Weekly reset admission]
C[Prior account snapshot] --> B
D[Reset-credit evidence] --> B
B --> E[Persisted reset candidate]
E --> F[Later exact refresh]
F --> G[Published usage snapshot]
G --> H[Menu bar and reset celebration]
Loading

Before merge

  • Add real behavior proof - Needs real behavior proof before merge: The contributor reports automated tests, but no after-fix real Codex refresh trace or redacted runtime artifact is attached; add one while redacting account details and tokens. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.
  • Expire candidates even when provider timestamps stall (P2) - The non-advancing-timestamp branch returns true before checking candidate age. A candidate persisted across relaunch therefore remains stored whenever OAuth keeps returning the same or older updatedAt, despite the stated 30-minute expiry. Check local expiry before this return and add the stalled-timestamp regression.
  • Resolve merge risk (P1) - A persisted candidate can survive beyond 30 minutes when provider timestamps stall or regress, leaving stale evidence stored until a later advancing observation.
  • Complete next step (P2) - A focused code repair can correct the candidate-expiry invariant, while the contributor separately supplies required real behavior proof.

Findings

  • [P2] Expire candidates even when provider timestamps stall — Sources/CodexBar/Providers/Codex/CodexWeeklyResetConfirmation.swift:324-326
Agent review details

Security

None.

Review metrics

Metric Value Why it matters
Changed surface 17 files, +2320/-254 lines The patch changes provider admission, persisted snapshots, and reset notifications together.
Production versus tests production +1018, tests +1302 The implementation has substantial state-model coverage, but the persisted lifetime edge remains untested.

Root-cause cluster

Relationship: fixed_by_candidate
Canonical: #3179
Summary: This PR explicitly implements the unchanged-positive-reset-credit publication case described by the canonical issue.

Members:

Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.

Merge-risk options

Maintainer options:

  1. Enforce wall-clock expiry (recommended)
    Store or inject a local observation time so an old persisted candidate is discarded even when the next provider payload has the same or older timestamp.
  2. Pause the delayed path
    Keep the existing publication guard until candidate expiry semantics and runtime proof are established.
Copy recommended automerge instruction
@clawsweeper automerge

Special instructions:
Preserve the candidate only within 30 minutes of local creation; add a relaunch regression where the provider timestamp does not advance.

Technical review

Best possible solution:

Make expiry depend on local persistence time, add a relaunch/stalled-timestamp regression, and attach redacted real refresh proof.

Do we have a high-confidence way to reproduce the issue?

Yes, from source: persist a candidate older than 30 minutes, then return a valid but non-advancing provider timestamp; the branch retains it without evaluating expiry.

Is this the best way to solve the issue?

No; delayed confirmation is reasonable, but expiry must be independent of provider timestamp progress before persisted relaunch recovery is safe.

Full review comments:

  • [P2] Expire candidates even when provider timestamps stall — Sources/CodexBar/Providers/Codex/CodexWeeklyResetConfirmation.swift:324-326
    The non-advancing-timestamp branch returns true before checking candidate age. A candidate persisted across relaunch therefore remains stored whenever OAuth keeps returning the same or older updatedAt, despite the stated 30-minute expiry. Check local expiry before this return and add the stalled-timestamp regression.
    Confidence: 0.93

Overall correctness: patch is incorrect
Overall confidence: 0.93

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning high; reviewed against b6a4ce968c16.

Labels

Label justifications:

  • P2: The patch affects user-visible Codex usage freshness but is bounded to a provider reset edge case.
  • merge-risk: 🚨 compatibility: The branch changes persisted account snapshot and reset-event semantics for existing Codex accounts.
  • rating: 🦪 silver shellfish: Overall readiness is 🦪 silver shellfish; proof is 🦪 silver shellfish and patch quality is 🦐 gold shrimp.
  • status: 📣 needs proof: The PR needs real behavior proof before ClawSweeper can clear the contributor ask. Needs real behavior proof before merge: The contributor reports automated tests, but no after-fix real Codex refresh trace or redacted runtime artifact is attached; add one while redacting account details and tokens. After adding proof, update the PR body; ClawSweeper should re-review automatically. If it does not, the PR author or someone with repository write access can comment @clawsweeper re-review.

Evidence

Acceptance criteria:

  • [P1] swift test --filter CodexWeeklyResetPublicationTests.
  • [P1] swift test --filter CodexWeeklyResetConfirmationTests.
  • [P1] swift test --filter UsageStoreCodexWeeklyResetCelebrationTests.
  • [P1] make check.
  • [P1] make test.

What I checked:

Likely related people:

  • Peter Steinberger: History includes the reset-confetti guard and recent release maintenance for the affected Codex reset path. (role: recent area contributor; confidence: high; commits: 6f84c4e9425b, 061593ca15d9; files: Sources/CodexBar/UsageStore+LimitResetCelebration.swift, Sources/CodexBar/Providers/Codex/CodexWeeklyResetConfirmation.swift)
  • luantu: Introduced the current-main zero-available-credit weekly reset confirmation extended here. (role: introduced adjacent reset-confirmation behavior; confidence: high; commits: dd6eefd964db; files: Sources/CodexBar/Providers/Codex/CodexWeeklyResetConfirmation.swift)

Rank-up moves

Optional improvements that raise the rating; they are not merge blockers.

  • Make candidate expiry independent of provider timestamp advancement and add the relaunch regression.
  • Attach a redacted real refresh trace showing the withheld snapshot publishes only after the later exact OAuth refresh.

Rating scale

Score Internal tier Crab rank Meaning
6/6 S 🦀 challenger crab Exceptional readiness
5/6 A 🦞 diamond lobster Very strong readiness
4/6 B 🐚 platinum hermit Good normal PR; ordinary maintainer review
3/6 C 🦐 gold shrimp Useful, but confidence is limited
2/6 D 🦪 silver shellfish Proof or implementation needs work
1/6 F 🧂 unranked krab Not merge-ready
N/A NA 🌊 off-meta tidepool Rating does not apply

Overall follows the weaker of proof and patch quality.
Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Workflow

  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

History

Review history (1 earlier review cycle)
  • reviewed 2026-08-24T18:40:57.360Z sha 8b4d413 :: needs real behavior proof before merge. :: none

@steipete steipete left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking this on — the confirmation-state-machine structure is a reasonable shape, all 46 new tests pass, and make check is clean. But a deep review found the core scenario from #3168/#3179 still isn't fixed, and the workaround path introduces two new failure modes. Details:

P0 — the reported 1 → 1 early-reset case remains stuck. CodexWeeklyResetConfirmation.swift:170 still rejects an early reset unless a credit was consumed, the previous inventory was explicitly zero, or the boundary is unchanged within one second. A real backend reset advances the boundary, and unchanged positive credit inventory satisfies none of those arms — which is exactly the residual case both issues describe. The existing test at CodexWeeklyResetPublicationTests.swift:467 actively asserts the scenario stays frozen at the previous reading, and none of the new tests feeds unchanged positive inventories across all three snapshots. Please add a test that reproduces #3179's table (1 available → 1 available, boundary advanced, fresh 0%) and make it publish.

P1 — the claimed relaunch migration doesn't repair the persisted baseline. UsageStore+PlanUtilization.swift:956 discards legacy confetti-detector state, but the stale usage snapshot is persisted separately and rehydrated at UsageStore.swift:534. The migration test only checks detector-state deletion, so upgrading and relaunching won't recover an already-stuck install.

P1 — false confetti on mundane corrections. UsageStore+LimitResetCelebration.swift:429 treats any crossing from >5% to ≤5% as a reset candidate regardless of magnitude — a 6% → 5% → 5% drift with unchanged plan and boundary celebrates a reset after 60 s. Gate on boundary advancement, credit consumption, or a substantial drop.

P1 — transient zero glitches publish immediately. UsageStore+CodexWeeklyResetConfirmation.swift:81 fires the confirmation fetch immediately and CodexWeeklyResetConfirmation.swift:185 accepts two matching zeros on account/plan/boundary match alone, so a brief backend cache glitch hitting both requests persists a wrong reading; the 60 s / 30 min windows only guard confetti, not publication.

Happy to re-review once the 1 → 1 case actually publishes and the persisted-baseline recovery is proven by a test.

@Zihao-Qi
Zihao-Qi force-pushed the codex/fix-codex-weekly-reset-refresh branch from 8b4d413 to 653628c Compare August 25, 2026 04:23
@Zihao-Qi

Zihao-Qi commented Aug 25, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the requested changes in 653628ce5:

  • the exact 1 available -> 1 available, advanced-boundary, fresh 0% case now publishes only on a later exact OAuth refresh at least 60 seconds after the private candidate was created
  • the candidate persists beside the stale account snapshot; a relaunch test proves recovery and candidate cleanup
  • the immediate same-request retry remains private, and expired or mismatched candidates fail closed
  • 6% -> 5% -> 5% now has an explicit no-confetti regression test
  • make check is clean and the full make test suite passed all 929 selections in 78 groups with zero retries or timeouts

Co-authored-by: Trim <womayeyonggugele@gmail.com>

@steipete steipete left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All requested weekly-reset cases now pass, including unchanged positive reset-credit inventory, advanced boundaries, persisted recovery across relaunch, and false-confetti suppression. Maintainer hardening additionally separates persisted local observation time from provider timestamps, expires stalled and regressing candidates reliably, rejects wrong-account or wrong-source stale observations, preserves sibling-account caches, and decodes preexisting snapshots compatibly. Focused account-ownership, confirmation, publication, celebration, and provider-architecture regressions plus make check are green.

@clawsweeper

clawsweeper Bot commented Aug 25, 2026

Copy link
Copy Markdown

ClawSweeper status: review started.

I am starting a fresh review of this pull request: Fix Codex weekly reset publication This is item 1/1 in the current shard. Shard 0/1.

This placeholder means the worker is alive and reading the current context. I will edit this same comment with the actual review when the claws are done clicking.

Crustacean status: shell secured, claws on keyboard, evidence pebbles being sorted.

@steipete
steipete merged commit 0a1aa53 into steipete:main Aug 25, 2026
9 checks passed
@steipete

Copy link
Copy Markdown
Owner

Landed in 0a1aa53 after addressing every maintainer review finding and additional account-safety defects. Exact OAuth weekly resets with unchanged positive reset-credit inventory now publish only after an independent later refresh; private account-scoped evidence survives relaunch without publishing transient zero readings or false reset confetti. Provider timestamps establish evidence ordering, while a separately persisted local observation clock enforces the 60-second minimum and 30-minute maximum even when provider timestamps stall, regress, or jump forward. Expired evidence is removed from persisted snapshots; stale observations from another account, plan, or source fail closed; and creating an active-account candidate preserves every sibling account snapshot. Legacy persisted candidates decode compatibly without cache migration. Verification: 198 focused account-ownership, weekly-reset confirmation, publication, celebration, persistence, and provider-architecture tests passed; make check passed with zero violations; the full local suite passed all 930 selections across 78 groups with zero failures, retries, or timeouts; and exact-head GitHub Actions run 32873319421 passed both macOS shards, Linux x64, arm64 and musl builds, lint, and security checks. Contributor credit and changelog included.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 compatibility 🚨 Merging this PR could break existing users, config, migrations, defaults, or upgrades. P2 Normal priority bug or improvement with limited blast radius. rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Codex early backend weekly reset remains stale when reset-credit inventory is unchanged and non-zero (#2790 follow-up)

2 participants