fix: let a full provision run compose with a published extract - #36
Conversation
`publish-extract` leaves the superseded extract beside the one it binds, exactly as it leaves the superseded runtime. The runtime check already looked past its rollback file, but the extracts check ran with no preserve predicate, so the extra `.sqlite` raised `existing output mismatch` and a later full provision could not run at all. Preserve only a superseded published extract: a top-level 0o444 file whose name matches the timestamped publication pattern and is not the extract this run staged. Anything else in the extract output still trips the mismatch, and the staged extract stays verified against what this run wrote. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 000d399d35
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return ( | ||
| "/" not in relative | ||
| and relative != active | ||
| and PUBLISHED_EXTRACT.fullmatch(relative) is not None |
There was a problem hiding this comment.
Limit preserved extracts to the provisioned authority
When provisioning one direct cell, this global pattern also preserves timestamped extracts belonging to either of the other direct authorities; for example, sro-evidence silently filters a top-level, mode-0444 cra-birth-20260812T090000Z.sqlite out of both inventory checks. Such a file cannot be a superseded publication from SRO's publish-extract, so a misplaced or foreign artifact is accepted instead of producing existing output mismatch. Scope the predicate to the active extract's authority prefix (or pass the current cell explicitly).
Useful? React with 👍 / 👎.
Each cell provisions its own extract output, so a publication named after another authority is state this run cannot account for and stays a refusal. Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 45a92cd463
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| and relative != active | ||
| and published.fullmatch(relative) is not None | ||
| and value[0] != "directory" | ||
| and value[1] == 0o444 |
There was a problem hiding this comment.
Validate preserved extracts before excluding them
When a same-authority rollback extract is corrupted or replaced while retaining a timestamp-shaped filename and mode 0444, the newly scoped predicate still excludes it from both inventory comparisons based solely on its name and mode. A later full provision therefore reports success while preserving an unusable or untrusted database that the retained rollback runtime may reference; validate each candidate's metadata/schema/integrity, or correlate it with verified rollback state, before treating it as a superseded publication.
Useful? React with 👍 / 👎.
Problem
publish-extractis the documented way to bind a later checkpoint. It appendsthe new extract under a fresh immutable filename and preserves the old file,
and it rewrites
runtime.yamlwhile leavingruntime.rollback-<old_id>.yamlbehind. Both leftovers are deliberate: a rollback needs them.
A later full
provisionrun then failed._check_install_treecompares thestaged tree against the destination, and the runtime side was already given
_preserve_extract_rollback, but the extracts side was called with nopreservepredicate at all. The preserved.sqlitecounted as an unexpectedfile and raised
existing output mismatch, so the escape hatch and the normalprovision path did not compose.
Fix
_preserve_superseded_extracts(active)looks past a superseded publishedextract and nothing else. A file qualifies only when it is:
/in the relative path),<prefix>-<YYYYmmddTHHMMSS[micros]>Z.sqlite),0o444.The staged extract is deliberately excluded, so it stays verified against what
this run wrote.
ROLLBACK_RUNTIMEand the newPUBLISHED_EXTRACTnow share onePUBLICATIONfragment, sincepublish-extractnames the extract and the runtimeit supersedes after the same publication.
ROLLBACK_RUNTIMEmatches exactly whatit matched before.
_stage_evidencenow returns the extract name it bound (orNonefor a cellthat holds no extract) so the caller can build the predicate.
Tests
Written first, red at the real call site before the fix:
test_provision_composes_with_a_published_extract— the regression. Lays outa genuine post-
publish-extracttree (both extracts at0o444, both runtimefiles) and asserts provision succeeds, the superseded extract survives
byte-identical at
0o444, and the extract directory ends at0o555.test_provision_still_refuses_an_unpublished_extract_file— drops asro-poverty-restored.sqliteinto the same tree and assertsexisting output mismatchstill raises.test_only_a_superseded_published_extract_is_preserved— the predicateitself: false for the active extract, a foreign prefix, a nested path, a
directory, mode
0o644, and a.baksuffix.Mutation-checked: widening
PUBLISHED_EXTRACTto^.*\.sqlite$fails 2 ofthem, so the narrowness is actually pinned.
Verification
uv run python -m unittest scripts.test_provision_hosted_runtime— 43 tests, OKjust test— 20 + 28 + 251 Python tests OK, 61 vitest tests passedjust lint— ruff clean, all gate scripts pass, portal 423 files / 0 errors,home 388 files / 0 errors
Note
scripts/provision-hosted-runtime.pyships inside the provisioner image, so thelive fleet does not pick this up from a merge alone. It needs a
release-candidate build and a digest re-pin.