Skip to content

fix: let a full provision run compose with a published extract - #36

Merged
jeremi merged 2 commits into
mainfrom
fix/provision-composes-with-published-extract
Aug 20, 2026
Merged

fix: let a full provision run compose with a published extract#36
jeremi merged 2 commits into
mainfrom
fix/provision-composes-with-published-extract

Conversation

@jeremi

@jeremi jeremi commented Aug 20, 2026

Copy link
Copy Markdown
Member

Problem

publish-extract is the documented way to bind a later checkpoint. It appends
the new extract under a fresh immutable filename and preserves the old file,
and it rewrites runtime.yaml while leaving runtime.rollback-<old_id>.yaml
behind. Both leftovers are deliberate: a rollback needs them.

A later full provision run then failed. _check_install_tree compares the
staged tree against the destination, and the runtime side was already given
_preserve_extract_rollback, but the extracts side was called with no
preserve predicate at all. The preserved .sqlite counted as an unexpected
file and raised existing output mismatch, so the escape hatch and the normal
provision path did not compose.

Fix

_preserve_superseded_extracts(active) looks past a superseded published
extract and nothing else. A file qualifies only when it is:

  • top level (no / in the relative path),
  • not the extract this run staged,
  • named for a timestamped publication (<prefix>-<YYYYmmddTHHMMSS[micros]>Z.sqlite),
  • a file rather than a directory,
  • mode 0o444.

The staged extract is deliberately excluded, so it stays verified against what
this run wrote. ROLLBACK_RUNTIME and the new PUBLISHED_EXTRACT now share one
PUBLICATION fragment, since publish-extract names the extract and the runtime
it supersedes after the same publication. ROLLBACK_RUNTIME matches exactly what
it matched before.

_stage_evidence now returns the extract name it bound (or None for a cell
that holds no extract) so the caller can build the predicate.

Tests

Written first, red at the real call site before the fix:

  • test_provision_composes_with_a_published_extract — the regression. Lays out
    a genuine post-publish-extract tree (both extracts at 0o444, both runtime
    files) and asserts provision succeeds, the superseded extract survives
    byte-identical at 0o444, and the extract directory ends at 0o555.
  • test_provision_still_refuses_an_unpublished_extract_file — drops a
    sro-poverty-restored.sqlite into the same tree and asserts
    existing output mismatch still raises.
  • test_only_a_superseded_published_extract_is_preserved — the predicate
    itself: false for the active extract, a foreign prefix, a nested path, a
    directory, mode 0o644, and a .bak suffix.

Mutation-checked: widening PUBLISHED_EXTRACT to ^.*\.sqlite$ fails 2 of
them, so the narrowness is actually pinned.

Verification

  • uv run python -m unittest scripts.test_provision_hosted_runtime — 43 tests, OK
  • just test — 20 + 28 + 251 Python tests OK, 61 vitest tests passed
  • just lint — ruff clean, all gate scripts pass, portal 423 files / 0 errors,
    home 388 files / 0 errors

Note

scripts/provision-hosted-runtime.py ships inside the provisioner image, so the
live fleet does not pick this up from a merge alone. It needs a
release-candidate build and a digest re-pin.

`publish-extract` leaves the superseded extract beside the one it binds,
exactly as it leaves the superseded runtime. The runtime check already
looked past its rollback file, but the extracts check ran with no
preserve predicate, so the extra `.sqlite` raised `existing output
mismatch` and a later full provision could not run at all.

Preserve only a superseded published extract: a top-level 0o444 file
whose name matches the timestamped publication pattern and is not the
extract this run staged. Anything else in the extract output still trips
the mismatch, and the staged extract stays verified against what this run
wrote.

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 000d399d35

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/provision-hosted-runtime.py Outdated
return (
"/" not in relative
and relative != active
and PUBLISHED_EXTRACT.fullmatch(relative) is not None

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Limit preserved extracts to the provisioned authority

When provisioning one direct cell, this global pattern also preserves timestamped extracts belonging to either of the other direct authorities; for example, sro-evidence silently filters a top-level, mode-0444 cra-birth-20260812T090000Z.sqlite out of both inventory checks. Such a file cannot be a superseded publication from SRO's publish-extract, so a misplaced or foreign artifact is accepted instead of producing existing output mismatch. Scope the predicate to the active extract's authority prefix (or pass the current cell explicitly).

Useful? React with 👍 / 👎.

Each cell provisions its own extract output, so a publication named after another authority is state this run cannot account for and stays a refusal.

Signed-off-by: Jeremi Joslin <jeremi@joslin.fr>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 45a92cd463

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +577 to +580
and relative != active
and published.fullmatch(relative) is not None
and value[0] != "directory"
and value[1] == 0o444

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Validate preserved extracts before excluding them

When a same-authority rollback extract is corrupted or replaced while retaining a timestamp-shaped filename and mode 0444, the newly scoped predicate still excludes it from both inventory comparisons based solely on its name and mode. A later full provision therefore reports success while preserving an unusable or untrusted database that the retained rollback runtime may reference; validate each candidate's metadata/schema/integrity, or correlate it with verified rollback state, before treating it as a superseded publication.

Useful? React with 👍 / 👎.

@jeremi
jeremi merged commit a61f83d into main Aug 20, 2026
1 check passed
@jeremi
jeremi deleted the fix/provision-composes-with-published-extract branch August 20, 2026 19:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant