Skip to content

feat(postgres): version PG extension from cloudsync.h, gate releases on a migration check#45

Open
andinux wants to merge 5 commits intomainfrom
feat/postgres-extension-migrations
Open

feat(postgres): version PG extension from cloudsync.h, gate releases on a migration check#45
andinux wants to merge 5 commits intomainfrom
feat/postgres-extension-migrations

Conversation

@andinux
Copy link
Copy Markdown
Collaborator

@andinux andinux commented Apr 23, 2026

Summary

Make the PostgreSQL extension version derive automatically from CLOUDSYNC_VERSION, ship per-release upgrade scripts so existing deployments can ALTER EXTENSION cloudsync UPDATE instead of
dropping/recreating, and fail CI when a release breaks the semver contract.

Why

Today default_version in cloudsync.control is pinned to '1.0' across all 16 releases. Every user's pg_extension.extversion reports '1.0' regardless of which .so they have loaded, no
per-release upgrade scripts exist, and no CI check flags missing ones. A version bump that changes SQL surface can silently leave users unable to reach the new release via ALTER EXTENSION cloudsync UPDATE — forcing them into a destructive DROP EXTENSION ... CASCADE; CREATE EXTENSION ... workaround or, worse, leaving them in a split-brain state where cloudsync.so is the new
code but pg_extension.extversion is stale and the SQL bindings in pg_proc may not match the new ABI.

What changes

Single source of truth. The PG extension version is derived from CLOUDSYNC_VERSION in src/cloudsync.h (what the rest of the repo already tracks). docker/Makefile.postgresql reads it and
generates cloudsync.control and cloudsync--<version>.sql at build time from new .control.in / .sql.in templates; the generated files are gitignored.

MAJOR.MINOR extension version. EXTVERSION exposes only the first two semver components — for example, 1.0.17 installs as extension version 1.0. PATCH bumps are binary-only: replace the
.so, no ALTER EXTENSION needed, installed_version stays put. MINOR/MAJOR bumps move EXTVERSION and require a per-release upgrade script. cloudsync_version() in the .so still reports
the full semver for debugging. This matches the convention used by mature PG extensions (PostGIS, pgvector, hstore, pg_stat_statements).

Per-release upgrade scripts. New src/postgresql/migrations/ directory holds hand-written cloudsync--<from>--<to>.sql files. postgres-install, postgres-package, and all three release
Dockerfiles (Dockerfile.release, Dockerfile.supabase, Dockerfile.supabase.release) install them alongside the main script via wildcard so every release ships the full chain.

Structural CI guard. scripts/check-postgres-migration.sh + a postgres-check-migration Make target + a new postgres-migration-check workflow job (which postgres-test and
postgres-build block on) enforce the contract in two modes:

  • Same EXTVERSION since the previous tag (PATCH release): render both the previous tag's install script and the current cloudsync.sql.in with their respective @EXTVERSION@ substitutions,
    diff them, and fail with a unified diff if they differ. Accidental SQL drift in a PATCH release would silently break users whose installed_version stays at the old value; this guard makes that
    impossible.
  • EXTVERSION moved (MINOR/MAJOR release): require a matching cloudsync--<prev>--<new>.sql in the migrations directory.

The check runs in <1s; gates every PR.

Docs. New top-level "Versioning" section in README.md describing the project-wide semver contract (PATCH never alters API) and the PG extension's MAJOR.MINOR catalog-version exception. New
"Upgrading a later release" subsection in docs/postgresql/quickstarts/postgres.md explaining when users need to run ALTER EXTENSION cloudsync UPDATE. Versioning policy (with
patch-vs-minor-vs-major decision table) in src/postgresql/migrations/README.md. Version-hardcoded filenames swept out of docker/README.md and docs/internal/supabase-flyio.md.

No version bump. This branch contains only build/CI infrastructure — no functional extension changes — so CLOUDSYNC_VERSION stays at 1.0.16. The next real fix or feature will bump the
version naturally.

Test plan

  • make postgres-check-migration passes against the current state ("patch-only release candidate; SQL surface identical")
  • Drift detection: appended a fake CREATE OR REPLACE FUNCTION cloudsync_oops() to cloudsync.sql.in → check fails with unified diff and remediation options
  • MINOR bump pass: temporarily set CLOUDSYNC_VERSION to 1.1.0 and added a stub cloudsync--1.0--1.1.sql → check passes
  • MINOR bump fail: same header bump without the migration file → check fails with the expected file path
  • End-to-end upgrade (run before the MAJOR.MINOR policy revision, when EXTVERSION still tracked the full semver): rebuilt the local debug container with a persistent volume at
    installed_version = '1.0', ran ALTER EXTENSION cloudsync UPDATE, confirmed installed_version moved to 1.0.17. Under the final MAJOR.MINOR scheme, this transition is a no-op (PATCH only);
    the upgrade path will be exercised live when the next MINOR release lands.
  • New postgres-migration-check job goes green in CI on this PR
  • Existing postgres-test (pg15 + pg17) and postgres-build matrix jobs stay green

andinux and others added 5 commits April 21, 2026 17:28
Before this change, default_version in the control file was frozen at
'1.0' across all 16 releases. Every user's pg_extension.extversion
reported '1.0' regardless of which .so they ran, no per-release upgrade
scripts existed, and no CI check flagged missing ones — so a version
bump could silently leave users unable to reach the new release via
`ALTER EXTENSION cloudsync UPDATE` and force them into a destructive
DROP EXTENSION ... CASCADE; CREATE EXTENSION ... workaround.

Single source of truth: the PG extension version is now derived from
CLOUDSYNC_VERSION in src/cloudsync.h (what the rest of the repo already
tracks). docker/Makefile.postgresql reads it and generates
cloudsync.control and cloudsync--<version>.sql at build time from new
.in templates; the generated files are gitignored.

Per-release upgrade scripts live under src/postgresql/migrations/ as
cloudsync--<from>--<to>.sql and are picked up by postgres-install,
postgres-package, and all three release Dockerfiles via wildcard.
Bootstrap: cloudsync--1.0--1.0.17.sql (comment-only, no SQL surface
changes) lets existing extversion='1.0' deployments upgrade cleanly.

CI gate: scripts/check-postgres-migration.sh + a postgres-check-migration
Make target + a new postgres-migration-check workflow job (both
postgres-test and postgres-build now block on it) resolve the previous
release's extversion from the most recent semver tag's control file (or
CLOUDSYNC_VERSION at that tag for new-scheme tags) and fail the build
if cloudsync--<prev>--<curr>.sql is missing. Sub-second; runs on every PR.

Also: bumped CLOUDSYNC_VERSION to 1.0.17; taught Dockerfile.supabase to
accept CLOUDSYNC_VERSION as a build arg for the image label; swept
version-hardcoded filenames out of docker/README.md,
docs/internal/supabase-flyio.md, and docs/postgresql/quickstarts/postgres.md.

Verified end-to-end: rebuilt the local debug container with a persistent
volume at installed_version='1.0', ran `ALTER EXTENSION cloudsync UPDATE;`,
confirmed installed_version moved to '1.0.17'.
Two fixes from review of the versioning branch.

P2 — parse-time tool dependency: change CLOUDSYNC_VERSION_FULL and
EXTVERSION from := (immediate) to = (recursive) so the sed/cut shell
calls only fire when something actually references them. Drop the
parse-time \$(error ...) guards and replace with a runtime check inside
postgres-check, which now also prints the resolved versions. Non-PG
targets like \`make help\` and \`make test\` no longer hard-fail when
src/cloudsync.h is unreadable or sed/cut are missing from PATH.

Caveat: Make still expands these variables at parse time wherever they
appear in a rule's target/prereqs (e.g. \$(PG_EXTENSION_SQL): ...), so
the shell calls themselves still happen — they just no longer crash
the run. Fully eliminating parse-time evaluation would require gating
the whole PG section on \$(MAKECMDGOALS), a larger structural change
not done here.

P3 — image label collision: the Supabase docker build was passing
EXTVERSION (now MAJOR.MINOR, e.g. '1.0') as the CLOUDSYNC_VERSION
build arg, which Dockerfile.supabase writes into
org.sqliteai.cloudsync.version. Result: 1.0.16 and 1.0.17 images
would carry the same label, breaking docker-inspect-style binary
identification across patch releases. Pass CLOUDSYNC_VERSION_FULL so
the label uniquely identifies the binary.

Verified:
- \`make postgres-check-migration\` still passes against the current state.
- \`make help\` exits 0 even when src/cloudsync.h is moved aside
  (was a hard-fail before).
- \`make postgres-check\` prints the resolved versions, e.g.
    "CloudSync version : 1.0.16 (extension version 1.0)"

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fully resolve the parse-time tool-dependency issue from the earlier
review. The previous fix switched EXTVERSION and CLOUDSYNC_VERSION_FULL
from := (immediate) to = (recursive), but Make still expanded them at
parse time because they appeared in a rule target and prerequisite
position:

  PG_EXTENSION_SQL: PG_EXTENSION_SQL_IN src/cloudsync.h
  postgres-build: postgres-check PG_EXTENSION_SQL PG_EXTENSION_CONTROL

Make needs to resolve rule targets and prerequisites at parse time to
build its rule database, so those references forced sed/cut to run on
every make invocation — including make help, make test, etc. With the
header missing, the parse-time sed calls printed stderr noise even
though non-PG targets succeeded.

This commit moves the template rendering into a phony target whose
name contains no version reference, and removes the PG_EXTENSION_SQL
and PG_EXTENSION_CONTROL references from postgres-build's prereqs.
All remaining EXTVERSION / PG_EXTENSION_SQL references now live
inside recipe bodies, which Make defers until a recipe actually runs.
Net: non-PG targets invoke zero sed/cut calls at parse time.

Hardening:
- Each sed now writes to a .tmp file and renames via mv after success,
  so a failed render cannot leave a half-written cloudsync.control or
  cloudsync--<ver>.sql in the PostgreSQL extension share directory on
  a subsequent install.
- postgres-generate-files depends on postgres-check, so the shell
  sanity-check messages (unreadable header, un-derivable EXTVERSION)
  still fire whenever the generator runs, not just on a direct
  postgres-check invocation.

Tradeoff: the previous file-rule form had incremental rebuild — the
SQL was only regenerated when the template or header changed. The
phony form regenerates on every invocation of postgres-build. sed
over the 300-line template is sub-millisecond and the cost is
dwarfed by the .so compile, so this is acceptable for the full
parse-time cleanliness it buys.

Verified:
- make help with src/cloudsync.h moved aside: exit 0, zero sed errors
  (previously exit 0 but two sed errors on stderr).
- make postgres-check-migration: still passes.
- make -p: EXTVERSION, CLOUDSYNC_VERSION_FULL, and PG_EXTENSION_SQL
  all remain unexpanded in the variable dump, confirming Make never
  resolved them at parse time.
- make postgres-generate-files: renders cloudsync.control with
  default_version = '1.0' and cloudsync--1.0.sql with -- Version 1.0.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…kstart

The "Upgrading a later release" section added to
docs/postgresql/quickstarts/postgres.md in 710d48d explains when PATCH
releases are transparent binary upgrades vs. when MINOR/MAJOR releases
require ALTER EXTENSION cloudsync UPDATE. Supabase self-hosted users
run the same extension and face the same decision when moving between
image tags, so mirror the section into supabase-self-hosted.md with
the phrasing adapted for their workflow: pull
sqlitecloud/sqlite-sync-supabase:<tag> and restart the db service,
rather than swap the .so on disk.

The top-level README "Versioning" section keeps its single link to
postgres.md as the canonical reference.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant