From 6292eee20102c3ca3d0cf57a962db48474fbbe4d Mon Sep 17 00:00:00 2001 From: Andrea Donetti Date: Tue, 21 Apr 2026 17:28:52 -0600 Subject: [PATCH 1/5] feat(postgres): derive version from cloudsync.h, ship upgrade scripts MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Before this change, default_version in the control file was frozen at '1.0' across all 16 releases. Every user's pg_extension.extversion reported '1.0' regardless of which .so they ran, no per-release upgrade scripts existed, and no CI check flagged missing ones — so a version bump could silently leave users unable to reach the new release via `ALTER EXTENSION cloudsync UPDATE` and force them into a destructive DROP EXTENSION ... CASCADE; CREATE EXTENSION ... workaround. Single source of truth: the PG extension version is now derived from CLOUDSYNC_VERSION in src/cloudsync.h (what the rest of the repo already tracks). docker/Makefile.postgresql reads it and generates cloudsync.control and cloudsync--.sql at build time from new .in templates; the generated files are gitignored. Per-release upgrade scripts live under src/postgresql/migrations/ as cloudsync----.sql and are picked up by postgres-install, postgres-package, and all three release Dockerfiles via wildcard. Bootstrap: cloudsync--1.0--1.0.17.sql (comment-only, no SQL surface changes) lets existing extversion='1.0' deployments upgrade cleanly. CI gate: scripts/check-postgres-migration.sh + a postgres-check-migration Make target + a new postgres-migration-check workflow job (both postgres-test and postgres-build now block on it) resolve the previous release's extversion from the most recent semver tag's control file (or CLOUDSYNC_VERSION at that tag for new-scheme tags) and fail the build if cloudsync----.sql is missing. Sub-second; runs on every PR. Also: bumped CLOUDSYNC_VERSION to 1.0.17; taught Dockerfile.supabase to accept CLOUDSYNC_VERSION as a build arg for the image label; swept version-hardcoded filenames out of docker/README.md, docs/internal/supabase-flyio.md, and docs/postgresql/quickstarts/postgres.md. Verified end-to-end: rebuilt the local debug container with a persistent volume at installed_version='1.0', ran `ALTER EXTENSION cloudsync UPDATE;`, confirmed installed_version moved to '1.0.17'. --- .github/workflows/main.yml | 17 +++ .gitignore | 5 + docker/Makefile.postgresql | 54 ++++++++- docker/README.md | 2 +- docker/postgresql/Dockerfile.release | 2 +- docker/postgresql/Dockerfile.supabase | 14 ++- docker/postgresql/Dockerfile.supabase.release | 4 +- docker/postgresql/cloudsync.control | 22 ---- docker/postgresql/cloudsync.control.in | 13 +++ docs/internal/supabase-flyio.md | 9 +- docs/postgresql/quickstarts/postgres.md | 4 +- scripts/check-postgres-migration.sh | 105 ++++++++++++++++++ src/cloudsync.h | 2 +- .../{cloudsync--1.0.sql => cloudsync.sql.in} | 2 +- src/postgresql/migrations/README.md | 39 +++++++ .../migrations/cloudsync--1.0--1.0.17.sql | 14 +++ 16 files changed, 265 insertions(+), 43 deletions(-) delete mode 100644 docker/postgresql/cloudsync.control create mode 100644 docker/postgresql/cloudsync.control.in create mode 100755 scripts/check-postgres-migration.sh rename src/postgresql/{cloudsync--1.0.sql => cloudsync.sql.in} (99%) create mode 100644 src/postgresql/migrations/README.md create mode 100644 src/postgresql/migrations/cloudsync--1.0--1.0.17.sql diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index 2c91615..74c86ba 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -239,8 +239,24 @@ jobs: path: dist/${{ matrix.name == 'apple-xcframework' && 'CloudSync.*' || 'cloudsync.*'}} if-no-files-found: error + postgres-migration-check: + if: ${{ !contains(github.event.head_commit.message, '[auto-update]') }} + runs-on: ubuntu-22.04 + name: postgresql migration script check + timeout-minutes: 2 + steps: + - uses: actions/checkout@v4.2.2 + with: + # Need full history + tags so `git describe` can find the previous + # release tag that the check script compares against. + fetch-depth: 0 + + - name: verify migration script for current CLOUDSYNC_VERSION + run: make postgres-check-migration + postgres-test: if: ${{ !contains(github.event.head_commit.message, '[auto-update]') }} + needs: [postgres-migration-check] runs-on: ubuntu-22.04 name: postgresql ${{ matrix.postgres_tag }} build + test timeout-minutes: 10 @@ -284,6 +300,7 @@ jobs: postgres-build: if: ${{ !contains(github.event.head_commit.message, '[auto-update]') }} + needs: [postgres-migration-check] runs-on: ${{ matrix.os }} name: postgresql${{ matrix.postgres_version }}-${{ matrix.name }}-${{ matrix.arch }} build timeout-minutes: 15 diff --git a/.gitignore b/.gitignore index a17d9d7..a8f72de 100644 --- a/.gitignore +++ b/.gitignore @@ -7,6 +7,11 @@ dist/ /curl/src openssl/ +# Generated PostgreSQL extension files (produced from .in templates by +# docker/Makefile.postgresql; version is derived from src/cloudsync.h) +/docker/postgresql/cloudsync.control +/src/postgresql/cloudsync--*.sql + # Test artifacts /coverage unittest diff --git a/docker/Makefile.postgresql b/docker/Makefile.postgresql index 8e1a514..56813b0 100644 --- a/docker/Makefile.postgresql +++ b/docker/Makefile.postgresql @@ -12,7 +12,12 @@ PG_INCLUDEDIR := $(shell $(PG_CONFIG) --includedir-server 2>/dev/null) # Extension metadata EXTENSION = cloudsync -EXTVERSION = 1.0 +# Read the extension version from src/cloudsync.h (CLOUDSYNC_VERSION) so the +# PG extension version stays in sync with the rest of the repo on every release. +EXTVERSION := $(shell sed -n 's/^\#define CLOUDSYNC_VERSION[[:space:]]*"\([^"]*\)".*/\1/p' src/cloudsync.h) +ifeq ($(strip $(EXTVERSION)),) +$(error Could not read CLOUDSYNC_VERSION from src/cloudsync.h) +endif # Detect OS for platform-specific settings ifneq ($(OS),Windows_NT) @@ -82,17 +87,32 @@ endif PG_EXTENSION_SQL = src/postgresql/$(EXTENSION)--$(EXTVERSION).sql PG_EXTENSION_CONTROL = docker/postgresql/$(EXTENSION).control +# Input templates (tracked). @EXTVERSION@ is substituted at build time. +PG_EXTENSION_SQL_IN = src/postgresql/$(EXTENSION).sql.in +PG_EXTENSION_CONTROL_IN = docker/postgresql/$(EXTENSION).control.in + +# Upgrade scripts (cloudsync----.sql) are hand-written per release. +PG_MIGRATIONS_DIR = src/postgresql/migrations +PG_MIGRATION_SQLS = $(wildcard $(PG_MIGRATIONS_DIR)/$(EXTENSION)--*--*.sql) + # ============================================================================ # PostgreSQL Build Targets # ============================================================================ -.PHONY: postgres-check postgres-build postgres-install postgres-package postgres-clean postgres-test \ +.PHONY: postgres-check postgres-check-migration postgres-build postgres-install postgres-package postgres-clean postgres-test \ postgres-docker-build postgres-docker-build-asan postgres-docker-run postgres-docker-run-asan postgres-docker-stop postgres-docker-rebuild \ postgres-docker-debug-build postgres-docker-debug-run postgres-docker-debug-rebuild \ postgres-docker-shell postgres-dev-rebuild postgres-help unittest-pg \ postgres-supabase-build postgres-supabase-rebuild postgres-supabase-run-smoke-test \ postgres-docker-run-smoke-test +# Verify that a cloudsync----.sql upgrade script exists for the +# current CLOUDSYNC_VERSION in src/cloudsync.h. Release-blocking: a missing +# upgrade script silently breaks ALTER EXTENSION cloudsync UPDATE for every +# existing deployment. Runs in <1s; safe to call on every PR. +postgres-check-migration: + @scripts/check-postgres-migration.sh + # Check if PostgreSQL is available postgres-check: @echo "Checking PostgreSQL installation..." @@ -102,9 +122,19 @@ postgres-check: @echo "Share directory: $(PG_SHAREDIR)" @echo "Include directory: $(PG_INCLUDEDIR)" +# Generate the versioned install script from the template +$(PG_EXTENSION_SQL): $(PG_EXTENSION_SQL_IN) src/cloudsync.h + @echo "Generating $@ (version $(EXTVERSION))" + @sed 's/@EXTVERSION@/$(EXTVERSION)/g' $(PG_EXTENSION_SQL_IN) > $@ + +# Generate the control file from the template +$(PG_EXTENSION_CONTROL): $(PG_EXTENSION_CONTROL_IN) src/cloudsync.h + @echo "Generating $@ (version $(EXTVERSION))" + @sed 's/@EXTVERSION@/$(EXTVERSION)/g' $(PG_EXTENSION_CONTROL_IN) > $@ + # Build PostgreSQL extension -postgres-build: postgres-check - @echo "Building PostgreSQL extension..." +postgres-build: postgres-check $(PG_EXTENSION_SQL) $(PG_EXTENSION_CONTROL) + @echo "Building PostgreSQL extension (version $(EXTVERSION))..." @echo "Compiling source files..." @for src in $(PG_ALL_SRC); do \ echo " CC $$src"; \ @@ -125,26 +155,37 @@ postgres-install: postgres-build install -m 644 $(PG_EXTENSION_SQL) $(PG_SHAREDIR)/extension/ @echo "Installing control file to $(PG_SHAREDIR)/extension/" install -m 644 $(PG_EXTENSION_CONTROL) $(PG_SHAREDIR)/extension/ + @if [ -n "$(PG_MIGRATION_SQLS)" ]; then \ + echo "Installing $(words $(PG_MIGRATION_SQLS)) migration script(s) to $(PG_SHAREDIR)/extension/"; \ + install -m 644 $(PG_MIGRATION_SQLS) $(PG_SHAREDIR)/extension/; \ + fi @echo "" @echo "Installation complete!" @echo "To use the extension, run in psql:" @echo " CREATE EXTENSION $(EXTENSION);" + @echo "To upgrade an existing installation, run in psql:" + @echo " ALTER EXTENSION $(EXTENSION) UPDATE;" # Package extension files for distribution PG_DIST_DIR = dist/postgresql postgres-package: postgres-build - @echo "Packaging PostgreSQL extension..." + @echo "Packaging PostgreSQL extension (version $(EXTVERSION))..." @mkdir -p $(PG_DIST_DIR) cp $(PG_EXTENSION_LIB) $(PG_DIST_DIR)/ cp $(PG_EXTENSION_SQL) $(PG_DIST_DIR)/ cp $(PG_EXTENSION_CONTROL) $(PG_DIST_DIR)/ + @if [ -n "$(PG_MIGRATION_SQLS)" ]; then \ + echo "Including $(words $(PG_MIGRATION_SQLS)) migration script(s)"; \ + cp $(PG_MIGRATION_SQLS) $(PG_DIST_DIR)/; \ + fi @echo "Package ready in $(PG_DIST_DIR)/" # Clean PostgreSQL build artifacts postgres-clean: @echo "Cleaning PostgreSQL build artifacts..." rm -f $(PG_OBJS) $(PG_EXTENSION_LIB) + rm -f $(PG_EXTENSION_SQL) $(PG_EXTENSION_CONTROL) @echo "Clean complete" # Test extension (requires running PostgreSQL) @@ -314,7 +355,7 @@ postgres-supabase-build: echo "Using base image: $$supabase_cli_image"; \ echo "Pulling fresh base image to avoid layer accumulation..."; \ docker pull "$$supabase_cli_image" 2>/dev/null || true; \ - docker build --build-arg SUPABASE_POSTGRES_TAG="$(SUPABASE_POSTGRES_TAG)" -f "$$tmp_dockerfile" -t "$$supabase_cli_image" .; \ + docker build --build-arg SUPABASE_POSTGRES_TAG="$(SUPABASE_POSTGRES_TAG)" --build-arg CLOUDSYNC_VERSION="$(EXTVERSION)" -f "$$tmp_dockerfile" -t "$$supabase_cli_image" .; \ rm -f "$$tmp_dockerfile"; \ echo "Build complete: $$supabase_cli_image" @@ -361,6 +402,7 @@ postgres-help: @echo "" @echo "Build & Install:" @echo " postgres-check - Verify PostgreSQL installation" + @echo " postgres-check-migration - Verify a migration script exists for the current version" @echo " postgres-build - Build extension (.so file)" @echo " postgres-install - Install extension to PostgreSQL" @echo " postgres-clean - Clean build artifacts" diff --git a/docker/README.md b/docker/README.md index 27188bc..2aaff4b 100644 --- a/docker/README.md +++ b/docker/README.md @@ -10,7 +10,7 @@ docker/ │ ├── Dockerfile # Custom PostgreSQL image │ ├── docker-compose.yml │ ├── init.sql # CloudSync metadata tables -│ └── cloudsync.control +│ └── cloudsync.control.in # template; cloudsync.control is generated at build time ``` ## Option 1: Standalone PostgreSQL diff --git a/docker/postgresql/Dockerfile.release b/docker/postgresql/Dockerfile.release index 12f5778..4d9e7aa 100644 --- a/docker/postgresql/Dockerfile.release +++ b/docker/postgresql/Dockerfile.release @@ -31,7 +31,7 @@ RUN case "${TARGETARCH}" in \ mkdir -p /tmp/cloudsync && \ tar -xzf /tmp/cloudsync.tar.gz -C /tmp/cloudsync && \ install -m 755 /tmp/cloudsync/cloudsync.so "$(pg_config --pkglibdir)/" && \ - install -m 644 /tmp/cloudsync/cloudsync--1.0.sql "$(pg_config --sharedir)/extension/" && \ + install -m 644 /tmp/cloudsync/cloudsync--*.sql "$(pg_config --sharedir)/extension/" && \ install -m 644 /tmp/cloudsync/cloudsync.control "$(pg_config --sharedir)/extension/" && \ rm -rf /tmp/cloudsync /tmp/cloudsync.tar.gz && \ apt-get purge -y curl && apt-get autoremove -y && rm -rf /var/lib/apt/lists/* diff --git a/docker/postgresql/Dockerfile.supabase b/docker/postgresql/Dockerfile.supabase index 0b5cd10..983a74a 100644 --- a/docker/postgresql/Dockerfile.supabase +++ b/docker/postgresql/Dockerfile.supabase @@ -30,13 +30,21 @@ RUN if [ ! -x "$CLOUDSYNC_PG_CONFIG" ]; then \ # Collect build artifacts (avoid installing into the Nix store) RUN mkdir -p /tmp/cloudsync-artifacts/lib /tmp/cloudsync-artifacts/extension && \ cp /tmp/cloudsync/cloudsync.so /tmp/cloudsync-artifacts/lib/ && \ - cp /tmp/cloudsync/src/postgresql/cloudsync--1.0.sql /tmp/cloudsync-artifacts/extension/ && \ - cp /tmp/cloudsync/docker/postgresql/cloudsync.control /tmp/cloudsync-artifacts/extension/ + cp /tmp/cloudsync/src/postgresql/cloudsync--*.sql /tmp/cloudsync-artifacts/extension/ && \ + cp /tmp/cloudsync/docker/postgresql/cloudsync.control /tmp/cloudsync-artifacts/extension/ && \ + # Include per-release upgrade scripts so ALTER EXTENSION ... UPDATE works + if ls /tmp/cloudsync/src/postgresql/migrations/cloudsync--*--*.sql 1>/dev/null 2>&1; then \ + cp /tmp/cloudsync/src/postgresql/migrations/cloudsync--*--*.sql /tmp/cloudsync-artifacts/extension/; \ + fi # Runtime image based on Supabase Postgres ARG SUPABASE_POSTGRES_TAG=17.6.1.071 FROM public.ecr.aws/supabase/postgres:${SUPABASE_POSTGRES_TAG} +# Extension version (derived from src/cloudsync.h by the Makefile and passed in +# as a build arg); used only for the image label. +ARG CLOUDSYNC_VERSION=unknown + # Match builder pg_config path ENV CLOUDSYNC_PG_CONFIG=/root/.nix-profile/bin/pg_config @@ -85,5 +93,5 @@ EXPOSE 5432 WORKDIR / # Add label with extension version -LABEL org.sqliteai.cloudsync.version="1.0" \ +LABEL org.sqliteai.cloudsync.version="${CLOUDSYNC_VERSION}" \ org.sqliteai.cloudsync.description="PostgreSQL with CloudSync CRDT extension" diff --git a/docker/postgresql/Dockerfile.supabase.release b/docker/postgresql/Dockerfile.supabase.release index 10036a4..5ef83f3 100644 --- a/docker/postgresql/Dockerfile.supabase.release +++ b/docker/postgresql/Dockerfile.supabase.release @@ -42,10 +42,10 @@ RUN case "${TARGETARCH}" in \ SHAREDIR_STD="/usr/share/postgresql" && \ install -d "$PKGLIBDIR" "$SHAREDIR_PGCONFIG/extension" && \ install -m 755 /tmp/cloudsync/cloudsync.so "$PKGLIBDIR/" && \ - install -m 644 /tmp/cloudsync/cloudsync--1.0.sql /tmp/cloudsync/cloudsync.control "$SHAREDIR_PGCONFIG/extension/" && \ + install -m 644 /tmp/cloudsync/cloudsync--*.sql /tmp/cloudsync/cloudsync.control "$SHAREDIR_PGCONFIG/extension/" && \ if [ "$SHAREDIR_STD" != "$SHAREDIR_PGCONFIG" ]; then \ install -d "$SHAREDIR_STD/extension" && \ - install -m 644 /tmp/cloudsync/cloudsync--1.0.sql /tmp/cloudsync/cloudsync.control "$SHAREDIR_STD/extension/"; \ + install -m 644 /tmp/cloudsync/cloudsync--*.sql /tmp/cloudsync/cloudsync.control "$SHAREDIR_STD/extension/"; \ fi && \ rm -rf /tmp/cloudsync /tmp/cloudsync.tar.gz && \ apt-get purge -y curl && apt-get autoremove -y && rm -rf /var/lib/apt/lists/* diff --git a/docker/postgresql/cloudsync.control b/docker/postgresql/cloudsync.control deleted file mode 100644 index 31304b8..0000000 --- a/docker/postgresql/cloudsync.control +++ /dev/null @@ -1,22 +0,0 @@ -# CloudSync PostgreSQL Extension Control File - -# Extension name -comment = 'CloudSync - CRDT-based multi-master database synchronization' - -# Default version -default_version = '1.0' - -# Can be loaded into an existing database -relocatable = true - -# Required PostgreSQL version -requires = '' - -# Superuser privileges required for installation -superuser = false - -# Modules to load -module_pathname = '$libdir/cloudsync' - -# Trusted extension (can be installed by non-superusers) -trusted = true diff --git a/docker/postgresql/cloudsync.control.in b/docker/postgresql/cloudsync.control.in new file mode 100644 index 0000000..704af37 --- /dev/null +++ b/docker/postgresql/cloudsync.control.in @@ -0,0 +1,13 @@ +# CloudSync PostgreSQL Extension Control File +# +# Generated from cloudsync.control.in by docker/Makefile.postgresql. +# Do not edit the generated file; edit the .in template instead. +# The version below is read from CLOUDSYNC_VERSION in src/cloudsync.h. + +comment = 'CloudSync - CRDT-based multi-master database synchronization' +default_version = '@EXTVERSION@' +relocatable = true +requires = '' +superuser = false +module_pathname = '$libdir/cloudsync' +trusted = true diff --git a/docs/internal/supabase-flyio.md b/docs/internal/supabase-flyio.md index e55e3a6..762d0ab 100644 --- a/docs/internal/supabase-flyio.md +++ b/docs/internal/supabase-flyio.md @@ -86,10 +86,10 @@ The `make postgres-supabase-build` command does the following: 1. **Pulls the official Supabase Postgres base image** (e.g., `public.ecr.aws/supabase/postgres:15.8.1.085`) — this is Supabase's standard PostgreSQL image that ships with ~30 extensions pre-installed (PostGIS, pgvector, etc.) 2. **Runs a multi-stage Docker build** using `docker/postgresql/Dockerfile.supabase`: - **Stage 1 (builder)**: Installs C build tools (`gcc`, `make`), copies the CloudSync source code (`src/`, `modules/`), and compiles `cloudsync.so` against Supabase's `pg_config` - - **Stage 2 (runtime)**: Starts from a clean Supabase Postgres image and copies in just three files: + - **Stage 2 (runtime)**: Starts from a clean Supabase Postgres image and copies in just three kinds of file: - `cloudsync.so` — the compiled extension binary - - `cloudsync.control` — tells PostgreSQL the extension's name and version - - `cloudsync--1.0.sql` — the SQL that defines all CloudSync functions + - `cloudsync.control` — tells PostgreSQL the extension's name and default version (generated at build time from `cloudsync.control.in`, with the version read from `src/cloudsync.h`) + - `cloudsync--.sql` — the SQL that defines all CloudSync functions for the current release (e.g. `cloudsync--1.0.16.sql`), plus any `cloudsync----.sql` upgrade scripts shipped under `src/postgresql/migrations/` 3. **Tags the result** with the same name as the base image, so it's a drop-in replacement To find the correct tag, clone the Supabase repo and check: @@ -118,7 +118,8 @@ Verify CloudSync is installed inside the image: ```bash docker run --rm /supabase-postgres-cloudsync:15.8.1.085 \ find / -name "cloudsync*" -type f 2>/dev/null -# Should list cloudsync.so, cloudsync.control, and cloudsync--1.0.sql +# Should list cloudsync.so, cloudsync.control, and cloudsync--.sql +# (plus any cloudsync----.sql upgrade scripts) # in /nix/store/...-postgresql-and-plugins-15.8/ paths ``` diff --git a/docs/postgresql/quickstarts/postgres.md b/docs/postgresql/quickstarts/postgres.md index e7b5c3d..6e14bab 100644 --- a/docs/postgresql/quickstarts/postgres.md +++ b/docs/postgresql/quickstarts/postgres.md @@ -49,11 +49,11 @@ docker compose up -d If you already run PostgreSQL directly on a VM or bare metal, download the release tarball that matches your operating system, CPU architecture, and PostgreSQL major version. -Extract the archive, then copy the three extension files into PostgreSQL's extension directories: +Extract the archive, then copy the extension files into PostgreSQL's extension directories. The tarball ships `cloudsync.control`, a `cloudsync--.sql` install script for the current release, and — from release 1.0.17 onward — any `cloudsync----.sql` upgrade scripts needed so existing installations can run `ALTER EXTENSION cloudsync UPDATE`. ```bash cp cloudsync.so "$(pg_config --pkglibdir)/" -cp cloudsync.control cloudsync--1.0.sql "$(pg_config --sharedir)/extension/" +cp cloudsync.control cloudsync--*.sql "$(pg_config --sharedir)/extension/" ``` Then connect to PostgreSQL and enable the extension: diff --git a/scripts/check-postgres-migration.sh b/scripts/check-postgres-migration.sh new file mode 100755 index 0000000..c426c4b --- /dev/null +++ b/scripts/check-postgres-migration.sh @@ -0,0 +1,105 @@ +#!/usr/bin/env bash +# +# Verify that the PostgreSQL extension has a valid upgrade script for the +# current CLOUDSYNC_VERSION in src/cloudsync.h. +# +# A missing cloudsync----.sql means existing deployments cannot run +# ALTER EXTENSION cloudsync UPDATE; +# to reach the new version. Worse, if a user replaces cloudsync.so without a +# valid upgrade path, they end up in a split-brain state where the .so on disk +# is new but pg_extension.extversion is stale and the SQL bindings in pg_proc +# may not match the ABI of the new binary. So this check is release-blocking. +# +# "Previous version" is derived from the most recent semver git tag reachable +# from HEAD: +# - For pre-1.0.17 tags the extension version is frozen at '1.0' because the +# static control file shipped default_version = '1.0' across every release. +# - For 1.0.17 and later, the control file is generated from cloudsync.h at +# build time, so we read CLOUDSYNC_VERSION from the tagged source tree. +# +# Exit codes: +# 0 - migration present (or no version bump since last tag; nothing to check) +# 1 - migration missing / required file not found +# 2 - misconfigured environment (no git, no header, unresolvable tag, etc.) + +set -euo pipefail + +repo_root=$(git rev-parse --show-toplevel 2>/dev/null || true) +if [ -z "$repo_root" ]; then + echo "Error: not inside a git working tree; cannot determine previous release tag." >&2 + exit 2 +fi +cd "$repo_root" + +header="src/cloudsync.h" +if [ ! -f "$header" ]; then + echo "Error: $header not found." >&2 + exit 2 +fi + +current_version=$(sed -n 's/^#define CLOUDSYNC_VERSION[[:space:]]*"\([^"]*\)".*/\1/p' "$header") +if [ -z "$current_version" ]; then + echo "Error: could not read CLOUDSYNC_VERSION from $header." >&2 + exit 2 +fi + +prev_tag=$(git describe --tags --abbrev=0 --match '[0-9]*.[0-9]*.[0-9]*' 2>/dev/null || true) +if [ -z "$prev_tag" ]; then + echo "No prior semver tag reachable from HEAD; skipping migration check." + echo "(This is expected on an initial release or a shallow clone without tags.)" + exit 0 +fi + +# Determine what pg_extension.extversion contains for deployments of prev_tag. +# Pre-new-scheme tags have a tracked control file with a literal default_version. +prev_version=$(git show "${prev_tag}:docker/postgresql/cloudsync.control" 2>/dev/null \ + | sed -n "s/^default_version = '\\([^']*\\)'.*/\\1/p" \ + | head -1 || true) + +# New-scheme tags generate the control file at build time; fall back to +# CLOUDSYNC_VERSION in the header at that tag. +if [ -z "$prev_version" ]; then + prev_version=$(git show "${prev_tag}:${header}" 2>/dev/null \ + | sed -n 's/^#define CLOUDSYNC_VERSION[[:space:]]*"\([^"]*\)".*/\1/p' || true) +fi + +if [ -z "$prev_version" ]; then + echo "Error: could not determine the extension version shipped at tag ${prev_tag}." >&2 + exit 2 +fi + +if [ "$prev_version" = "$current_version" ]; then + echo "CLOUDSYNC_VERSION unchanged since ${prev_tag} (${prev_version})." + echo "No migration script required." + exit 0 +fi + +expected="src/postgresql/migrations/cloudsync--${prev_version}--${current_version}.sql" +if [ ! -f "$expected" ]; then + cat >&2 < ${current_version} covered)." diff --git a/src/cloudsync.h b/src/cloudsync.h index 1bf0b53..196b8bb 100644 --- a/src/cloudsync.h +++ b/src/cloudsync.h @@ -18,7 +18,7 @@ extern "C" { #endif -#define CLOUDSYNC_VERSION "1.0.16" +#define CLOUDSYNC_VERSION "1.0.17" #define CLOUDSYNC_MAX_TABLENAME_LEN 512 #define CLOUDSYNC_VALUE_NOTSET -1 diff --git a/src/postgresql/cloudsync--1.0.sql b/src/postgresql/cloudsync.sql.in similarity index 99% rename from src/postgresql/cloudsync--1.0.sql rename to src/postgresql/cloudsync.sql.in index 763ac9b..edfa4d3 100644 --- a/src/postgresql/cloudsync--1.0.sql +++ b/src/postgresql/cloudsync.sql.in @@ -1,5 +1,5 @@ -- CloudSync Extension for PostgreSQL --- Version 1.0 +-- Version @EXTVERSION@ -- Complain if script is sourced in psql, rather than via CREATE EXTENSION \echo Use "CREATE EXTENSION cloudsync" to load this file. \quit diff --git a/src/postgresql/migrations/README.md b/src/postgresql/migrations/README.md new file mode 100644 index 0000000..0ed1e59 --- /dev/null +++ b/src/postgresql/migrations/README.md @@ -0,0 +1,39 @@ +# CloudSync PostgreSQL Migration Scripts + +This directory holds PostgreSQL extension upgrade scripts of the form: + + cloudsync----.sql + +PostgreSQL uses these to execute `ALTER EXTENSION cloudsync UPDATE` by chaining +one or more files to reach the target version. + +## When to add a script + +For every release that bumps `CLOUDSYNC_VERSION` in `src/cloudsync.h` and +introduces SQL-level changes (new functions, changed signatures, dropped +objects, new views/triggers/casts, etc.), add: + + cloudsync----.sql + +If a release has no SQL-level changes, you still need the file — an empty-ish +upgrade script (a single comment is fine) so `pg_extension_update_paths` reports +a valid path from the previous release. + +## Rules + +- Use `CREATE OR REPLACE` for every function — the underlying C symbol may have + changed even when the SQL signature didn't. +- Drop removed objects explicitly (`DROP FUNCTION IF EXISTS ...`). +- Never run `CREATE EXTENSION`-style bootstrap inside an upgrade script. +- Objects created inside an upgrade script are automatically attached to the + extension via `pg_depend`. +- Scripts are packaged and installed by `make postgres-install` / + `make postgres-package` via a wildcard; no need to list them anywhere. + +## Verifying the chain + +After rebuilding, inside the PG container: + + SELECT * FROM pg_extension_update_paths('cloudsync'); + +All `source -> target` rows should show a non-NULL `path`. diff --git a/src/postgresql/migrations/cloudsync--1.0--1.0.17.sql b/src/postgresql/migrations/cloudsync--1.0--1.0.17.sql new file mode 100644 index 0000000..a5f6a39 --- /dev/null +++ b/src/postgresql/migrations/cloudsync--1.0--1.0.17.sql @@ -0,0 +1,14 @@ +-- CloudSync PostgreSQL extension upgrade script: 1.0 -> 1.0.17 +-- +-- Existing deployments installed under the static 1.0 default_version need a +-- valid upgrade path to reach 1.0.17, the first release to track +-- CLOUDSYNC_VERSION from src/cloudsync.h as the extension version. +-- +-- No SQL-level changes were introduced between 1.0 and 1.0.17 -- the install +-- script surface is identical. This file exists solely to advertise a valid +-- upgrade path so users can run: +-- +-- ALTER EXTENSION cloudsync UPDATE; +-- +-- instead of the destructive DROP EXTENSION ... CASCADE; CREATE EXTENSION ... +-- workaround. From 710d48df8994b64ae91aa8499d34a8212109d7f4 Mon Sep 17 00:00:00 2001 From: Andrea Donetti Date: Wed, 22 Apr 2026 16:34:20 -0600 Subject: [PATCH 2/5] refactor(postgres): scope EXTVERSION to MAJOR.MINOR with drift guard --- README.md | 6 + docker/Makefile.postgresql | 15 +- docs/postgresql/quickstarts/postgres.md | 23 +++ scripts/check-postgres-migration.sh | 187 +++++++++++++----- src/cloudsync.h | 2 +- src/postgresql/migrations/README.md | 57 ++++-- .../migrations/cloudsync--1.0--1.0.17.sql | 14 -- 7 files changed, 221 insertions(+), 83 deletions(-) delete mode 100644 src/postgresql/migrations/cloudsync--1.0--1.0.17.sql diff --git a/README.md b/README.md index 99ea349..b42aaee 100644 --- a/README.md +++ b/README.md @@ -212,6 +212,12 @@ Part of the **[SQLite AI](https://sqlite.ai)** ecosystem: | **[SQLite-JS](https://github.com/sqliteai/sqlite-js)** | Custom SQLite functions in JavaScript | | **[Liteparser](https://github.com/sqliteai/liteparser)** | Fully compliant SQLite SQL parser | +## Versioning + +This project follows [semver](https://semver.org/). The single source of truth is `CLOUDSYNC_VERSION` in `src/cloudsync.h`; all packaged artifacts (NPM, Maven, pub.dev, Swift, Docker, native tarballs) inherit this version. PATCH releases never alter the exposed API — they ship bug fixes, performance improvements, and internal changes only. + +The PostgreSQL extension differs only in how it surfaces the version: its catalog version (`default_version` / `installed_version`) exposes `MAJOR.MINOR` only, so PATCH releases are transparent binary upgrades and only MINOR/MAJOR releases need `ALTER EXTENSION cloudsync UPDATE`. The `cloudsync_version()` SQL function always reports the full semver of the loaded `.so`. See the [PostgreSQL upgrade docs](docs/postgresql/quickstarts/postgres.md#upgrading-a-later-release) for the user-facing procedure. + ## License This project is licensed under the [Elastic License 2.0](./LICENSE.md). For production or managed service use, [contact SQLite Cloud, Inc](mailto:info@sqlitecloud.io) for a commercial license. diff --git a/docker/Makefile.postgresql b/docker/Makefile.postgresql index 56813b0..a9cda6c 100644 --- a/docker/Makefile.postgresql +++ b/docker/Makefile.postgresql @@ -12,12 +12,19 @@ PG_INCLUDEDIR := $(shell $(PG_CONFIG) --includedir-server 2>/dev/null) # Extension metadata EXTENSION = cloudsync -# Read the extension version from src/cloudsync.h (CLOUDSYNC_VERSION) so the -# PG extension version stays in sync with the rest of the repo on every release. -EXTVERSION := $(shell sed -n 's/^\#define CLOUDSYNC_VERSION[[:space:]]*"\([^"]*\)".*/\1/p' src/cloudsync.h) -ifeq ($(strip $(EXTVERSION)),) +# Read the binary version (full semver) from src/cloudsync.h, and derive +# EXTVERSION as just MAJOR.MINOR. Rationale: PATCH bumps are binary-only +# (no SQL surface change, no ALTER EXTENSION UPDATE needed); MINOR/MAJOR bumps +# are the SQL-surface-changing ones that require a per-release upgrade script. +# cloudsync_version() in the .so still reports the full semver for debugging. +CLOUDSYNC_VERSION_FULL := $(shell sed -n 's/^\#define CLOUDSYNC_VERSION[[:space:]]*"\([^"]*\)".*/\1/p' src/cloudsync.h) +ifeq ($(strip $(CLOUDSYNC_VERSION_FULL)),) $(error Could not read CLOUDSYNC_VERSION from src/cloudsync.h) endif +EXTVERSION := $(shell echo '$(CLOUDSYNC_VERSION_FULL)' | cut -d. -f1-2) +ifeq ($(strip $(EXTVERSION)),) +$(error Could not derive MAJOR.MINOR from CLOUDSYNC_VERSION '$(CLOUDSYNC_VERSION_FULL)') +endif # Detect OS for platform-specific settings ifneq ($(OS),Windows_NT) diff --git a/docs/postgresql/quickstarts/postgres.md b/docs/postgresql/quickstarts/postgres.md index 6e14bab..8de217c 100644 --- a/docs/postgresql/quickstarts/postgres.md +++ b/docs/postgresql/quickstarts/postgres.md @@ -80,6 +80,29 @@ psql -U postgres -d postgres -c "SELECT cloudsync_version();" If the extension is installed correctly, PostgreSQL returns the CloudSync version string. +### Upgrading a later release + +CloudSync uses the first two components of its semver as the PostgreSQL extension version (for example, `1.0.17` installs as extension version `1.0`). How you upgrade depends on which component changed: + +- **PATCH release** (e.g. `1.0.17 → 1.0.18`): pull the new Docker image or replace the extension files on disk and restart PostgreSQL. No SQL-level upgrade is needed — `installed_version` stays at `1.0` and the new binary takes over on reconnect. `SELECT cloudsync_version();` confirms the new semver. +- **MINOR or MAJOR release** (e.g. `1.0.x → 1.1.0`): pull the new artifacts as above, then run once per database: + + ```sql + ALTER EXTENSION cloudsync UPDATE; + ``` + + PostgreSQL applies any `cloudsync----.sql` upgrade scripts shipped with the release and moves `installed_version` to the new value. + +You can check the current state at any time: + +```sql +SELECT name, default_version, installed_version +FROM pg_available_extensions +WHERE name = 'cloudsync'; +``` + +If `installed_version` is behind `default_version` after a release, run `ALTER EXTENSION cloudsync UPDATE;` to catch up. + --- ## Step 3: Register Your Database in the CloudSync Dashboard diff --git a/scripts/check-postgres-migration.sh b/scripts/check-postgres-migration.sh index c426c4b..f14acd9 100755 --- a/scripts/check-postgres-migration.sh +++ b/scripts/check-postgres-migration.sh @@ -1,26 +1,36 @@ #!/usr/bin/env bash # -# Verify that the PostgreSQL extension has a valid upgrade script for the -# current CLOUDSYNC_VERSION in src/cloudsync.h. +# Enforce the PostgreSQL extension versioning contract on every PR/push. # -# A missing cloudsync----.sql means existing deployments cannot run -# ALTER EXTENSION cloudsync UPDATE; -# to reach the new version. Worse, if a user replaces cloudsync.so without a -# valid upgrade path, they end up in a split-brain state where the .so on disk -# is new but pg_extension.extversion is stale and the SQL bindings in pg_proc -# may not match the ABI of the new binary. So this check is release-blocking. +# The extension version (default_version in cloudsync.control, and the +# cloudsync--.sql filename) is MAJOR.MINOR only — it's derived from the +# first two components of CLOUDSYNC_VERSION in src/cloudsync.h. The full +# semver of the binary is reported by the cloudsync_version() SQL function. # -# "Previous version" is derived from the most recent semver git tag reachable -# from HEAD: -# - For pre-1.0.17 tags the extension version is frozen at '1.0' because the -# static control file shipped default_version = '1.0' across every release. -# - For 1.0.17 and later, the control file is generated from cloudsync.h at -# build time, so we read CLOUDSYNC_VERSION from the tagged source tree. +# Contract: +# - PATCH bumps (e.g. 1.0.16 -> 1.0.17) keep EXTVERSION the same ('1.0'). +# Binary-only release; no SQL surface changes allowed; no user action +# needed after swapping the .so. +# - MINOR / MAJOR bumps (e.g. 1.0.x -> 1.1.0 or 1.x -> 2.0) move EXTVERSION. +# Must ship a matching cloudsync----.sql upgrade script so +# existing deployments can ALTER EXTENSION cloudsync UPDATE. +# +# This script runs in one of two modes depending on whether EXTVERSION moved +# since the most recent ancestor semver tag: +# +# (a) EXTVERSION unchanged (patch release): +# diff the current cloudsync.sql.in against the previous tag's install +# script. If they differ, fail: SQL surface changed without a MINOR +# bump, which would silently break users whose pg_extension.extversion +# stays at the old value. +# +# (b) EXTVERSION changed (minor/major release): +# require src/postgresql/migrations/cloudsync----.sql. # # Exit codes: -# 0 - migration present (or no version bump since last tag; nothing to check) -# 1 - migration missing / required file not found -# 2 - misconfigured environment (no git, no header, unresolvable tag, etc.) +# 0 - contract satisfied (no bump, or bump with migration present) +# 1 - contract violated (SQL drift without bump, or missing migration) +# 2 - misconfigured environment (no git, missing header, unresolvable tag) set -euo pipefail @@ -32,17 +42,24 @@ fi cd "$repo_root" header="src/cloudsync.h" -if [ ! -f "$header" ]; then - echo "Error: $header not found." >&2 - exit 2 -fi +sql_template="src/postgresql/cloudsync.sql.in" + +[ -f "$header" ] || { echo "Error: $header not found." >&2; exit 2; } +[ -f "$sql_template" ] || { echo "Error: $sql_template not found." >&2; exit 2; } -current_version=$(sed -n 's/^#define CLOUDSYNC_VERSION[[:space:]]*"\([^"]*\)".*/\1/p' "$header") -if [ -z "$current_version" ]; then +# Read full semver from the header and derive MAJOR.MINOR. +current_full=$(sed -n 's/^#define CLOUDSYNC_VERSION[[:space:]]*"\([^"]*\)".*/\1/p' "$header") +if [ -z "$current_full" ]; then echo "Error: could not read CLOUDSYNC_VERSION from $header." >&2 exit 2 fi +current_ext=$(printf '%s\n' "$current_full" | cut -d. -f1-2) +if [ -z "$current_ext" ]; then + echo "Error: could not derive MAJOR.MINOR from CLOUDSYNC_VERSION '$current_full'." >&2 + exit 2 +fi +# Find the latest ancestor semver tag. prev_tag=$(git describe --tags --abbrev=0 --match '[0-9]*.[0-9]*.[0-9]*' 2>/dev/null || true) if [ -z "$prev_tag" ]; then echo "No prior semver tag reachable from HEAD; skipping migration check." @@ -50,56 +67,120 @@ if [ -z "$prev_tag" ]; then exit 0 fi -# Determine what pg_extension.extversion contains for deployments of prev_tag. -# Pre-new-scheme tags have a tracked control file with a literal default_version. -prev_version=$(git show "${prev_tag}:docker/postgresql/cloudsync.control" 2>/dev/null \ - | sed -n "s/^default_version = '\\([^']*\\)'.*/\\1/p" \ - | head -1 || true) - -# New-scheme tags generate the control file at build time; fall back to -# CLOUDSYNC_VERSION in the header at that tag. -if [ -z "$prev_version" ]; then - prev_version=$(git show "${prev_tag}:${header}" 2>/dev/null \ - | sed -n 's/^#define CLOUDSYNC_VERSION[[:space:]]*"\([^"]*\)".*/\1/p' || true) +# Resolve what EXTVERSION the previous release shipped with. +# Pre-new-scheme tags: tracked cloudsync.control held default_version. +# New-scheme tags: control is generated; read CLOUDSYNC_VERSION from header +# at that tag and truncate. +prev_full=$(git show "${prev_tag}:${header}" 2>/dev/null \ + | sed -n 's/^#define CLOUDSYNC_VERSION[[:space:]]*"\([^"]*\)".*/\1/p' || true) + +prev_ext=$(git show "${prev_tag}:docker/postgresql/cloudsync.control" 2>/dev/null \ + | sed -n "s/^default_version = '\\([^']*\\)'.*/\\1/p" \ + | head -1 || true) + +if [ -z "$prev_ext" ] && [ -n "$prev_full" ]; then + prev_ext=$(printf '%s\n' "$prev_full" | cut -d. -f1-2) fi -if [ -z "$prev_version" ]; then - echo "Error: could not determine the extension version shipped at tag ${prev_tag}." >&2 +if [ -z "$prev_ext" ]; then + echo "Error: could not determine EXTVERSION at tag ${prev_tag}." >&2 exit 2 fi -if [ "$prev_version" = "$current_version" ]; then - echo "CLOUDSYNC_VERSION unchanged since ${prev_tag} (${prev_version})." - echo "No migration script required." - exit 0 +# --------------------------------------------------------------------------- +# Mode (a): EXTVERSION unchanged — verify SQL surface didn't drift. +# --------------------------------------------------------------------------- +if [ "$prev_ext" = "$current_ext" ]; then + # Produce a normalized view of the previous release's install script. + if git cat-file -e "${prev_tag}:${sql_template}" 2>/dev/null; then + # New-scheme tag: substitute @EXTVERSION@ with that tag's EXTVERSION. + prev_sql=$(git show "${prev_tag}:${sql_template}" \ + | sed "s/@EXTVERSION@/${prev_ext}/g") + else + # Old-scheme tag: the literal cloudsync--.sql was tracked. + prev_install_path="src/postgresql/cloudsync--${prev_ext}.sql" + if ! git cat-file -e "${prev_tag}:${prev_install_path}" 2>/dev/null; then + echo "Error: could not find previous install script at ${prev_tag}:${prev_install_path}." >&2 + exit 2 + fi + prev_sql=$(git show "${prev_tag}:${prev_install_path}") + fi + + # Current install script, rendered (substitute @EXTVERSION@ -> current_ext). + curr_sql=$(sed "s/@EXTVERSION@/${current_ext}/g" "$sql_template") + + if [ "$prev_sql" = "$curr_sql" ]; then + echo "OK: patch-only release candidate." + echo " EXTVERSION unchanged at '${current_ext}' since ${prev_tag}." + echo " Binary semver: ${prev_full:-unknown} -> ${current_full}." + echo " SQL surface identical; no migration script required." + exit 0 + fi + + cat >&2 < ${current_full} + +The install script (src/postgresql/cloudsync.sql.in) differs from what the +previous release shipped, but EXTVERSION is still '${current_ext}'. Existing +deployments have pg_extension.extversion = '${current_ext}' and will NOT run +any upgrade script when they swap in the new .so, so the new SQL bindings +won't be applied to their catalog. This silently breaks users. + +Pick one: + + 1. Bump MINOR in src/cloudsync.h (e.g. 1.0.x -> 1.1.0), and add + src/postgresql/migrations/cloudsync--${current_ext}--.sql + with the DDL deltas (CREATE OR REPLACE FUNCTION ..., etc.). This is the + correct choice for any intentional SQL-surface change. + + 2. Revert the SQL-level change in cloudsync.sql.in if it was accidental + (e.g. a refactor that went further than intended). + +Diff (previous -> current, normalized): + +EOF + # Show a readable diff; fall back to a terse message if diff is unavailable. + if command -v diff >/dev/null 2>&1; then + diff -u <(printf '%s\n' "$prev_sql") <(printf '%s\n' "$curr_sql") >&2 || true + else + echo "(install 'diff' to see line-level changes)" >&2 + fi + exit 1 fi -expected="src/postgresql/migrations/cloudsync--${prev_version}--${current_version}.sql" +# --------------------------------------------------------------------------- +# Mode (b): EXTVERSION changed — require a migration file. +# --------------------------------------------------------------------------- +expected="src/postgresql/migrations/cloudsync--${prev_ext}--${current_ext}.sql" if [ ! -f "$expected" ]; then cat >&2 < ${current_version} covered)." +echo "OK: ${expected} exists (upgrade path ${prev_ext} -> ${current_ext} covered)." +echo " Binary semver: ${prev_full:-unknown} -> ${current_full}." diff --git a/src/cloudsync.h b/src/cloudsync.h index 196b8bb..1bf0b53 100644 --- a/src/cloudsync.h +++ b/src/cloudsync.h @@ -18,7 +18,7 @@ extern "C" { #endif -#define CLOUDSYNC_VERSION "1.0.17" +#define CLOUDSYNC_VERSION "1.0.16" #define CLOUDSYNC_MAX_TABLENAME_LEN 512 #define CLOUDSYNC_VALUE_NOTSET -1 diff --git a/src/postgresql/migrations/README.md b/src/postgresql/migrations/README.md index 0ed1e59..497bf77 100644 --- a/src/postgresql/migrations/README.md +++ b/src/postgresql/migrations/README.md @@ -7,22 +7,47 @@ This directory holds PostgreSQL extension upgrade scripts of the form: PostgreSQL uses these to execute `ALTER EXTENSION cloudsync UPDATE` by chaining one or more files to reach the target version. -## When to add a script +## Versioning model -For every release that bumps `CLOUDSYNC_VERSION` in `src/cloudsync.h` and -introduces SQL-level changes (new functions, changed signatures, dropped -objects, new views/triggers/casts, etc.), add: +The PostgreSQL extension version (`default_version` in `cloudsync.control`) is +**`MAJOR.MINOR`** only — derived from the first two components of +`CLOUDSYNC_VERSION` in `src/cloudsync.h`. The full semver of the compiled +binary is reported by the `cloudsync_version()` SQL function. - cloudsync----.sql +| Release kind | Example | EXTVERSION moves? | Upgrade script? | User action | +| -------------------------------- | --------------- | ----------------- | --------------- | ------------------ | +| PATCH bump (binary only) | 1.0.16 → 1.0.17 | No (stays `1.0`) | Not required | Swap the `.so`. | +| MINOR bump (SQL surface changes) | 1.0.x → 1.1.0 | Yes (`1.0` → `1.1`) | Required | `ALTER EXTENSION cloudsync UPDATE;` | +| MAJOR bump | 1.x → 2.0.0 | Yes (`1.x` → `2.0`) | Required | `ALTER EXTENSION cloudsync UPDATE;` | -If a release has no SQL-level changes, you still need the file — an empty-ish -upgrade script (a single comment is fine) so `pg_extension_update_paths` reports -a valid path from the previous release. +CI enforces this contract via `scripts/check-postgres-migration.sh`: -## Rules +- PATCH releases: the script diffs the current `cloudsync.sql.in` against the + previous release's install script. If they differ, the build fails — + accidental SQL surface drift in a PATCH release would silently break users + whose `pg_extension.extversion` would otherwise stay at the old EXTVERSION. +- MINOR/MAJOR releases: the script requires a matching + `cloudsync----.sql` in this directory. -- Use `CREATE OR REPLACE` for every function — the underlying C symbol may have - changed even when the SQL signature didn't. +## When to add an upgrade script + +Add one **only when `EXTVERSION` changes** — i.e. when you bump MINOR or MAJOR +in `src/cloudsync.h`. The filename is literally +`cloudsync----.sql`, not the full semver. + +Examples: + + 1.0.17 -> 1.1.0 → cloudsync--1.0--1.1.sql + 1.1.5 -> 2.0.0 → cloudsync--1.1--2.0.sql + +PATCH-level releases (1.0.16 → 1.0.17 → 1.0.18 → …) require **no file** — the +catalog's `installed_version` stays at the MAJOR.MINOR, and the `.so` swap is +a transparent binary upgrade. + +## Rules for upgrade script content + +- Use `CREATE OR REPLACE` for every function — the underlying C symbol may + have changed even when the SQL signature didn't. - Drop removed objects explicitly (`DROP FUNCTION IF EXISTS ...`). - Never run `CREATE EXTENSION`-style bootstrap inside an upgrade script. - Objects created inside an upgrade script are automatically attached to the @@ -37,3 +62,13 @@ After rebuilding, inside the PG container: SELECT * FROM pg_extension_update_paths('cloudsync'); All `source -> target` rows should show a non-NULL `path`. + +## I intended a PATCH but the CI check says "SQL surface drift" + +Either: + +1. You meant to change SQL. Bump MINOR in `src/cloudsync.h` and add + `cloudsync----.sql`. +2. You didn't. Revert the change in `src/postgresql/cloudsync.sql.in`. + +The CI error message prints a unified diff to help you decide which. diff --git a/src/postgresql/migrations/cloudsync--1.0--1.0.17.sql b/src/postgresql/migrations/cloudsync--1.0--1.0.17.sql deleted file mode 100644 index a5f6a39..0000000 --- a/src/postgresql/migrations/cloudsync--1.0--1.0.17.sql +++ /dev/null @@ -1,14 +0,0 @@ --- CloudSync PostgreSQL extension upgrade script: 1.0 -> 1.0.17 --- --- Existing deployments installed under the static 1.0 default_version need a --- valid upgrade path to reach 1.0.17, the first release to track --- CLOUDSYNC_VERSION from src/cloudsync.h as the extension version. --- --- No SQL-level changes were introduced between 1.0 and 1.0.17 -- the install --- script surface is identical. This file exists solely to advertise a valid --- upgrade path so users can run: --- --- ALTER EXTENSION cloudsync UPDATE; --- --- instead of the destructive DROP EXTENSION ... CASCADE; CREATE EXTENSION ... --- workaround. From 952322c0e1d013c0950b721b8d325e13652306d1 Mon Sep 17 00:00:00 2001 From: Andrea Donetti Date: Wed, 22 Apr 2026 17:27:15 -0600 Subject: [PATCH 3/5] fix(postgres): lazy version expansion; full semver for image label MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two fixes from review of the versioning branch. P2 — parse-time tool dependency: change CLOUDSYNC_VERSION_FULL and EXTVERSION from := (immediate) to = (recursive) so the sed/cut shell calls only fire when something actually references them. Drop the parse-time \$(error ...) guards and replace with a runtime check inside postgres-check, which now also prints the resolved versions. Non-PG targets like \`make help\` and \`make test\` no longer hard-fail when src/cloudsync.h is unreadable or sed/cut are missing from PATH. Caveat: Make still expands these variables at parse time wherever they appear in a rule's target/prereqs (e.g. \$(PG_EXTENSION_SQL): ...), so the shell calls themselves still happen — they just no longer crash the run. Fully eliminating parse-time evaluation would require gating the whole PG section on \$(MAKECMDGOALS), a larger structural change not done here. P3 — image label collision: the Supabase docker build was passing EXTVERSION (now MAJOR.MINOR, e.g. '1.0') as the CLOUDSYNC_VERSION build arg, which Dockerfile.supabase writes into org.sqliteai.cloudsync.version. Result: 1.0.16 and 1.0.17 images would carry the same label, breaking docker-inspect-style binary identification across patch releases. Pass CLOUDSYNC_VERSION_FULL so the label uniquely identifies the binary. Verified: - \`make postgres-check-migration\` still passes against the current state. - \`make help\` exits 0 even when src/cloudsync.h is moved aside (was a hard-fail before). - \`make postgres-check\` prints the resolved versions, e.g. "CloudSync version : 1.0.16 (extension version 1.0)" Co-Authored-By: Claude Opus 4.7 (1M context) --- docker/Makefile.postgresql | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-) diff --git a/docker/Makefile.postgresql b/docker/Makefile.postgresql index a9cda6c..8c66709 100644 --- a/docker/Makefile.postgresql +++ b/docker/Makefile.postgresql @@ -17,14 +17,12 @@ EXTENSION = cloudsync # (no SQL surface change, no ALTER EXTENSION UPDATE needed); MINOR/MAJOR bumps # are the SQL-surface-changing ones that require a per-release upgrade script. # cloudsync_version() in the .so still reports the full semver for debugging. -CLOUDSYNC_VERSION_FULL := $(shell sed -n 's/^\#define CLOUDSYNC_VERSION[[:space:]]*"\([^"]*\)".*/\1/p' src/cloudsync.h) -ifeq ($(strip $(CLOUDSYNC_VERSION_FULL)),) -$(error Could not read CLOUDSYNC_VERSION from src/cloudsync.h) -endif -EXTVERSION := $(shell echo '$(CLOUDSYNC_VERSION_FULL)' | cut -d. -f1-2) -ifeq ($(strip $(EXTVERSION)),) -$(error Could not derive MAJOR.MINOR from CLOUDSYNC_VERSION '$(CLOUDSYNC_VERSION_FULL)') -endif +# +# Recursive (=) rather than immediate (:=) assignment so the shell calls only +# fire when a PG-related target actually references these variables. Non-PG +# targets parse this included makefile without invoking sed/cut at all. +CLOUDSYNC_VERSION_FULL = $(shell sed -n 's/^\#define CLOUDSYNC_VERSION[[:space:]]*"\([^"]*\)".*/\1/p' src/cloudsync.h) +EXTVERSION = $(shell echo '$(CLOUDSYNC_VERSION_FULL)' | cut -d. -f1-2) # Detect OS for platform-specific settings ifneq ($(OS),Windows_NT) @@ -124,7 +122,10 @@ postgres-check-migration: postgres-check: @echo "Checking PostgreSQL installation..." @which $(PG_CONFIG) > /dev/null || (echo "Error: pg_config not found. Install postgresql-server-dev." && exit 1) + @[ -n "$(CLOUDSYNC_VERSION_FULL)" ] || (echo "Error: could not read CLOUDSYNC_VERSION from src/cloudsync.h" && exit 1) + @[ -n "$(EXTVERSION)" ] || (echo "Error: could not derive MAJOR.MINOR EXTVERSION from CLOUDSYNC_VERSION '$(CLOUDSYNC_VERSION_FULL)'" && exit 1) @echo "PostgreSQL version: $$($(PG_CONFIG) --version)" + @echo "CloudSync version : $(CLOUDSYNC_VERSION_FULL) (extension version $(EXTVERSION))" @echo "Extension directory: $(PG_PKGLIBDIR)" @echo "Share directory: $(PG_SHAREDIR)" @echo "Include directory: $(PG_INCLUDEDIR)" @@ -362,7 +363,7 @@ postgres-supabase-build: echo "Using base image: $$supabase_cli_image"; \ echo "Pulling fresh base image to avoid layer accumulation..."; \ docker pull "$$supabase_cli_image" 2>/dev/null || true; \ - docker build --build-arg SUPABASE_POSTGRES_TAG="$(SUPABASE_POSTGRES_TAG)" --build-arg CLOUDSYNC_VERSION="$(EXTVERSION)" -f "$$tmp_dockerfile" -t "$$supabase_cli_image" .; \ + docker build --build-arg SUPABASE_POSTGRES_TAG="$(SUPABASE_POSTGRES_TAG)" --build-arg CLOUDSYNC_VERSION="$(CLOUDSYNC_VERSION_FULL)" -f "$$tmp_dockerfile" -t "$$supabase_cli_image" .; \ rm -f "$$tmp_dockerfile"; \ echo "Build complete: $$supabase_cli_image" From fbc9326045c7cf8dd98e6d7e2cb465e428793245 Mon Sep 17 00:00:00 2001 From: Andrea Donetti Date: Thu, 23 Apr 2026 08:26:54 -0600 Subject: [PATCH 4/5] fix(postgres): render templates via phony target, no parse-time shell MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fully resolve the parse-time tool-dependency issue from the earlier review. The previous fix switched EXTVERSION and CLOUDSYNC_VERSION_FULL from := (immediate) to = (recursive), but Make still expanded them at parse time because they appeared in a rule target and prerequisite position: PG_EXTENSION_SQL: PG_EXTENSION_SQL_IN src/cloudsync.h postgres-build: postgres-check PG_EXTENSION_SQL PG_EXTENSION_CONTROL Make needs to resolve rule targets and prerequisites at parse time to build its rule database, so those references forced sed/cut to run on every make invocation — including make help, make test, etc. With the header missing, the parse-time sed calls printed stderr noise even though non-PG targets succeeded. This commit moves the template rendering into a phony target whose name contains no version reference, and removes the PG_EXTENSION_SQL and PG_EXTENSION_CONTROL references from postgres-build's prereqs. All remaining EXTVERSION / PG_EXTENSION_SQL references now live inside recipe bodies, which Make defers until a recipe actually runs. Net: non-PG targets invoke zero sed/cut calls at parse time. Hardening: - Each sed now writes to a .tmp file and renames via mv after success, so a failed render cannot leave a half-written cloudsync.control or cloudsync--.sql in the PostgreSQL extension share directory on a subsequent install. - postgres-generate-files depends on postgres-check, so the shell sanity-check messages (unreadable header, un-derivable EXTVERSION) still fire whenever the generator runs, not just on a direct postgres-check invocation. Tradeoff: the previous file-rule form had incremental rebuild — the SQL was only regenerated when the template or header changed. The phony form regenerates on every invocation of postgres-build. sed over the 300-line template is sub-millisecond and the cost is dwarfed by the .so compile, so this is acceptable for the full parse-time cleanliness it buys. Verified: - make help with src/cloudsync.h moved aside: exit 0, zero sed errors (previously exit 0 but two sed errors on stderr). - make postgres-check-migration: still passes. - make -p: EXTVERSION, CLOUDSYNC_VERSION_FULL, and PG_EXTENSION_SQL all remain unexpanded in the variable dump, confirming Make never resolved them at parse time. - make postgres-generate-files: renders cloudsync.control with default_version = '1.0' and cloudsync--1.0.sql with -- Version 1.0. Co-Authored-By: Claude Opus 4.7 (1M context) --- docker/Makefile.postgresql | 27 ++++++++++++++++----------- 1 file changed, 16 insertions(+), 11 deletions(-) diff --git a/docker/Makefile.postgresql b/docker/Makefile.postgresql index 8c66709..f5303bd 100644 --- a/docker/Makefile.postgresql +++ b/docker/Makefile.postgresql @@ -104,7 +104,7 @@ PG_MIGRATION_SQLS = $(wildcard $(PG_MIGRATIONS_DIR)/$(EXTENSION)--*--*.sql) # PostgreSQL Build Targets # ============================================================================ -.PHONY: postgres-check postgres-check-migration postgres-build postgres-install postgres-package postgres-clean postgres-test \ +.PHONY: postgres-check postgres-check-migration postgres-generate-files postgres-build postgres-install postgres-package postgres-clean postgres-test \ postgres-docker-build postgres-docker-build-asan postgres-docker-run postgres-docker-run-asan postgres-docker-stop postgres-docker-rebuild \ postgres-docker-debug-build postgres-docker-debug-run postgres-docker-debug-rebuild \ postgres-docker-shell postgres-dev-rebuild postgres-help unittest-pg \ @@ -130,18 +130,22 @@ postgres-check: @echo "Share directory: $(PG_SHAREDIR)" @echo "Include directory: $(PG_INCLUDEDIR)" -# Generate the versioned install script from the template -$(PG_EXTENSION_SQL): $(PG_EXTENSION_SQL_IN) src/cloudsync.h - @echo "Generating $@ (version $(EXTVERSION))" - @sed 's/@EXTVERSION@/$(EXTVERSION)/g' $(PG_EXTENSION_SQL_IN) > $@ - -# Generate the control file from the template -$(PG_EXTENSION_CONTROL): $(PG_EXTENSION_CONTROL_IN) src/cloudsync.h - @echo "Generating $@ (version $(EXTVERSION))" - @sed 's/@EXTVERSION@/$(EXTVERSION)/g' $(PG_EXTENSION_CONTROL_IN) > $@ +# Render the versioned .sql install script and .control file from their .in +# templates. This is a phony target (rather than file rules keyed on +# $(PG_EXTENSION_SQL) / $(PG_EXTENSION_CONTROL)) so that $(EXTVERSION) never +# appears in a rule's target or prerequisite position — those expansions +# happen at parse time and would force sed/cut to run on every `make` +# invocation, including for non-PG goals. Here, the references live inside +# the recipe body and are only evaluated when this target actually fires. +# Writes via a .tmp + atomic mv so a failed sed can't leave a half-rendered +# output file in the extension share dir. +postgres-generate-files: postgres-check + @echo "Rendering extension files at version $(EXTVERSION) (binary $(CLOUDSYNC_VERSION_FULL))" + @sed 's/@EXTVERSION@/$(EXTVERSION)/g' $(PG_EXTENSION_SQL_IN) > $(PG_EXTENSION_SQL).tmp && mv $(PG_EXTENSION_SQL).tmp $(PG_EXTENSION_SQL) + @sed 's/@EXTVERSION@/$(EXTVERSION)/g' $(PG_EXTENSION_CONTROL_IN) > $(PG_EXTENSION_CONTROL).tmp && mv $(PG_EXTENSION_CONTROL).tmp $(PG_EXTENSION_CONTROL) # Build PostgreSQL extension -postgres-build: postgres-check $(PG_EXTENSION_SQL) $(PG_EXTENSION_CONTROL) +postgres-build: postgres-generate-files @echo "Building PostgreSQL extension (version $(EXTVERSION))..." @echo "Compiling source files..." @for src in $(PG_ALL_SRC); do \ @@ -411,6 +415,7 @@ postgres-help: @echo "Build & Install:" @echo " postgres-check - Verify PostgreSQL installation" @echo " postgres-check-migration - Verify a migration script exists for the current version" + @echo " postgres-generate-files - Render cloudsync.control and cloudsync--.sql from templates" @echo " postgres-build - Build extension (.so file)" @echo " postgres-install - Install extension to PostgreSQL" @echo " postgres-clean - Clean build artifacts" From cd86c71b0522596dcbad11a294f433f57d621f48 Mon Sep 17 00:00:00 2001 From: Andrea Donetti Date: Thu, 23 Apr 2026 10:07:58 -0600 Subject: [PATCH 5/5] docs(postgres): mirror upgrade procedure to self-hosted Supabase quickstart The "Upgrading a later release" section added to docs/postgresql/quickstarts/postgres.md in 710d48d explains when PATCH releases are transparent binary upgrades vs. when MINOR/MAJOR releases require ALTER EXTENSION cloudsync UPDATE. Supabase self-hosted users run the same extension and face the same decision when moving between image tags, so mirror the section into supabase-self-hosted.md with the phrasing adapted for their workflow: pull sqlitecloud/sqlite-sync-supabase: and restart the db service, rather than swap the .so on disk. The top-level README "Versioning" section keeps its single link to postgres.md as the canonical reference. Co-Authored-By: Claude Opus 4.7 (1M context) --- .../quickstarts/supabase-self-hosted.md | 23 +++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/docs/postgresql/quickstarts/supabase-self-hosted.md b/docs/postgresql/quickstarts/supabase-self-hosted.md index a5dd047..30f7d2d 100644 --- a/docs/postgresql/quickstarts/supabase-self-hosted.md +++ b/docs/postgresql/quickstarts/supabase-self-hosted.md @@ -81,6 +81,29 @@ docker compose exec db psql -U supabase_admin -d postgres -c "SELECT cloudsync_v If the extension is installed correctly, PostgreSQL returns the CloudSync version string. +### Upgrading a later release + +CloudSync uses the first two components of its semver as the PostgreSQL extension version (for example, `1.0.17` installs as extension version `1.0`). How you upgrade depends on which component changed: + +- **PATCH release** (e.g. `1.0.17 → 1.0.18`): pull the matching `sqlitecloud/sqlite-sync-supabase:` image and restart the `db` service. No SQL-level upgrade is needed — `installed_version` stays at `1.0` and the new binary takes over on reconnect. `SELECT cloudsync_version();` confirms the new semver. +- **MINOR or MAJOR release** (e.g. `1.0.x → 1.1.0`): pull the new image and restart as above, then run once per database: + + ```sql + ALTER EXTENSION cloudsync UPDATE; + ``` + + PostgreSQL applies any `cloudsync----.sql` upgrade scripts shipped with the release and moves `installed_version` to the new value. + +You can check the current state at any time: + +```sql +SELECT name, default_version, installed_version +FROM pg_available_extensions +WHERE name = 'cloudsync'; +``` + +If `installed_version` is behind `default_version` after a release, run `ALTER EXTENSION cloudsync UPDATE;` to catch up. + --- ## Step 3: Register Your Database in the CloudSync Dashboard