From b70255d97d7fc9ee79c65598ad732c841c59cbda Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Tue, 4 Aug 2026 15:31:15 +0100 Subject: [PATCH 01/47] fix(cli): port db reset --experimental remote schema-files path to native TS (CLI-1958) Ports Go's apply.MigrateAndSeed EXPERIMENTAL declarative branch (apps/cli-go/internal/migration/apply/apply.go:19,51-68) for db reset's remote (--linked / remote --db-url) path, replacing the last Go-binary delegation on that command. A versionless --experimental / SUPABASE_EXPERIMENTAL reset with pg-delta not enabled now applies [db.migrations].schema_paths files directly (legacyApplySchemaFiles) instead of replaying timestamped migrations, faithfully reproducing two undocumented Go quirks: an empty schema_paths default silently applies nothing, and a partial glob failure is swallowed once at least one pattern matches. Hoists the Glob.SQLFiles traversal (legacySqlFilesGlob) out of the seed pipeline into shared/ so both [db.seed].sql_paths and the new [db.migrations].schema_paths resolve through one port of Go's glob semantics. Exposes schema_paths from the db-config TOML reader with the same env-override/remote-block-merge handling as the sibling seed field. Removes the remaining LegacyGoProxy delegation from db reset's handler and runtime layer now that both the remote and local paths are fully native (the local path's own schema-files branch still runs behind the existing db __db-bootstrap seam, out of scope here). --- apps/cli/docs/go-cli-porting-status.md | 2 +- .../legacy/commands/db/reset/SIDE_EFFECTS.md | 121 +++-- .../legacy/commands/db/reset/reset.errors.ts | 8 +- .../legacy/commands/db/reset/reset.handler.ts | 134 ++---- .../db/reset/reset.integration.test.ts | 453 ++++++++++-------- .../legacy/commands/db/reset/reset.layers.ts | 5 +- .../shared/legacy-db-config.toml-read.ts | 31 ++ .../legacy/shared/legacy-migration-apply.ts | 58 +++ apps/cli/src/legacy/shared/legacy-seed-ops.ts | 156 +----- .../legacy/shared/legacy-sql-files-glob.ts | 173 +++++++ 10 files changed, 633 insertions(+), 508 deletions(-) create mode 100644 apps/cli/src/legacy/shared/legacy-sql-files-glob.ts diff --git a/apps/cli/docs/go-cli-porting-status.md b/apps/cli/docs/go-cli-porting-status.md index 19f91e2a70..66d54f2a52 100644 --- a/apps/cli/docs/go-cli-porting-status.md +++ b/apps/cli/docs/go-cli-porting-status.md @@ -87,7 +87,7 @@ These commands exist in the TS CLI today but have no direct top-level equivalent | `db lint` | `ported` | `legacy/commands/db/lint/` | `n/a` | `n/a` | Native TS port. Runs `plpgsql_check` in a rolled-back transaction via LegacyDbConnection; emits Go-parity pretty JSON. | | `db pull` | `ported` | `legacy/commands/db/pull/` | `n/a` | `n/a` | Native TS port. Native pg-delta / migra migration + `--declarative` pg-delta export; reconciles `schema_migrations`. The initial-migra pull dumps the remote schema natively (`pg_dump`) then appends the migra diff. Only `--experimental` (structured dump) still delegates to Go, pending a TS PostgreSQL DDL parser for `format.WriteStructuredSchemas`. | | `db push` | `ported` | `legacy/commands/db/push/` | `n/a` | `n/a` | Native TS port. Connects local/linked/`--db-url`; pushes pending migrations, `--include-seed` seeds (`seed_files` hash tracking), `--include-roles`, `[db.vault]` secrets including decrypted `encrypted:` values; `--dry-run`; best-effort pg-delta migrations-catalog cache (warning-only on failure). Pipeline-incompatible statements (`CREATE INDEX CONCURRENTLY`, `VACUUM`, …) run standalone outside the batch transaction — from the closed Go PR supabase/cli#5156, also ported into `apps/cli-go` (CLI-1989 ruling). | -| `db reset` | `ported` | `legacy/commands/db/reset/` | `n/a` | `n/a` | Remote path native (drop user schemas, vault upsert, MigrateAndSeed, `--version`/`--last`, `--sql-paths` seed override). Local path native: running check, recreate + migrate + seed via the hidden Go `db __db-bootstrap` seam, storage-gated bucket seeding (reuses `seed buckets`), git-branch `Finished…` line. Only the niche `--experimental` remote schema-files path still delegates to the Go binary (telemetry-disabled). Pipeline-incompatible statements run standalone outside the batch transaction, same as `db push` (closed Go PR supabase/cli#5156, CLI-1989 ruling). | +| `db reset` | `ported` | `legacy/commands/db/reset/` | `n/a` | `n/a` | Remote path native (drop user schemas, vault upsert, MigrateAndSeed — including the `--experimental` declarative `[db.migrations].schema_paths` apply branch, CLI-1958 — `--version`/`--last`, `--sql-paths` seed override). Local path native: running check, recreate + migrate + seed via the hidden Go `db __db-bootstrap` seam (that seam still forwards `--experimental` for its own schema-files branch, CLI-1955 scope), storage-gated bucket seeding (reuses `seed buckets`), git-branch `Finished…` line. Pipeline-incompatible statements run standalone outside the batch transaction, same as `db push` (closed Go PR supabase/cli#5156, CLI-1989 ruling). | | `db start` | `ported` | `legacy/commands/db/start/` | `n/a` | `n/a` | Native TS port. Validates config, checks "already running" (prints Go's line), else delegates the container bootstrap (create + health + initial schema/roles/migrations/seed + `_current_branch`) to the hidden Go `db __db-bootstrap --mode start` seam. No status table / `cli_stack_started` (those are `supabase start`). `--from-backup` supported. | | `inspect report` | `ported` | `legacy/commands/inspect/report/` | `n/a` | `n/a` | Native TS port. Runs every inspect query via server-side `COPY ... CSV`, writes 14 CSVs under `//`, then renders a Go-parity Glamour rules summary (bounded csvq-subset evaluator; custom `[experimental.inspect.rules]` supported). | | `inspect db db-stats` | `ported` | `legacy/commands/inspect/db/db-stats/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | diff --git a/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md b/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md index fd5cf2f93b..ea6afa5889 100644 --- a/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md +++ b/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md @@ -2,12 +2,16 @@ Native TypeScript port of `apps/cli-go/internal/db/reset/reset.go`. Reinitialises a database from local migrations (plus seed). The **remote** path (`--linked`, or a -remote `--db-url`) is native: drop all user schemas, upsert vault secrets, then -re-apply migrations and seed. The **local** path (`--local`/default, or a `--db-url` +remote `--db-url`) is native: drop all user schemas, upsert vault secrets, then either +re-apply migrations (the default) or, on a versionless `--experimental`/ +`SUPABASE_EXPERIMENTAL` reset with pg-delta not enabled, apply the declarative +`[db.migrations].schema_paths` files instead (Go's `apply.MigrateAndSeed` EXPERIMENTAL +branch, CLI-1958), then seed. The **local** path (`--local`/default, or a `--db-url` pointing at the local stack) is also native: TS orchestrates the running check, messages, bucket seeding, and git-branch line, while the container-recreate -primitives run behind the hidden Go `db __db-bootstrap` seam. Only the niche -**`--experimental`** remote schema-files path still delegates to the Go binary. +primitives run behind the hidden Go `db __db-bootstrap` seam — that seam forwards +`--experimental` to the Go child itself, so the local path's own schema-files branch +is still handled by Go (CLI-1955 scope, unaffected by CLI-1958). ## Files Read @@ -19,6 +23,7 @@ primitives run behind the hidden Go `db __db-bootstrap` seam. Only the niche | `~/.supabase//project-ref` | plain text | `--linked`, to resolve the ref | | `~/.supabase/access-token` | plain text | `--linked`, when `SUPABASE_ACCESS_TOKEN` unset and a temp role is minted | | seed files from `--sql-paths` or `[db.seed].sql_paths` | SQL | when seeding is enabled (not `--no-seed`); `--sql-paths` overrides config | +| schema files from `[db.migrations].schema_paths` | SQL | remote path only, when the `--experimental` schema-files branch is taken (see Notes) | | `/supabase/buckets/` | files | local path, when storage is up and `[storage.buckets]` configure objects | ## Files Written @@ -29,31 +34,32 @@ primitives run behind the hidden Go `db __db-bootstrap` seam. Only the niche | `~/.supabase/telemetry.json` | JSON | always (post-run telemetry flush) | On the local path the Go seam additionally recreates the `supabase_db_` -container/volume and applies the initial schema (`SetupLocalDatabase`); the -`--experimental` remote path produces whatever the delegated Go binary writes. +container/volume and applies the initial schema (`SetupLocalDatabase`). ## Subprocesses -| Command | When | Purpose | -| --------------------------------------------------------------------------- | ----------------------------------- | ----------------------------------------------------------------------- | -| `docker container inspect supabase_db_` | local path | `AssertSupabaseDbIsRunning` probe (Podman fallback) | -| `supabase-go db __db-bootstrap --mode recreate [--version ] [--no-seed]` | local path | recreate container + init schema + migrate + seed + restart services | -| `supabase-go db __db-bootstrap --mode await-storage` | local path | storage health gate before bucket seeding (`ready` / `absent`) | -| `supabase-go db reset --linked\|--db-url … [--no-seed]` | `--experimental` remote, no version | the un-ported experimental schema-files apply path (telemetry disabled) | +| Command | When | Purpose | +| --------------------------------------------------------------------------- | ---------- | -------------------------------------------------------------------- | +| `docker container inspect supabase_db_` | local path | `AssertSupabaseDbIsRunning` probe (Podman fallback) | +| `supabase-go db __db-bootstrap --mode recreate [--version ] [--no-seed]` | local path | recreate container + init schema + migrate + seed + restart services | +| `supabase-go db __db-bootstrap --mode await-storage` | local path | storage health gate before bucket seeding (`ready` / `absent`) | The seam subprocesses run with `SUPABASE_TELEMETRY_DISABLED=1`, stderr inherited; -`--network-id` / a flag-selected `--profile` are forwarded. +`--network-id` / a flag-selected `--profile` (plus `--experimental`, so the seam's own +`MigrateAndSeed` takes Go's schema-files branch on a versionless local reset) are +forwarded. ## Database Mutations ### Remote path (native, in TS) -| Statement | When | -| ------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------ | -| `drop.sql` `DO` block (drops user schemas/extensions/public objects, truncates auth/migrations) | always, first | -| `SELECT vault.update_secret(...)` / `vault.create_secret(...)` | when `[db.vault]` has syncable secrets | -| migration statements + `schema_migrations` history insert (per file, transactional; pipeline-incompatible statements run standalone — see Notes) | when `[db.migrations].enabled`, for migrations `≤ --version` | -| seed statements + `seed_files` hash upsert | when `[db.seed].enabled` and not `--no-seed` | +| Statement | When | +| ------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------- | +| `drop.sql` `DO` block (drops user schemas/extensions/public objects, truncates auth/migrations) | always, first | +| `SELECT vault.update_secret(...)` / `vault.create_secret(...)` | when `[db.vault]` has syncable secrets | +| schema-file statements (no history bookkeeping, no `RESET ALL` between files) | `--experimental` + no resolved version + pg-delta not enabled (see Notes) | +| migration statements + `schema_migrations` history insert (per file, transactional; pipeline-incompatible statements run standalone — see Notes) | otherwise, when `[db.migrations].enabled`, for migrations `≤ --version` | +| seed statements + `seed_files` hash upsert | when `[db.seed].enabled` and not `--no-seed` (runs after either branch above) | ### Local path (inside the Go seam) @@ -81,35 +87,36 @@ races a restarting gateway. | `SUPABASE_ACCESS_TOKEN` | auth token for the `--linked` resolver path | no (falls back to keyring → `~/.supabase/access-token`) | | `SUPABASE_DB_PASSWORD` | password for the linked/remote connection | no | | `SUPABASE_YES` | auto-confirm the reset prompt | no (also `--yes`) | -| `SUPABASE_EXPERIMENTAL` | routes the experimental schema-files path to Go | no (also `--experimental`) | +| `SUPABASE_EXPERIMENTAL` | selects the remote schema-files apply branch | no (also `--experimental`) | | `SUPABASE_PROJECT_ID` | overrides the local container id (`utils.DbId`) | no | ## Exit Codes -| Code | Condition | -| -------------------- | -------------------------------------------------------------------------------------------------------------------------- | -| `0` | success | -| `1` | mutually exclusive target flags (`[db-url linked local]`) | -| `1` | `--version` + `--last` together (`[last version]`) | -| `1` | `--version` not an integer (`invalid version number`) | -| `1` | `--version` has no matching migration file | -| `1` | local: database not running (`supabase start is not running.`) | -| `1` | user declined the reset confirmation (`context canceled`) | -| `1` | `config.toml` parse failure | -| `1` | drop / migrate / seed / vault apply failure, or connection error | -| child's exact code\* | local: container recreate / storage health-gate failure (seam), or `--experimental`/`--linked` delegate (proxy) child exit | - -\* The `db __db-bootstrap` seam and the `--experimental` remote delegate both -propagate the spawned `supabase-go` child's real exit code (e.g. `130` after a -Ctrl-C mid-recreate) instead of collapsing every failure to `1` — in every -`--output-format` (CLI-1879). +| Code | Condition | +| -------------------- | ----------------------------------------------------------------------------------------- | +| `0` | success | +| `1` | mutually exclusive target flags (`[db-url linked local]`) | +| `1` | `--version` + `--last` together (`[last version]`) | +| `1` | `--version` not an integer (`invalid version number`) | +| `1` | `--version` has no matching migration file | +| `1` | local: database not running (`supabase start is not running.`) | +| `1` | user declined the reset confirmation (`context canceled`) | +| `1` | `config.toml` parse failure | +| `1` | drop / migrate / seed / vault apply failure, or connection error | +| `1` | no `[db.migrations].schema_paths` pattern matched anything on the `--experimental` branch | +| child's exact code\* | local: container recreate / storage health-gate failure (seam) | + +\* The `db __db-bootstrap` seam propagates the spawned `supabase-go` child's real exit +code (e.g. `130` after a Ctrl-C mid-recreate) instead of collapsing every failure to +`1` — in every `--output-format` (CLI-1879). ## Output -The remote path prints `Resetting remote database…` to **stderr**, then the -drop/migrate/seed progress (`Applying migration …`, `Seeding data from …`). Go -connects with `io.Discard`, so there is **no** `Connecting to … database…` line and -**no** `Finished …` line on the remote path. +The remote path prints `Resetting remote database…` to **stderr**, then either the +schema-files branch's apply (no per-file progress — Go's `applySchemaFiles` prints +nothing, CLI-1958) or the migrate/seed progress (`Applying migration …`, `Seeding +data from …`). Go connects with `io.Discard`, so there is **no** `Connecting to … +database…` line and **no** `Finished …` line on the remote path. The local path prints `Resetting local database…` to **stderr**, then the seam's `Recreating database...` / `Restarting containers...` progress, and finally @@ -118,9 +125,8 @@ in Aqua). ### `--output-format text` (Go CLI compatible) -Byte-matches Go's stderr progress for both the remote and local paths. The -`--experimental` remote path passes the delegated Go binary's output through -unchanged. +Byte-matches Go's stderr progress for both the remote and local paths, including the +silent (no-progress-line) `--experimental` schema-files apply. ### `--output-format json` / `stream-json` @@ -152,6 +158,27 @@ path has no confirmation prompt. - `--last n` reverts the most recent `n` migrations; if `n ≥ total`, the reset target version becomes `-` (revert everything). Mutually exclusive with `--version`. - `--db-url`, `--linked`, and `--local` (default true) are mutually exclusive. -- **Known interim**: only `--experimental` remote resets run via the Go binary; the - best-effort pg-delta catalog cache (inside the seam) is not surfaced (no output - impact). `encrypted:` vault secrets are skipped on the remote path. +- **`--experimental` remote schema-files apply** (Go's `apply.MigrateAndSeed` + EXPERIMENTAL branch, `apps/cli-go/internal/migration/apply/apply.go:19,51-68`; + ported natively CLI-1958): taken on the remote path when `--experimental` / + `SUPABASE_EXPERIMENTAL` is set, no `--version`/`--last` resolved a version, AND + `[experimental.pgdelta].enabled` is NOT set. A hard `if`/`else if` in Go — taking + this branch means timestamped migrations never run at all, even when + `[db.migrations].schema_paths` matches nothing. Faithfully reproduces two + undocumented Go quirks: (1) the `schema_paths` default is `[]`, so a stock project + running an experimental remote reset silently applies NOTHING (drops schemas, + seeds, but replays no SQL) rather than falling back to migrations; (2) a partial + glob failure (some patterns match, others don't) is silently dropped — only a + TOTAL failure (no pattern matches anything) aborts the reset, with Go's joined + `no files matched pattern: …` text and no `CmdSuggestion`. A per-file apply + failure attaches Go's `See schema file: ` suggestion. No progress line is + printed per file (Go's `applySchemaFiles` has no output), no migration-history row + is inserted, and no `RESET ALL` runs between files. Seeding still runs afterward, + unconditionally, exactly as on the migrations branch. The best-effort pg-delta + catalog-cache warning (`down.go:58-59`, gated on `SUPABASE_EXPERIMENTAL_PG_DELTA`) + is not ported (no output impact) — same known gap as the migrations branch. + `encrypted:` vault secrets are skipped on the remote path (both branches). +- The local path's own `--experimental` schema-files branch is still handled by the + Go child behind the `db __db-bootstrap` seam (CLI-1955 scope) — this command's + handler forwards `--experimental` to that seam but does not implement the branch + itself for the local target. diff --git a/apps/cli/src/legacy/commands/db/reset/reset.errors.ts b/apps/cli/src/legacy/commands/db/reset/reset.errors.ts index 80d2cba332..9ecae3c60f 100644 --- a/apps/cli/src/legacy/commands/db/reset/reset.errors.ts +++ b/apps/cli/src/legacy/commands/db/reset/reset.errors.ts @@ -51,9 +51,15 @@ export class LegacyDbResetCancelledError extends Data.TaggedError("LegacyDbReset readonly message: string; }> {} -/** A drop / migrate / seed / vault statement failed during the remote reset. */ +/** + * A drop / migrate / seed / vault statement failed during the remote reset. `suggestion` + * is Go's `CmdSuggestion` — set only by the `--experimental` schema-files apply branch + * (`"See schema file: "`, `apply.go:63`); every other apply failure on this + * command leaves it unset, matching Go. + */ export class LegacyDbResetApplyError extends Data.TaggedError("LegacyDbResetApplyError")<{ readonly message: string; + readonly suggestion?: string; }> {} /** diff --git a/apps/cli/src/legacy/commands/db/reset/reset.handler.ts b/apps/cli/src/legacy/commands/db/reset/reset.handler.ts index af07850513..f701aa5539 100644 --- a/apps/cli/src/legacy/commands/db/reset/reset.handler.ts +++ b/apps/cli/src/legacy/commands/db/reset/reset.handler.ts @@ -7,7 +7,6 @@ import { legacyResolveExperimentalWithProjectEnv, legacyResolveYesWithProjectEnv, } from "../../../../shared/legacy/global-flags.ts"; -import { LegacyGoProxy } from "../../../../shared/legacy/go-proxy.service.ts"; import { CONTEXT_CANCELED_MESSAGE } from "../../../../shared/output/errors.ts"; import { Output } from "../../../../shared/output/output.service.ts"; import { legacyAqua, legacyYellow } from "../../../shared/legacy-colors.ts"; @@ -20,13 +19,13 @@ import { legacyResolveSeedSqlPath, } from "../../../shared/legacy-db-config.toml-read.ts"; import { LegacyDbConnection } from "../../../shared/legacy-db-connection.service.ts"; -import { legacyApplyMigrations } from "../../../shared/legacy-migration-apply.ts"; +import { + legacyApplyMigrations, + legacyApplySchemaFiles, +} from "../../../shared/legacy-migration-apply.ts"; import { legacyParseMigrationVersion } from "../../../shared/legacy-migration-timestamp.format.ts"; import { legacyPromptYesNo } from "../../../../shared/legacy/legacy-prompt-yes-no.ts"; -import { - type LegacyDbConnType, - resolveLegacyDbTargetFlags, -} from "../../../shared/legacy-db-target-flags.ts"; +import { resolveLegacyDbTargetFlags } from "../../../shared/legacy-db-target-flags.ts"; import { LegacyLinkedProjectCache } from "../../../telemetry/legacy-linked-project-cache.service.ts"; import { LegacyTelemetryState } from "../../../telemetry/legacy-telemetry-state.service.ts"; import { legacyDropUserSchemas } from "../shared/legacy-drop-schemas.ts"; @@ -51,59 +50,31 @@ import { const MIGRATE_FILE_PATTERN = /^([0-9]+)_(.*)\.sql$/u; -const applyError = (message: string) => new LegacyDbResetApplyError({ message }); +const applyError = (message: string, suggestion?: string) => + new LegacyDbResetApplyError({ message, ...(suggestion !== undefined ? { suggestion } : {}) }); /** Go's `toLogMessage` (`internal/db/reset/reset.go:88-91`). */ const toLogMessage = (version: string): string => version.length > 0 ? ` to version: ${version}` : "..."; -/** - * Rebuilds the `db reset` argv for the remaining Go-delegated path: a remote - * `--experimental` reset with no resolved version. Only the flags reachable on - * that path are forwarded — `--local` always takes the native path, and a set - * `--version`/`--last` resolves a non-empty version which disables the experimental - * delegation (a degenerate `--last 0` resolves to "" and is behaviourally identical - * whether or not it is forwarded, so it is omitted). - * - * The target selector is forwarded from the RESOLVED `connType`, not the raw `--linked` - * boolean: the parent's `resolveLegacyDbTargetFlags` follows Cobra's `Changed` semantics, so - * `--linked=false` selects the linked/remote target (this path is remote-only). Forwarding - * only when `flags.linked === true` would drop the selector for `--linked=false` and let the - * Go child fall back to its local default — resetting the wrong database. - */ -const buildResetArgs = ( - flags: LegacyDbResetFlags, - connType: LegacyDbConnType, - yes: boolean, -): Array => { - const args = ["db", "reset"]; - if (Option.isSome(flags.dbUrl)) args.push("--db-url", flags.dbUrl.value); - else if (connType === "linked") args.push("--linked"); - if (flags.noSeed) args.push("--no-seed"); - for (const p of flags.sqlPaths) args.push("--sql-paths", p); - // Forward the parent's RESOLVED `yes` as a bound flag. Go's `--yes` beats `AutomaticEnv`, - // so `--yes=false` overrides an inherited `SUPABASE_YES=true` (the child no longer - // auto-confirms a reset the user protected with `--yes=false`), while `--yes=true` honors - // an explicit `--yes` / env even in machine mode where the child's stdin is ignored. - // `--yes=false` still prompts on a TTY (Go's PromptYesNo only short-circuits on true), so - // this matches the default behavior when neither flag nor env is set. - args.push(`--yes=${yes}`); - return args; -}; - /** * `supabase db reset` — reinitialise a database from local migrations (+ seed). * - * Strict 1:1 port of `apps/cli-go/internal/db/reset/reset.go`. The remote path - * (`--linked` / a remote `--db-url`) is native. The local path (and the niche - * `--experimental` schema-files path) delegate to the Go binary as a documented - * interim until the container-bootstrap seam is ported (CLI-1325 Stage 3). + * Strict 1:1 port of `apps/cli-go/internal/db/reset/reset.go`. Both the remote path + * (`--linked` / a remote `--db-url`) and the local path (`--local`/default, or a + * `--db-url` pointing at the local stack) are native TS. On the remote path, a + * versionless `--experimental` (or `SUPABASE_EXPERIMENTAL`) reset with pg-delta NOT + * enabled takes Go's EXPERIMENTAL declarative schema-files branch of + * `apply.MigrateAndSeed` (`legacyApplySchemaFiles`, CLI-1958) instead of replaying + * timestamped migrations. The local path's container-recreate primitives still run + * behind the hidden Go `db __db-bootstrap` seam (CLI-1955); that seam forwards + * `--experimental` to the Go child itself, so the local path's own schema-files + * branch is out of scope here. */ export const legacyDbReset = Effect.fn("legacy.db.reset")(function* (flags: LegacyDbResetFlags) { const output = yield* Output; const resolver = yield* LegacyDbConfigResolver; const dbConn = yield* LegacyDbConnection; - const proxy = yield* LegacyGoProxy; const seam = yield* LegacyDbBootstrapSeam; const cliConfig = yield* LegacyCliConfig; const telemetryState = yield* LegacyTelemetryState; @@ -234,35 +205,6 @@ export const legacyDbReset = Effect.fn("legacy.db.reset")(function* (flags: Lega } const connType = target.connType ?? "local"; - // Single source of truth for "does this reset delegate to the Go child?" — - // checked at both delegation sites below (before `resolve()` for a linked - // target, after it for a `--db-url` target) so the two call sites can never - // drift apart. - const shouldDelegateExperimental = experimental && resolvedVersion === ""; - - // Delegates the remaining `--experimental` schema-files apply path - // (`apply.MigrateAndSeed`, not ported) to the Go child. In text mode inherit - // its stdio. Under a machine-output mode (`--output-format json|stream-json`) - // the Go child emits no TS envelope, so suppress its stdout (capture + discard) - // and emit the same structured success the native local and remote paths do, - // keeping the JSON contract consistent across all reset paths. - const delegateExperimentalReset = () => - Effect.gen(function* () { - const env = { SUPABASE_TELEMETRY_DISABLED: "1" }; - if (output.format === "text") { - yield* proxy.exec(buildResetArgs(flags, connType, yes), { env }); - } else { - // Machine-output mode is non-interactive: give the Go child a non-TTY stdin - // (`stdin: "ignore"`) so it can't block on (or be answered at) Go's - // destructive reset prompt — it takes the default `false`, matching the - // native reset path which suppresses prompts under json/stream-json. - yield* proxy.execCapture(buildResetArgs(flags, connType, yes), { env, stdin: "ignore" }); - yield* output.success("Reset remote database.", { - target: "remote", - version: resolvedVersion, - }); - } - }); // Go's ParseDatabaseConfig runs LoadProjectRef BEFORE the fallible linked // resolution (db_url.go:87-95), and Execute() writes the linked-project cache @@ -272,21 +214,6 @@ export const legacyDbReset = Effect.fn("legacy.db.reset")(function* (flags: Lega if (connType === "linked") { const refResolver = yield* LegacyProjectRefResolver; linkedRefForCache = yield* refResolver.loadProjectRef(Option.none()); - - // A linked target is never local (`resolver.resolve()`'s "linked" branch - // always returns `isLocal: false`), so the delegated-experimental check can - // run BEFORE calling `resolve()`. This matters: for `connType === "linked"`, - // `resolve()` mints/verifies a temporary Postgres login role over the - // Management API — and the delegated Go child re-runs that exact same - // `ParseDatabaseConfig` work itself once delegation happens. Calling - // `resolve()` here would mint the temp role twice for zero downstream use on - // this branch (Go's own reset flow mints it exactly once, as part of the code - // path being delegated to — confirmed against `apps/cli-go/internal/utils/ - // flags/db_url.go`'s `NewDbConfigWithPassword`/`initLoginRole`). CLI-1879. - if (shouldDelegateExperimental) { - yield* delegateExperimentalReset(); - return; - } } const cfg = yield* resolver.resolve({ dbUrl: flags.dbUrl, connType, dnsResolver }); @@ -385,22 +312,12 @@ export const legacyDbReset = Effect.fn("legacy.db.reset")(function* (flags: Lega } // Re-confirm `linkedRefForCache` from the now-resolved `cfg.ref` for the native - // remote linked path below (a linked+experimental+versionless target already - // delegated and returned above, before `resolve()` was ever called — see the - // `connType === "linked"` block earlier in this function). A `connType === - // "db-url"` target leaves `linkedRefForCache` as whatever the pre-load block - // set (nothing, for `db-url`), since this assignment only fires when linked. + // remote path below. A `connType === "db-url"` target leaves `linkedRefForCache` + // as whatever the pre-load block set (nothing, for `db-url`), since this + // assignment only fires when linked. const linkedRef = Option.getOrUndefined(cfg.ref ?? Option.none()); if (connType === "linked" && linkedRef !== undefined) linkedRefForCache = linkedRef; - // Remaining remote target: a `--db-url` pointing at a non-local host (the - // `connType === "linked"` case already delegated above, before `resolve()`, - // without resolving a connection at all). - if (shouldDelegateExperimental) { - yield* delegateExperimentalReset(); - return; - } - // Single Go-parity config load (`flags.LoadConfig` → `config.Load` + `Validate`): // decodes the whole config with Go's env-expansion + `strconv.ParseBool` weak typing // (so `enabled = "env(SEED_ENABLED)"` etc. load like Go), applies `SUPABASE_*` @@ -437,7 +354,16 @@ export const legacyDbReset = Effect.fn("legacy.db.reset")(function* (flags: Lega yield* legacyDropUserSchemas(session, applyError); yield* legacyUpsertVaultSecrets(session, vaultSecrets); - if (toml.migrationsEnabled) { + // Go's three-conjunct EXPERIMENTAL gate (`apply.MigrateAndSeed`, `apply.go:19`): + // `--experimental`/`SUPABASE_EXPERIMENTAL` + no resolved version + pg-delta NOT + // enabled. A hard `if`/`else if` in Go (`apply.go:19-27`) — taking the + // schema-files branch means timestamped migrations never run at all, even when + // the glob matches nothing (Go's `schema_paths = []` default silently applies + // NOTHING rather than falling back to migrations — CLI-1958). + const useSchemaFiles = experimental && resolvedVersion === "" && !toml.pgDelta.enabled; + if (useSchemaFiles) { + yield* legacyApplySchemaFiles(session, fs, path, workdir, toml.schemaPaths, applyError); + } else if (toml.migrationsEnabled) { const locals = yield* legacyListLocalMigrations(fs, path, migrationsDir); // LoadPartialMigrations filter: version === "" || v <= version. const pending = locals.filter((p) => { diff --git a/apps/cli/src/legacy/commands/db/reset/reset.integration.test.ts b/apps/cli/src/legacy/commands/db/reset/reset.integration.test.ts index 3f710a511a..713928aa2e 100644 --- a/apps/cli/src/legacy/commands/db/reset/reset.integration.test.ts +++ b/apps/cli/src/legacy/commands/db/reset/reset.integration.test.ts @@ -30,7 +30,6 @@ import { LegacyExperimentalFlag, LegacyYesFlag, } from "../../../../shared/legacy/global-flags.ts"; -import { LegacyGoProxy } from "../../../../shared/legacy/go-proxy.service.ts"; import { LegacyGoChildExitError } from "../../../../shared/legacy/legacy-go-child-exit.error.ts"; import type { OutputFormat } from "../../../../shared/output/types.ts"; import { LegacyDbConfigResolver } from "../../../shared/legacy-db-config.service.ts"; @@ -72,10 +71,8 @@ const DEFAULT_FLAGS: LegacyDbResetFlags = { /** * Tracks every `resolve`/`resolvePoolerFallback` invocation so tests can prove a - * connection was (or, for the delegated-experimental path, was NOT) resolved — - * `resolve()` mints/verifies a temporary Postgres login role over the Management - * API, so calling it on a path that immediately discards the result is wasted - * (and duplicated) work (CLI-1879). + * connection was resolved exactly once per reset — `resolve()` mints/verifies a + * temporary Postgres login role over the Management API for a `--linked` target. */ function mockResolver(opts: { isLocal: boolean; @@ -116,7 +113,12 @@ function mockResolver(opts: { }; } -function mockConnection(opts: { remoteSeeds?: Readonly> }) { +function mockConnection(opts: { + remoteSeeds?: Readonly>; + /** When set, an `exec` whose SQL contains this substring fails instead of succeeding. */ + execFailsOn?: string; + execFailsMessage?: string; +}) { const execs: Array = []; const queries: Array<{ sql: string; params?: ReadonlyArray }> = []; const layer = Layer.succeed(LegacyDbConnection, { @@ -126,9 +128,13 @@ function mockConnection(opts: { remoteSeeds?: Readonly> } copyToCsv: () => Effect.succeed(new Uint8Array()), queryRaw: () => Effect.succeed({ fields: [], rows: [], commandTag: "" }), exec: (sql: string): Effect.Effect => - Effect.sync(() => { - execs.push(sql); - }), + opts.execFailsOn !== undefined && sql.includes(opts.execFailsOn) + ? Effect.fail( + new LegacyDbExecError({ message: opts.execFailsMessage ?? "syntax error" }), + ) + : Effect.sync(() => { + execs.push(sql); + }), query: ( sql: string, params?: ReadonlyArray, @@ -225,42 +231,6 @@ const mockStorageHttp = Layer.succeed( ), ); -/** - * `execCaptureExitCode`, when set, makes `execCapture` fail with a - * `LegacyGoChildExitError` carrying that code instead of succeeding — simulating - * a delegated Go child exiting non-zero under a machine-output mode (CLI-1879). - */ -function mockProxy(opts: { execCaptureExitCode?: number } = {}) { - const calls: Array<{ args: ReadonlyArray; env?: Record }> = []; - const layer = Layer.succeed(LegacyGoProxy, { - exec: (args, execOpts) => - Effect.sync(() => { - calls.push({ args, env: execOpts?.env }); - }), - execCapture: (args, execOpts) => - Effect.sync(() => { - calls.push({ args, env: execOpts?.env }); - }).pipe( - Effect.flatMap(() => - opts.execCaptureExitCode !== undefined - ? Effect.fail( - new LegacyGoChildExitError({ - exitCode: opts.execCaptureExitCode, - message: `supabase-go exited with code ${opts.execCaptureExitCode}`, - }), - ) - : Effect.succeed(""), - ), - ), - }); - return { - layer, - get calls() { - return calls; - }, - }; -} - function setup( workdir: string, opts: { @@ -273,13 +243,14 @@ function setup( ref?: string; experimental?: boolean; remoteSeeds?: Readonly>; + execFailsOn?: string; + execFailsMessage?: string; yes?: boolean; omitRef?: boolean; resolveFails?: boolean; running?: boolean; storageReady?: boolean; awaitStorageReadyExitCode?: number; - execCaptureExitCode?: number; }, ) { if (opts.toml !== undefined) { @@ -294,7 +265,6 @@ function setup( const out = mockOutput({ format: opts.format ?? "text", promptConfirmResponses: opts.confirm }); const conn = mockConnection(opts); - const proxy = mockProxy({ execCaptureExitCode: opts.execCaptureExitCode }); const seam = mockBootstrapSeam({ running: opts.running, storageReady: opts.storageReady, @@ -315,7 +285,6 @@ function setup( const layer = Layer.mergeAll( out.layer, conn.layer, - proxy.layer, seam.layer, resolver.layer, mockLegacyCliConfig({ workdir }), @@ -347,7 +316,7 @@ function setup( telemetry.layer, linkedCache.layer, ); - return { layer, out, conn, proxy, seam, telemetry, linkedCache, resolver }; + return { layer, out, conn, seam, telemetry, linkedCache, resolver }; } const migrationFile = (version: string, body = "create table t ();") => ({ @@ -358,7 +327,7 @@ describe("legacy db reset", () => { const tmp = useLegacyTempWorkdir("supabase-db-reset-"); it.live("resets the local database via the bootstrap seam", () => { - const { layer, out, seam, proxy } = setup(tmp.current, { + const { layer, out, seam } = setup(tmp.current, { toml: 'project_id = "test"\n', args: ["db", "reset"], isLocal: true, @@ -366,8 +335,6 @@ describe("legacy db reset", () => { }); return Effect.gen(function* () { yield* legacyDbReset(DEFAULT_FLAGS).pipe(Effect.provide(layer)); - // Native path — no Go delegation. - expect(proxy.calls).toHaveLength(0); expect(out.stderrText).toContain("Resetting local database..."); expect(seam.recreateCalls).toEqual([{ version: "", noSeed: false, sqlPaths: [] }]); // Storage gate checked; with no buckets configured nothing is seeded. @@ -961,75 +928,187 @@ describe("legacy db reset", () => { }); }); - it.live("delegates an experimental remote reset to the Go binary", () => { - const { layer, proxy } = setup(tmp.current, { - toml: 'project_id = "test"\n', - experimental: true, - }); - return Effect.gen(function* () { - yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe(Effect.provide(layer)); - expect(proxy.calls).toHaveLength(1); - expect(proxy.calls[0]!.args).toEqual(["db", "reset", "--linked", "--yes=false"]); - expect(proxy.calls[0]!.env).toEqual({ SUPABASE_TELEMETRY_DISABLED: "1" }); - }); - }); + it.live( + "applies configured schema files instead of replaying migrations on an experimental remote reset", + () => { + // `--linked=false` still selects the linked/remote target (Cobra `Changed` + // semantics) — exercised here alongside the schema-files branch itself. + const { layer, out, conn, resolver, linkedCache } = setup(tmp.current, { + toml: 'project_id = "test"\n\n[db.migrations]\nschema_paths = ["schemas/*.sql"]\n', + files: { + "supabase/schemas/01_users.sql": "create table schema_users ();", + ...migrationFile("20240101000000", "create table migrated_table ();"), + "supabase/seed.sql": "insert into t values (1);", + }, + experimental: true, + args: ["db", "reset", "--linked=false"], + confirm: [true], + ref: LEGACY_VALID_REF, + }); + return Effect.gen(function* () { + yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: false }).pipe(Effect.provide(layer)); + // The configured schema file ran... + expect(conn.execs.some((s) => s.includes("create table schema_users"))).toBe(true); + // ...but the timestamped migration did NOT — Go's `if`/`else if` is mutually + // exclusive (`apply.go:19-27`); taking the schema-files branch means migrations + // never run at all. + expect(conn.execs.some((s) => s.includes("create table migrated_table"))).toBe(false); + expect(out.stderrText).not.toContain("Applying migration"); + // Seeding still runs afterward — Go's `applySeedFiles` sits outside the + // if/else if (`apply.go:26`). + expect(out.stderrText).toContain("Seeding data from supabase/seed.sql..."); + // A real connection is resolved now — this is a fully native path, not a + // delegated one that discarded the resolve (CLI-1958 removed the delegate). + expect(resolver.calls).toBe(1); + expect(linkedCache.cached).toBe(true); + expect(linkedCache.cachedRef).toBe(LEGACY_VALID_REF); + }); + }, + ); - it.live("does not resolve a linked DB connection before delegating an experimental reset", () => { - const { layer, proxy, resolver } = setup(tmp.current, { - toml: 'project_id = "test"\n', - experimental: true, - }); - return Effect.gen(function* () { - yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe(Effect.provide(layer)); - expect(proxy.calls).toHaveLength(1); - // The delegated Go child re-runs its own connection resolution (including - // minting/verifying the temp login role) once it starts — the TS wrapper - // must not do that same Management-API work first only to discard it (CLI-1879). - expect(resolver.calls).toBe(0); - }); - }); + it.live( + "silently applies nothing when schema_paths is unset on an experimental remote reset (Go's undocumented default-config behavior)", + () => { + // Go's `schema_paths` default is `[]` (`pkg/config/templates/config.toml:64`). + // With no patterns to glob, `SQLFiles` returns a nil error, so `applySchemaFiles` + // is a silent no-op — Go does NOT fall back to replaying migrations (`apply.go: + // 19-27` is a hard `if`/`else if`). + const { layer, out, conn } = setup(tmp.current, { + toml: 'project_id = "test"\n', + files: migrationFile("20240101000000", "create table migrated_table ();"), + experimental: true, + confirm: [true], + }); + return Effect.gen(function* () { + yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe(Effect.provide(layer)); + // Schemas are still dropped (ResetAll drops before MigrateAndSeed)... + expect(conn.execs.some((s) => s.includes("drop schema if exists"))).toBe(true); + // ...but the local migration is silently skipped, not applied. + expect(conn.execs.some((s) => s.includes("create table migrated_table"))).toBe(false); + expect(out.stderrText).not.toContain("Applying migration"); + }); + }, + ); - it.live("still caches the linked ref when delegating an experimental reset", () => { - // `linkedRefForCache` is pre-loaded via `LegacyProjectRefResolver.loadProjectRef` - // separately from `resolver.resolve()`, specifically so the post-run - // linked-project-cache finalizer still fires on this path even though - // `resolve()` itself is skipped entirely (CLI-1879). - const { layer, linkedCache } = setup(tmp.current, { - toml: 'project_id = "test"\n', + it.live( + "replays migrations instead of schema files on an experimental remote reset when pg-delta is enabled", + () => { + const { layer, out, conn } = setup(tmp.current, { + toml: 'project_id = "test"\n\n[db.migrations]\nschema_paths = ["schemas/*.sql"]\n\n[experimental.pgdelta]\nenabled = true\n', + files: { + "supabase/schemas/01_users.sql": "create table schema_users ();", + ...migrationFile("20240101000000", "create table migrated_table ();"), + }, + experimental: true, + confirm: [true], + }); + return Effect.gen(function* () { + yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe(Effect.provide(layer)); + // `IsPgDeltaEnabled()` disables the schema-files branch (`apply.go:19`) even + // though `--experimental` and `schema_paths` are both set. + expect(conn.execs.some((s) => s.includes("create table migrated_table"))).toBe(true); + expect(conn.execs.some((s) => s.includes("create table schema_users"))).toBe(false); + expect(out.stderrText).toContain("Applying migration"); + }); + }, + ); + + it.live( + "replays migrations instead of schema files on an experimental remote reset with a resolved version", + () => { + const { layer, conn } = setup(tmp.current, { + toml: 'project_id = "test"\n\n[db.migrations]\nschema_paths = ["schemas/*.sql"]\n', + files: { + "supabase/schemas/01_users.sql": "create table schema_users ();", + ...migrationFile("20240101000000", "create table migrated_table ();"), + }, + experimental: true, + confirm: [true], + }); + return Effect.gen(function* () { + yield* legacyDbReset({ + ...DEFAULT_FLAGS, + linked: true, + version: Option.some("20240101000000"), + }).pipe(Effect.provide(layer)); + // A resolved --version disables the schema-files branch (`apply.go:19` requires + // `len(version) == 0`), even with `--experimental` set. + expect(conn.execs.some((s) => s.includes("create table migrated_table"))).toBe(true); + expect(conn.execs.some((s) => s.includes("create table schema_users"))).toBe(false); + }); + }, + ); + + it.live( + "fails an experimental remote reset when no schema_paths pattern matches anything", + () => { + const { layer, conn } = setup(tmp.current, { + toml: 'project_id = "test"\n\n[db.migrations]\nschema_paths = ["nomatch/*.sql"]\n', + experimental: true, + confirm: [true], + }); + return Effect.gen(function* () { + const exit = yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe( + Effect.provide(layer), + Effect.exit, + ); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const cause = JSON.stringify(exit.cause); + expect(cause).toContain("no files matched pattern: supabase/nomatch/*.sql"); + // No CmdSuggestion on this failure mode — only a per-file exec failure sets one. + expect(cause).not.toContain("See schema file"); + } + // Schemas were already dropped before the failed apply step (Go's ResetAll order). + expect(conn.execs.some((s) => s.includes("drop schema if exists"))).toBe(true); + }); + }, + ); + + it.live("ignores a partial schema_paths glob failure once at least one pattern matches", () => { + // Go's `applySchemaFiles` only surfaces the joined glob error when NO pattern + // matched anything at all (`apply.go:53-55`); a partial failure is silently dropped. + const { layer, out, conn } = setup(tmp.current, { + toml: 'project_id = "test"\n\n[db.migrations]\nschema_paths = ["schemas/*.sql", "typo/*.sql"]\n', + files: { + "supabase/schemas/01_users.sql": "create table schema_users ();", + // Present so the (unrelated) seed glob's own "no files matched" WARN line + // doesn't show up and get confused with the schema-files warning below. + "supabase/seed.sql": "insert into t values (1);", + }, experimental: true, - ref: LEGACY_VALID_REF, + confirm: [true], }); return Effect.gen(function* () { yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe(Effect.provide(layer)); - expect(linkedCache.cached).toBe(true); - expect(linkedCache.cachedRef).toBe(LEGACY_VALID_REF); + expect(conn.execs.some((s) => s.includes("create table schema_users"))).toBe(true); + expect(out.stderrText).not.toContain("no files matched pattern"); }); }); it.live( - "surfaces a delegated experimental-reset child failure as a LegacyGoChildExitError under json output", + "attaches Go's schema-file suggestion when a schema file fails to apply on an experimental remote reset", () => { const { layer } = setup(tmp.current, { - toml: 'project_id = "test"\n', + toml: 'project_id = "test"\n\n[db.migrations]\nschema_paths = ["schemas/*.sql"]\n', + files: { "supabase/schemas/01_users.sql": "not valid sql;" }, experimental: true, - format: "json", - execCaptureExitCode: 3, + confirm: [true], + execFailsOn: "not valid sql", + execFailsMessage: 'syntax error at or near "not"', }); return Effect.gen(function* () { - // Under json/stream-json, the delegated path uses `execCapture` (non-text - // branch of `delegateExperimentalReset`) — this must flow through the normal - // Effect failure channel (reachable by `withJsonErrorHandling` at the - // command-wiring layer) instead of an immediate `ProcessControl.exit()` that a - // handler-level test could never observe (CLI-1879). const exit = yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe( Effect.provide(layer), Effect.exit, ); expect(Exit.isFailure(exit)).toBe(true); if (Exit.isFailure(exit)) { - const error = Cause.squash(exit.cause); - expect(error).toBeInstanceOf(LegacyGoChildExitError); - expect((error as LegacyGoChildExitError).exitCode).toBe(3); + const cause = JSON.stringify(exit.cause); + expect(cause).toContain("syntax error at or near"); + // Go's `CmdSuggestion = "See schema file: "` (`apply.go:63`). + expect(cause).toContain("See schema file:"); + expect(cause).toContain("supabase/schemas/01_users.sql"); } }); }, @@ -1062,76 +1141,29 @@ describe("legacy db reset", () => { }, ); - it.live("forwards the linked selector to the delegate even for --linked=false", () => { - // Cobra `Changed` semantics: `--linked=false` still selects the linked/remote target in - // the parent, so the delegated argv must carry `--linked` — otherwise the Go child falls - // back to its local default and resets the wrong database. - const { layer, proxy } = setup(tmp.current, { - toml: 'project_id = "test"\n', - experimental: true, - args: ["db", "reset", "--linked=false"], - }); - return Effect.gen(function* () { - yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: false }).pipe(Effect.provide(layer)); - expect(proxy.calls).toHaveLength(1); - expect(proxy.calls[0]!.args).toEqual(["db", "reset", "--linked", "--yes=false"]); - }); - }); - - it.live("forwards --yes=false to the delegate even when SUPABASE_YES is set", () => { - // Explicit `--yes=false` beats `AutomaticEnv` in Go; the delegated child must receive the - // bound false flag so an inherited `SUPABASE_YES=true` doesn't auto-confirm the reset and - // drop the remote schemas the user tried to protect. - const previous = process.env["SUPABASE_YES"]; - process.env["SUPABASE_YES"] = "true"; - const { layer, proxy } = setup(tmp.current, { - toml: 'project_id = "test"\n', - experimental: true, - args: ["db", "reset", "--linked", "--yes=false"], - }); - return Effect.gen(function* () { - yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe(Effect.provide(layer)); - expect(proxy.calls[0]!.args).toContain("--yes=false"); - }).pipe( - Effect.ensuring( - Effect.sync(() => { - if (previous === undefined) delete process.env["SUPABASE_YES"]; - else process.env["SUPABASE_YES"] = previous; - }), - ), - ); - }); - - it.live("forwards --yes=true to the delegate when --yes is set", () => { - const { layer, proxy } = setup(tmp.current, { - toml: 'project_id = "test"\n', - experimental: true, - args: ["db", "reset", "--linked", "--yes"], - yes: true, - }); - return Effect.gen(function* () { - yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe(Effect.provide(layer)); - expect(proxy.calls[0]!.args).toContain("--yes=true"); - }); - }); - it.live( - "takes the experimental delegate path via SUPABASE_EXPERIMENTAL in the project .env", + "takes the native experimental schema-files path via SUPABASE_EXPERIMENTAL in the project .env", () => { - // Go loads nested env before reset.Run reads viper EXPERIMENTAL, so the versionless remote - // reset delegates to the Go binary rather than replaying migrations natively. + // Go loads nested env before `reset.Run` reads viper's EXPERIMENTAL, so a + // `SUPABASE_EXPERIMENTAL` set only in `supabase/.env` reaches the native + // three-conjunct gate the same way an explicit `--experimental` does. const previous = process.env["SUPABASE_EXPERIMENTAL"]; delete process.env["SUPABASE_EXPERIMENTAL"]; - const { layer, proxy, conn } = setup(tmp.current, { - toml: 'project_id = "test"\n', - files: { "supabase/.env": "SUPABASE_EXPERIMENTAL=true\n" }, + const { layer, out, conn } = setup(tmp.current, { + toml: 'project_id = "test"\n\n[db.migrations]\nschema_paths = ["schemas/*.sql"]\n', + files: { + "supabase/.env": "SUPABASE_EXPERIMENTAL=true\n", + "supabase/schemas/01_users.sql": "create table schema_users ();", + ...migrationFile("20240101000000", "create table migrated_table ();"), + }, + confirm: [true], // No experimental flag / shell env — only the project .env sets it. }); return Effect.gen(function* () { yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe(Effect.provide(layer)); - expect(proxy.calls).toHaveLength(1); - // Delegated, so the native remote path never dropped schemas. - expect(conn.execs.some((s) => s.includes("drop schema if exists"))).toBe(false); + expect(conn.execs.some((s) => s.includes("create table schema_users"))).toBe(true); + expect(conn.execs.some((s) => s.includes("create table migrated_table"))).toBe(false); + expect(out.stderrText).not.toContain("Applying migration"); }).pipe( Effect.ensuring( Effect.sync(() => { @@ -1160,32 +1192,30 @@ describe("legacy db reset", () => { }); }); - it.live("forwards --db-url and --no-seed on an experimental remote db-url reset", () => { - const { layer, proxy, resolver } = setup(tmp.current, { - toml: 'project_id = "test"\n', - experimental: true, - args: ["db", "reset", "--db-url", "postgresql://db.example.com:5432/postgres"], - }); - return Effect.gen(function* () { - yield* legacyDbReset({ - ...DEFAULT_FLAGS, - dbUrl: Option.some("postgresql://db.example.com:5432/postgres"), - noSeed: true, - }).pipe(Effect.provide(layer)); - expect(proxy.calls[0]!.args).toEqual([ - "db", - "reset", - "--db-url", - "postgresql://db.example.com:5432/postgres", - "--no-seed", - "--yes=false", - ]); - // Unlike the `connType === "linked"` branch above, a `--db-url` target still - // resolves a connection before delegating — the pre-delegation skip (CLI-1879) - // is scoped to the linked branch only, not "never call resolve when delegating". - expect(resolver.calls).toBe(1); - }); - }); + it.live( + "applies configured schema files and skips seeding on an experimental remote --db-url reset", + () => { + const { layer, conn, resolver } = setup(tmp.current, { + toml: 'project_id = "test"\n\n[db.migrations]\nschema_paths = ["schemas/*.sql"]\n', + files: { "supabase/schemas/01_users.sql": "create table schema_users ();" }, + experimental: true, + args: ["db", "reset", "--db-url", "postgresql://db.example.com:5432/postgres"], + confirm: [true], + }); + return Effect.gen(function* () { + yield* legacyDbReset({ + ...DEFAULT_FLAGS, + dbUrl: Option.some("postgresql://db.example.com:5432/postgres"), + noSeed: true, + }).pipe(Effect.provide(layer)); + expect(conn.execs.some((s) => s.includes("create table schema_users"))).toBe(true); + expect(conn.execs.some((s) => s.includes("insert into"))).toBe(false); + // A `--db-url` target always resolved a real connection, same as before — + // this is no longer delegated at all (CLI-1958). + expect(resolver.calls).toBe(1); + }); + }, + ); it.live("passes --no-seed and the resolved --last version to the recreate seam", () => { const { layer, seam } = setup(tmp.current, { @@ -1420,25 +1450,30 @@ describe("legacy db reset", () => { }); }); - it.live("forwards --sql-paths to the Go binary on an experimental remote reset", () => { - const { layer, proxy } = setup(tmp.current, { - toml: 'project_id = "test"\n', - experimental: true, - }); - return Effect.gen(function* () { - yield* legacyDbReset({ - ...DEFAULT_FLAGS, - linked: true, - sqlPaths: ["custom-seed.sql"], - }).pipe(Effect.provide(layer)); - expect(proxy.calls[0]!.args).toEqual([ - "db", - "reset", - "--linked", - "--sql-paths", - "custom-seed.sql", - "--yes=false", - ]); - }); - }); + it.live( + "seeds from --sql-paths on an experimental remote reset, independently of the schema-files apply", + () => { + // `--sql-paths` overrides `[db.seed].sql_paths` regardless of which branch of + // `apply.MigrateAndSeed` ran — Go's `applySeedFiles` sits outside the if/else if + // (`apply.go:26`), and the seed override is resolved entirely upstream of it. + const { layer, out, conn } = setup(tmp.current, { + toml: 'project_id = "test"\n\n[db.migrations]\nschema_paths = ["schemas/*.sql"]\n', + files: { + "supabase/schemas/01_users.sql": "create table schema_users ();", + "supabase/custom-seed.sql": "insert into t values (2);", + }, + experimental: true, + confirm: [true], + }); + return Effect.gen(function* () { + yield* legacyDbReset({ + ...DEFAULT_FLAGS, + linked: true, + sqlPaths: ["custom-seed.sql"], + }).pipe(Effect.provide(layer)); + expect(conn.execs.some((s) => s.includes("create table schema_users"))).toBe(true); + expect(out.stderrText).toContain("Seeding data from supabase/custom-seed.sql..."); + }); + }, + ); }); diff --git a/apps/cli/src/legacy/commands/db/reset/reset.layers.ts b/apps/cli/src/legacy/commands/db/reset/reset.layers.ts index 7cf648f3fe..2820a6b69c 100644 --- a/apps/cli/src/legacy/commands/db/reset/reset.layers.ts +++ b/apps/cli/src/legacy/commands/db/reset/reset.layers.ts @@ -19,8 +19,9 @@ import { legacyDbBootstrapSeamLayer } from "../shared/legacy-db-bootstrap.seam.l * Runtime layer for `supabase db reset`. Same composition as `db push` / `db lint`: * the Postgres connection, the db-config resolver, project-ref resolution, and the * linked-project cache, all over the lazy management-API factory so the local / - * `--db-url` paths never resolve an access token at layer-build time. `LegacyGoProxy` - * (used to delegate the local / experimental reset paths) is ambient from the root. + * `--db-url` paths never resolve an access token at layer-build time. Both the local + * and remote reset paths (including the `--experimental` remote schema-files apply, + * CLI-1958) are fully native — no `LegacyGoProxy` dependency remains. */ const cliConfig = legacyCliConfigLayer.pipe(Layer.provide(legacyDebugLoggerLayer)); const httpClient = legacyHttpClientLayer.pipe(Layer.provide(legacyDebugLoggerLayer)); diff --git a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts index 74582272eb..0693f6f22a 100644 --- a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts +++ b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts @@ -98,6 +98,14 @@ export interface LegacyDbTomlValues { readonly baseline: LegacyBaselineTomlConfig; /** `[db.migrations] enabled` (default true) — gates `up`/`down` migration apply. */ readonly migrationsEnabled: boolean; + /** + * `[db.migrations] schema_paths` glob patterns, default `[]`, each supabase-prefixed + * when relative (Go's `config.resolve`, `config.go:976-980`) — the same resolution + * `[db.seed].sql_paths` gets, but resolved unconditionally (not gated on + * `db.migrations.enabled`). Feeds `apply.MigrateAndSeed`'s EXPERIMENTAL declarative + * branch (`legacyApplySchemaFiles`) — see `db reset`'s `--experimental` remote path. + */ + readonly schemaPaths: ReadonlyArray; /** `[db.seed]` enabled + supabase-prefixed `sql_paths` globs — used by `down`. */ readonly seed: LegacyDbSeedTomlConfig; /** `[db.vault]` secrets (name → resolved value) — upserted by `up`/`down`. */ @@ -283,6 +291,7 @@ const LEGACY_ENV_OVERRIDABLE_KEYS: ReadonlyArray = [ "db.shadow_port", "db.major_version", "db.migrations.enabled", + "db.migrations.schema_paths", "db.seed.enabled", "db.seed.sql_paths", "auth.enabled", @@ -1860,6 +1869,27 @@ const readDbTomlCore = Effect.fnUntraced(function* ( // resolve each to Go's config-load form (absolute verbatim, relative supabase-joined). const seedSqlPaths = sqlPathPatterns.map((pattern) => legacyResolveSeedSqlPath(path, pattern)); + // `[db.migrations] schema_paths` — Go default `[]` (`pkg/config/templates/config.toml:64`), + // resolved through the exact same decode + env-expand + supabase-join pipeline as + // `[db.seed].sql_paths` above, but UNCONDITIONALLY (Go's resolve loop for schema_paths, + // `config.go:976-980`, is not gated on `db.migrations.enabled` the way the seed loop is + // gated on `db.seed.enabled`, `config.go:968-975`). + const rawSchemaPaths = migrationsRaw?.["schema_paths"]; + const schemaPathsOverride = remoteOverrideKeys.has("db.migrations.schema_paths") + ? undefined + : envOverride("SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS"); + const schemaPathPatterns = + schemaPathsOverride !== undefined + ? splitGoSeedPaths(schemaPathsOverride) + : Array.isArray(rawSchemaPaths) + ? rawSchemaPaths + .filter((pattern): pattern is string => typeof pattern === "string") + .map((pattern) => legacyExpandEnv(pattern, lookup)) + : typeof rawSchemaPaths === "string" + ? splitGoSeedPaths(rawSchemaPaths) + : []; + const schemaPaths = schemaPathPatterns.map((pattern) => legacyResolveSeedSqlPath(path, pattern)); + // `[db.vault]` secrets: env-expand each value, then decrypt dotenvx `encrypted:` // ciphertext. `resolved` mirrors Go's `len(SHA256) > 0` gate (Go sets SHA256 only // after a successful decrypt-or-passthrough; `UpsertVaultSecrets` upserts only @@ -1953,6 +1983,7 @@ const readDbTomlCore = Effect.fnUntraced(function* ( vaultNames, }, migrationsEnabled, + schemaPaths, seed: { enabled: seedEnabled, sqlPaths: seedSqlPaths }, vault, appliedRemote, diff --git a/apps/cli/src/legacy/shared/legacy-migration-apply.ts b/apps/cli/src/legacy/shared/legacy-migration-apply.ts index a6b3bef1e9..0975a33d1c 100644 --- a/apps/cli/src/legacy/shared/legacy-migration-apply.ts +++ b/apps/cli/src/legacy/shared/legacy-migration-apply.ts @@ -1,6 +1,7 @@ import { Data, Effect, type FileSystem, type Path } from "effect"; import { Output } from "../../shared/output/output.service.ts"; +import { legacyBold } from "./legacy-colors.ts"; import type { LegacyDbExecError } from "./legacy-db-connection.errors.ts"; import type { LegacyDbSession } from "./legacy-db-connection.service.ts"; import { @@ -8,6 +9,7 @@ import { MIGRATE_FILE_PATTERN, legacyCreateMigrationTable, } from "./legacy-migration-history.ts"; +import { legacySqlFilesGlob } from "./legacy-sql-files-glob.ts"; import { legacySplitAndTrim } from "./legacy-sql-split.ts"; /** @@ -355,3 +357,59 @@ export const legacyExecSqlFile = ( filePath: string, mapError: (message: string) => E, ): Effect.Effect => execMigrationBatch(session, fs, path, filePath, mapError, true); + +/** + * Applies Go's EXPERIMENTAL declarative schema-files branch of `apply.MigrateAndSeed` + * (`apps/cli-go/internal/migration/apply/apply.go:19,51-68`). Reads `[db.migrations] + * schema_paths` (already resolved to Go's config-load form — supabase-joined when + * relative, verbatim when absolute) via the shared `Glob.SQLFiles` port + * ({@link legacySqlFilesGlob}), then runs each matched file's statements with + * {@link legacyExecSqlFile} in glob order — no history table, no history row, and no + * `RESET ALL` between files, matching Go's `schema.Version = ""` discard (`apply.go:61`) + * and the fact that `ExecBatch` (unlike `ApplyMigrations`) never resets connection state. + * + * Callers gate the call on Go's three-conjunct condition (`--experimental` + no resolved + * version + pg-delta NOT enabled, `apply.go:19`) themselves — this function only performs + * the branch's body, mirroring `applySchemaFiles`'s own signature (it never re-checks the + * gate). It is the caller's responsibility to skip `legacyApplyMigrations` entirely when + * this is called (Go's `if`/`else if` is mutually exclusive, `apply.go:19-27`). + * + * Faithfully reproduces two undocumented, unfixed-upstream Go quirks that are load-bearing + * for the strict 1:1 contract (CLI-1958): + * - **Empty `schema_paths` (the `supabase init` default) silently applies nothing** and + * returns success — `Config.Db.Migrations.SchemaPaths.SQLFiles` returns a `nil` error + * when there are zero patterns to glob (`errors.Join()` with no arguments is `nil`), so + * `applySchemaFiles` returns `nil` too (`apply.go:53-54`). + * - **A PARTIAL glob failure is silently dropped**: per-pattern warnings are only + * surfaced (as the returned failure) when NO pattern matched anything at all + * (`declared` empty, `apply.go:53-55`); once at least one file is found, every other + * pattern's warning is discarded — unlike the seed path's `WARN:` line. + * + * On a per-file exec failure, attaches Go's `CmdSuggestion = "See schema file: "` + * (`apply.go:63`) via the optional second argument of `mapError`. + */ +export const legacyApplySchemaFiles = ( + session: LegacyDbSession, + fs: FileSystem.FileSystem, + path: Path.Path, + workdir: string, + schemaPaths: ReadonlyArray, + mapError: (message: string, suggestion?: string) => E, +): Effect.Effect => + Effect.gen(function* () { + const { files, warnings } = yield* legacySqlFilesGlob(fs, path, schemaPaths, workdir); + if (files.length === 0) { + // Go: `if len(declared) == 0 { return err }` — `err` is `nil` when there were no + // patterns to glob at all, and the joined per-pattern warnings otherwise. + if (warnings.length > 0) { + return yield* Effect.fail(mapError(warnings.join("\n"))); + } + return; + } + for (const file of files) { + const absolutePath = path.isAbsolute(file) ? file : path.join(workdir, file); + yield* legacyExecSqlFile(session, fs, path, absolutePath, (message) => + mapError(message, `See schema file: ${legacyBold(file)}`), + ); + } + }); diff --git a/apps/cli/src/legacy/shared/legacy-seed-ops.ts b/apps/cli/src/legacy/shared/legacy-seed-ops.ts index bbea4d4fcc..d430766cbe 100644 --- a/apps/cli/src/legacy/shared/legacy-seed-ops.ts +++ b/apps/cli/src/legacy/shared/legacy-seed-ops.ts @@ -5,7 +5,7 @@ import { Output } from "../../shared/output/output.service.ts"; import type { LegacyDbExecError } from "./legacy-db-connection.errors.ts"; import type { LegacyDbSession } from "./legacy-db-connection.service.ts"; import { legacyCreateSeedTable } from "./legacy-migration-history.ts"; -import { LEGACY_BAD_PATTERN_MESSAGE, legacyPathMatch } from "./legacy-path-match.ts"; +import { legacySqlFilesGlob } from "./legacy-sql-files-glob.ts"; import { legacySplitAndTrim } from "./legacy-sql-split.ts"; /** @@ -26,8 +26,6 @@ export interface LegacySeedFile { readonly dirty: boolean; } -const META_CHARS = /[*?[\\]/u; - /** Result of resolving `[db.seed].sql_paths` against the workspace. */ interface LegacyGlobResult { /** Workdir-relative, forward-slashed matches, deduplicated in pattern order. */ @@ -38,13 +36,15 @@ interface LegacyGlobResult { /** * Resolves seed glob patterns to existing files, porting Go's `config.Glob.Files` - * over `fs.Glob` (`pkg/config/config.go:102-124`). Each pattern is first joined - * under the `supabase/` directory (Go resolves `sql_paths` at config load, - * `config.go:884`). Matches per pattern are sorted; the overall result preserves - * first-seen order across patterns. A pattern that matches nothing, or is malformed - * (Go's `path.ErrBadPattern`, e.g. an unterminated `[` class), contributes a warning - * but is not fatal — mirroring `fs.Glob`'s up-front `Match(pattern, "")` validation - * (`io/fs/glob.go`) and the sibling seed pipeline's `legacy-seed.ts:resolveSeedFiles`. + * over `fs.Glob` (`pkg/config/config.go:102-124`) via the shared {@link legacySqlFilesGlob} + * traversal (also used by `[db.migrations].schema_paths`, `legacy-migration-apply.ts`). + * Each pattern is first joined under the `supabase/` directory (Go resolves `sql_paths` + * at config load, `config.go:884`) — that resolution happens once, upstream, via + * `legacyResolveSeedSqlPath`; this function globs the already-resolved patterns + * verbatim. Per-pattern warnings (`no files matched pattern: …` / malformed glob) are + * joined with Go's `errors.Join` newline semantics and surfaced unconditionally — the + * seed path always warns, unlike the schema-files apply path (see + * `legacyApplySchemaFiles`), which only surfaces a warning when it is the ONLY outcome. */ const legacyGlobSeedFiles = Effect.fnUntraced(function* ( fs: FileSystem.FileSystem, @@ -52,145 +52,13 @@ const legacyGlobSeedFiles = Effect.fnUntraced(function* ( patterns: ReadonlyArray, workdir: string, ) { - const seen = new Set(); - const files: Array = []; - const errors: Array = []; - - for (const rawPattern of patterns) { - // Patterns arrive already resolved to Go's config-load form (relative entries - // supabase/-joined, absolute preserved) via `legacyResolveSeedSqlPath` — the reader - // for `[db.seed].sql_paths`, the caller for `--sql-paths`. Go's `config.Glob.Files` - // globs those resolved paths without re-prefixing (`config.go:102-124`), so only - // normalize separators here; re-joining `supabase/` would double-prefix. - const pattern = toSlash(rawPattern); - // Go's `fs.Glob` validates the whole pattern up front (`Match(pattern, "")`); a - // malformed glob is reported as `failed to glob files: ` and - // contributes no matches, rather than the misleading "no files matched" below. - if (legacyPathMatch(pattern, "").badPattern) { - errors.push(`failed to glob files: ${LEGACY_BAD_PATTERN_MESSAGE}`); - continue; - } - const matches = yield* globOne(fs, path, workdir, pattern); - if (matches.length === 0) { - errors.push(`no files matched pattern: ${pattern}`); - continue; - } - for (const match of [...matches].sort()) { - const fp = toSlash(match); - // Go's `GetPendingSeeds` globs via `Glob.SQLFiles`, which `Stat`s each match: a - // directory is expanded to its regular `.sql` files recursively (`walkMatchedDir`, - // sorted) while a file match is kept verbatim (`config.go:157-183`). Without this a - // directory `sql_paths` entry (e.g. `["seeds"]`) would flow into - // `readFileString()` and fail — Go's `db push --include-seed` / remote reset - // seed the directory's SQL children instead. - const matchType = yield* fs.stat(path.isAbsolute(fp) ? fp : path.join(workdir, fp)).pipe( - Effect.map((info) => info.type), - Effect.orElseSucceed(() => "File" as const), - ); - if (matchType === "Directory") { - for (const file of yield* legacyWalkSeedSqlFiles(fs, path, workdir, fp)) { - if (!seen.has(file)) { - seen.add(file); - files.push(file); - } - } - continue; - } - if (!seen.has(fp)) { - seen.add(fp); - files.push(fp); - } - } - } - + const { files, warnings } = yield* legacySqlFilesGlob(fs, path, patterns, workdir); return { files, - warning: errors.length > 0 ? Option.some(errors.join("\n")) : Option.none(), + warning: warnings.length > 0 ? Option.some(warnings.join("\n")) : Option.none(), } satisfies LegacyGlobResult; }); -const toSlash = (p: string): string => p.replaceAll("\\", "/"); - -/** Splits a forward-slashed path into its directory prefix and final element. */ -const splitPath = (p: string): { readonly dir: string; readonly file: string } => { - const slash = p.lastIndexOf("/"); - return slash === -1 ? { dir: "", file: p } : { dir: p.slice(0, slash), file: p.slice(slash + 1) }; -}; - -/** Faithful port of Go's `fs.Glob` for one pattern, rooted at `workdir`. */ -const globOne = ( - fs: FileSystem.FileSystem, - path: Path.Path, - workdir: string, - pattern: string, -): Effect.Effect, never> => - Effect.gen(function* () { - // Absolute patterns resolve against the filesystem root (Go preserves absolute - // seed paths); relative ones are rooted at the workdir. - const resolve = (p: string): string => (path.isAbsolute(p) ? p : path.join(workdir, p)); - // No metacharacters: a direct existence check (Go's `fs.Glob` fast path). - if (!META_CHARS.test(pattern)) { - const exists = yield* fs.exists(resolve(pattern)).pipe(Effect.orElseSucceed(() => false)); - return exists ? [pattern] : []; - } - const { dir, file } = splitPath(pattern); - // Resolve the directory level first (recursively if it, too, is a glob). - const dirs = - dir === "" || !META_CHARS.test(dir) ? [dir] : yield* globOne(fs, path, workdir, dir); - const result: Array = []; - for (const d of dirs) { - const absDir = d === "" ? workdir : resolve(d); - const names = yield* fs - .readDirectory(absDir) - .pipe(Effect.orElseSucceed((): ReadonlyArray => [])); - for (const name of names) { - if (legacyPathMatch(file, name).matched) { - result.push(d === "" ? name : `${d}/${name}`); - } - } - } - return result; - }); - -/** - * Recursively collects the regular `.sql` files under a matched seed directory, porting - * Go's `walkMatchedDir` with the `SQLFiles` include filter (`entry.Type().IsRegular() && - * filepath.Ext(path) == ".sql"`, `config.go:126-131,194-211`). Paths are workdir-relative - * (matching the glob output), forward-slashed, and sorted for deterministic application. - */ -const legacyWalkSeedSqlFiles = ( - fs: FileSystem.FileSystem, - path: Path.Path, - workdir: string, - dir: string, -): Effect.Effect, never> => - Effect.gen(function* () { - const collected: Array = []; - const walk = (rel: string): Effect.Effect => - Effect.gen(function* () { - const absDir = path.isAbsolute(rel) ? rel : path.join(workdir, rel); - const names = yield* fs - .readDirectory(absDir) - .pipe(Effect.orElseSucceed((): ReadonlyArray => [])); - for (const name of names) { - const childRel = `${rel}/${name}`; - const childType = yield* fs - .stat(path.isAbsolute(childRel) ? childRel : path.join(workdir, childRel)) - .pipe( - Effect.map((info) => info.type), - Effect.orElseSucceed(() => "Unknown" as const), - ); - if (childType === "Directory") { - yield* walk(childRel); - } else if (childType === "File" && childRel.endsWith(".sql")) { - collected.push(toSlash(childRel)); - } - } - }); - yield* walk(dir); - return collected.sort(); - }); - /** `SELECT path, hash FROM supabase_migrations.seed_files`, `42P01` → empty map. */ const readRemoteSeeds = (session: LegacyDbSession) => session.query(SELECT_SEED_TABLE).pipe( diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts new file mode 100644 index 0000000000..d6a2c3cd15 --- /dev/null +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts @@ -0,0 +1,173 @@ +import { Effect, type FileSystem, type Path } from "effect"; + +import { LEGACY_BAD_PATTERN_MESSAGE, legacyPathMatch } from "./legacy-path-match.ts"; + +const META_CHARS = /[*?[\\]/u; + +const toSlash = (p: string): string => p.replaceAll("\\", "/"); + +/** Splits a forward-slashed path into its directory prefix and final element. */ +const splitPath = (p: string): { readonly dir: string; readonly file: string } => { + const slash = p.lastIndexOf("/"); + return slash === -1 ? { dir: "", file: p } : { dir: p.slice(0, slash), file: p.slice(slash + 1) }; +}; + +/** Faithful port of Go's `fs.Glob` for one pattern, rooted at `workdir`. */ +const globOne = ( + fs: FileSystem.FileSystem, + path: Path.Path, + workdir: string, + pattern: string, +): Effect.Effect, never> => + Effect.gen(function* () { + // Absolute patterns resolve against the filesystem root; relative ones are + // rooted at the workdir. + const resolve = (p: string): string => (path.isAbsolute(p) ? p : path.join(workdir, p)); + // No metacharacters: a direct existence check (Go's `fs.Glob` fast path). + if (!META_CHARS.test(pattern)) { + const exists = yield* fs.exists(resolve(pattern)).pipe(Effect.orElseSucceed(() => false)); + return exists ? [pattern] : []; + } + const { dir, file } = splitPath(pattern); + // Resolve the directory level first (recursively if it, too, is a glob). + const dirs = + dir === "" || !META_CHARS.test(dir) ? [dir] : yield* globOne(fs, path, workdir, dir); + const result: Array = []; + for (const d of dirs) { + const absDir = d === "" ? workdir : resolve(d); + const names = yield* fs + .readDirectory(absDir) + .pipe(Effect.orElseSucceed((): ReadonlyArray => [])); + for (const name of names) { + if (legacyPathMatch(file, name).matched) { + result.push(d === "" ? name : `${d}/${name}`); + } + } + } + return result; + }); + +/** + * Recursively collects the regular `.sql` files under a matched directory, porting + * Go's `walkMatchedDir` with the `SQLFiles` include filter (`entry.Type().IsRegular() && + * filepath.Ext(path) == ".sql"`, `apps/cli-go/pkg/config/config.go:126-131,194-211`). + * Paths are workdir-relative (matching the glob output), forward-slashed, and sorted + * for deterministic application. + */ +const legacyWalkSqlFiles = ( + fs: FileSystem.FileSystem, + path: Path.Path, + workdir: string, + dir: string, +): Effect.Effect, never> => + Effect.gen(function* () { + const collected: Array = []; + const walk = (rel: string): Effect.Effect => + Effect.gen(function* () { + const absDir = path.isAbsolute(rel) ? rel : path.join(workdir, rel); + const names = yield* fs + .readDirectory(absDir) + .pipe(Effect.orElseSucceed((): ReadonlyArray => [])); + for (const name of names) { + const childRel = `${rel}/${name}`; + const childType = yield* fs + .stat(path.isAbsolute(childRel) ? childRel : path.join(workdir, childRel)) + .pipe( + Effect.map((info) => info.type), + Effect.orElseSucceed(() => "Unknown" as const), + ); + if (childType === "Directory") { + yield* walk(childRel); + } else if (childType === "File" && childRel.endsWith(".sql")) { + collected.push(toSlash(childRel)); + } + } + }); + yield* walk(dir); + return collected.sort(); + }); + +/** Result of resolving SQL-file glob patterns against the workspace. */ +interface LegacySqlFilesGlobResult { + /** Workdir-relative, forward-slashed matches, deduplicated in first-seen order across patterns. */ + readonly files: ReadonlyArray; + /** + * Per-pattern problems (`no files matched pattern: …` / `failed to glob files: …`), + * in pattern order. Never fatal by itself — callers decide when a warning matters + * (e.g. the seed path always surfaces it; the schema-files apply path only surfaces + * it when NO pattern matched anything at all, mirroring Go's `apply.go:53-55`). + */ + readonly warnings: ReadonlyArray; +} + +/** + * Resolves SQL-file glob patterns to existing files, porting Go's `config.Glob.SQLFiles` + * over `fs.Glob` (`apps/cli-go/pkg/config/config.go:123-211`). Shared by `[db.seed].sql_paths` + * (via `legacyGlobSeedFiles`, `legacy-seed-ops.ts`) and `[db.migrations].schema_paths` (via + * `legacyApplySchemaFiles`, `legacy-migration-apply.ts`) — both Go fields resolve through the + * exact same `Glob` type and `SQLFiles` method, so the traversal logic lives here once. + * + * Each pattern is matched independently: matches are sorted per-pattern (Go's + * `sort.Strings`, `config.go:155`), but the overall result preserves cross-pattern + * DECLARATION order (no global re-sort), with first-seen dedup. A directory match is + * expanded to its regular `.sql` files, recursively, sorted by full path + * (`walkMatchedDir`); a non-directory match is kept verbatim regardless of extension. A + * pattern that matches nothing, or is malformed (Go's `path.ErrBadPattern`, e.g. an + * unterminated `[` class), contributes a warning but does not stop the loop — mirroring + * `fs.Glob`'s up-front `Match(pattern, "")` validation (`io/fs/glob.go`). + * + * Patterns are assumed already resolved to Go's config-load form (a relative entry + * `supabase/`-joined, an absolute entry verbatim) — callers resolve that once at + * config-read time (`legacyResolveSeedSqlPath`), matching Go's `config.resolve` step, + * which runs once at config load, before any glob. + */ +export const legacySqlFilesGlob = Effect.fnUntraced(function* ( + fs: FileSystem.FileSystem, + path: Path.Path, + patterns: ReadonlyArray, + workdir: string, +) { + const seen = new Set(); + const files: Array = []; + const warnings: Array = []; + + for (const rawPattern of patterns) { + const pattern = toSlash(rawPattern); + // Go's `fs.Glob` validates the whole pattern up front (`Match(pattern, "")`); a + // malformed glob is reported as `failed to glob files: ` and + // contributes no matches, rather than the misleading "no files matched" below. + if (legacyPathMatch(pattern, "").badPattern) { + warnings.push(`failed to glob files: ${LEGACY_BAD_PATTERN_MESSAGE}`); + continue; + } + const matches = yield* globOne(fs, path, workdir, pattern); + if (matches.length === 0) { + warnings.push(`no files matched pattern: ${pattern}`); + continue; + } + for (const match of [...matches].sort()) { + const fp = toSlash(match); + // A directory match is expanded to its regular `.sql` files recursively + // (`walkMatchedDir`, sorted); a file match is kept verbatim (`config.go:157-183`). + const matchType = yield* fs.stat(path.isAbsolute(fp) ? fp : path.join(workdir, fp)).pipe( + Effect.map((info) => info.type), + Effect.orElseSucceed(() => "File" as const), + ); + if (matchType === "Directory") { + for (const file of yield* legacyWalkSqlFiles(fs, path, workdir, fp)) { + if (!seen.has(file)) { + seen.add(file); + files.push(file); + } + } + continue; + } + if (!seen.has(fp)) { + seen.add(fp); + files.push(fp); + } + } + } + + return { files, warnings } satisfies LegacySqlFilesGlobResult; +}); From 7d3327859ec79b13689eb143b9557ffa28f6a6c8 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Tue, 4 Aug 2026 15:32:26 +0100 Subject: [PATCH 02/47] docs(cli): reformat go-cli-porting-status.md table (oxfmt column widths) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Whitespace-only fix following the db reset (CLI-1958) note update — oxfmt recomputes column widths across the whole markdown table. --- apps/cli/docs/go-cli-porting-status.md | 88 +++++++++++++------------- 1 file changed, 44 insertions(+), 44 deletions(-) diff --git a/apps/cli/docs/go-cli-porting-status.md b/apps/cli/docs/go-cli-porting-status.md index 66d54f2a52..d08ca4089b 100644 --- a/apps/cli/docs/go-cli-porting-status.md +++ b/apps/cli/docs/go-cli-porting-status.md @@ -80,51 +80,51 @@ These commands exist in the TS CLI today but have no direct top-level equivalent ## Database -| Old command | TS status | TS command path or `missing` | Missing flags/params | Extra TS flags/params | Notes | -| --------------------------------- | --------- | -------------------------------------------------- | -------------------- | --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `db diff` | `ported` | `legacy/commands/db/diff/` | `n/a` | `n/a` | Native TS port. Native pg-delta / migra diff via edge-runtime against a Go-seam-provisioned live shadow (`db __shadow`); `--use-pgadmin` / `--use-pg-schema` delegate to the Go binary. | -| `db dump` | `ported` | `legacy/commands/db/dump/` | `n/a` | `n/a` | Native TS port. Streams `pg_dump`/`pg_dumpall` via a Docker container (`LegacyDockerRun`); schema/data/role modes, `--dry-run` script print, IPv4 transaction-pooler fallback. | -| `db lint` | `ported` | `legacy/commands/db/lint/` | `n/a` | `n/a` | Native TS port. Runs `plpgsql_check` in a rolled-back transaction via LegacyDbConnection; emits Go-parity pretty JSON. | -| `db pull` | `ported` | `legacy/commands/db/pull/` | `n/a` | `n/a` | Native TS port. Native pg-delta / migra migration + `--declarative` pg-delta export; reconciles `schema_migrations`. The initial-migra pull dumps the remote schema natively (`pg_dump`) then appends the migra diff. Only `--experimental` (structured dump) still delegates to Go, pending a TS PostgreSQL DDL parser for `format.WriteStructuredSchemas`. | -| `db push` | `ported` | `legacy/commands/db/push/` | `n/a` | `n/a` | Native TS port. Connects local/linked/`--db-url`; pushes pending migrations, `--include-seed` seeds (`seed_files` hash tracking), `--include-roles`, `[db.vault]` secrets including decrypted `encrypted:` values; `--dry-run`; best-effort pg-delta migrations-catalog cache (warning-only on failure). Pipeline-incompatible statements (`CREATE INDEX CONCURRENTLY`, `VACUUM`, …) run standalone outside the batch transaction — from the closed Go PR supabase/cli#5156, also ported into `apps/cli-go` (CLI-1989 ruling). | +| Old command | TS status | TS command path or `missing` | Missing flags/params | Extra TS flags/params | Notes | +| --------------------------------- | --------- | -------------------------------------------------- | -------------------- | --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `db diff` | `ported` | `legacy/commands/db/diff/` | `n/a` | `n/a` | Native TS port. Native pg-delta / migra diff via edge-runtime against a Go-seam-provisioned live shadow (`db __shadow`); `--use-pgadmin` / `--use-pg-schema` delegate to the Go binary. | +| `db dump` | `ported` | `legacy/commands/db/dump/` | `n/a` | `n/a` | Native TS port. Streams `pg_dump`/`pg_dumpall` via a Docker container (`LegacyDockerRun`); schema/data/role modes, `--dry-run` script print, IPv4 transaction-pooler fallback. | +| `db lint` | `ported` | `legacy/commands/db/lint/` | `n/a` | `n/a` | Native TS port. Runs `plpgsql_check` in a rolled-back transaction via LegacyDbConnection; emits Go-parity pretty JSON. | +| `db pull` | `ported` | `legacy/commands/db/pull/` | `n/a` | `n/a` | Native TS port. Native pg-delta / migra migration + `--declarative` pg-delta export; reconciles `schema_migrations`. The initial-migra pull dumps the remote schema natively (`pg_dump`) then appends the migra diff. Only `--experimental` (structured dump) still delegates to Go, pending a TS PostgreSQL DDL parser for `format.WriteStructuredSchemas`. | +| `db push` | `ported` | `legacy/commands/db/push/` | `n/a` | `n/a` | Native TS port. Connects local/linked/`--db-url`; pushes pending migrations, `--include-seed` seeds (`seed_files` hash tracking), `--include-roles`, `[db.vault]` secrets including decrypted `encrypted:` values; `--dry-run`; best-effort pg-delta migrations-catalog cache (warning-only on failure). Pipeline-incompatible statements (`CREATE INDEX CONCURRENTLY`, `VACUUM`, …) run standalone outside the batch transaction — from the closed Go PR supabase/cli#5156, also ported into `apps/cli-go` (CLI-1989 ruling). | | `db reset` | `ported` | `legacy/commands/db/reset/` | `n/a` | `n/a` | Remote path native (drop user schemas, vault upsert, MigrateAndSeed — including the `--experimental` declarative `[db.migrations].schema_paths` apply branch, CLI-1958 — `--version`/`--last`, `--sql-paths` seed override). Local path native: running check, recreate + migrate + seed via the hidden Go `db __db-bootstrap` seam (that seam still forwards `--experimental` for its own schema-files branch, CLI-1955 scope), storage-gated bucket seeding (reuses `seed buckets`), git-branch `Finished…` line. Pipeline-incompatible statements run standalone outside the batch transaction, same as `db push` (closed Go PR supabase/cli#5156, CLI-1989 ruling). | -| `db start` | `ported` | `legacy/commands/db/start/` | `n/a` | `n/a` | Native TS port. Validates config, checks "already running" (prints Go's line), else delegates the container bootstrap (create + health + initial schema/roles/migrations/seed + `_current_branch`) to the hidden Go `db __db-bootstrap --mode start` seam. No status table / `cli_stack_started` (those are `supabase start`). `--from-backup` supported. | -| `inspect report` | `ported` | `legacy/commands/inspect/report/` | `n/a` | `n/a` | Native TS port. Runs every inspect query via server-side `COPY ... CSV`, writes 14 CSVs under `//`, then renders a Go-parity Glamour rules summary (bounded csvq-subset evaluator; custom `[experimental.inspect.rules]` supported). | -| `inspect db db-stats` | `ported` | `legacy/commands/inspect/db/db-stats/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | -| `inspect db replication-slots` | `ported` | `legacy/commands/inspect/db/replication-slots/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | -| `inspect db locks` | `ported` | `legacy/commands/inspect/db/locks/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | -| `inspect db blocking` | `ported` | `legacy/commands/inspect/db/blocking/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | -| `inspect db outliers` | `ported` | `legacy/commands/inspect/db/outliers/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | -| `inspect db calls` | `ported` | `legacy/commands/inspect/db/calls/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | -| `inspect db index-stats` | `ported` | `legacy/commands/inspect/db/index-stats/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | -| `inspect db long-running-queries` | `ported` | `legacy/commands/inspect/db/long-running-queries/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | -| `inspect db bloat` | `ported` | `legacy/commands/inspect/db/bloat/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | -| `inspect db role-stats` | `ported` | `legacy/commands/inspect/db/role-stats/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | -| `inspect db vacuum-stats` | `ported` | `legacy/commands/inspect/db/vacuum-stats/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | -| `inspect db table-stats` | `ported` | `legacy/commands/inspect/db/table-stats/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | -| `inspect db traffic-profile` | `ported` | `legacy/commands/inspect/db/traffic-profile/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | -| `inspect db cache-hit` | `ported` | `legacy/commands/inspect/db/cache-hit/` | `n/a` | `n/a` | Native TS port. Deprecated (use db-stats); routes to the active query. | -| `inspect db index-usage` | `ported` | `legacy/commands/inspect/db/index-usage/` | `n/a` | `n/a` | Native TS port. Deprecated (use index-stats); routes to the active query. | -| `inspect db total-index-size` | `ported` | `legacy/commands/inspect/db/total-index-size/` | `n/a` | `n/a` | Native TS port. Deprecated (use index-stats); routes to the active query. | -| `inspect db index-sizes` | `ported` | `legacy/commands/inspect/db/index-sizes/` | `n/a` | `n/a` | Native TS port. Deprecated (use index-stats); routes to the active query. | -| `inspect db table-sizes` | `ported` | `legacy/commands/inspect/db/table-sizes/` | `n/a` | `n/a` | Native TS port. Deprecated (use table-stats); routes to the active query. | -| `inspect db table-index-sizes` | `ported` | `legacy/commands/inspect/db/table-index-sizes/` | `n/a` | `n/a` | Native TS port. Deprecated (use table-stats); routes to the active query. | -| `inspect db total-table-sizes` | `ported` | `legacy/commands/inspect/db/total-table-sizes/` | `n/a` | `n/a` | Native TS port. Deprecated (use table-stats); routes to the active query. | -| `inspect db unused-indexes` | `ported` | `legacy/commands/inspect/db/unused-indexes/` | `n/a` | `n/a` | Native TS port. Deprecated (use index-stats); routes to the active query. | -| `inspect db table-record-counts` | `ported` | `legacy/commands/inspect/db/table-record-counts/` | `n/a` | `n/a` | Native TS port. Deprecated (use table-stats); routes to the active query. | -| `inspect db seq-scans` | `ported` | `legacy/commands/inspect/db/seq-scans/` | `n/a` | `n/a` | Native TS port. Deprecated (use index-stats); routes to the active query. | -| `inspect db role-configs` | `ported` | `legacy/commands/inspect/db/role-configs/` | `n/a` | `n/a` | Native TS port. Deprecated (use role-stats); routes to the active query. | -| `inspect db role-connections` | `ported` | `legacy/commands/inspect/db/role-connections/` | `n/a` | `n/a` | Native TS port. Deprecated (use role-stats); routes to the active query. | -| `migration down` | `ported` | `legacy/commands/migration/down/` | `n/a` | `n/a` | Native TS port. Revert prompt → drop user schemas → vault upsert → migrate&seed to the target version; defaults to `--local`. Skips Go's pgcache catalog write. | -| `migration fetch` | `ported` | `legacy/commands/migration/fetch/` | `n/a` | `n/a` | Native TS port. Reads `schema_migrations` and writes `supabase/migrations/_.sql`; overwrite prompt for a non-empty dir. | -| `migration list` | `ported` | `legacy/commands/migration/list/` | `n/a` | `n/a` | Native TS port. Merges remote `schema_migrations` with local files into a Glamour ASCII table (Local / Remote / Time-UTC columns); defaults to `--linked`. | -| `migration new` | `ported` | `legacy/commands/migration/new/` | `n/a` | `n/a` | Native TS port. Writes `supabase/migrations/_.sql` (mode 0644) from piped stdin; no DB/API. | -| `migration repair` | `ported` | `legacy/commands/migration/repair/` | `n/a` | `n/a` | Native TS port. Transactional create-table + TRUNCATE/UPSERT/DELETE; applied mode reads local files; repair-all prompt; defaults to `--linked`. | -| `migration squash` | `missing` | `missing` | `n/a` | `n/a` | No native TS implementation yet. Wrapped in legacy shell. | -| `migration up` | `ported` | `legacy/commands/migration/up/` | `n/a` | `n/a` | Native TS port. Computes pending migrations, upserts `[db.vault]`, applies each transactionally (pipeline-incompatible statements run standalone — closed Go PR supabase/cli#5156, ported into `apps/cli-go`, CLI-1989 ruling); `--include-all` for out-of-order; defaults to `--local`. Does not seed (matches Go). | -| `seed buckets` | `ported` | `legacy/commands/seed/buckets/` | `n/a` | `n/a` | Native TS port. Local-only (Go's `seed` defines no `--project-ref`, so the ref is always empty): seeds `[storage.buckets]` + `[storage.vector]` against the local Storage service gateway; remote/analytics paths are unreachable and omitted. `--linked`/`--local` accepted for surface parity (both seed local). Vector graceful-skip WARNINGs ported. | -| `test db` | `ported` | `legacy/commands/test/db/` | `n/a` | `n/a` | Native TS port. `--db-url`/`--local`/`--linked` + variadic paths; runs `supabase/pg_prove:3.36` via `docker run`; pgTAP enable/disable via `@effect/sql-pg`. `--network-id` override and `[images]` config override not modeled (documented divergences). | -| `test new` | `ported` | `legacy/commands/test/new/` | `n/a` | `n/a` | Native TS port. Writes `supabase/tests/_test.sql` from the embedded pgtap template; `--template` (pgtap). | +| `db start` | `ported` | `legacy/commands/db/start/` | `n/a` | `n/a` | Native TS port. Validates config, checks "already running" (prints Go's line), else delegates the container bootstrap (create + health + initial schema/roles/migrations/seed + `_current_branch`) to the hidden Go `db __db-bootstrap --mode start` seam. No status table / `cli_stack_started` (those are `supabase start`). `--from-backup` supported. | +| `inspect report` | `ported` | `legacy/commands/inspect/report/` | `n/a` | `n/a` | Native TS port. Runs every inspect query via server-side `COPY ... CSV`, writes 14 CSVs under `//`, then renders a Go-parity Glamour rules summary (bounded csvq-subset evaluator; custom `[experimental.inspect.rules]` supported). | +| `inspect db db-stats` | `ported` | `legacy/commands/inspect/db/db-stats/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | +| `inspect db replication-slots` | `ported` | `legacy/commands/inspect/db/replication-slots/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | +| `inspect db locks` | `ported` | `legacy/commands/inspect/db/locks/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | +| `inspect db blocking` | `ported` | `legacy/commands/inspect/db/blocking/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | +| `inspect db outliers` | `ported` | `legacy/commands/inspect/db/outliers/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | +| `inspect db calls` | `ported` | `legacy/commands/inspect/db/calls/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | +| `inspect db index-stats` | `ported` | `legacy/commands/inspect/db/index-stats/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | +| `inspect db long-running-queries` | `ported` | `legacy/commands/inspect/db/long-running-queries/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | +| `inspect db bloat` | `ported` | `legacy/commands/inspect/db/bloat/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | +| `inspect db role-stats` | `ported` | `legacy/commands/inspect/db/role-stats/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | +| `inspect db vacuum-stats` | `ported` | `legacy/commands/inspect/db/vacuum-stats/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | +| `inspect db table-stats` | `ported` | `legacy/commands/inspect/db/table-stats/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | +| `inspect db traffic-profile` | `ported` | `legacy/commands/inspect/db/traffic-profile/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | +| `inspect db cache-hit` | `ported` | `legacy/commands/inspect/db/cache-hit/` | `n/a` | `n/a` | Native TS port. Deprecated (use db-stats); routes to the active query. | +| `inspect db index-usage` | `ported` | `legacy/commands/inspect/db/index-usage/` | `n/a` | `n/a` | Native TS port. Deprecated (use index-stats); routes to the active query. | +| `inspect db total-index-size` | `ported` | `legacy/commands/inspect/db/total-index-size/` | `n/a` | `n/a` | Native TS port. Deprecated (use index-stats); routes to the active query. | +| `inspect db index-sizes` | `ported` | `legacy/commands/inspect/db/index-sizes/` | `n/a` | `n/a` | Native TS port. Deprecated (use index-stats); routes to the active query. | +| `inspect db table-sizes` | `ported` | `legacy/commands/inspect/db/table-sizes/` | `n/a` | `n/a` | Native TS port. Deprecated (use table-stats); routes to the active query. | +| `inspect db table-index-sizes` | `ported` | `legacy/commands/inspect/db/table-index-sizes/` | `n/a` | `n/a` | Native TS port. Deprecated (use table-stats); routes to the active query. | +| `inspect db total-table-sizes` | `ported` | `legacy/commands/inspect/db/total-table-sizes/` | `n/a` | `n/a` | Native TS port. Deprecated (use table-stats); routes to the active query. | +| `inspect db unused-indexes` | `ported` | `legacy/commands/inspect/db/unused-indexes/` | `n/a` | `n/a` | Native TS port. Deprecated (use index-stats); routes to the active query. | +| `inspect db table-record-counts` | `ported` | `legacy/commands/inspect/db/table-record-counts/` | `n/a` | `n/a` | Native TS port. Deprecated (use table-stats); routes to the active query. | +| `inspect db seq-scans` | `ported` | `legacy/commands/inspect/db/seq-scans/` | `n/a` | `n/a` | Native TS port. Deprecated (use index-stats); routes to the active query. | +| `inspect db role-configs` | `ported` | `legacy/commands/inspect/db/role-configs/` | `n/a` | `n/a` | Native TS port. Deprecated (use role-stats); routes to the active query. | +| `inspect db role-connections` | `ported` | `legacy/commands/inspect/db/role-connections/` | `n/a` | `n/a` | Native TS port. Deprecated (use role-stats); routes to the active query. | +| `migration down` | `ported` | `legacy/commands/migration/down/` | `n/a` | `n/a` | Native TS port. Revert prompt → drop user schemas → vault upsert → migrate&seed to the target version; defaults to `--local`. Skips Go's pgcache catalog write. | +| `migration fetch` | `ported` | `legacy/commands/migration/fetch/` | `n/a` | `n/a` | Native TS port. Reads `schema_migrations` and writes `supabase/migrations/_.sql`; overwrite prompt for a non-empty dir. | +| `migration list` | `ported` | `legacy/commands/migration/list/` | `n/a` | `n/a` | Native TS port. Merges remote `schema_migrations` with local files into a Glamour ASCII table (Local / Remote / Time-UTC columns); defaults to `--linked`. | +| `migration new` | `ported` | `legacy/commands/migration/new/` | `n/a` | `n/a` | Native TS port. Writes `supabase/migrations/_.sql` (mode 0644) from piped stdin; no DB/API. | +| `migration repair` | `ported` | `legacy/commands/migration/repair/` | `n/a` | `n/a` | Native TS port. Transactional create-table + TRUNCATE/UPSERT/DELETE; applied mode reads local files; repair-all prompt; defaults to `--linked`. | +| `migration squash` | `missing` | `missing` | `n/a` | `n/a` | No native TS implementation yet. Wrapped in legacy shell. | +| `migration up` | `ported` | `legacy/commands/migration/up/` | `n/a` | `n/a` | Native TS port. Computes pending migrations, upserts `[db.vault]`, applies each transactionally (pipeline-incompatible statements run standalone — closed Go PR supabase/cli#5156, ported into `apps/cli-go`, CLI-1989 ruling); `--include-all` for out-of-order; defaults to `--local`. Does not seed (matches Go). | +| `seed buckets` | `ported` | `legacy/commands/seed/buckets/` | `n/a` | `n/a` | Native TS port. Local-only (Go's `seed` defines no `--project-ref`, so the ref is always empty): seeds `[storage.buckets]` + `[storage.vector]` against the local Storage service gateway; remote/analytics paths are unreachable and omitted. `--linked`/`--local` accepted for surface parity (both seed local). Vector graceful-skip WARNINGs ported. | +| `test db` | `ported` | `legacy/commands/test/db/` | `n/a` | `n/a` | Native TS port. `--db-url`/`--local`/`--linked` + variadic paths; runs `supabase/pg_prove:3.36` via `docker run`; pgTAP enable/disable via `@effect/sql-pg`. `--network-id` override and `[images]` config override not modeled (documented divergences). | +| `test new` | `ported` | `legacy/commands/test/new/` | `n/a` | `n/a` | Native TS port. Writes `supabase/tests/_test.sql` from the embedded pgtap template; `--template` (pgtap). | ## Code Generation From 873c3d35b8bcc72e3d4c470c4992d1e548ca4a02 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Tue, 4 Aug 2026 16:16:06 +0100 Subject: [PATCH 03/47] fix(cli): address review findings on db reset --experimental native port (CLI-1958) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Fixes the items three reviewers (go-parity, engineer, architect) converged on: - Correct legacyMigrateAndSeed's stale docstring: the EXPERIMENTAL schema-files branch is reachable from start's fresh-volume setup (version: ""), not just migration down, and is deliberately deferred to CLI-2040, not unreachable. - Route legacy-seed.ts's resolveSeedFiles through the shared legacySqlFilesGlob instead of a third hand-rolled glob copy, fixing a silent directory-expansion gap on the migration down/start seed path. Remove legacy-seed-ops.ts's now- redundant legacyGlobSeedFiles/LegacyGlobResult pass-through shim. - Add Go's GlobOption surface (skipEmptyGlobs/errorOnAllSkipped) to legacySqlFilesGlob for db diff's upcoming declarative path; this issue's own callers pass no options, so behavior is unchanged. - Fix legacy-sql-files-glob.ts's toSlash to only convert on win32, matching Go's filepath.ToSlash (a no-op on non-Windows) — otherwise routing legacy-seed.ts's backslash-escape patterns through the shared glob would corrupt them. - Add reset.integration.test.ts coverage for schema_paths declaration order across multiple patterns and directory-entry expansion, plus toml-read tests for SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS's env-override, string, non-string- filter, and remote-suppression branches. - Correct reset.layers.ts's docstring: the local reset path still reaches LegacyGoProxy through the bootstrap seam (CLI-1955's scope), only the remote path is fully native. --- .../db/reset/reset.integration.test.ts | 50 +++++++++++ .../legacy/commands/db/reset/reset.layers.ts | 8 +- .../legacy-db-config.toml-read.unit.test.ts | 85 +++++++++++++++++++ .../legacy/shared/legacy-migrate-and-seed.ts | 14 ++- apps/cli/src/legacy/shared/legacy-seed-ops.ts | 55 +++--------- apps/cli/src/legacy/shared/legacy-seed.ts | 83 +++--------------- .../legacy/shared/legacy-sql-files-glob.ts | 69 +++++++++++++-- 7 files changed, 238 insertions(+), 126 deletions(-) diff --git a/apps/cli/src/legacy/commands/db/reset/reset.integration.test.ts b/apps/cli/src/legacy/commands/db/reset/reset.integration.test.ts index 713928aa2e..45b7e82268 100644 --- a/apps/cli/src/legacy/commands/db/reset/reset.integration.test.ts +++ b/apps/cli/src/legacy/commands/db/reset/reset.integration.test.ts @@ -966,6 +966,56 @@ describe("legacy db reset", () => { }, ); + it.live( + "applies schema files across multiple schema_paths patterns in declaration order, sorted within each pattern", + () => { + // Go sorts matches WITHIN each pattern (`sort.Strings`, `config.go:155`) but + // preserves DECLARATION order ACROSS patterns (no global re-sort) — `zz/*.sql`'s + // files must all run before `aa/*.sql`'s, even though "aa" sorts before "zz". + const { layer, conn } = setup(tmp.current, { + toml: 'project_id = "test"\n\n[db.migrations]\nschema_paths = ["zz/*.sql", "aa/*.sql"]\n', + files: { + "supabase/zz/b.sql": "create table zz_b ();", + "supabase/zz/a.sql": "create table zz_a ();", + "supabase/aa/b.sql": "create table aa_b ();", + "supabase/aa/a.sql": "create table aa_a ();", + }, + experimental: true, + confirm: [true], + }); + return Effect.gen(function* () { + yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe(Effect.provide(layer)); + const order = conn.execs + .map((s) => /create table (\w+) \(\)/.exec(s)?.[1]) + .filter((name): name is string => name !== undefined); + expect(order).toEqual(["zz_a", "zz_b", "aa_a", "aa_b"]); + }); + }, + ); + + it.live( + "expands a schema_paths directory entry to its nested .sql files on an experimental remote reset", + () => { + // `[db.migrations].schema_paths` resolves through Go's `Glob.SQLFiles` (not + // `Glob.Files`), which expands a directory match to its regular `.sql` files, + // recursively — unlike a plain glob pattern. + const { layer, conn } = setup(tmp.current, { + toml: 'project_id = "test"\n\n[db.migrations]\nschema_paths = ["some-dir"]\n', + files: { + "supabase/some-dir/01_top.sql": "create table dir_top ();", + "supabase/some-dir/nested/02_nested.sql": "create table dir_nested ();", + }, + experimental: true, + confirm: [true], + }); + return Effect.gen(function* () { + yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe(Effect.provide(layer)); + expect(conn.execs.some((s) => s.includes("create table dir_top"))).toBe(true); + expect(conn.execs.some((s) => s.includes("create table dir_nested"))).toBe(true); + }); + }, + ); + it.live( "silently applies nothing when schema_paths is unset on an experimental remote reset (Go's undocumented default-config behavior)", () => { diff --git a/apps/cli/src/legacy/commands/db/reset/reset.layers.ts b/apps/cli/src/legacy/commands/db/reset/reset.layers.ts index 2820a6b69c..23e0d88514 100644 --- a/apps/cli/src/legacy/commands/db/reset/reset.layers.ts +++ b/apps/cli/src/legacy/commands/db/reset/reset.layers.ts @@ -19,9 +19,11 @@ import { legacyDbBootstrapSeamLayer } from "../shared/legacy-db-bootstrap.seam.l * Runtime layer for `supabase db reset`. Same composition as `db push` / `db lint`: * the Postgres connection, the db-config resolver, project-ref resolution, and the * linked-project cache, all over the lazy management-API factory so the local / - * `--db-url` paths never resolve an access token at layer-build time. Both the local - * and remote reset paths (including the `--experimental` remote schema-files apply, - * CLI-1958) are fully native — no `LegacyGoProxy` dependency remains. + * `--db-url` paths never resolve an access token at layer-build time. The remote + * reset path (including the `--experimental` remote schema-files apply, CLI-1958) + * is fully native. The local path's container primitives still reach `LegacyGoProxy` + * through the bootstrap seam below (`legacyDbBootstrapSeamLayer`, ambient from the + * root) — that native port is CLI-1955's scope, not this one. */ const cliConfig = legacyCliConfigLayer.pipe(Layer.provide(legacyDebugLoggerLayer)); const httpClient = legacyHttpClientLayer.pipe(Layer.provide(legacyDebugLoggerLayer)); diff --git a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts index 09b88bdc40..70f34f8bdd 100644 --- a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts @@ -293,6 +293,91 @@ describe("legacyReadDbToml", () => { ); }); + it.effect( + "honors SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS over the TOML array (comma split, no trim)", + () => { + const previous = process.env["SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS"]; + process.env["SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS"] = "a.sql, b.sql"; + const dir = withConfig(["[db.migrations]", 'schema_paths = ["ignored.sql"]', ""].join("\n")); + return read(dir).pipe( + Effect.tap((v) => + Effect.sync(() => { + expect(v.schemaPaths).toEqual(["supabase/a.sql", "supabase/ b.sql"]); + }), + ), + Effect.ensuring( + Effect.sync(() => { + if (previous === undefined) delete process.env["SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS"]; + else process.env["SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS"] = previous; + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "decodes a STRING db.migrations.schema_paths via StringToSliceHookFunc (comma, no trim)", + () => { + const dir = withConfig(["[db.migrations]", 'schema_paths = "a.sql,b.sql"', ""].join("\n")); + return read(dir).pipe( + Effect.tap((v) => + Effect.sync(() => { + expect(v.schemaPaths).toEqual(["supabase/a.sql", "supabase/b.sql"]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect("filters non-string db.migrations.schema_paths array elements", () => { + const dir = withConfig( + ["[db.migrations]", 'schema_paths = [42, "schemas/*.sql"]', ""].join("\n"), + ); + return read(dir).pipe( + Effect.tap((v) => + Effect.sync(() => { + expect(v.schemaPaths).toEqual(["supabase/schemas/*.sql"]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }); + + it.effect( + "an explicit remote db.migrations.schema_paths beats SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS", + () => { + // Go applies each matched-remote key via v.Set (override tier) above AutomaticEnv + // (config.go:635-637), so an explicit remote value wins over the env var. + const ref = "schmschmschmschmschm"; + const previous = process.env["SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS"]; + process.env["SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS"] = "env-only.sql"; + const dir = withConfig( + [ + "[remotes.prod]", + `project_id = "${ref}"`, + 'db.migrations.schema_paths = ["remote-only.sql"]', + "", + ].join("\n"), + ); + return readRef(dir, ref).pipe( + Effect.tap((v) => + Effect.sync(() => { + expect(v.schemaPaths).toEqual(["supabase/remote-only.sql"]); + }), + ), + Effect.ensuring( + Effect.sync(() => { + if (previous === undefined) delete process.env["SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS"]; + else process.env["SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS"] = previous; + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + it.effect("decodes a numeric db.seed.enabled = 0 as false (Go weak-bool decode)", () => { const dir = withConfig(["[db.seed]", "enabled = 0", ""].join("\n")); return read(dir).pipe( diff --git a/apps/cli/src/legacy/shared/legacy-migrate-and-seed.ts b/apps/cli/src/legacy/shared/legacy-migrate-and-seed.ts index 809af32b95..6071bb2e63 100644 --- a/apps/cli/src/legacy/shared/legacy-migrate-and-seed.ts +++ b/apps/cli/src/legacy/shared/legacy-migrate-and-seed.ts @@ -15,10 +15,16 @@ export interface LegacyMigrateAndSeedConfig { /** * Reapplies local migrations up to `version`, then runs seed files. Port of Go's * `apply.MigrateAndSeed` (`internal/migration/apply/apply.go:16`) for the - * `version`-set path (the EXPERIMENTAL declarative `applySchemaFiles` branch is - * unreachable from `migration down`, which always passes a concrete version, so - * it is intentionally not ported). Migration apply is gated on - * `db.migrations.enabled`; seeding on `db.seed.enabled` (inside the seed helper). + * `version`-set path only — the EXPERIMENTAL declarative `applySchemaFiles` branch + * is NOT ported here. That branch is unreachable from `migration down` (which always + * passes a concrete version), but IS reachable from `start`'s fresh-volume setup + * (`commands/start/lib/db-setup.ts`, which calls this with `version: ""`), where real + * Go WOULD take it on `--experimental`. This is a known, pre-existing parity gap + * (not introduced by this PR) deliberately left out of CLI-1958's scope, tracked as + * CLI-2040 ("`supabase start --experimental` doesn't apply schema files on a fresh + * volume, parity gap with Go's `SetupLocalDatabase`") — see that issue's own PR for + * the port. Migration apply is gated on `db.migrations.enabled`; seeding on + * `db.seed.enabled` (inside the seed helper). */ export const legacyMigrateAndSeed = ( session: LegacyDbSession, diff --git a/apps/cli/src/legacy/shared/legacy-seed-ops.ts b/apps/cli/src/legacy/shared/legacy-seed-ops.ts index d430766cbe..d95bdba572 100644 --- a/apps/cli/src/legacy/shared/legacy-seed-ops.ts +++ b/apps/cli/src/legacy/shared/legacy-seed-ops.ts @@ -1,5 +1,5 @@ import { createHash } from "node:crypto"; -import { Effect, type FileSystem, Option, type Path } from "effect"; +import { Effect, type FileSystem, type Path } from "effect"; import { Output } from "../../shared/output/output.service.ts"; import type { LegacyDbExecError } from "./legacy-db-connection.errors.ts"; @@ -26,39 +26,6 @@ export interface LegacySeedFile { readonly dirty: boolean; } -/** Result of resolving `[db.seed].sql_paths` against the workspace. */ -interface LegacyGlobResult { - /** Workdir-relative, forward-slashed matches, deduplicated in pattern order. */ - readonly files: ReadonlyArray; - /** Per-pattern warnings (`no files matched pattern: …`), joined by Go's `errors.Join`. */ - readonly warning: Option.Option; -} - -/** - * Resolves seed glob patterns to existing files, porting Go's `config.Glob.Files` - * over `fs.Glob` (`pkg/config/config.go:102-124`) via the shared {@link legacySqlFilesGlob} - * traversal (also used by `[db.migrations].schema_paths`, `legacy-migration-apply.ts`). - * Each pattern is first joined under the `supabase/` directory (Go resolves `sql_paths` - * at config load, `config.go:884`) — that resolution happens once, upstream, via - * `legacyResolveSeedSqlPath`; this function globs the already-resolved patterns - * verbatim. Per-pattern warnings (`no files matched pattern: …` / malformed glob) are - * joined with Go's `errors.Join` newline semantics and surfaced unconditionally — the - * seed path always warns, unlike the schema-files apply path (see - * `legacyApplySchemaFiles`), which only surfaces a warning when it is the ONLY outcome. - */ -const legacyGlobSeedFiles = Effect.fnUntraced(function* ( - fs: FileSystem.FileSystem, - path: Path.Path, - patterns: ReadonlyArray, - workdir: string, -) { - const { files, warnings } = yield* legacySqlFilesGlob(fs, path, patterns, workdir); - return { - files, - warning: warnings.length > 0 ? Option.some(warnings.join("\n")) : Option.none(), - } satisfies LegacyGlobResult; -}); - /** `SELECT path, hash FROM supabase_migrations.seed_files`, `42P01` → empty map. */ const readRemoteSeeds = (session: LegacyDbSession) => session.query(SELECT_SEED_TABLE).pipe( @@ -80,10 +47,16 @@ const isUndefinedTable = (error: LegacyDbExecError): boolean => /** * Resolves the pending seed files for `db push --include-seed`. Mirrors Go's - * `GetPendingSeeds` (`pkg/migration/seed.go:34-63`): glob the configured paths - * (warn, don't fail, on empty patterns), read the remote `seed_files` hashes, - * and emit each local file that is new (`dirty=false`) or hash-changed - * (`dirty=true`); files whose hash already matches are skipped. + * `GetPendingSeeds` (`pkg/migration/seed.go:34-63`): glob the configured paths via + * the shared {@link legacySqlFilesGlob} traversal (also used by `[db.migrations]. + * schema_paths`, `legacy-migration-apply.ts`, and by `legacy-seed.ts`'s own + * `resolveSeedFiles` for the `migration down`/`start` seed step), warn — don't fail — + * on empty patterns, read the remote `seed_files` hashes, and emit each local file + * that is new (`dirty=false`) or hash-changed (`dirty=true`); files whose hash + * already matches are skipped. Per-pattern warnings are joined with Go's `errors.Join` + * newline semantics and surfaced unconditionally (`seed.go:36-38`) — unlike the + * schema-files apply path (see `legacyApplySchemaFiles`), which only surfaces a + * warning when it is the ONLY outcome. */ export const legacyGetPendingSeeds = Effect.fnUntraced(function* ( session: LegacyDbSession, @@ -93,9 +66,9 @@ export const legacyGetPendingSeeds = Effect.fnUntraced(function* ( workdir: string, ) { const output = yield* Output; - const { files, warning } = yield* legacyGlobSeedFiles(fs, path, patterns, workdir); - if (Option.isSome(warning)) { - yield* output.raw(`WARN: ${warning.value}\n`, "stderr"); + const { files, warnings } = yield* legacySqlFilesGlob(fs, path, patterns, workdir); + if (warnings.length > 0) { + yield* output.raw(`WARN: ${warnings.join("\n")}\n`, "stderr"); } const pending: Array = []; if (files.length === 0) return pending; diff --git a/apps/cli/src/legacy/shared/legacy-seed.ts b/apps/cli/src/legacy/shared/legacy-seed.ts index dc67437c48..ca1a6eb3e0 100644 --- a/apps/cli/src/legacy/shared/legacy-seed.ts +++ b/apps/cli/src/legacy/shared/legacy-seed.ts @@ -8,7 +8,7 @@ import { legacyReadSeedTable, UPSERT_SEED_FILE, } from "./legacy-migration-history.ts"; -import { LEGACY_BAD_PATTERN_MESSAGE, legacyPathMatch } from "./legacy-path-match.ts"; +import { legacySqlFilesGlob } from "./legacy-sql-files-glob.ts"; import { legacySplitAndTrim } from "./legacy-sql-split.ts"; /** Applying a seed file failed (Go's `SeedData` / `ExecBatchWithCache` errors). */ @@ -33,14 +33,6 @@ interface LegacyPendingSeed { readonly dirty: boolean; } -// Go's `io/fs.hasMeta` magic-character set is `*`, `?`, `[`, and `\` (escape) — -// `glob.go` `hasMeta`. `\` must count so a pattern whose only meta syntax is a -// backslash escape (e.g. `foo\.sql`, `seed\*.sql`) is globbed via `legacyPathMatch` -// (which handles the escape) instead of being treated as a literal filename and -// missing the real file. Go applies `filepath.ToSlash` before globbing, so a `\` -// here is always a glob escape, never a path separator. -const hasMeta = (pattern: string): boolean => /[*?[\\]/u.test(pattern); - // Go globs/reads seed paths through an OS-root-rooted `afero.NewOsFs`, where the // CLI's "workdir" is just `os.Chdir(workdir)` (`internal/utils/misc.go`) — which // only affects RELATIVE paths. An absolute `[db.seed].sql_paths` entry, preserved @@ -52,46 +44,14 @@ const resolveUnderWorkdir = (path: Path.Path, workdir: string, p: string): strin path.isAbsolute(p) ? p : path.join(workdir, p); /** - * Resolves a single glob pattern against the workdir, returning the matched - * paths RELATIVE to the workdir (so `seed_files.path` stays Go-compatible). - * Mirrors Go's `fs.Glob`: a literal pattern returns itself iff it exists; a - * pattern with metacharacters lists each parent directory and matches per - * segment via `legacyPathMatch` (Go's `path.Match`). The caller validates the - * whole pattern up front, so a malformed class never reaches here. + * Resolves `[db.seed].sql_paths` to existing files, porting Go's `config.Glob.SQLFiles` + * (`pkg/migration/seed.go:35`, via the shared {@link legacySqlFilesGlob} traversal — + * also used by `legacyGetPendingSeeds` (`legacy-seed-ops.ts`) for the same Go field on + * the `db push`/`db reset` path, and by `legacyApplySchemaFiles` (`legacy-migration-apply.ts`) + * for `[db.migrations].schema_paths`). Go's `GetPendingSeeds` prints a single unconditional + * `WARN: ` line for any glob problem (`seed.go:36-38`) — unlike the schema-files + * apply path, which only warns when NO pattern matched anything at all. */ -const globPattern = ( - fs: FileSystem.FileSystem, - path: Path.Path, - workdir: string, - pattern: string, -): Effect.Effect> => - Effect.gen(function* () { - if (!hasMeta(pattern)) { - const exists = yield* fs - .exists(resolveUnderWorkdir(path, workdir, pattern)) - .pipe(Effect.orElseSucceed(() => false)); - return exists ? [pattern] : []; - } - const slash = pattern.lastIndexOf("/"); - const dirPattern = slash === -1 ? "" : pattern.slice(0, slash); - const filePattern = slash === -1 ? pattern : pattern.slice(slash + 1); - const dirs = hasMeta(dirPattern) - ? yield* globPattern(fs, path, workdir, dirPattern) - : [dirPattern]; - const result: Array = []; - for (const dir of dirs) { - const absDir = dir.length === 0 ? workdir : resolveUnderWorkdir(path, workdir, dir); - const names = yield* fs.readDirectory(absDir).pipe(Effect.orElseSucceed(() => [])); - for (const name of names) { - if (legacyPathMatch(filePattern, name).matched) { - result.push(dir.length === 0 ? name : `${dir}/${name}`); - } - } - } - return result; - }); - -/** Go's `config.Glob.Files`: glob each pattern, sort, dedup; warn on bad/no-match. */ const resolveSeedFiles = ( fs: FileSystem.FileSystem, path: Path.Path, @@ -100,30 +60,9 @@ const resolveSeedFiles = ( ) => Effect.gen(function* () { const output = yield* Output; - const seen = new Set(); - const result: Array = []; - const unmatched: Array = []; - for (const pattern of patterns) { - // Go's `fs.Glob` validates the whole pattern up front (`Match(pattern, "")`); - // a malformed glob is reported as `failed to glob files: ` and - // contributes no matches, exactly like `Glob.Files`'s error branch. - if (legacyPathMatch(pattern, "").badPattern) { - unmatched.push(`failed to glob files: ${LEGACY_BAD_PATTERN_MESSAGE}`); - continue; - } - const matches = [...(yield* globPattern(fs, path, workdir, pattern))].sort(); - if (matches.length === 0) unmatched.push(`no files matched pattern: ${pattern}`); - for (const match of matches) { - if (!seen.has(match)) { - seen.add(match); - result.push(match); - } - } - } - // Go collects all glob errors into one `errors.Join` and prints a single - // `WARN: ` line (`config.Glob.Files` → `seed.go:37`), not one per pattern. - if (unmatched.length > 0) yield* output.raw(`WARN: ${unmatched.join("\n")}\n`, "stderr"); - return result; + const { files, warnings } = yield* legacySqlFilesGlob(fs, path, patterns, workdir); + if (warnings.length > 0) yield* output.raw(`WARN: ${warnings.join("\n")}\n`, "stderr"); + return files; }); /** diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts index d6a2c3cd15..fa4f63a066 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts @@ -4,7 +4,18 @@ import { LEGACY_BAD_PATTERN_MESSAGE, legacyPathMatch } from "./legacy-path-match const META_CHARS = /[*?[\\]/u; -const toSlash = (p: string): string => p.replaceAll("\\", "/"); +// Go's `config.hasGlobMeta` (`apps/cli-go/pkg/config/config.go:211-213`) — a DIFFERENT, +// narrower set than `META_CHARS` above (which mirrors `io/fs.hasMeta`'s `path.Match` +// escape handling and includes `\`). Only used to gate `WithSkipEmptyGlobs()` below. +const GLOB_META_CHARS = /[*?[]/u; + +// Go's `filepath.ToSlash` replaces `os.PathSeparator` with `/` — a no-op on the +// non-Windows platforms this shell mostly runs on, since their separator already IS +// `/`. Gating on `win32` (rather than converting unconditionally) matters: on +// non-Windows a `\` in a pattern is never a path separator, only a `path.Match` +// escape (`foo\.sql`, `seed\*.sql`), and unconditionally slashing it would corrupt +// that escape — see `legacyPathMatch`'s escape handling below. +const toSlash = (p: string): string => (process.platform === "win32" ? p.replaceAll("\\", "/") : p); /** Splits a forward-slashed path into its directory prefix and final element. */ const splitPath = (p: string): { readonly dir: string; readonly file: string } => { @@ -100,12 +111,42 @@ interface LegacySqlFilesGlobResult { readonly warnings: ReadonlyArray; } +/** + * Mirrors Go's `GlobOption`s (`apps/cli-go/pkg/config/config.go:100-117`). Neither + * caller in this shared module needs them today (`legacyApplySchemaFiles`'s + * `[db.migrations].schema_paths`, and the two `[db.seed].sql_paths` callers — + * `legacyGetPendingSeeds` and `legacy-seed.ts`'s `resolveSeedFiles` — all pass none, + * matching Go's own zero-option call sites, `pkg/migration/seed.go:35` and + * `internal/migration/apply/apply.go:52`). Added for `db diff`'s declarative path, + * which calls `Glob.files` `WithSkipEmptyGlobs()` + `WithErrorOnAllSkippedGlobs()`. + */ +export interface LegacySqlFilesGlobOptions { + /** + * Go's `WithSkipEmptyGlobs()`: a pattern that contains a glob metacharacter + * (`config.hasGlobMeta` — `*`, `?`, `[`; NOT `\`, unlike `META_CHARS` above) and + * matches nothing is silently skipped — no "no files matched pattern" warning — + * unless `errorOnAllSkipped` retroactively un-skips it (below). A LITERAL pattern + * (no glob metacharacter) that doesn't exist always warns, regardless of this flag. + */ + readonly skipEmptyGlobs?: boolean; + /** + * Go's `WithErrorOnAllSkippedGlobs()`: only meaningful together with + * `skipEmptyGlobs`. If the overall result ends up empty AND at least one pattern + * was silently skipped, every skipped pattern's silence is retroactively turned + * back into a "no files matched pattern" warning — so a `skipEmptyGlobs` caller + * can still detect the "nothing matched anything" case. + */ + readonly errorOnAllSkipped?: boolean; +} + /** * Resolves SQL-file glob patterns to existing files, porting Go's `config.Glob.SQLFiles` - * over `fs.Glob` (`apps/cli-go/pkg/config/config.go:123-211`). Shared by `[db.seed].sql_paths` - * (via `legacyGlobSeedFiles`, `legacy-seed-ops.ts`) and `[db.migrations].schema_paths` (via - * `legacyApplySchemaFiles`, `legacy-migration-apply.ts`) — both Go fields resolve through the - * exact same `Glob` type and `SQLFiles` method, so the traversal logic lives here once. + * over `fs.Glob` (`apps/cli-go/pkg/config/config.go:123-211`). Shared by + * `[db.seed].sql_paths` (via `legacyGetPendingSeeds`, `legacy-seed-ops.ts`, and + * `legacy-seed.ts`'s `resolveSeedFiles`) and `[db.migrations].schema_paths` (via + * `legacyApplySchemaFiles`, `legacy-migration-apply.ts`) — all three Go call sites + * resolve through the exact same `Glob` type and `SQLFiles` method, so the traversal + * logic lives here once. * * Each pattern is matched independently: matches are sorted per-pattern (Go's * `sort.Strings`, `config.go:155`), but the overall result preserves cross-pattern @@ -126,10 +167,14 @@ export const legacySqlFilesGlob = Effect.fnUntraced(function* ( path: Path.Path, patterns: ReadonlyArray, workdir: string, + options?: LegacySqlFilesGlobOptions, ) { + const skipEmptyGlobs = options?.skipEmptyGlobs ?? false; + const errorOnAllSkipped = options?.errorOnAllSkipped ?? false; const seen = new Set(); const files: Array = []; const warnings: Array = []; + const skipped: Array = []; for (const rawPattern of patterns) { const pattern = toSlash(rawPattern); @@ -142,7 +187,11 @@ export const legacySqlFilesGlob = Effect.fnUntraced(function* ( } const matches = yield* globOne(fs, path, workdir, pattern); if (matches.length === 0) { - warnings.push(`no files matched pattern: ${pattern}`); + if (skipEmptyGlobs && GLOB_META_CHARS.test(pattern)) { + skipped.push(pattern); + } else { + warnings.push(`no files matched pattern: ${pattern}`); + } continue; } for (const match of [...matches].sort()) { @@ -169,5 +218,13 @@ export const legacySqlFilesGlob = Effect.fnUntraced(function* ( } } + // Go: `if opts.errorOnAllSkipped && len(result) == 0 && len(skipped) > 0` — only + // escalate silently-skipped patterns back into warnings when NOTHING matched at all. + if (errorOnAllSkipped && files.length === 0 && skipped.length > 0) { + for (const pattern of skipped) { + warnings.push(`no files matched pattern: ${pattern}`); + } + } + return { files, warnings } satisfies LegacySqlFilesGlobResult; }); From 8ff44852890f7c9e2d259cca63da5eaa2de32922 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Tue, 4 Aug 2026 17:36:48 +0100 Subject: [PATCH 04/47] docs(cli): reformat go-cli-porting-status.md table (oxfmt column widths) --- apps/cli/docs/go-cli-porting-status.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/apps/cli/docs/go-cli-porting-status.md b/apps/cli/docs/go-cli-porting-status.md index 9a71496f0f..4571c360d3 100644 --- a/apps/cli/docs/go-cli-porting-status.md +++ b/apps/cli/docs/go-cli-porting-status.md @@ -85,7 +85,7 @@ These commands exist in the TS CLI today but have no direct top-level equivalent | `db diff` | `ported` | `legacy/commands/db/diff/` | `n/a` | `n/a` | Native TS port. Native pg-delta / migra diff via edge-runtime against a Go-seam-provisioned live shadow (`db __shadow`); `--use-pgadmin` / `--use-pg-schema` delegate to the Go binary. | | `db dump` | `ported` | `legacy/commands/db/dump/` | `n/a` | `n/a` | Native TS port. Streams `pg_dump`/`pg_dumpall` via a Docker container (`LegacyDockerRun`); schema/data/role modes, `--dry-run` script print, IPv4 transaction-pooler fallback. | | `db lint` | `ported` | `legacy/commands/db/lint/` | `n/a` | `n/a` | Native TS port. Runs `plpgsql_check` in a rolled-back transaction via LegacyDbConnection; emits Go-parity pretty JSON. | -| `db pull` | `ported` | `legacy/commands/db/pull/` | `n/a` | `n/a` | Native TS port. Native pg-delta / migra migration + `--declarative` pg-delta export; reconciles `schema_migrations`. The initial-migra pull dumps the remote schema natively (`pg_dump`) then appends the migra diff. Only `--experimental` (structured dump) still delegates to Go, deprecated in favor of `--declarative` (CLI-1957) — it needs a TS PostgreSQL DDL parser for Go's `format.WriteStructuredSchemas` that has no equivalent in this repo, and `--declarative` already delivers the same per-object schema split via pg-delta catalog introspection. | +| `db pull` | `ported` | `legacy/commands/db/pull/` | `n/a` | `n/a` | Native TS port. Native pg-delta / migra migration + `--declarative` pg-delta export; reconciles `schema_migrations`. The initial-migra pull dumps the remote schema natively (`pg_dump`) then appends the migra diff. Only `--experimental` (structured dump) still delegates to Go, deprecated in favor of `--declarative` (CLI-1957) — it needs a TS PostgreSQL DDL parser for Go's `format.WriteStructuredSchemas` that has no equivalent in this repo, and `--declarative` already delivers the same per-object schema split via pg-delta catalog introspection. | | `db push` | `ported` | `legacy/commands/db/push/` | `n/a` | `n/a` | Native TS port. Connects local/linked/`--db-url`; pushes pending migrations, `--include-seed` seeds (`seed_files` hash tracking), `--include-roles`, `[db.vault]` secrets including decrypted `encrypted:` values; `--dry-run`; best-effort pg-delta migrations-catalog cache (warning-only on failure). Pipeline-incompatible statements (`CREATE INDEX CONCURRENTLY`, `VACUUM`, …) run standalone outside the batch transaction — from the closed Go PR supabase/cli#5156, also ported into `apps/cli-go` (CLI-1989 ruling). | | `db reset` | `ported` | `legacy/commands/db/reset/` | `n/a` | `n/a` | Remote path native (drop user schemas, vault upsert, MigrateAndSeed — including the `--experimental` declarative `[db.migrations].schema_paths` apply branch, CLI-1958 — `--version`/`--last`, `--sql-paths` seed override). Local path native: running check, recreate + migrate + seed via the hidden Go `db __db-bootstrap` seam (that seam still forwards `--experimental` for its own schema-files branch, CLI-1955 scope), storage-gated bucket seeding (reuses `seed buckets`), git-branch `Finished…` line. Pipeline-incompatible statements run standalone outside the batch transaction, same as `db push` (closed Go PR supabase/cli#5156, CLI-1989 ruling). | | `db start` | `ported` | `legacy/commands/db/start/` | `n/a` | `n/a` | Native TS port. Validates config, checks "already running" (prints Go's line), else delegates the container bootstrap (create + health + initial schema/roles/migrations/seed + `_current_branch`) to the hidden Go `db __db-bootstrap --mode start` seam. No status table / `cli_stack_started` (those are `supabase start`). `--from-backup` supported. | From 4bdb3bda68404ea7409b399232f247e26e6e2dc9 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Wed, 5 Aug 2026 11:40:07 +0100 Subject: [PATCH 05/47] fix(cli): match Go's Glob.SQLFiles symlink and empty-pattern handling (CLI-1958) `legacySqlFilesGlob`/`legacyWalkSqlFiles` diverged from Go's `config.Glob.SQLFiles` (`apps/cli-go/pkg/config/config.go:123-211`) in two ways, both confirmed empirically against a built `apps/cli-go` probe: - Directory expansion re-`stat`ed each child, following symlinks. Go's `fs.WalkDir` types children from their parent's `ReadDir` entry (Lstat-based), so a symlinked `.sql` file or subdirectory below a matched schema/seed directory is never included or recursed into (`io/fs/walk.go:114-115`). Detect this with `readLink` (succeeds only for symlinks) before falling back to `stat`. - An empty pattern (e.g. `schema_paths = [""]`) resolved via `path.join(workdir, "")` to the workdir itself and reported a match. Go's `Lstat("")` fails, so `fs.Glob`/`afero.Glob` always report no match for an empty pattern. Short-circuit on `pattern.length === 0`. Review: PR #6062 (chatgpt-codex-connector), threads on legacy-sql-files-glob.ts:92 and :40. --- .../legacy/shared/legacy-sql-files-glob.ts | 34 +++++- .../shared/legacy-sql-files-glob.unit.test.ts | 104 ++++++++++++++++++ 2 files changed, 132 insertions(+), 6 deletions(-) create mode 100644 apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts index fa4f63a066..f35147155e 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts @@ -31,6 +31,14 @@ const globOne = ( pattern: string, ): Effect.Effect, never> => Effect.gen(function* () { + // Go's `fs.Glob`/`afero.Glob` resolve a literal (no-metacharacter) pattern via + // `Lstat`, which errors on an empty path — so `""` always yields no matches. An + // unguarded `path.join(workdir, "")` resolves to `workdir` itself, which would + // wrongly report the workdir as a match for an empty `schema_paths`/`sql_paths` + // entry (e.g. `schema_paths = [""]`). + if (pattern.length === 0) { + return []; + } // Absolute patterns resolve against the filesystem root; relative ones are // rooted at the workdir. const resolve = (p: string): string => (path.isAbsolute(p) ? p : path.join(workdir, p)); @@ -81,12 +89,26 @@ const legacyWalkSqlFiles = ( .pipe(Effect.orElseSucceed((): ReadonlyArray => [])); for (const name of names) { const childRel = `${rel}/${name}`; - const childType = yield* fs - .stat(path.isAbsolute(childRel) ? childRel : path.join(workdir, childRel)) - .pipe( - Effect.map((info) => info.type), - Effect.orElseSucceed(() => "Unknown" as const), - ); + const childAbs = path.isAbsolute(childRel) ? childRel : path.join(workdir, childRel); + // Go's `fs.WalkDir` types each child from the parent's `ReadDir` entry + // (`os.ReadDir`'s Lstat-based `DirEntry`) and never re-`Stat`s through it — + // so a symlinked file or subdirectory found below the matched root is + // neither included nor recursed into, regardless of what it points to + // (`io/fs/walk.go:114-115`: only the matched root itself, resolved once by + // the caller before reaching this walk, may be a symlink). `readLink` + // succeeds only for symlinks, so use it as the no-follow probe in place of + // the `Lstat` this FileSystem service doesn't expose. + const isSymlink = yield* fs.readLink(childAbs).pipe( + Effect.map(() => true), + Effect.orElseSucceed(() => false), + ); + if (isSymlink) { + continue; + } + const childType = yield* fs.stat(childAbs).pipe( + Effect.map((info) => info.type), + Effect.orElseSucceed(() => "Unknown" as const), + ); if (childType === "Directory") { yield* walk(childRel); } else if (childType === "File" && childRel.endsWith(".sql")) { diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts new file mode 100644 index 0000000000..7fa4d05cb9 --- /dev/null +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts @@ -0,0 +1,104 @@ +import { mkdirSync, mkdtempSync, rmSync, symlinkSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { BunServices } from "@effect/platform-bun"; +import { describe, expect, it } from "@effect/vitest"; +import { Effect, FileSystem, Path } from "effect"; + +import { legacySqlFilesGlob } from "./legacy-sql-files-glob.ts"; + +const run = (patterns: ReadonlyArray, workdir: string) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + return yield* legacySqlFilesGlob(fs, path, patterns, workdir); + }).pipe(Effect.provide(BunServices.layer)); + +describe("legacySqlFilesGlob", () => { + it.effect( + "treats an empty pattern as no match, not the workdir itself (Go fs.Glob parity)", + () => { + // Go's `fs.Glob`/`afero.Glob` resolve a no-metacharacter pattern via `Lstat`, which + // errors on an empty path — an empty `schema_paths`/`sql_paths` entry (e.g. + // `schema_paths = [""]`) always yields no matches, never the workdir itself. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-empty-")); + return run([""], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual([]); + expect(result.warnings).toEqual(["no files matched pattern: "]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "does not follow a symlinked .sql file below a matched directory (Go WalkDir parity)", + () => { + // Go's `Glob.SQLFiles` expands a matched directory with `fs.WalkDir`, which types + // each child from its parent's `ReadDir` entry (`os.ReadDir`'s Lstat-based + // `DirEntry`) and never re-`Stat`s through it — so a symlinked `.sql` file is + // never included, regardless of what it points to. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-symlink-file-")); + const schemasDir = join(dir, "schemas"); + mkdirSync(schemasDir); + writeFileSync(join(schemasDir, "real.sql"), "select 1;"); + const outsideDir = join(dir, "outside"); + mkdirSync(outsideDir); + writeFileSync(join(outsideDir, "evil.sql"), "select 2;"); + symlinkSync(join(outsideDir, "evil.sql"), join(schemasDir, "linked.sql")); + return run(["schemas"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["schemas/real.sql"]); + expect(result.warnings).toEqual([]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "does not recurse into a symlinked subdirectory below a matched directory (Go WalkDir parity)", + () => { + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-symlink-dir-")); + const schemasDir = join(dir, "schemas"); + mkdirSync(schemasDir); + writeFileSync(join(schemasDir, "real.sql"), "select 1;"); + const outsideSubdir = join(dir, "outside-subdir"); + mkdirSync(outsideSubdir); + writeFileSync(join(outsideSubdir, "nested.sql"), "select 3;"); + symlinkSync(outsideSubdir, join(schemasDir, "linked-dir")); + return run(["schemas"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["schemas/real.sql"]); + expect(result.warnings).toEqual([]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect("still expands a real (non-symlinked) nested directory recursively", () => { + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-nested-")); + const schemasDir = join(dir, "schemas"); + const nestedDir = join(schemasDir, "nested"); + mkdirSync(nestedDir, { recursive: true }); + writeFileSync(join(schemasDir, "a.sql"), "select 1;"); + writeFileSync(join(nestedDir, "b.sql"), "select 2;"); + return run(["schemas"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["schemas/a.sql", "schemas/nested/b.sql"]); + expect(result.warnings).toEqual([]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }); +}); From 7020d74f0f8f7342ffdf780cd0550b2f933c55ed Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Wed, 5 Aug 2026 11:40:16 +0100 Subject: [PATCH 06/47] fix(cli): weakly coerce non-string schema_paths/sql_paths entries (CLI-1958) `[db.migrations].schema_paths` and `[db.seed].sql_paths` decode through Go's `v.UnmarshalExact` (`apps/cli-go/pkg/config/config.go:749-756`), whose decoder config never overrides `WeaklyTypedInput`, so viper's `defaultDecoderConfig` default of `true` stands. mapstructure's `decodeString` therefore weakly converts a non-string scalar array element (bool to "1"/"0", a number to its decimal string) instead of erroring or dropping it. The TS reader filtered non-string entries out instead, silently dropping schemas. Verified empirically against a built `apps/cli-go` probe: `schema_paths = [42, true, "schemas/*.sql"]` resolves to `supabase/{42,1,schemas/*.sql}`. Review: PR #6062 (chatgpt-codex-connector), thread on legacy-db-config.toml-read.ts:1887. --- .../shared/legacy-db-config.toml-read.ts | 26 ++++++++- .../legacy-db-config.toml-read.unit.test.ts | 53 ++++++++++++++----- 2 files changed, 64 insertions(+), 15 deletions(-) diff --git a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts index 0693f6f22a..3c9c20b0d7 100644 --- a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts +++ b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts @@ -1851,6 +1851,26 @@ const readDbTomlCore = Effect.fnUntraced(function* ( const expanded = legacyExpandEnv(value, lookup); return expanded.length === 0 ? [] : expanded.split(","); }; + // Go decodes both `[db.seed].sql_paths` and `[db.migrations].schema_paths` as + // `config.Glob` (`[]string`) through the SAME mapstructure `UnmarshalExact` call + // (`config.go:749-756`), whose decoder config never sets `WeaklyTypedInput: false` + // — viper's `defaultDecoderConfig` defaults it to `true` and nothing here overrides + // it. So a non-string array element isn't dropped: `decodeString` + // (`github.com/go-viper/mapstructure/v2@v2.5.0/mapstructure.go:729-780`) weakly + // converts a bool to `"1"`/`"0"` and a number to its decimal string, THEN the + // result flows through the same env-expand/resolve pipeline as a real string + // entry. Verified empirically against `apps/cli-go` (`schema_paths = [42]` resolves + // to `supabase/42`, `schema_paths = [true]` to `supabase/1`). A non-scalar element + // (nested array/table) is mapstructure's `UnconvertibleTypeError`, which aborts the + // ENTIRE config load with `failed to parse config: ...` rather than dropping just + // that element — out of scope for a path list nobody nests a table inside, so it is + // filtered out like before rather than replicating mapstructure's decode-error text. + const legacyWeakCoerceGlobEntry = (value: unknown): string | undefined => { + if (typeof value === "string") return value; + if (typeof value === "boolean") return value ? "1" : "0"; + if (typeof value === "number") return String(value); + return undefined; + }; const rawSqlPaths = seedRaw?.["sql_paths"]; const sqlPathsOverride = remoteOverrideKeys.has("db.seed.sql_paths") ? undefined @@ -1860,7 +1880,8 @@ const readDbTomlCore = Effect.fnUntraced(function* ( ? splitGoSeedPaths(sqlPathsOverride) : Array.isArray(rawSqlPaths) ? rawSqlPaths - .filter((pattern): pattern is string => typeof pattern === "string") + .map((pattern) => legacyWeakCoerceGlobEntry(pattern)) + .filter((pattern): pattern is string => pattern !== undefined) .map((pattern) => legacyExpandEnv(pattern, lookup)) : typeof rawSqlPaths === "string" ? splitGoSeedPaths(rawSqlPaths) @@ -1883,7 +1904,8 @@ const readDbTomlCore = Effect.fnUntraced(function* ( ? splitGoSeedPaths(schemaPathsOverride) : Array.isArray(rawSchemaPaths) ? rawSchemaPaths - .filter((pattern): pattern is string => typeof pattern === "string") + .map((pattern) => legacyWeakCoerceGlobEntry(pattern)) + .filter((pattern): pattern is string => pattern !== undefined) .map((pattern) => legacyExpandEnv(pattern, lookup)) : typeof rawSchemaPaths === "string" ? splitGoSeedPaths(rawSchemaPaths) diff --git a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts index 70f34f8bdd..0fb7d5b5ed 100644 --- a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts @@ -293,6 +293,24 @@ describe("legacyReadDbToml", () => { ); }); + it.effect( + "weakly coerces non-string db.seed.sql_paths array elements (Go mapstructure parity)", + () => { + // Same `config.Glob` decode path as schema_paths below — a bool/number element + // is coerced to its Go string form ("1"/"0" for bool, decimal for a number), + // not dropped. + const dir = withConfig(["[db.seed]", 'sql_paths = [42, true, "seed.sql"]', ""].join("\n")); + return read(dir).pipe( + Effect.tap((v) => + Effect.sync(() => { + expect(v.seed.sqlPaths).toEqual(["supabase/42", "supabase/1", "supabase/seed.sql"]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + it.effect( "honors SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS over the TOML array (comma split, no trim)", () => { @@ -331,19 +349,28 @@ describe("legacyReadDbToml", () => { }, ); - it.effect("filters non-string db.migrations.schema_paths array elements", () => { - const dir = withConfig( - ["[db.migrations]", 'schema_paths = [42, "schemas/*.sql"]', ""].join("\n"), - ); - return read(dir).pipe( - Effect.tap((v) => - Effect.sync(() => { - expect(v.schemaPaths).toEqual(["supabase/schemas/*.sql"]); - rmSync(dir, { recursive: true, force: true }); - }), - ), - ); - }); + it.effect( + "weakly coerces non-string db.migrations.schema_paths array elements (Go mapstructure parity)", + () => { + // Go's `v.UnmarshalExact` never sets `WeaklyTypedInput: false`, so viper's + // `defaultDecoderConfig` default of `true` stands — mapstructure's `decodeString` + // coerces a bool to "1"/"0" and a number to its decimal string rather than + // erroring or dropping the element. Verified empirically against `apps/cli-go`: + // `schema_paths = [42, true, "schemas/*.sql"]` resolves to + // `supabase/{42,1,schemas/*.sql}`, not a filtered two-element list. + const dir = withConfig( + ["[db.migrations]", 'schema_paths = [42, true, "schemas/*.sql"]', ""].join("\n"), + ); + return read(dir).pipe( + Effect.tap((v) => + Effect.sync(() => { + expect(v.schemaPaths).toEqual(["supabase/42", "supabase/1", "supabase/schemas/*.sql"]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); it.effect( "an explicit remote db.migrations.schema_paths beats SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS", From 4024640aabafcb9b67c7fb45fc2fbab22cb60b3c Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Wed, 5 Aug 2026 13:07:43 +0100 Subject: [PATCH 07/47] fix(cli): propagate SQL-glob walk failures and gate schema-file suggestion to exec-phase (CLI-1958) Two Go-parity gaps in the experimental `db reset` remote schema-files path, both verified empirically against apps/cli-go: - legacySqlFilesGlob/legacyWalkSqlFiles silently treated a directory-read failure during walk as an empty match. Go's fs.WalkDir propagates a ReadDir error and walkMatchedDir wraps it as "failed to walk matched directory: ...", which applySchemaFiles only discards when at least one OTHER file was still found (declared non-empty); with nothing else matched, Go aborts before applying anything. The walk failure now surfaces as a warning via Effect.result/Result.isFailure, so the existing files.length === 0 gate correctly turns it fatal instead of reporting silent success after schemas are already dropped. - legacyApplySchemaFiles attached Go's CmdSuggestion ("See schema file: ...") to every legacyExecSqlFile failure, but Go's applySchemaFiles only sets CmdSuggestion after ExecBatch (statement execution) fails -- a NewMigrationFromFile (file-read) failure returns before CmdSuggestion is ever touched. execMigrationBatch's mapError callback now carries a "read"/"exec" phase tag so the suggestion is attached only on exec-phase failures. Extracted the ad hoc errMessage helper (legacy-migration-apply.ts) into shared legacy-error-message.ts so legacy-sql-files-glob.ts can reuse it without a circular import. --- .../db/reset/reset.integration.test.ts | 70 +++++- .../src/legacy/shared/legacy-error-message.ts | 11 + .../legacy/shared/legacy-migration-apply.ts | 200 ++++++++++-------- .../legacy/shared/legacy-sql-files-glob.ts | 37 +++- .../shared/legacy-sql-files-glob.unit.test.ts | 68 +++++- 5 files changed, 285 insertions(+), 101 deletions(-) create mode 100644 apps/cli/src/legacy/shared/legacy-error-message.ts diff --git a/apps/cli/src/legacy/commands/db/reset/reset.integration.test.ts b/apps/cli/src/legacy/commands/db/reset/reset.integration.test.ts index 45b7e82268..a8f5d19182 100644 --- a/apps/cli/src/legacy/commands/db/reset/reset.integration.test.ts +++ b/apps/cli/src/legacy/commands/db/reset/reset.integration.test.ts @@ -1,4 +1,4 @@ -import { mkdirSync, writeFileSync } from "node:fs"; +import { chmodSync, mkdirSync, writeFileSync } from "node:fs"; import { dirname, join } from "node:path"; import { BunServices } from "@effect/platform-bun"; @@ -1164,6 +1164,74 @@ describe("legacy db reset", () => { }, ); + const isRoot = typeof process.getuid === "function" && process.getuid() === 0; + + it.live.skipIf(isRoot)( + "does not attach the schema-file suggestion when a schema file cannot be READ on an experimental remote reset", + () => { + // Go's `NewMigrationFromFile` (the file-read/parse step, `apply.go:57-59`) returns + // BEFORE `CmdSuggestion` is ever set — only a later `ExecBatch` (statement + // execution) failure attaches it (`apply.go:61-63`). A file that glob-matches but + // can't be read (permissions changed after the glob) must fail WITHOUT the + // suggestion, unlike the exec-failure case above. + const schemaFile = join(tmp.current, "supabase", "schemas", "01_users.sql"); + const { layer, conn } = setup(tmp.current, { + toml: 'project_id = "test"\n\n[db.migrations]\nschema_paths = ["schemas/*.sql"]\n', + files: { "supabase/schemas/01_users.sql": "create table schema_users ();" }, + experimental: true, + confirm: [true], + }); + chmodSync(schemaFile, 0o000); + return Effect.gen(function* () { + const exit = yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe( + Effect.provide(layer), + Effect.exit, + ); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const cause = JSON.stringify(exit.cause); + expect(cause).not.toContain("See schema file"); + } + // The statement was never reached, so it was never executed. + expect(conn.execs.some((s) => s.includes("create table schema_users"))).toBe(false); + }).pipe(Effect.ensuring(Effect.sync(() => chmodSync(schemaFile, 0o644)))); + }, + ); + + it.live.skipIf(isRoot)( + "fails an experimental remote reset (without silently succeeding) when a matched schema_paths directory cannot be walked", + () => { + // Go's `fs.WalkDir` stops on the first `ReadDir` failure and `applySchemaFiles` + // only silently drops that error when at least one OTHER file was still found + // (`apply.go:53-55`); with a single pattern matching only the unreadable + // directory, `declared` stays empty and Go aborts the command — it must not + // report success having applied nothing. Verified empirically against `apps/cli-go`. + const schemasDir = join(tmp.current, "supabase", "schemas"); + const { layer, conn } = setup(tmp.current, { + toml: 'project_id = "test"\n\n[db.migrations]\nschema_paths = ["schemas"]\n', + files: { "supabase/schemas/01_users.sql": "create table schema_users ();" }, + experimental: true, + confirm: [true], + }); + chmodSync(schemasDir, 0o000); + return Effect.gen(function* () { + const exit = yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe( + Effect.provide(layer), + Effect.exit, + ); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const cause = JSON.stringify(exit.cause); + expect(cause).toContain("failed to walk matched directory"); + expect(cause).not.toContain("See schema file"); + } + // Schemas were already dropped before the failed apply step (Go's ResetAll order). + expect(conn.execs.some((s) => s.includes("drop schema if exists"))).toBe(true); + expect(conn.execs.some((s) => s.includes("create table schema_users"))).toBe(false); + }).pipe(Effect.ensuring(Effect.sync(() => chmodSync(schemasDir, 0o755)))); + }, + ); + it.live( "propagates the storage-ready check's exact exit code and still flushes telemetry on a local reset", () => { diff --git a/apps/cli/src/legacy/shared/legacy-error-message.ts b/apps/cli/src/legacy/shared/legacy-error-message.ts new file mode 100644 index 0000000000..d0b4a72bb7 --- /dev/null +++ b/apps/cli/src/legacy/shared/legacy-error-message.ts @@ -0,0 +1,11 @@ +/** + * Best-effort extraction of a human-readable message from an unknown thrown/failed + * value — an Effect `PlatformError`, a driver error, a plain `Error`, or anything else. + * Shared by every legacy module that wraps a raw Effect/driver failure into Go-style + * error text (Go's own `err.Error()` equivalent), so wording stays consistent across + * call sites instead of each one re-deriving its own fallback. + */ +export const legacyErrorMessage = (e: unknown): string => + typeof e === "object" && e !== null && "message" in e && typeof e.message === "string" + ? e.message + : String(e); diff --git a/apps/cli/src/legacy/shared/legacy-migration-apply.ts b/apps/cli/src/legacy/shared/legacy-migration-apply.ts index 0975a33d1c..d84e676108 100644 --- a/apps/cli/src/legacy/shared/legacy-migration-apply.ts +++ b/apps/cli/src/legacy/shared/legacy-migration-apply.ts @@ -4,6 +4,7 @@ import { Output } from "../../shared/output/output.service.ts"; import { legacyBold } from "./legacy-colors.ts"; import type { LegacyDbExecError } from "./legacy-db-connection.errors.ts"; import type { LegacyDbSession } from "./legacy-db-connection.service.ts"; +import { legacyErrorMessage } from "./legacy-error-message.ts"; import { INSERT_MIGRATION_VERSION, MIGRATE_FILE_PATTERN, @@ -99,11 +100,6 @@ type LegacyBatchItem = | { readonly kind: "exec"; readonly sql: string } | { readonly kind: "version" }; -const errMessage = (e: unknown): string => - typeof e === "object" && e !== null && "message" in e && typeof e.message === "string" - ? e.message - : String(e); - const utf8ByteLength = (value: string): number => new TextEncoder().encode(value).length; /** @@ -160,95 +156,110 @@ const execMigrationBatch = ( fs: FileSystem.FileSystem, path: Path.Path, migrationPath: string, - mapError: (message: string) => E, + mapError: (message: string, phase: "read" | "exec") => E, forceNoVersion: boolean, ): Effect.Effect => Effect.gen(function* () { - const content = yield* fs.readFileString(migrationPath); - const statements = legacySplitAndTrim(content); - const filename = path.basename(migrationPath); - const matches = MIGRATE_FILE_PATTERN.exec(filename); - const version = forceNoVersion ? "" : (matches?.[1] ?? ""); - const name = matches?.[2] ?? ""; + // Go's `MigrationFile.ExecBatch` receives an already-read/parsed file (the read + // happens earlier, in `NewMigrationFromFile`) — so a read failure here is a + // DIFFERENT error class than a statement-execution failure below. Tagged "read" so + // callers that attach a suggestion only around execution failures (`apply.go:61-63`) + // can tell the two apart. + const content = yield* fs + .readFileString(migrationPath) + .pipe(Effect.mapError((error) => mapError(legacyErrorMessage(error), "read"))); - // Mirror Go's `MigrationFile.ExecBatch` error context (`pkg/migration/file.go:88-113`): - // on a failed statement, render the `^` caret under the server-reported error - // position, the `Detail` line when present, the SQLSTATE-42704 extension hint, - // then `At statement: ` and the (caret-marked) statement text. The - // structured `detail`/`position` fields are only set by the driver for server - // ErrorResponses, mirroring Go's `errors.As(err, &pgErr)` gate. - const atStatement = (e: LegacyDbExecError, index: number, stat: string) => { - const marked = legacyMarkError(stat, e.position ?? 0); - const msg: Array = []; - if (e.detail !== undefined && e.detail.length > 0) { - msg.push(e.detail); - } - // Provide helpful hint for extension type errors (SQLSTATE 42704: undefined_object) - const typeName = TYPE_NAME_PATTERN.exec(e.message)?.[1]; - if (typeName !== undefined && e.code === "42704" && !typeName.includes(".")) { - msg.push(""); - msg.push("Hint: This type may be defined in a schema that's not in your search_path."); - msg.push(" Use schema-qualified type references to avoid this error:"); - msg.push(` CREATE TABLE example (col extensions.${typeName});`); - msg.push(" Learn more: supabase migration new --help"); - } - msg.push(`At statement: ${index}`, marked); - return new Error(`${errMessage(e)}\n${msg.join("\n")}`); - }; + // Everything below mirrors Go's `(*MigrationFile).ExecBatch` (`pkg/migration/file.go`), + // which runs against an already-read file — so every failure from here on is an + // execution failure, tagged "exec" (as opposed to the "read" failure above, which + // mirrors `NewMigrationFromFile`). Only execution failures get `CmdSuggestion` + // (`apply.go:61-63`); callers rely on this tag to replicate that split. + yield* Effect.gen(function* () { + const statements = legacySplitAndTrim(content); + const filename = path.basename(migrationPath); + const matches = MIGRATE_FILE_PATTERN.exec(filename); + const version = forceNoVersion ? "" : (matches?.[1] ?? ""); + const name = matches?.[2] ?? ""; + + // Mirror Go's `MigrationFile.ExecBatch` error context (`pkg/migration/file.go:88-113`): + // on a failed statement, render the `^` caret under the server-reported error + // position, the `Detail` line when present, the SQLSTATE-42704 extension hint, + // then `At statement: ` and the (caret-marked) statement text. The + // structured `detail`/`position` fields are only set by the driver for server + // ErrorResponses, mirroring Go's `errors.As(err, &pgErr)` gate. + const atStatement = (e: LegacyDbExecError, index: number, stat: string) => { + const marked = legacyMarkError(stat, e.position ?? 0); + const msg: Array = []; + if (e.detail !== undefined && e.detail.length > 0) { + msg.push(e.detail); + } + // Provide helpful hint for extension type errors (SQLSTATE 42704: undefined_object) + const typeName = TYPE_NAME_PATTERN.exec(e.message)?.[1]; + if (typeName !== undefined && e.code === "42704" && !typeName.includes(".")) { + msg.push(""); + msg.push("Hint: This type may be defined in a schema that's not in your search_path."); + msg.push(" Use schema-qualified type references to avoid this error:"); + msg.push(` CREATE TABLE example (col extensions.${typeName});`); + msg.push(" Learn more: supabase migration new --help"); + } + msg.push(`At statement: ${index}`, marked); + return new Error(`${legacyErrorMessage(e)}\n${msg.join("\n")}`); + }; - // `executed` is the global statement index of the next statement to run, so the - // error context stays accurate across flushed batches and standalone statements - // (Go threads the same counter through `ExecBatch`). - let pending: ReadonlyArray = []; - let executed = 0; + // `executed` is the global statement index of the next statement to run, so the + // error context stays accurate across flushed batches and standalone statements + // (Go threads the same counter through `ExecBatch`). + let pending: ReadonlyArray = []; + let executed = 0; - const flushBatch = Effect.gen(function* () { - if (pending.length === 0) return; - const items = pending; - pending = []; - const base = executed; - const body = Effect.gen(function* () { - for (const [offset, item] of items.entries()) { - const index = base + offset; - if (item.kind === "version") { - // Go defaults to the version-insert statement when all listed statements succeed. - yield* session - .query(INSERT_MIGRATION_VERSION, [version, name, statements]) - .pipe( - Effect.mapError((cause) => atStatement(cause, index, INSERT_MIGRATION_VERSION)), - ); - } else { - yield* session - .exec(item.sql) - .pipe(Effect.mapError((cause) => atStatement(cause, index, item.sql))); + const flushBatch = Effect.gen(function* () { + if (pending.length === 0) return; + const items = pending; + pending = []; + const base = executed; + const body = Effect.gen(function* () { + for (const [offset, item] of items.entries()) { + const index = base + offset; + if (item.kind === "version") { + // Go defaults to the version-insert statement when all listed statements succeed. + yield* session + .query(INSERT_MIGRATION_VERSION, [version, name, statements]) + .pipe( + Effect.mapError((cause) => atStatement(cause, index, INSERT_MIGRATION_VERSION)), + ); + } else { + yield* session + .exec(item.sql) + .pipe(Effect.mapError((cause) => atStatement(cause, index, item.sql))); + } } - } - yield* session.exec("COMMIT"); + yield* session.exec("COMMIT"); + }); + yield* session.exec("BEGIN"); + yield* body.pipe(Effect.tapError(() => session.exec("ROLLBACK").pipe(Effect.ignore))); + executed += items.length; }); - yield* session.exec("BEGIN"); - yield* body.pipe(Effect.tapError(() => session.exec("ROLLBACK").pipe(Effect.ignore))); - executed += items.length; - }); - for (const statement of statements) { - if (legacyIsPipelineIncompatible(statement)) { - // Flush the open batch, then run the incompatible statement on its own (no - // surrounding transaction) so PostgreSQL accepts it. - yield* flushBatch; - const index = executed; - yield* session - .exec(statement) - .pipe(Effect.mapError((cause) => atStatement(cause, index, statement))); - executed += 1; - } else { - pending = [...pending, { kind: "exec", sql: statement }]; + for (const statement of statements) { + if (legacyIsPipelineIncompatible(statement)) { + // Flush the open batch, then run the incompatible statement on its own (no + // surrounding transaction) so PostgreSQL accepts it. + yield* flushBatch; + const index = executed; + yield* session + .exec(statement) + .pipe(Effect.mapError((cause) => atStatement(cause, index, statement))); + executed += 1; + } else { + pending = [...pending, { kind: "exec", sql: statement }]; + } } - } - if (version.length > 0) { - pending = [...pending, { kind: "version" }]; - } - yield* flushBatch; - }).pipe(Effect.mapError((error) => mapError(errMessage(error)))); + if (version.length > 0) { + pending = [...pending, { kind: "version" }]; + } + yield* flushBatch; + }).pipe(Effect.mapError((error) => mapError(legacyErrorMessage(error), "exec"))); + }); /** * Go's per-migration connection reset (`apply.go:65-69`): `RESET ALL` clears any @@ -261,7 +272,7 @@ const resetConnectionState = ( session: LegacyDbSession, mapError: (message: string) => E, ): Effect.Effect => - session.exec("RESET ALL").pipe(Effect.mapError((e) => mapError(errMessage(e)))); + session.exec("RESET ALL").pipe(Effect.mapError((e) => mapError(legacyErrorMessage(e)))); /** * Applies a single migration file to the connected database and records it in @@ -283,7 +294,7 @@ export const legacyApplyMigrationFile = ( Effect.gen(function* () { yield* resetConnectionState(session, mapError); yield* legacyCreateMigrationTable(session).pipe( - Effect.mapError((e) => mapError(errMessage(e))), + Effect.mapError((e) => mapError(legacyErrorMessage(e))), ); yield* execMigrationBatch(session, fs, path, migrationPath, mapError, false); }); @@ -305,7 +316,7 @@ export const legacyApplyMigrations = ( const output = yield* Output; if (pending.length === 0) return; yield* legacyCreateMigrationTable(session).pipe( - Effect.mapError((e) => mapError(errMessage(e))), + Effect.mapError((e) => mapError(legacyErrorMessage(e))), ); for (const migrationPath of pending) { yield* output.raw(`Applying migration ${path.basename(migrationPath)}...\n`, "stderr"); @@ -355,7 +366,7 @@ export const legacyExecSqlFile = ( fs: FileSystem.FileSystem, path: Path.Path, filePath: string, - mapError: (message: string) => E, + mapError: (message: string, phase: "read" | "exec") => E, ): Effect.Effect => execMigrationBatch(session, fs, path, filePath, mapError, true); /** @@ -385,8 +396,11 @@ export const legacyExecSqlFile = ( * (`declared` empty, `apply.go:53-55`); once at least one file is found, every other * pattern's warning is discarded — unlike the seed path's `WARN:` line. * - * On a per-file exec failure, attaches Go's `CmdSuggestion = "See schema file: "` - * (`apply.go:63`) via the optional second argument of `mapError`. + * On a per-file EXECUTION failure only, attaches Go's `CmdSuggestion = "See schema file: + * "` (`apply.go:63`) via the optional second argument of `mapError`. A file-READ + * failure (Go's `NewMigrationFromFile`, `apply.go:57-59`) returns before `CmdSuggestion` is + * ever set, so it must NOT carry the suggestion — {@link legacyExecSqlFile}'s `mapError` + * receives the `"read"`/`"exec"` phase precisely so this call site can tell them apart. */ export const legacyApplySchemaFiles = ( session: LegacyDbSession, @@ -408,8 +422,10 @@ export const legacyApplySchemaFiles = ( } for (const file of files) { const absolutePath = path.isAbsolute(file) ? file : path.join(workdir, file); - yield* legacyExecSqlFile(session, fs, path, absolutePath, (message) => - mapError(message, `See schema file: ${legacyBold(file)}`), + yield* legacyExecSqlFile(session, fs, path, absolutePath, (message, phase) => + phase === "exec" + ? mapError(message, `See schema file: ${legacyBold(file)}`) + : mapError(message), ); } }); diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts index f35147155e..bb26a56a85 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts @@ -1,5 +1,6 @@ -import { Effect, type FileSystem, type Path } from "effect"; +import { Effect, type FileSystem, type Path, Result } from "effect"; +import { legacyErrorMessage } from "./legacy-error-message.ts"; import { LEGACY_BAD_PATTERN_MESSAGE, legacyPathMatch } from "./legacy-path-match.ts"; const META_CHARS = /[*?[\\]/u; @@ -72,21 +73,33 @@ const globOne = ( * filepath.Ext(path) == ".sql"`, `apps/cli-go/pkg/config/config.go:126-131,194-211`). * Paths are workdir-relative (matching the glob output), forward-slashed, and sorted * for deterministic application. + * + * A `ReadDir` failure anywhere in the tree (e.g. a permissions error on a nested + * directory) fails the WHOLE walk with `failed to walk matched directory: `, + * discarding every file collected so far — Go's `fs.WalkDir` callback returns that + * `err` unchanged, which stops the traversal immediately and makes `walkMatchedDir` + * return `(nil, err)` rather than the partial list (`config.go:196-208`; verified + * empirically: an unreadable matched directory makes `Glob.SQLFiles` return a + * `failed to walk matched directory: ...` error with zero files, not an empty match). */ const legacyWalkSqlFiles = ( fs: FileSystem.FileSystem, path: Path.Path, workdir: string, dir: string, -): Effect.Effect, never> => +): Effect.Effect, string> => Effect.gen(function* () { const collected: Array = []; - const walk = (rel: string): Effect.Effect => + const walk = (rel: string): Effect.Effect => Effect.gen(function* () { const absDir = path.isAbsolute(rel) ? rel : path.join(workdir, rel); const names = yield* fs .readDirectory(absDir) - .pipe(Effect.orElseSucceed((): ReadonlyArray => [])); + .pipe( + Effect.mapError( + (error) => `failed to walk matched directory: ${legacyErrorMessage(error)}`, + ), + ); for (const name of names) { const childRel = `${rel}/${name}`; const childAbs = path.isAbsolute(childRel) ? childRel : path.join(workdir, childRel); @@ -125,8 +138,9 @@ interface LegacySqlFilesGlobResult { /** Workdir-relative, forward-slashed matches, deduplicated in first-seen order across patterns. */ readonly files: ReadonlyArray; /** - * Per-pattern problems (`no files matched pattern: …` / `failed to glob files: …`), - * in pattern order. Never fatal by itself — callers decide when a warning matters + * Per-pattern/per-match problems (`no files matched pattern: …` / `failed to glob + * files: …` / `failed to walk matched directory: …`), in pattern order. Never fatal + * by itself — callers decide when a warning matters * (e.g. the seed path always surfaces it; the schema-files apply path only surfaces * it when NO pattern matched anything at all, mirroring Go's `apply.go:53-55`). */ @@ -225,7 +239,16 @@ export const legacySqlFilesGlob = Effect.fnUntraced(function* ( Effect.orElseSucceed(() => "File" as const), ); if (matchType === "Directory") { - for (const file of yield* legacyWalkSqlFiles(fs, path, workdir, fp)) { + // Go: `if err != nil { allErrors = append(allErrors, err); continue }` — a walk + // failure on this match becomes a warning (never a hard Effect failure, like + // every other per-match/per-pattern problem here) and the loop moves on to the + // next match, exactly like a malformed pattern or a "no files matched" miss. + const walked = yield* legacyWalkSqlFiles(fs, path, workdir, fp).pipe(Effect.result); + if (Result.isFailure(walked)) { + warnings.push(walked.failure); + continue; + } + for (const file of walked.success) { if (!seen.has(file)) { seen.add(file); files.push(file); diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts index 7fa4d05cb9..a7a86a7087 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts @@ -1,4 +1,4 @@ -import { mkdirSync, mkdtempSync, rmSync, symlinkSync, writeFileSync } from "node:fs"; +import { chmodSync, mkdirSync, mkdtempSync, rmSync, symlinkSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { BunServices } from "@effect/platform-bun"; @@ -101,4 +101,70 @@ describe("legacySqlFilesGlob", () => { ), ); }); + + const isRoot = typeof process.getuid === "function" && process.getuid() === 0; + + it.effect.skipIf(isRoot)( + "surfaces a directory-read failure during walk as a warning instead of an empty match (Go WalkDir parity)", + () => { + // Go's `fs.WalkDir` returns the `ReadDir` error from its walkFn unchanged, which + // stops the walk immediately; `walkMatchedDir` then wraps it as `failed to walk + // matched directory: ...` and discards every file already found — never an empty + // (successful) match. Verified empirically against `apps/cli-go`: an unreadable + // matched directory makes `Glob.SQLFiles` return that error with zero files. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-walk-fail-")); + const schemasDir = join(dir, "schemas"); + mkdirSync(schemasDir); + writeFileSync(join(schemasDir, "a.sql"), "select 1;"); + chmodSync(schemasDir, 0o000); + return run(["schemas"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual([]); + expect(result.warnings).toHaveLength(1); + expect(result.warnings[0]).toMatch(/^failed to walk matched directory: /); + }), + ), + Effect.ensuring( + Effect.sync(() => { + chmodSync(schemasDir, 0o755); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect.skipIf(isRoot)( + "keeps files from a sibling pattern when only one matched directory fails to walk", + () => { + // Go: `if err != nil { allErrors = append(allErrors, err); continue }` — a walk + // failure on one match doesn't stop the loop over the REST of the matches/patterns; + // whether it's ultimately fatal is the caller's decision (`legacyApplySchemaFiles`'s + // `len(declared) == 0` gate), not this function's. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-walk-fail-partial-")); + const goodDir = join(dir, "good"); + const badDir = join(dir, "bad"); + mkdirSync(goodDir); + mkdirSync(badDir); + writeFileSync(join(goodDir, "a.sql"), "select 1;"); + writeFileSync(join(badDir, "b.sql"), "select 2;"); + chmodSync(badDir, 0o000); + return run(["good", "bad"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["good/a.sql"]); + expect(result.warnings).toHaveLength(1); + expect(result.warnings[0]).toMatch(/^failed to walk matched directory: /); + }), + ), + Effect.ensuring( + Effect.sync(() => { + chmodSync(badDir, 0o755); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); }); From 1fa150808032bf97394a7f26205fb2eb5cfff66a Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Wed, 5 Aug 2026 13:07:55 +0100 Subject: [PATCH 08/47] fix(cli): abort config load on non-scalar schema_paths/sql_paths entries (CLI-1958) Go's config.Glob decode (UnmarshalExact, config.go:749-756) weakly coerces a bool/number array element but hits mapstructure's UnconvertibleTypeError for a non-scalar one (nested array/table), which aborts the ENTIRE config load with "failed to parse config: decoding failed due to the following error(s): ...". Verified empirically against apps/cli-go: `schema_paths = [[]]` / `[{path = "x.sql"}]` both fail config.Load with that exact message before any schema is dropped; multiple bad entries are aggregated in one message. legacyWeakCoerceGlobEntry previously filtered these elements out silently, which on an experimental remote reset could drop remote schemas and then apply zero files while reporting success. legacyReadDbToml now fails the whole config load with the byte-matching mapstructure-style message instead. --- .../shared/legacy-db-config.toml-read.ts | 44 +++++++++-- .../legacy-db-config.toml-read.unit.test.ts | 73 +++++++++++++++++++ 2 files changed, 112 insertions(+), 5 deletions(-) diff --git a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts index 3c9c20b0d7..ee9b86561e 100644 --- a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts +++ b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts @@ -1860,21 +1860,52 @@ const readDbTomlCore = Effect.fnUntraced(function* ( // converts a bool to `"1"`/`"0"` and a number to its decimal string, THEN the // result flows through the same env-expand/resolve pipeline as a real string // entry. Verified empirically against `apps/cli-go` (`schema_paths = [42]` resolves - // to `supabase/42`, `schema_paths = [true]` to `supabase/1`). A non-scalar element - // (nested array/table) is mapstructure's `UnconvertibleTypeError`, which aborts the - // ENTIRE config load with `failed to parse config: ...` rather than dropping just - // that element — out of scope for a path list nobody nests a table inside, so it is - // filtered out like before rather than replicating mapstructure's decode-error text. + // to `supabase/42`, `schema_paths = [true]` to `supabase/1`). const legacyWeakCoerceGlobEntry = (value: unknown): string | undefined => { if (typeof value === "string") return value; if (typeof value === "boolean") return value ? "1" : "0"; if (typeof value === "number") return String(value); return undefined; }; + // A non-scalar element (nested array/table, e.g. `schema_paths = [[]]` or + // `[{path = "x.sql"}]`) is mapstructure's `UnconvertibleTypeError` instead of a weak + // conversion, and mapstructure reports every offending index from the SAME + // `UnmarshalExact` call together, joined by its own `Error.Error()` — which is what + // aborts the entire config load, not just that one array. Verified empirically + // against `apps/cli-go`: `schema_paths = [[]]` / `[{path = "x.sql"}]` both fail with + // `failed to parse config: decoding failed due to the following error(s):\n\n'db. + // migrations.schema_paths[0]' expected type 'string', got unconvertible type + // '[]interface {}'` / `'map[string]interface {}'` respectively — never silently + // dropping the element and continuing with an empty/partial glob list. + const legacyGoUnconvertibleType = (value: unknown): string | undefined => + Array.isArray(value) + ? "[]interface {}" + : typeof value === "object" && value !== null + ? "map[string]interface {}" + : undefined; + const legacyFailOnUnconvertibleGlobEntries = ( + keyPath: string, + values: ReadonlyArray, + ): Effect.Effect => { + const issues = values.flatMap((value, index) => { + const goType = legacyGoUnconvertibleType(value); + return goType === undefined + ? [] + : [`'${keyPath}[${index}]' expected type 'string', got unconvertible type '${goType}'`]; + }); + return issues.length === 0 + ? Effect.void + : fail( + `failed to parse config: decoding failed due to the following error(s):\n\n${issues.join("\n")}`, + ); + }; const rawSqlPaths = seedRaw?.["sql_paths"]; const sqlPathsOverride = remoteOverrideKeys.has("db.seed.sql_paths") ? undefined : envOverride("SUPABASE_DB_SEED_SQL_PATHS"); + if (sqlPathsOverride === undefined && Array.isArray(rawSqlPaths)) { + yield* legacyFailOnUnconvertibleGlobEntries("db.seed.sql_paths", rawSqlPaths); + } const sqlPathPatterns = sqlPathsOverride !== undefined ? splitGoSeedPaths(sqlPathsOverride) @@ -1899,6 +1930,9 @@ const readDbTomlCore = Effect.fnUntraced(function* ( const schemaPathsOverride = remoteOverrideKeys.has("db.migrations.schema_paths") ? undefined : envOverride("SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS"); + if (schemaPathsOverride === undefined && Array.isArray(rawSchemaPaths)) { + yield* legacyFailOnUnconvertibleGlobEntries("db.migrations.schema_paths", rawSchemaPaths); + } const schemaPathPatterns = schemaPathsOverride !== undefined ? splitGoSeedPaths(schemaPathsOverride) diff --git a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts index 0fb7d5b5ed..61bc3165c4 100644 --- a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts @@ -372,6 +372,79 @@ describe("legacyReadDbToml", () => { }, ); + it.effect( + "aborts the whole config load on a non-scalar db.migrations.schema_paths element (Go mapstructure UnconvertibleTypeError)", + () => { + // Unlike a bool/number (weakly coerced above), a nested array/table is + // mapstructure's `UnconvertibleTypeError`, which fails `UnmarshalExact` entirely + // rather than dropping just that element. Verified empirically against + // `apps/cli-go`: `schema_paths = [[]]` fails config load with exactly this + // message, never resolving to an empty/partial glob list. + const dir = withConfig(["[db.migrations]", "schema_paths = [[]]", ""].join("\n")); + return read(dir).pipe( + Effect.exit, + Effect.tap((exit) => + Effect.sync(() => { + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + expect(JSON.stringify(exit.cause)).toContain( + "failed to parse config: decoding failed due to the following error(s):\\n\\n'db.migrations.schema_paths[0]' expected type 'string', got unconvertible type '[]interface {}'", + ); + } + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "aborts the whole config load on a table db.migrations.schema_paths element, reporting every bad index (Go mapstructure parity)", + () => { + // Verified empirically against `apps/cli-go`: a second bad entry is reported + // alongside the first (mapstructure aggregates every `UnmarshalExact` error from + // the same decode call), and an inline table decodes as `map[string]interface {}`. + const dir = withConfig( + ["[db.migrations]", 'schema_paths = ["schemas/*.sql", { path = "x.sql" }]', ""].join("\n"), + ); + return read(dir).pipe( + Effect.exit, + Effect.tap((exit) => + Effect.sync(() => { + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + expect(JSON.stringify(exit.cause)).toContain( + "'db.migrations.schema_paths[1]' expected type 'string', got unconvertible type 'map[string]interface {}'", + ); + } + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "aborts the whole config load on a non-scalar db.seed.sql_paths element (same UnmarshalExact call as schema_paths)", + () => { + const dir = withConfig(["[db.seed]", "sql_paths = [[]]", ""].join("\n")); + return read(dir).pipe( + Effect.exit, + Effect.tap((exit) => + Effect.sync(() => { + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + expect(JSON.stringify(exit.cause)).toContain( + "'db.seed.sql_paths[0]' expected type 'string', got unconvertible type '[]interface {}'", + ); + } + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + it.effect( "an explicit remote db.migrations.schema_paths beats SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS", () => { From e781d2e5ba470890189e8ad3dde9760c90665767 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Wed, 5 Aug 2026 14:13:54 +0100 Subject: [PATCH 09/47] fix(cli): weakly coerce top-level scalar schema_paths/sql_paths (CLI-1958) Go's mapstructure decode is weakly typed on the whole `[]string` field, not just on array elements: a top-level scalar (e.g. `schema_paths = 42`) is wrapped into a synthetic single-element array and decoded through the same per-element rules as a real array entry, and a non-empty table fails with the same unconvertible-type error an array element would. The native reader only applied that weak coercion to array elements, so a scalar fell through to the empty/default fallback instead of resolving (and potentially warning) like Go does. Verified empirically against apps/cli-go. --- .../shared/legacy-db-config.toml-read.ts | 44 +++++++++- .../legacy-db-config.toml-read.unit.test.ts | 85 +++++++++++++++++++ 2 files changed, 127 insertions(+), 2 deletions(-) diff --git a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts index ee9b86561e..32441390de 100644 --- a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts +++ b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts @@ -1899,6 +1899,37 @@ const readDbTomlCore = Effect.fnUntraced(function* ( `failed to parse config: decoding failed due to the following error(s):\n\n${issues.join("\n")}`, ); }; + // A TOP-LEVEL raw value that is neither an array nor a string (e.g. + // `schema_paths = 42`/`true`, or a stray inline table) still reaches + // mapstructure's `decodeSlice`, which is weakly typed the same way an array + // ELEMENT is (see `legacyWeakCoerceGlobEntry` above): a zero-length map + // decodes straight to an empty slice; anything else is wrapped into a + // synthetic single-element `[]any{value}` and decoded through the exact + // same per-element rules as a real array entry — a scalar weakly coerces, + // an unconvertible value (map/array) fails with `'[0]' expected + // type 'string', got unconvertible type '...'` (mapstructure always + // reports the synthetic wrapped index, which is `0`). Verified empirically + // against `apps/cli-go`: `schema_paths = 42` → `["42"]`, `= true` → + // `["1"]`, `= {}` → `[]`, `[db.migrations.schema_paths]\nfoo = "bar"` → + // `failed to parse config: … 'db.migrations.schema_paths[0]' expected type + // 'string', got unconvertible type 'map[string]interface {}'`. Never + // called with `undefined` — an absent key has its own Go-matching default + // per caller below, so callers guard that case before reaching here. + const legacyResolveScalarGlobFallback = ( + keyPath: string, + value: unknown, + ): Effect.Effect, LegacyDbConfigLoadError> => + Effect.gen(function* () { + if (typeof value === "object" && value !== null && Object.keys(value).length === 0) { + return []; + } + const coerced = legacyWeakCoerceGlobEntry(value); + if (coerced !== undefined) { + return [coerced]; + } + yield* legacyFailOnUnconvertibleGlobEntries(keyPath, [value]); + return []; + }); const rawSqlPaths = seedRaw?.["sql_paths"]; const sqlPathsOverride = remoteOverrideKeys.has("db.seed.sql_paths") ? undefined @@ -1916,7 +1947,11 @@ const readDbTomlCore = Effect.fnUntraced(function* ( .map((pattern) => legacyExpandEnv(pattern, lookup)) : typeof rawSqlPaths === "string" ? splitGoSeedPaths(rawSqlPaths) - : ["seed.sql"]; + : rawSqlPaths === undefined + ? ["seed.sql"] + : (yield* legacyResolveScalarGlobFallback("db.seed.sql_paths", rawSqlPaths)).map( + (pattern) => legacyExpandEnv(pattern, lookup), + ); // Patterns are already env-expanded above (Go's LoadEnvHook runs before the split); // resolve each to Go's config-load form (absolute verbatim, relative supabase-joined). const seedSqlPaths = sqlPathPatterns.map((pattern) => legacyResolveSeedSqlPath(path, pattern)); @@ -1943,7 +1978,12 @@ const readDbTomlCore = Effect.fnUntraced(function* ( .map((pattern) => legacyExpandEnv(pattern, lookup)) : typeof rawSchemaPaths === "string" ? splitGoSeedPaths(rawSchemaPaths) - : []; + : rawSchemaPaths === undefined + ? [] + : (yield* legacyResolveScalarGlobFallback( + "db.migrations.schema_paths", + rawSchemaPaths, + )).map((pattern) => legacyExpandEnv(pattern, lookup)); const schemaPaths = schemaPathPatterns.map((pattern) => legacyResolveSeedSqlPath(path, pattern)); // `[db.vault]` secrets: env-expand each value, then decrypt dotenvx `encrypted:` diff --git a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts index 61bc3165c4..898e99763c 100644 --- a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts @@ -372,6 +372,91 @@ describe("legacyReadDbToml", () => { }, ); + it.effect( + "weakly coerces a TOP-LEVEL scalar db.migrations.schema_paths (Go mapstructure weak-decode of a []string field)", + () => { + // Go's `decodeSlice` wraps a non-array/non-string value into a synthetic + // single-element `[]any{value}` and decodes it through the same per-element + // rules as a real array entry — it does NOT fall back to the `[]` default the + // way an absent key does. Verified empirically against `apps/cli-go`: + // `schema_paths = 42` → `["42"]` (resolves to `supabase/42`), `= true` → `["1"]`. + const dirNumber = withConfig(["[db.migrations]", "schema_paths = 42", ""].join("\n")); + const dirBool = withConfig(["[db.migrations]", "schema_paths = true", ""].join("\n")); + return Effect.all([read(dirNumber), read(dirBool)]).pipe( + Effect.tap(([numberResult, boolResult]) => + Effect.sync(() => { + expect(numberResult.schemaPaths).toEqual(["supabase/42"]); + expect(boolResult.schemaPaths).toEqual(["supabase/1"]); + rmSync(dirNumber, { recursive: true, force: true }); + rmSync(dirBool, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "treats a TOP-LEVEL empty-table db.migrations.schema_paths as no patterns (Go mapstructure zero-length-map special case)", + () => { + // Go's `decodeSlice` special-cases a zero-length map BEFORE the generic weak-typing + // wrap above: it decodes straight to an empty slice. Verified empirically against + // `apps/cli-go`: `schema_paths = {}` → `[]`. + const dir = withConfig(["[db.migrations]", "schema_paths = {}", ""].join("\n")); + return read(dir).pipe( + Effect.tap((v) => + Effect.sync(() => { + expect(v.schemaPaths).toEqual([]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "aborts the whole config load on a TOP-LEVEL table db.migrations.schema_paths (Go mapstructure UnconvertibleTypeError, synthetic index 0)", + () => { + // A non-empty map isn't weakly coercible, so Go's `decodeSlice` wraps it into + // `[]any{value}` and fails decoding element 0 the same way a nested-array/table + // ARRAY element does. Verified empirically against `apps/cli-go`: + // `[db.migrations.schema_paths]\nfoo = "bar"` fails with this exact message. + const dir = withConfig(["[db.migrations.schema_paths]", 'foo = "bar"', ""].join("\n")); + return read(dir).pipe( + Effect.exit, + Effect.tap((exit) => + Effect.sync(() => { + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + expect(JSON.stringify(exit.cause)).toContain( + "'db.migrations.schema_paths[0]' expected type 'string', got unconvertible type 'map[string]interface {}'", + ); + } + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "weakly coerces a TOP-LEVEL scalar db.seed.sql_paths instead of falling back to the ['seed.sql'] default", + () => { + // The absent-key default (`["seed.sql"]`) only applies when the key is missing + // entirely — a PRESENT scalar still goes through Go's weak-decode wrap, same as + // schema_paths above. Verified empirically against `apps/cli-go`: + // `[db.seed]\nenabled = true\nsql_paths = 42` → `["42"]`, not `["seed.sql"]`. + const dir = withConfig(["[db.seed]", "enabled = true", "sql_paths = 42", ""].join("\n")); + return read(dir).pipe( + Effect.tap((v) => + Effect.sync(() => { + expect(v.seed.sqlPaths).toEqual(["supabase/42"]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + it.effect( "aborts the whole config load on a non-scalar db.migrations.schema_paths element (Go mapstructure UnconvertibleTypeError)", () => { From efbc9a070b44bf0ed71385f7676db324860e4139 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Wed, 5 Aug 2026 14:14:04 +0100 Subject: [PATCH 10/47] fix(cli): stop misreading stat failures and absolute glob roots (CLI-1958) Two Go-parity gaps in the shared SQL-file globber: - A matched path that fails to stat (a broken symlink, or a file that disappears between the glob and the stat) was falling back to treating it as a regular file. Go's Glob.SQLFiles records a "failed to stat matched file" warning and skips the path instead; the fallback here let a later read of the nonexistent path turn a warned-but-otherwise-successful reset into a hard apply error. - Splitting an absolute pattern whose meta character is in the first path component (e.g. `/*.sql`, `/tmp*/*.sql`) collapsed the root directory to `""`, which the globber treats as "use the workdir". Go's real runtime glob path (afero.IOFS.Glob -> afero.Glob) explicitly special-cases a bare `/` and keeps it, so the pattern resolves against the filesystem root, not cwd. Verified empirically against apps/cli-go. --- .../legacy/shared/legacy-sql-files-glob.ts | 46 +++++++++-- .../shared/legacy-sql-files-glob.unit.test.ts | 79 +++++++++++++++++++ 2 files changed, 118 insertions(+), 7 deletions(-) diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts index bb26a56a85..3d1553529b 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts @@ -18,10 +18,28 @@ const GLOB_META_CHARS = /[*?[]/u; // that escape — see `legacyPathMatch`'s escape handling below. const toSlash = (p: string): string => (process.platform === "win32" ? p.replaceAll("\\", "/") : p); -/** Splits a forward-slashed path into its directory prefix and final element. */ +/** + * Splits a forward-slashed path into its directory prefix and final element. + * + * A bare root prefix is kept as `"/"`, never chopped to `""`. This mirrors + * the real runtime glob path — `config.Glob.SQLFiles`'s `fs.Glob` call + * resolves to `afero.IOFS.Glob` (it implements `fs.GlobFS`), which delegates + * to `afero.Glob`/`match.go`'s `filepath.Split` followed by a switch on + * `dir` that leaves a bare `filepath.Separator` alone — every OTHER trailing + * separator is chopped, but the root one is deliberately preserved. Verified + * empirically against `apps/cli-go`: with cwd elsewhere, a pattern rooted at + * `/`, with a metacharacter in the FIRST component after the root slash + * (e.g. `tmp` + wildcard + `probe-dir` + wildcard + `.sql`), still resolves + * that first component against the filesystem ROOT, not cwd. Collapsing this + * to `dir: ""` would make `globOne` below treat such an absolute root-level + * pattern as relative to the workdir instead of the filesystem root. + */ const splitPath = (p: string): { readonly dir: string; readonly file: string } => { const slash = p.lastIndexOf("/"); - return slash === -1 ? { dir: "", file: p } : { dir: p.slice(0, slash), file: p.slice(slash + 1) }; + if (slash === -1) return { dir: "", file: p }; + return slash === 0 + ? { dir: "/", file: p.slice(1) } + : { dir: p.slice(0, slash), file: p.slice(slash + 1) }; }; /** Faithful port of Go's `fs.Glob` for one pattern, rooted at `workdir`. */ @@ -60,7 +78,10 @@ const globOne = ( .pipe(Effect.orElseSucceed((): ReadonlyArray => [])); for (const name of names) { if (legacyPathMatch(file, name).matched) { - result.push(d === "" ? name : `${d}/${name}`); + // `d === "/"` is the filesystem root (from `splitPath`'s preserved root + // prefix, above) — join without a doubled slash, matching Go's + // `filepath.Join("/", name)`. + result.push(d === "" ? name : d === "/" ? `/${name}` : `${d}/${name}`); } } } @@ -234,10 +255,21 @@ export const legacySqlFilesGlob = Effect.fnUntraced(function* ( const fp = toSlash(match); // A directory match is expanded to its regular `.sql` files recursively // (`walkMatchedDir`, sorted); a file match is kept verbatim (`config.go:157-183`). - const matchType = yield* fs.stat(path.isAbsolute(fp) ? fp : path.join(workdir, fp)).pipe( - Effect.map((info) => info.type), - Effect.orElseSucceed(() => "File" as const), - ); + // Go: `if err != nil { allErrors = append(allErrors, errors.Errorf("failed to + // stat matched file: %w", err)); continue }` (`config.go:157-161`) — a match + // that disappears (or is a broken symlink) between the glob and this stat + // becomes a warning and is skipped, same as a walk failure below. Falling back + // to treating it as a regular file (the previous behaviour here) would instead + // hand a nonexistent path to the caller's later read, turning a warned-but- + // otherwise-successful reset into a hard apply error. + const statResult = yield* fs + .stat(path.isAbsolute(fp) ? fp : path.join(workdir, fp)) + .pipe(Effect.result); + if (Result.isFailure(statResult)) { + warnings.push(`failed to stat matched file: ${legacyErrorMessage(statResult.failure)}`); + continue; + } + const matchType = statResult.success.type; if (matchType === "Directory") { // Go: `if err != nil { allErrors = append(allErrors, err); continue }` — a walk // failure on this match becomes a warning (never a hard Effect failure, like diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts index a7a86a7087..537d0d7ffa 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts @@ -84,6 +84,85 @@ describe("legacySqlFilesGlob", () => { }, ); + it.effect( + "surfaces a stat failure on a matched file as a warning instead of treating it as a regular file (Go parity)", + () => { + // Go: `if info, err := fs.Stat(fsys, fp); err != nil { allErrors = append(allErrors, + // errors.Errorf("failed to stat matched file: %w", err)); continue }` (config.go:157-161) + // — a match that disappears (or is a broken symlink) between the glob and this stat + // becomes a warning and is skipped entirely, never silently treated as a regular file. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-stat-fail-")); + const schemasDir = join(dir, "schemas"); + mkdirSync(schemasDir); + writeFileSync(join(schemasDir, "good.sql"), "select 1;"); + symlinkSync(join(schemasDir, "does-not-exist.sql"), join(schemasDir, "broken.sql")); + return run(["schemas/*.sql"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["schemas/good.sql"]); + expect(result.warnings).toHaveLength(1); + expect(result.warnings[0]).toMatch(/^failed to stat matched file: /); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "keeps a bare root ('/') as the directory when a glob pattern's meta character is in the first path component (Go afero.Glob parity)", + () => { + // Go's real runtime glob path — `config.Glob.SQLFiles`'s `fs.Glob` call resolves to + // `afero.IOFS.Glob` (it implements `fs.GlobFS`), which delegates to `afero.Glob` + // (`match.go`): `filepath.Split` followed by a switch that leaves a bare + // `filepath.Separator` alone — every OTHER trailing separator is chopped, but the + // root one is deliberately preserved. Verified empirically against `apps/cli-go`: + // with cwd elsewhere, a pattern rooted at "/" with a metacharacter in the first + // component after the root slash still resolves against the filesystem ROOT, not + // cwd. A canary file placed in the WORKDIR (never the real "/") proves this native + // port does not fall back to treating the root component as workdir-relative. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-abs-root-")); + writeFileSync(join(dir, "__legacy_sql_glob_canary__.sql"), "select 1;"); + return run(["/*__legacy_sql_glob_canary__*.sql"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual([]); + expect(result.warnings).toEqual([ + "no files matched pattern: /*__legacy_sql_glob_canary__*.sql", + ]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "recurses through a root-anchored directory component without falling back to the workdir (Go afero.Glob parity)", + () => { + // Same bug as above, but for a two-level pattern (`/foo*/*.sql`) — the recursive + // call that resolves the "foo*" directory component must also treat "/" as the + // real filesystem root, not "" (which `globOne` maps to the workdir). The workdir + // here contains a subdirectory that WOULD match "foo*" if (and only if) the + // recursive call incorrectly fell back to reading the workdir instead of "/". + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-abs-root-nested-")); + const canaryDir = join(dir, "__legacy_sql_glob_root_canary_dir__"); + mkdirSync(canaryDir); + writeFileSync(join(canaryDir, "a.sql"), "select 1;"); + return run(["/__legacy_sql_glob_root_canary_dir__*/*.sql"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual([]); + expect(result.warnings).toEqual([ + "no files matched pattern: /__legacy_sql_glob_root_canary_dir__*/*.sql", + ]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + it.effect("still expands a real (non-symlinked) nested directory recursively", () => { const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-nested-")); const schemasDir = join(dir, "schemas"); From 971b6660f84f586f677daef4cb772a0280eae2ce Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Wed, 5 Aug 2026 15:08:35 +0100 Subject: [PATCH 11/47] fix(cli): normalize doubled slash under trailing-slash matched dirs (review: CLI-1958) Go's fs.WalkDir builds each child path via path.Join, which runs path.Clean and collapses a doubled `/`. The native walkMatchedDir port instead string-concatenated `${rel}/${name}`, so a literal schema_paths/sql_paths directory entry ending in `/` (e.g. "/tmp/schemas/") produced "/tmp/schemas//a.sql" instead of Go's "/tmp/schemas/a.sql" for every child. Verified empirically: a scratch apps/cli-go probe calling config.Glob{"/"}.SQLFiles(...) against a real trailing-slash directory returns the single-slash path. --- .../legacy/shared/legacy-sql-files-glob.ts | 15 ++++++++++- .../shared/legacy-sql-files-glob.unit.test.ts | 27 +++++++++++++++++++ 2 files changed, 41 insertions(+), 1 deletion(-) diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts index 3d1553529b..6901ee0126 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts @@ -103,6 +103,19 @@ const globOne = ( * empirically: an unreadable matched directory makes `Glob.SQLFiles` return a * `failed to walk matched directory: ...` error with zero files, not an empty match). */ +// Go's `fs.WalkDir` builds each child path via `path.Join(dirname, name)` +// (`io/fs/walk.go`), and `path.Join` runs `path.Clean` on the joined result — which +// collapses a doubled `/` down to one. A bare string-concatenated `${rel}/${name}` +// does NOT collapse it, so a matched directory whose OWN path already ends in `/` +// (e.g. a literal `schema_paths = ["/tmp/schemas/"]`/`sql_paths` entry — `fs.Glob`'s +// no-metacharacter fast path returns that pattern verbatim, trailing slash and all) +// would otherwise produce `/tmp/schemas//a.sql` here instead of Go's +// `/tmp/schemas/a.sql`. Verified empirically: an `apps/cli-go` probe importing +// `pkg/config` directly and calling `Glob{"/"}.SQLFiles(...)` against a +// trailing-slash absolute directory returns the single-slash path, not a doubled one. +const joinRelChild = (rel: string, name: string): string => + rel.endsWith("/") ? `${rel.replace(/\/+$/, "")}/${name}` : `${rel}/${name}`; + const legacyWalkSqlFiles = ( fs: FileSystem.FileSystem, path: Path.Path, @@ -122,7 +135,7 @@ const legacyWalkSqlFiles = ( ), ); for (const name of names) { - const childRel = `${rel}/${name}`; + const childRel = joinRelChild(rel, name); const childAbs = path.isAbsolute(childRel) ? childRel : path.join(workdir, childRel); // Go's `fs.WalkDir` types each child from the parent's `ReadDir` entry // (`os.ReadDir`'s Lstat-based `DirEntry`) and never re-`Stat`s through it — diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts index 537d0d7ffa..42b58422a1 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts @@ -163,6 +163,33 @@ describe("legacySqlFilesGlob", () => { }, ); + it.effect( + "normalizes a doubled slash when the matched directory itself has a trailing slash (Go path.Join parity)", + () => { + // Go's `fs.WalkDir` builds each child path via `path.Join(dirname, name)` + // (`io/fs/walk.go`), and `path.Join` runs `path.Clean` on the result, collapsing a + // doubled `/`. A literal (no-metacharacter) `schema_paths`/`sql_paths` entry like + // `"schemas/"` resolves via `fs.Glob`'s fast path to the pattern VERBATIM, trailing + // slash and all — so the walk over its children must not produce `schemas//a.sql`. + // Verified empirically against `apps/cli-go`: a scratch probe calling + // `config.Glob{"/"}.SQLFiles(...)` on a real trailing-slash directory returns + // the single-slash path, not a doubled one. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-trailing-slash-")); + const schemasDir = join(dir, "schemas"); + mkdirSync(schemasDir); + writeFileSync(join(schemasDir, "a.sql"), "select 1;"); + return run(["schemas/"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["schemas/a.sql"]); + expect(result.warnings).toEqual([]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + it.effect("still expands a real (non-symlinked) nested directory recursively", () => { const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-nested-")); const schemasDir = join(dir, "schemas"); From 1dabdb6ade2532c444c33a200cb15e4caec9c64d Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Wed, 5 Aug 2026 15:08:45 +0100 Subject: [PATCH 12/47] docs(cli): document SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS in db reset SIDE_EFFECTS (review: CLI-1958) The env var table omitted SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS even though this change makes the native remote --experimental schema-files branch resolve [db.migrations].schema_paths through legacy-db-config.toml-read.ts's generic AutomaticEnv-override reader (LEGACY_ENV_OVERRIDABLE_KEYS), which this PR newly added for that key. No dedicated CLI flag exists for schema_paths, so the env var is the only non-config-file override surface. --- .../src/legacy/commands/db/reset/SIDE_EFFECTS.md | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md b/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md index ea6afa5889..98a0189af9 100644 --- a/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md +++ b/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md @@ -82,13 +82,14 @@ races a restarting gateway. ## Environment Variables -| Variable | Purpose | Required? | -| ----------------------- | ----------------------------------------------- | ------------------------------------------------------- | -| `SUPABASE_ACCESS_TOKEN` | auth token for the `--linked` resolver path | no (falls back to keyring → `~/.supabase/access-token`) | -| `SUPABASE_DB_PASSWORD` | password for the linked/remote connection | no | -| `SUPABASE_YES` | auto-confirm the reset prompt | no (also `--yes`) | -| `SUPABASE_EXPERIMENTAL` | selects the remote schema-files apply branch | no (also `--experimental`) | -| `SUPABASE_PROJECT_ID` | overrides the local container id (`utils.DbId`) | no | +| Variable | Purpose | Required? | +| ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------- | +| `SUPABASE_ACCESS_TOKEN` | auth token for the `--linked` resolver path | no (falls back to keyring → `~/.supabase/access-token`) | +| `SUPABASE_DB_PASSWORD` | password for the linked/remote connection | no | +| `SUPABASE_YES` | auto-confirm the reset prompt | no (also `--yes`) | +| `SUPABASE_EXPERIMENTAL` | selects the remote schema-files apply branch | no (also `--experimental`) | +| `SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS` | overrides `[db.migrations].schema_paths` (viper `AutomaticEnv`, beats the config-file value) for the schema-files apply branch | no (no dedicated flag — config-file-only otherwise) | +| `SUPABASE_PROJECT_ID` | overrides the local container id (`utils.DbId`) | no | ## Exit Codes From 2502f3544fe7bb9132de30e500f6ff93bf66eef3 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Wed, 5 Aug 2026 15:43:13 +0100 Subject: [PATCH 13/47] fix(cli): drop './' prefix when a walked SQL directory cleans to '.' (review: CLI-1958) Go's fs.WalkDir joins child paths with path.Join, which cleans a bare `.` root away entirely. schema_paths/sql_paths entries like [".."] can resolve to exactly "." via config's own path.Join(SupabaseDirPath, ..), so joinRelChild must special-case rel === "." to match Go's foo.sql instead of ./foo.sql -- otherwise the seed_files.path hash key diverges between Go and native tooling, causing seeds to needlessly re-run. --- .../legacy/shared/legacy-sql-files-glob.ts | 38 +++++++++++++------ .../shared/legacy-sql-files-glob.unit.test.ts | 31 +++++++++++++++ 2 files changed, 58 insertions(+), 11 deletions(-) diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts index 6901ee0126..3ed12ac33e 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts @@ -104,17 +104,33 @@ const globOne = ( * `failed to walk matched directory: ...` error with zero files, not an empty match). */ // Go's `fs.WalkDir` builds each child path via `path.Join(dirname, name)` -// (`io/fs/walk.go`), and `path.Join` runs `path.Clean` on the joined result — which -// collapses a doubled `/` down to one. A bare string-concatenated `${rel}/${name}` -// does NOT collapse it, so a matched directory whose OWN path already ends in `/` -// (e.g. a literal `schema_paths = ["/tmp/schemas/"]`/`sql_paths` entry — `fs.Glob`'s -// no-metacharacter fast path returns that pattern verbatim, trailing slash and all) -// would otherwise produce `/tmp/schemas//a.sql` here instead of Go's -// `/tmp/schemas/a.sql`. Verified empirically: an `apps/cli-go` probe importing -// `pkg/config` directly and calling `Glob{"/"}.SQLFiles(...)` against a -// trailing-slash absolute directory returns the single-slash path, not a doubled one. -const joinRelChild = (rel: string, name: string): string => - rel.endsWith("/") ? `${rel.replace(/\/+$/, "")}/${name}` : `${rel}/${name}`; +// (`io/fs/walk.go`), and `path.Join` runs `path.Clean` on the joined result, which: +// +// 1. Collapses a doubled `/` down to one. A bare string-concatenated `${rel}/${name}` +// does NOT collapse it, so a matched directory whose OWN path already ends in `/` +// (e.g. a literal `schema_paths = ["/tmp/schemas/"]`/`sql_paths` entry — `fs.Glob`'s +// no-metacharacter fast path returns that pattern verbatim, trailing slash and all) +// would otherwise produce `/tmp/schemas//a.sql` here instead of Go's +// `/tmp/schemas/a.sql`. Verified empirically: an `apps/cli-go` probe importing +// `pkg/config` directly and calling `Glob{"/"}.SQLFiles(...)` against a +// trailing-slash absolute directory returns the single-slash path, not a doubled one. +// 2. Drops a bare `.` root entirely rather than joining it as a prefix. A matched +// directory can clean to exactly `.` — e.g. `[db.migrations].schema_paths = [".."]` +// or `[db.seed].sql_paths = [".."]`, which `baseConfig.resolve`'s own +// `path.Join(builder.SupabaseDirPath, pattern)` collapses to `.` +// (`apps/cli-go/pkg/config/config.go:969-980`) — and a bare string-concatenated +// `${rel}/${name}` would produce `./foo.sql` there instead of Go's `foo.sql`. This +// matters beyond cosmetics: for seeds, the walked path becomes the +// `supabase_migrations.seed_files.path` hash key, so a `./`-prefixed TS path would +// never match an already-recorded Go-CLI key and would re-run/re-record the seed. +// Verified empirically: `path.Join(".", "foo.sql")` and a real `fs.WalkDir` rooted at +// `.` both drop the `./` prefix entirely, while `path.Join("..", "foo.sql")` stays +// `../foo.sql` — only a bare `.` collapses this way; `..` is not further cleanable +// against nothing and joins normally, so it needs no special case here. +const joinRelChild = (rel: string, name: string): string => { + if (rel === ".") return name; + return rel.endsWith("/") ? `${rel.replace(/\/+$/, "")}/${name}` : `${rel}/${name}`; +}; const legacyWalkSqlFiles = ( fs: FileSystem.FileSystem, diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts index 42b58422a1..1d2f9b7213 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts @@ -190,6 +190,37 @@ describe("legacySqlFilesGlob", () => { }, ); + it.effect( + "drops the './' prefix when the matched directory cleans to '.' (Go path.Join parity)", + () => { + // Go's `fs.WalkDir` builds each child path via `path.Join(dirname, name)`, and + // `path.Join` runs `path.Clean`, which drops a bare `.` root entirely rather than + // joining it as a prefix. A matched directory can clean to exactly `.` — e.g. + // `[db.migrations].schema_paths = [".."]`/`[db.seed].sql_paths = [".."]`, which + // `baseConfig.resolve`'s own `path.Join(builder.SupabaseDirPath, pattern)` collapses + // to `.` (`apps/cli-go/pkg/config/config.go:969-980`) — so the walk over its children + // must record `a.sql`, not `./a.sql`. This matters beyond cosmetics: for seeds, the + // walked path becomes the `supabase_migrations.seed_files.path` hash key, so a + // `./`-prefixed path would never match an already-recorded Go-CLI key. Verified + // empirically: `path.Join(".", "foo.sql")` and a real `fs.WalkDir` rooted at `.` both + // drop the `./` prefix entirely. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-dot-root-")); + writeFileSync(join(dir, "a.sql"), "select 1;"); + const nestedDir = join(dir, "nested"); + mkdirSync(nestedDir); + writeFileSync(join(nestedDir, "b.sql"), "select 2;"); + return run(["."], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["a.sql", "nested/b.sql"]); + expect(result.warnings).toEqual([]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + it.effect("still expands a real (non-symlinked) nested directory recursively", () => { const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-nested-")); const schemasDir = join(dir, "schemas"); From f0fc52cfe8560e2757f832e6edf3f4cf194fb417 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Wed, 5 Aug 2026 17:07:14 +0100 Subject: [PATCH 14/47] fix(cli): clean walked SQL child paths like Go's path.Join (review: CLI-1958) joinRelChild string-concatenated rel/name instead of running Go's path.Clean-equivalent lexical cleaning, so a directory configured with a cleanable segment (e.g. "/tmp/x/../schemas") produced a literal ".."-containing path instead of matching Go's fs.WalkDir output. Verified empirically that Node's path.join matches Go's path.Join byte-for-byte across dot-root, trailing-slash, and embedded "."/".." cases, so delegate to the injected Path service instead of special-casing more segment shapes. --- .../legacy/shared/legacy-sql-files-glob.ts | 54 +++++++++---------- .../shared/legacy-sql-files-glob.unit.test.ts | 26 +++++++++ 2 files changed, 52 insertions(+), 28 deletions(-) diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts index 3ed12ac33e..b15969cdb4 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts @@ -103,34 +103,32 @@ const globOne = ( * empirically: an unreadable matched directory makes `Glob.SQLFiles` return a * `failed to walk matched directory: ...` error with zero files, not an empty match). */ -// Go's `fs.WalkDir` builds each child path via `path.Join(dirname, name)` -// (`io/fs/walk.go`), and `path.Join` runs `path.Clean` on the joined result, which: +// Go's `fs.WalkDir` builds each child path via `path.Join(dirname, name)` (`io/fs/walk.go`), +// and `path.Join` runs `path.Clean` on the joined result — collapsing doubled slashes, +// dropping a bare `.` root, and lexically resolving `.`/`..` segments anywhere else in the +// path (e.g. an absolute `schema_paths`/`sql_paths` directory configured as `/tmp/./schemas` +// or `/tmp/x/../schemas`). Node's `path.join` (via this module's injected `Path.Path` +// service, backed by `node:path`) runs the same POSIX lexical-cleaning algorithm and was +// verified empirically to match Go's `path.Join` byte-for-byte across every case this walk +// can hit — dot-root, trailing slash, embedded `.`/`..`, and doubled slashes — so delegating +// to it (rather than accumulating more hand-rolled special cases here) closes this whole +// class of cleaning edge cases at once: // -// 1. Collapses a doubled `/` down to one. A bare string-concatenated `${rel}/${name}` -// does NOT collapse it, so a matched directory whose OWN path already ends in `/` -// (e.g. a literal `schema_paths = ["/tmp/schemas/"]`/`sql_paths` entry — `fs.Glob`'s -// no-metacharacter fast path returns that pattern verbatim, trailing slash and all) -// would otherwise produce `/tmp/schemas//a.sql` here instead of Go's -// `/tmp/schemas/a.sql`. Verified empirically: an `apps/cli-go` probe importing -// `pkg/config` directly and calling `Glob{"/"}.SQLFiles(...)` against a -// trailing-slash absolute directory returns the single-slash path, not a doubled one. -// 2. Drops a bare `.` root entirely rather than joining it as a prefix. A matched -// directory can clean to exactly `.` — e.g. `[db.migrations].schema_paths = [".."]` -// or `[db.seed].sql_paths = [".."]`, which `baseConfig.resolve`'s own -// `path.Join(builder.SupabaseDirPath, pattern)` collapses to `.` -// (`apps/cli-go/pkg/config/config.go:969-980`) — and a bare string-concatenated -// `${rel}/${name}` would produce `./foo.sql` there instead of Go's `foo.sql`. This -// matters beyond cosmetics: for seeds, the walked path becomes the -// `supabase_migrations.seed_files.path` hash key, so a `./`-prefixed TS path would -// never match an already-recorded Go-CLI key and would re-run/re-record the seed. -// Verified empirically: `path.Join(".", "foo.sql")` and a real `fs.WalkDir` rooted at -// `.` both drop the `./` prefix entirely, while `path.Join("..", "foo.sql")` stays -// `../foo.sql` — only a bare `.` collapses this way; `..` is not further cleanable -// against nothing and joins normally, so it needs no special case here. -const joinRelChild = (rel: string, name: string): string => { - if (rel === ".") return name; - return rel.endsWith("/") ? `${rel.replace(/\/+$/, "")}/${name}` : `${rel}/${name}`; -}; +// Go: path.Join(".", "foo.sql") = "foo.sql" +// path.Join("/tmp/schemas/", "a.sql") = "/tmp/schemas/a.sql" +// path.Join("/tmp/./schemas", "a.sql") = "/tmp/schemas/a.sql" +// path.Join("/tmp/x/../schemas", "a.sql") = "/tmp/schemas/a.sql" +// path.Join("..", "foo.sql") = "../foo.sql" +// Node: path.join(".", "foo.sql") = "foo.sql" +// path.join("/tmp/schemas/", "a.sql") = "/tmp/schemas/a.sql" +// path.join("/tmp/./schemas", "a.sql") = "/tmp/schemas/a.sql" +// path.join("/tmp/x/../schemas", "a.sql") = "/tmp/schemas/a.sql" +// path.join("..", "foo.sql") = "../foo.sql" +// +// For seeds, the walked path becomes the `supabase_migrations.seed_files.path` hash key, so +// any of these cleaning differences would make a TS-walked path fail to match an +// already-recorded Go-CLI key and re-run/re-record the seed. +const joinRelChild = (path: Path.Path, rel: string, name: string): string => path.join(rel, name); const legacyWalkSqlFiles = ( fs: FileSystem.FileSystem, @@ -151,7 +149,7 @@ const legacyWalkSqlFiles = ( ), ); for (const name of names) { - const childRel = joinRelChild(rel, name); + const childRel = joinRelChild(path, rel, name); const childAbs = path.isAbsolute(childRel) ? childRel : path.join(workdir, childRel); // Go's `fs.WalkDir` types each child from the parent's `ReadDir` entry // (`os.ReadDir`'s Lstat-based `DirEntry`) and never re-`Stat`s through it — diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts index 1d2f9b7213..aa5b951c26 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts @@ -221,6 +221,32 @@ describe("legacySqlFilesGlob", () => { }, ); + it.effect( + "cleans a '..'-segment matched directory when walking its children (Go path.Join parity)", + () => { + // Go's `fs.WalkDir` builds each child path via `path.Join(dirname, name)`, and + // `path.Join` runs `path.Clean`, which lexically resolves an embedded `..` segment — + // not just a bare `.` root or a trailing slash. A matched directory can contain a + // `..` anywhere, e.g. `[db.migrations].schema_paths = ["nested/../schemas"]`, and the + // walk over its children must record `schemas/a.sql`, not `nested/../schemas/a.sql`. + // Verified empirically: `path.Join("/tmp/x/../schemas", "a.sql")` and Node's + // `path.join("/tmp/x/../schemas", "a.sql")` both clean to `/tmp/schemas/a.sql`. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-dotdot-segment-")); + const schemasDir = join(dir, "schemas"); + mkdirSync(schemasDir); + writeFileSync(join(schemasDir, "a.sql"), "select 1;"); + return run(["nested/../schemas"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["schemas/a.sql"]); + expect(result.warnings).toEqual([]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + it.effect("still expands a real (non-symlinked) nested directory recursively", () => { const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-nested-")); const schemasDir = join(dir, "schemas"); From 95f3108e4f6305eae237a6f913dedca1ac3f8bef Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Wed, 5 Aug 2026 17:08:10 +0100 Subject: [PATCH 15/47] fix(cli): keep a raced '.sql' child declared when its stat fails (review: CLI-1958) Go's fs.WalkDir types each child from the DirEntry its parent ReadDir already returned and never re-Stats through it, so a .sql file removed between ReadDir and the walk callback's own visit stays in Go's declared file list; only the later, real file-open fails loudly. This port's follow-up fs.stat call opens a race window Go doesn't have, and on failure silently classified the child as Unknown and dropped it with no warning - an experimental reset whose only schema file hit this race would "succeed" having applied nothing. Verified empirically with a scratch filepath.WalkDir probe that deletes a sibling .sql file between ReadDir and that file's own visit: Go still reports it IsRegular from the cached DirEntry, keeps it declared, and the later os.Open fails with "no such file or directory" - never a silent drop. Match that outcome: on a stat failure, best-effort include a '.sql'- named child anyway and let the real downstream read surface the failure. --- .../legacy/shared/legacy-sql-files-glob.ts | 25 +++++++++-- .../shared/legacy-sql-files-glob.unit.test.ts | 41 +++++++++++++++++++ 2 files changed, 62 insertions(+), 4 deletions(-) diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts index b15969cdb4..4b6495d789 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts @@ -166,10 +166,27 @@ const legacyWalkSqlFiles = ( if (isSymlink) { continue; } - const childType = yield* fs.stat(childAbs).pipe( - Effect.map((info) => info.type), - Effect.orElseSucceed(() => "Unknown" as const), - ); + const statResult = yield* fs.stat(childAbs).pipe(Effect.result); + if (Result.isFailure(statResult)) { + // TOCTOU: this `.sql` child existed a moment ago in `names` (this directory's + // `readDirectory` snapshot) but is gone by the time we stat it here — e.g. removed + // by a concurrent process. Go never hits this window: `fs.WalkDir` decides + // file-vs-directory from the SAME `DirEntry` its parent `ReadDir` already + // returned and never re-`Stat`s a child, so a `.sql` file that disappears here + // stays in Go's declared file list, and only the later, real file-open fails + // loudly. Verified empirically: a scratch `filepath.WalkDir` probe that deletes a + // sibling `.sql` file between `ReadDir` and that file's own visit still reports + // it `IsRegular` from the cached entry, keeps it in `declared`, and the + // subsequent `os.Open` on it fails with "no such file or directory" — never a + // silent drop. Losing the stat here must not silently drop the file and let the + // walk "succeed" having applied nothing — best-effort include it, matching Go's + // outcome, and let the real downstream read surface the failure. + if (childRel.endsWith(".sql")) { + collected.push(toSlash(childRel)); + } + continue; + } + const childType = statResult.success.type; if (childType === "Directory") { yield* walk(childRel); } else if (childType === "File" && childRel.endsWith(".sql")) { diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts index aa5b951c26..0f6fde1b33 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts @@ -247,6 +247,47 @@ describe("legacySqlFilesGlob", () => { }, ); + it.effect( + "still includes a '.sql' child whose stat fails after it's already listed (Go WalkDir parity)", + () => { + // Go's `fs.WalkDir` types each child from the parent's `ReadDir`-returned `DirEntry` + // and never re-`Stat`s through it, so a `.sql` file that disappears between `ReadDir` + // and its own visit stays in Go's declared file list — only the later, real file-open + // fails. Simulate the stat failure directly (mocking a real race is flaky) by pointing + // the matched directory at one that lists a child but whose child path is unreadable: + // a broken symlink target used as a bare filename via a `readLink` failure isn't + // enough here (that's the earlier symlink test), so exercise the `fs.stat` failure + // path itself by removing the file the instant after `readDirectory` returns it, via + // a `FileSystem` layer that deletes on first `stat` call for that path. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-stat-race-")); + const schemasDir = join(dir, "schemas"); + mkdirSync(schemasDir); + const racyFile = join(schemasDir, "racy.sql"); + writeFileSync(racyFile, "select 1;"); + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const racyFs: FileSystem.FileSystem = { + ...fs, + stat: (p: string) => + p === racyFile + ? Effect.sync(() => rmSync(racyFile)).pipe(Effect.andThen(fs.stat(p))) + : fs.stat(p), + }; + return yield* legacySqlFilesGlob(racyFs, path, ["schemas"], dir); + }).pipe( + Effect.provide(BunServices.layer), + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["schemas/racy.sql"]); + expect(result.warnings).toEqual([]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + it.effect("still expands a real (non-symlinked) nested directory recursively", () => { const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-nested-")); const schemasDir = join(dir, "schemas"); From 40f597fb4893a6479e5d3db2c5f639ff97a811ee Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Wed, 5 Aug 2026 18:18:01 +0100 Subject: [PATCH 16/47] fix(cli): clean direct SQL glob matches like Go's afero.Glob (review: CLI-1958) config.Glob.SQLFiles's real runtime path (afero.IOFS.Glob -> afero.Glob's glob() helper) appends each direct match as filepath.Join(dir, n), which cleans dot/dot-dot segments and doubled slashes. globOne's direct-match construction only concatenated d/name, so an absolute schema_paths/ sql_paths glob like /tmp/./schemas/*.sql or /tmp/x/../schemas/*.sql recorded the uncleaned path verbatim, drifting schema-file suggestions and the seed_files.path hash key from Go. Reuse joinRelChild (the Path-service clean-join already used for walked-child construction) instead of adding another special case. Verified empirically with a scratch afero.Glob probe against apps/cli-go's pinned afero v1.15.0: all three shapes (dot segment, dot-dot segment, doubled slash) resolve to the cleaned path. --- .../legacy/shared/legacy-sql-files-glob.ts | 79 +++++++++++------- .../shared/legacy-sql-files-glob.unit.test.ts | 80 +++++++++++++++++++ 2 files changed, 128 insertions(+), 31 deletions(-) diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts index 4b6495d789..edda56f247 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts @@ -18,6 +18,48 @@ const GLOB_META_CHARS = /[*?[]/u; // that escape — see `legacyPathMatch`'s escape handling below. const toSlash = (p: string): string => (process.platform === "win32" ? p.replaceAll("\\", "/") : p); +// Joins a matched directory (or glob-split directory prefix) with a child/entry name, +// delegating to the injected `Path.Path` service for Go's `path.Join`-equivalent +// cleaning. Two Go call sites build a path exactly this way, and both need the same +// cleaning: +// +// - Direct glob-match construction (`globOne`, below): the real runtime glob path — +// `config.Glob.SQLFiles`'s `fs.Glob(fsys, pattern)` call (`apps/cli-go/pkg/config/ +// config.go:145`) resolves to `afero.IOFS.Glob` (it implements `fs.GlobFS`), which +// delegates to `afero.Glob` (`github.com/spf13/afero@v1.15.0/iofs.go:56-65`). Its +// `glob()` helper appends each match as `filepath.Join(dir, n)` +// (`match.go:99`), so a glob whose directory portion has a cleanable segment +// (`/tmp/./schemas/*.sql`, `/tmp/x/../schemas/*.sql`, a doubled `/tmp/schemas//*.sql`) +// still records the CLEANED path, not a raw concatenation. Verified empirically: a +// scratch `afero.Glob` probe against all three shapes above returns +// `.../tmp/schemas/a.sql` in every case. +// - Walked-child construction (`legacyWalkSqlFiles`, below): Go's `fs.WalkDir` builds +// each child path via `path.Join(dirname, name)` (`io/fs/walk.go`). +// +// `path.Join`/`filepath.Join` both run Clean on the joined result — collapsing doubled +// slashes, dropping a bare `.` root, and lexically resolving `.`/`..` segments anywhere +// else in the path. Node's `path.join` (via this module's injected `Path.Path` service, +// backed by `node:path`) runs the same POSIX lexical-cleaning algorithm and was verified +// empirically to match byte-for-byte across every case either call site can hit — +// dot-root, trailing slash, embedded `.`/`..`, and doubled slashes: +// +// Go: path.Join(".", "foo.sql") = "foo.sql" +// filepath.Join("/tmp/schemas/", "a.sql") = "/tmp/schemas/a.sql" +// filepath.Join("/tmp/./schemas", "a.sql") = "/tmp/schemas/a.sql" +// filepath.Join("/tmp/x/../schemas", "a.sql") = "/tmp/schemas/a.sql" +// path.Join("..", "foo.sql") = "../foo.sql" +// Node: path.join(".", "foo.sql") = "foo.sql" +// path.join("/tmp/schemas/", "a.sql") = "/tmp/schemas/a.sql" +// path.join("/tmp/./schemas", "a.sql") = "/tmp/schemas/a.sql" +// path.join("/tmp/x/../schemas", "a.sql") = "/tmp/schemas/a.sql" +// path.join("..", "foo.sql") = "../foo.sql" +// +// For seeds, the resulting path becomes the `supabase_migrations.seed_files.path` hash +// key, so any of these cleaning differences would make a TS-resolved path fail to match +// an already-recorded Go-CLI key and re-run/re-record the seed; for schema files it +// changes what path is suggested on an apply failure. +const joinRelChild = (path: Path.Path, rel: string, name: string): string => path.join(rel, name); + /** * Splits a forward-slashed path into its directory prefix and final element. * @@ -78,10 +120,12 @@ const globOne = ( .pipe(Effect.orElseSucceed((): ReadonlyArray => [])); for (const name of names) { if (legacyPathMatch(file, name).matched) { - // `d === "/"` is the filesystem root (from `splitPath`'s preserved root - // prefix, above) — join without a doubled slash, matching Go's - // `filepath.Join("/", name)`. - result.push(d === "" ? name : d === "/" ? `/${name}` : `${d}/${name}`); + // `joinRelChild` (above) is the same Path-service clean-join the walked-child + // path below uses — see its comment for why a raw `${d}/${name}` concatenation + // isn't enough here (Go's `afero.Glob` cleans via `filepath.Join(dir, n)`, so + // `d === "/"`, `d === ""`, and a `dir` containing `.`/`..`/doubled-slash + // segments must all clean the same way as the walked-child case does). + result.push(toSlash(joinRelChild(path, d, name))); } } } @@ -103,33 +147,6 @@ const globOne = ( * empirically: an unreadable matched directory makes `Glob.SQLFiles` return a * `failed to walk matched directory: ...` error with zero files, not an empty match). */ -// Go's `fs.WalkDir` builds each child path via `path.Join(dirname, name)` (`io/fs/walk.go`), -// and `path.Join` runs `path.Clean` on the joined result — collapsing doubled slashes, -// dropping a bare `.` root, and lexically resolving `.`/`..` segments anywhere else in the -// path (e.g. an absolute `schema_paths`/`sql_paths` directory configured as `/tmp/./schemas` -// or `/tmp/x/../schemas`). Node's `path.join` (via this module's injected `Path.Path` -// service, backed by `node:path`) runs the same POSIX lexical-cleaning algorithm and was -// verified empirically to match Go's `path.Join` byte-for-byte across every case this walk -// can hit — dot-root, trailing slash, embedded `.`/`..`, and doubled slashes — so delegating -// to it (rather than accumulating more hand-rolled special cases here) closes this whole -// class of cleaning edge cases at once: -// -// Go: path.Join(".", "foo.sql") = "foo.sql" -// path.Join("/tmp/schemas/", "a.sql") = "/tmp/schemas/a.sql" -// path.Join("/tmp/./schemas", "a.sql") = "/tmp/schemas/a.sql" -// path.Join("/tmp/x/../schemas", "a.sql") = "/tmp/schemas/a.sql" -// path.Join("..", "foo.sql") = "../foo.sql" -// Node: path.join(".", "foo.sql") = "foo.sql" -// path.join("/tmp/schemas/", "a.sql") = "/tmp/schemas/a.sql" -// path.join("/tmp/./schemas", "a.sql") = "/tmp/schemas/a.sql" -// path.join("/tmp/x/../schemas", "a.sql") = "/tmp/schemas/a.sql" -// path.join("..", "foo.sql") = "../foo.sql" -// -// For seeds, the walked path becomes the `supabase_migrations.seed_files.path` hash key, so -// any of these cleaning differences would make a TS-walked path fail to match an -// already-recorded Go-CLI key and re-run/re-record the seed. -const joinRelChild = (path: Path.Path, rel: string, name: string): string => path.join(rel, name); - const legacyWalkSqlFiles = ( fs: FileSystem.FileSystem, path: Path.Path, diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts index 0f6fde1b33..82085d7436 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts @@ -247,6 +247,86 @@ describe("legacySqlFilesGlob", () => { }, ); + it.effect( + "cleans a direct glob match whose directory portion has a '.' segment (Go afero.Glob parity)", + () => { + // Distinct from the walked-child cleaning above: this pattern's glob metacharacter + // (`*`) is in the FINAL component, so `globOne` matches `a.sql` directly against + // the directory entries of `schemas/.` — it never goes through + // `legacyWalkSqlFiles`. Go's real runtime glob path resolves through + // `afero.IOFS.Glob` -> `afero.Glob`'s `glob()` helper, which appends each match as + // `filepath.Join(dir, n)` (`match.go:99`) — so the recorded match is the CLEANED + // `schemas/a.sql`, not a raw `schemas/./a.sql` concatenation. Verified empirically: + // a scratch `afero.Glob(fs, ".../tmp/./schemas/*.sql")` probe against a real + // filesystem returns the cleaned path. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-direct-dot-segment-")); + const schemasDir = join(dir, "schemas"); + mkdirSync(schemasDir); + writeFileSync(join(schemasDir, "a.sql"), "select 1;"); + return run(["schemas/./*.sql"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["schemas/a.sql"]); + expect(result.warnings).toEqual([]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "cleans a direct glob match whose directory portion has a '..' segment (Go afero.Glob parity)", + () => { + // Same distinction as above (a direct match via `globOne`, not a walked directory + // expansion), but for an embedded `..` rather than a `.` segment — e.g. an absolute + // `schema_paths`/`sql_paths` entry like `/tmp/x/../schemas/*.sql`. `filepath.Join` + // lexically resolves `..` the same way it drops a bare `.` root, so Go still records + // the cleaned `schemas/a.sql`, not `nested/../schemas/a.sql`. For seed files, that + // recorded path is the `supabase_migrations.seed_files.path` hash key, so leaving it + // uncleaned would make a TS-resolved match fail to line up with an already-recorded + // Go-CLI key and re-run/re-record the seed. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-direct-dotdot-segment-")); + const schemasDir = join(dir, "schemas"); + mkdirSync(schemasDir); + writeFileSync(join(schemasDir, "a.sql"), "select 1;"); + return run(["nested/../schemas/*.sql"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["schemas/a.sql"]); + expect(result.warnings).toEqual([]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "normalizes a doubled slash for a direct glob match under a trailing-slash directory component (Go afero.Glob parity)", + () => { + // Same distinction again: `splitPath` on `"schemas//*.sql"` yields a `dir` of + // `"schemas/"` (a single trailing slash survives the split), so the old raw + // `` `${d}/${name}` `` concatenation inserted a SECOND slash on top of it + // (`"schemas//a.sql"`). `filepath.Join`/`path.join` collapse doubled slashes + // regardless of where they came from, so the recorded match must be the + // single-slash `schemas/a.sql`. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-direct-doubled-slash-")); + const schemasDir = join(dir, "schemas"); + mkdirSync(schemasDir); + writeFileSync(join(schemasDir, "a.sql"), "select 1;"); + return run(["schemas//*.sql"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["schemas/a.sql"]); + expect(result.warnings).toEqual([]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + it.effect( "still includes a '.sql' child whose stat fails after it's already listed (Go WalkDir parity)", () => { From a3a846c871e3b3dc0cc31aebe7692a3052c9c946 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Wed, 5 Aug 2026 19:21:22 +0100 Subject: [PATCH 17/47] fix(cli): preserve Windows drive roots when splitting SQL glob patterns (review: CLI-1958) Go's filepath.Split keeps "C:/" intact for a drive-root pattern (volumeNameLen treats "C:" as the volume, and the following separator stays attached to dir). This split was chopping it to "C:" instead, which Node's path.isAbsolute treats as drive-relative rather than absolute, so globOne would join it under the workdir instead of resolving the real drive root. --- .../legacy/shared/legacy-sql-files-glob.ts | 21 ++++++- .../shared/legacy-sql-files-glob.unit.test.ts | 55 ++++++++++++++++++- 2 files changed, 71 insertions(+), 5 deletions(-) diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts index edda56f247..b272539991 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts @@ -75,13 +75,28 @@ const joinRelChild = (path: Path.Path, rel: string, name: string): string => pat * that first component against the filesystem ROOT, not cwd. Collapsing this * to `dir: ""` would make `globOne` below treat such an absolute root-level * pattern as relative to the workdir instead of the filesystem root. + * + * On Windows, a bare drive-root prefix (`"C:/"`) needs the exact same + * preservation, for the same reason but a different mechanism: Go's + * `filepath.Split` treats `"C:"` as the volume name (`volumeNameLen`, + * `internal/filepathlite/path_windows.go`) and always keeps the following + * separator attached to `dir` — verified against that source directly, since + * there is no Windows machine available to run the compiled stdlib on: + * `Split("C:/*.sql")` returns `dir: "C:/"`, not `dir: "C:"`. Chopping the + * separator here would matter downstream: Node's `path.isAbsolute("C:")` is + * `false` (a bare drive letter is a *drive-relative* path in Windows + * semantics, not absolute), so `globOne`'s `resolve()` would wrongly `join` + * it under the workdir instead of resolving the real drive root, while + * `path.isAbsolute("C:/")` is `true`. */ const splitPath = (p: string): { readonly dir: string; readonly file: string } => { const slash = p.lastIndexOf("/"); if (slash === -1) return { dir: "", file: p }; - return slash === 0 - ? { dir: "/", file: p.slice(1) } - : { dir: p.slice(0, slash), file: p.slice(slash + 1) }; + if (slash === 0) return { dir: "/", file: p.slice(1) }; + if (process.platform === "win32" && slash === 2 && p.charAt(1) === ":") { + return { dir: p.slice(0, 3), file: p.slice(3) }; + } + return { dir: p.slice(0, slash), file: p.slice(slash + 1) }; }; /** Faithful port of Go's `fs.Glob` for one pattern, rooted at `workdir`. */ diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts index 82085d7436..9e327b2654 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts @@ -1,9 +1,9 @@ import { chmodSync, mkdirSync, mkdtempSync, rmSync, symlinkSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; -import { BunServices } from "@effect/platform-bun"; +import { BunFileSystem, BunPath, BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; -import { Effect, FileSystem, Path } from "effect"; +import { Effect, FileSystem, Layer, Path } from "effect"; import { legacySqlFilesGlob } from "./legacy-sql-files-glob.ts"; @@ -163,6 +163,57 @@ describe("legacySqlFilesGlob", () => { }, ); + it.effect( + "preserves a Windows drive root ('C:/') as the directory when splitting a glob pattern (Go filepath.Split parity)", + () => { + // Go's `filepath.Split` treats `"C:"` as the volume name on Windows + // (`volumeNameLen`, `internal/filepathlite/path_windows.go`) and always + // keeps the following separator attached to `dir` — so `Split("C:/*.sql")` + // returns `dir: "C:/"`, not `dir: "C:"` (verified directly against that + // stdlib source; there is no Windows machine available to run the + // compiled binary on). Losing the trailing slash matters: Node's + // `path.isAbsolute("C:")` is `false` (a bare drive letter is + // *drive-relative*, not absolute, in Windows semantics), so `globOne`'s + // `resolve()` would wrongly `join` it under the workdir instead of + // resolving the real drive root. Force win32 path semantics + // (`BunPath.layerWin32`) and this module's own `process.platform` gate + // so the test exercises the same branch a real Windows install takes. + const originalPlatform = process.platform; + Object.defineProperty(process, "platform", { value: "win32" }); + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-drive-root-")); + writeFileSync(join(dir, "a.sql"), "select 1;"); + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + // A "C:/" drive root doesn't exist on this (non-Windows) test host, so + // fake just the two calls that must resolve against it, reusing a real + // file's stat info to avoid hand-rolling a `File.Info`. + const realFileInfo = yield* fs.stat(join(dir, "a.sql")); + const driveRootFs: FileSystem.FileSystem = { + ...fs, + readDirectory: (p: string) => + p === "C:/" ? Effect.succeed(["a.sql"]) : fs.readDirectory(p), + stat: (p: string) => (p === "C:/a.sql" ? Effect.succeed(realFileInfo) : fs.stat(p)), + }; + return yield* legacySqlFilesGlob(driveRootFs, path, ["C:/*.sql"], dir); + }).pipe( + Effect.provide(Layer.mergeAll(BunFileSystem.layer, BunPath.layerWin32)), + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["C:/a.sql"]); + expect(result.warnings).toEqual([]); + }), + ), + Effect.ensuring( + Effect.sync(() => { + Object.defineProperty(process, "platform", { value: originalPlatform }); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + it.effect( "normalizes a doubled slash when the matched directory itself has a trailing slash (Go path.Join parity)", () => { From 35204d55113ca85d8a6864c10cccef7e337ca2dc Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Wed, 5 Aug 2026 19:21:32 +0100 Subject: [PATCH 18/47] fix(cli): wrap migration read failures with Go's parse-file error text (review: CLI-1958) Go's NewMigrationFromFile/parseFile wraps an open failure as "failed to open migration file: %w" before ApplyMigrations/applySchemaFiles ever gets a chance to attach a CmdSuggestion. This read-phase mapping only forwarded the raw platform error, so stderr/JSON output omitted the Go prefix. Matches the same wrapping already used by legacyReadMigrationFile and other read paths. --- .../legacy/shared/legacy-migration-apply.ts | 16 +++++++---- .../legacy-migration-apply.unit.test.ts | 27 +++++++++++++++++++ 2 files changed, 38 insertions(+), 5 deletions(-) diff --git a/apps/cli/src/legacy/shared/legacy-migration-apply.ts b/apps/cli/src/legacy/shared/legacy-migration-apply.ts index d84e676108..dac8e32421 100644 --- a/apps/cli/src/legacy/shared/legacy-migration-apply.ts +++ b/apps/cli/src/legacy/shared/legacy-migration-apply.ts @@ -161,13 +161,19 @@ const execMigrationBatch = ( ): Effect.Effect => Effect.gen(function* () { // Go's `MigrationFile.ExecBatch` receives an already-read/parsed file (the read - // happens earlier, in `NewMigrationFromFile`) — so a read failure here is a - // DIFFERENT error class than a statement-execution failure below. Tagged "read" so - // callers that attach a suggestion only around execution failures (`apply.go:61-63`) - // can tell the two apart. + // happens earlier, in `NewMigrationFromFile`/`parseFile`, which wraps the open + // failure as `"failed to open migration file: %w"`, `pkg/migration/file.go:57-58`) + // — so a read failure here is a DIFFERENT error class than a statement-execution + // failure below, and needs the same Go prefix so stderr/JSON errors don't surface + // the bare platform error text. Tagged "read" so callers that attach a suggestion + // only around execution failures (`apply.go:61-63`) can tell the two apart. const content = yield* fs .readFileString(migrationPath) - .pipe(Effect.mapError((error) => mapError(legacyErrorMessage(error), "read"))); + .pipe( + Effect.mapError((error) => + mapError(`failed to open migration file: ${legacyErrorMessage(error)}`, "read"), + ), + ); // Everything below mirrors Go's `(*MigrationFile).ExecBatch` (`pkg/migration/file.go`), // which runs against an already-read file — so every failure from here on is an diff --git a/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts b/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts index 749fb9b70b..71d359cbe4 100644 --- a/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts @@ -131,6 +131,33 @@ describe("legacyApplyMigrationFile", () => { ); }); + it.effect( + "wraps a read failure with Go's parse-file error text (Go NewMigrationFromFile parity)", + () => { + // Go's `NewMigrationFromFile`/`parseFile` wraps the open failure as + // `"failed to open migration file: %w"` (`pkg/migration/file.go:57-58`) before + // `ApplyMigrations`/`applySchemaFiles` ever get a chance to attach a + // `CmdSuggestion` — a read failure here must carry the same prefix, not the bare + // platform error text. + const dir = mkdtempSync(join(tmpdir(), "legacy-apply-read-fail-")); + const missingFile = join(dir, "20240101120000_missing.sql"); + const { session } = fakeSession(); + return run(session, missingFile).pipe( + Effect.exit, + Effect.tap((exit) => + Effect.sync(() => { + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const msg = JSON.stringify(exit.cause); + expect(msg).toContain("failed to open migration file: "); + } + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + it.effect("runs a pipeline-incompatible statement outside the surrounding transaction", () => { const dir = mkdtempSync(join(tmpdir(), "legacy-apply-")); const file = join(dir, "20240101120000_add_index.sql"); From 5331b1b78f14b13ce8237630351661269cc25cdb Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Wed, 5 Aug 2026 19:21:41 +0100 Subject: [PATCH 19/47] docs(cli): correct db reset SIDE_EFFECTS on encrypted vault secrets (review: CLI-1958) legacyCheckDbToml decrypts encrypted: vault secrets into toml.vault, and the remote path calls legacyUpsertVaultSecrets unconditionally before either branch runs. The doc said these secrets were "skipped", hiding a real vault-mutation side effect from reviewers/e2e coverage. --- apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md b/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md index 98a0189af9..5c409c20a3 100644 --- a/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md +++ b/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md @@ -178,7 +178,9 @@ path has no confirmation prompt. unconditionally, exactly as on the migrations branch. The best-effort pg-delta catalog-cache warning (`down.go:58-59`, gated on `SUPABASE_EXPERIMENTAL_PG_DELTA`) is not ported (no output impact) — same known gap as the migrations branch. - `encrypted:` vault secrets are skipped on the remote path (both branches). + `encrypted:` vault secrets are NOT skipped on the remote path — `legacyCheckDbToml` + decrypts them into `toml.vault`, and `legacyUpsertVaultSecrets` upserts the decrypted + values unconditionally, before either branch (schema-files or migrations) runs. - The local path's own `--experimental` schema-files branch is still handled by the Go child behind the `db __db-bootstrap` seam (CLI-1955 scope) — this command's handler forwards `--experimental` to that seam but does not implement the branch From f37fe32b494dc40e8de8ba5de1326ad2ee9dc138 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Wed, 5 Aug 2026 19:21:50 +0100 Subject: [PATCH 20/47] fix(cli): format large numeric schema_paths entries as fixed decimal (review: CLI-1958) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Go's mapstructure weak-decode renders a float via strconv.FormatFloat(v, 'f', -1, 64) — always fixed decimal notation. JS's bare String(value) switches to scientific notation once magnitude crosses 1e21 (or drops below 1e-6), recording/searching for the wrong file path on the experimental reset path. Expand JS's own exponential notation back into fixed form instead of re-deriving digits, since both algorithms already agree on the shortest round-tripping digit sequence. --- .../shared/legacy-db-config.toml-read.ts | 24 ++++++++++++++++++- .../legacy-db-config.toml-read.unit.test.ts | 22 +++++++++++++++++ 2 files changed, 45 insertions(+), 1 deletion(-) diff --git a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts index 32441390de..d0fdaf0f59 100644 --- a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts +++ b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts @@ -1851,6 +1851,28 @@ const readDbTomlCore = Effect.fnUntraced(function* ( const expanded = legacyExpandEnv(value, lookup); return expanded.length === 0 ? [] : expanded.split(","); }; + // Go's `decodeString` renders a weakly-converted float via `strconv.FormatFloat(v, + // 'f', -1, 64)` (`mapstructure.go:747-748`, format `'f'`) — ALWAYS fixed decimal + // notation, never scientific, regardless of magnitude. JS's `String(value)` agrees + // for ordinary magnitudes (both use the same shortest-round-trip digit sequence — + // `String()` just chooses "e" notation once `|value| >= 1e21` or `< 1e-6`, which + // `FormatFloat('f', …)` never does). Verified empirically: `strconv.FormatFloat(1e21, + // 'f', -1, 64)` returns `"1000000000000000000000"`, not `"1e+21"`. Expand JS's own + // exponential notation back into fixed notation instead of re-deriving the digits, + // since `Number.prototype.toString()`/`toExponential()` already computed the same + // shortest round-tripping digit sequence Go's algorithm would — only the notation + // differs. + const legacyFormatGoWeakFloat = (value: number): string => { + const str = value.toString(); + const match = /^(-?)(\d+)(?:\.(\d+))?e([+-]\d+)$/.exec(str); + if (match === null) return str; + const [, sign = "", intPart = "", fracPart = "", expStr = "0"] = match; + const digits = intPart + fracPart; + const pointPos = intPart.length + Number(expStr); + if (pointPos <= 0) return `${sign}0.${"0".repeat(-pointPos)}${digits}`; + if (pointPos >= digits.length) return `${sign}${digits}${"0".repeat(pointPos - digits.length)}`; + return `${sign}${digits.slice(0, pointPos)}.${digits.slice(pointPos)}`; + }; // Go decodes both `[db.seed].sql_paths` and `[db.migrations].schema_paths` as // `config.Glob` (`[]string`) through the SAME mapstructure `UnmarshalExact` call // (`config.go:749-756`), whose decoder config never sets `WeaklyTypedInput: false` @@ -1864,7 +1886,7 @@ const readDbTomlCore = Effect.fnUntraced(function* ( const legacyWeakCoerceGlobEntry = (value: unknown): string | undefined => { if (typeof value === "string") return value; if (typeof value === "boolean") return value ? "1" : "0"; - if (typeof value === "number") return String(value); + if (typeof value === "number") return legacyFormatGoWeakFloat(value); return undefined; }; // A non-scalar element (nested array/table, e.g. `schema_paths = [[]]` or diff --git a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts index 898e99763c..0d345ba87b 100644 --- a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts @@ -372,6 +372,28 @@ describe("legacyReadDbToml", () => { }, ); + it.effect( + "formats a large numeric db.migrations.schema_paths entry as fixed decimal, not scientific notation (Go strconv.FormatFloat parity)", + () => { + // Go's `decodeString` renders a weakly-converted float via + // `strconv.FormatFloat(v, 'f', -1, 64)` — format `'f'` is ALWAYS fixed decimal, + // never scientific, regardless of magnitude. JS's bare `String(1e21)` switches to + // exponential notation ("1e+21") once the magnitude crosses 1e21, which would + // record (and later search for) the wrong file path. Verified empirically against + // Go's stdlib: `strconv.FormatFloat(1e21, 'f', -1, 64)` returns + // `"1000000000000000000000"`, not `"1e+21"`. + const dir = withConfig(["[db.migrations]", "schema_paths = [1e21]", ""].join("\n")); + return read(dir).pipe( + Effect.tap((v) => + Effect.sync(() => { + expect(v.schemaPaths).toEqual(["supabase/1000000000000000000000"]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + it.effect( "weakly coerces a TOP-LEVEL scalar db.migrations.schema_paths (Go mapstructure weak-decode of a []string field)", () => { From 3da79a8294d4a442d5d2dd86e5e9bcfae95bf7b4 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Wed, 5 Aug 2026 20:30:40 +0100 Subject: [PATCH 21/47] fix(cli): report workdir-relative path in schema-file read errors (review: CLI-1958) legacyApplySchemaFiles read a matched schema file through an absolute path (needed since this module never process.chdir's), but wrapped read failures embedded that same absolute path in the message. Go opens the workdir-relative fp directly (its cwd is always the workdir via ChangeWorkDir) and reports "open supabase/...: ...". Thread the glob's already-relative match through execMigrationBatch/legacyExecSqlFile as an optional display path so the wrapped message substitutes it in, without touching the other (pre-existing) execMigrationBatch callers. --- .../legacy/shared/legacy-migration-apply.ts | 54 ++++++++++++++----- .../legacy-migration-apply.unit.test.ts | 49 ++++++++++++++++- 2 files changed, 90 insertions(+), 13 deletions(-) diff --git a/apps/cli/src/legacy/shared/legacy-migration-apply.ts b/apps/cli/src/legacy/shared/legacy-migration-apply.ts index dac8e32421..7ed9e3b949 100644 --- a/apps/cli/src/legacy/shared/legacy-migration-apply.ts +++ b/apps/cli/src/legacy/shared/legacy-migration-apply.ts @@ -158,6 +158,7 @@ const execMigrationBatch = ( migrationPath: string, mapError: (message: string, phase: "read" | "exec") => E, forceNoVersion: boolean, + displayPath: string = migrationPath, ): Effect.Effect => Effect.gen(function* () { // Go's `MigrationFile.ExecBatch` receives an already-read/parsed file (the read @@ -167,13 +168,26 @@ const execMigrationBatch = ( // failure below, and needs the same Go prefix so stderr/JSON errors don't surface // the bare platform error text. Tagged "read" so callers that attach a suggestion // only around execution failures (`apply.go:61-63`) can tell the two apart. - const content = yield* fs - .readFileString(migrationPath) - .pipe( - Effect.mapError((error) => - mapError(`failed to open migration file: ${legacyErrorMessage(error)}`, "read"), - ), - ); + // + // Go opens `fp` — the workdir-RELATIVE form `[db.migrations].schema_paths`/ + // `[db.seed].sql_paths` already resolved to at config-load time — because Go's + // process cwd is always the workdir (`ChangeWorkDir`, `cmd/root.go:104`). This + // module deliberately never `process.chdir`s (only `bootstrap` does, as its own + // documented one-off), so callers must pass an ABSOLUTE `migrationPath` for the + // real read to work — but that means the platform error's embedded path is + // absolute too. When it differs from `displayPath` (the caller's Go-equivalent + // relative path), substitute it in so the wrapped message still reports the + // relative form Go would, not a leaked local temp/absolute path. + const content = yield* fs.readFileString(migrationPath).pipe( + Effect.mapError((error) => { + const rawMessage = legacyErrorMessage(error); + const message = + displayPath === migrationPath + ? rawMessage + : rawMessage.split(migrationPath).join(displayPath); + return mapError(`failed to open migration file: ${message}`, "read"); + }), + ); // Everything below mirrors Go's `(*MigrationFile).ExecBatch` (`pkg/migration/file.go`), // which runs against an already-read file — so every failure from here on is an @@ -366,6 +380,11 @@ export const legacySeedGlobals = ( * would print an extra line Go never prints. Callers write the in-memory SQL * constant to a temp file first (this module only reads files, like * `execMigrationBatch`'s other callers). + * + * `displayPath`, when given, is the path a read-failure's wrapped message should + * report instead of `filePath` — see `execMigrationBatch`'s comment on why the two + * can differ (an absolute path is required for the real read, but Go's equivalent + * error names the workdir-relative form). */ export const legacyExecSqlFile = ( session: LegacyDbSession, @@ -373,7 +392,9 @@ export const legacyExecSqlFile = ( path: Path.Path, filePath: string, mapError: (message: string, phase: "read" | "exec") => E, -): Effect.Effect => execMigrationBatch(session, fs, path, filePath, mapError, true); + displayPath?: string, +): Effect.Effect => + execMigrationBatch(session, fs, path, filePath, mapError, true, displayPath); /** * Applies Go's EXPERIMENTAL declarative schema-files branch of `apply.MigrateAndSeed` @@ -428,10 +449,19 @@ export const legacyApplySchemaFiles = ( } for (const file of files) { const absolutePath = path.isAbsolute(file) ? file : path.join(workdir, file); - yield* legacyExecSqlFile(session, fs, path, absolutePath, (message, phase) => - phase === "exec" - ? mapError(message, `See schema file: ${legacyBold(file)}`) - : mapError(message), + // `file` is already Go's `fp` form (workdir-relative when the declared pattern + // was relative, verbatim when absolute) — pass it through as the display path so + // a read failure reports it instead of the `absolutePath` the real read needs. + yield* legacyExecSqlFile( + session, + fs, + path, + absolutePath, + (message, phase) => + phase === "exec" + ? mapError(message, `See schema file: ${legacyBold(file)}`) + : mapError(message), + file, ); } }); diff --git a/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts b/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts index 71d359cbe4..032636bed6 100644 --- a/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts @@ -1,4 +1,4 @@ -import { mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { chmodSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { BunServices } from "@effect/platform-bun"; @@ -9,6 +9,7 @@ import { mockOutput } from "../../../tests/helpers/mocks.ts"; import type { LegacyDbSession } from "./legacy-db-connection.service.ts"; import { legacyApplyMigrationFile, + legacyApplySchemaFiles, legacyIsPipelineIncompatible, legacyMarkError, legacySeedGlobals, @@ -464,3 +465,49 @@ describe("legacySeedGlobals", () => { ); }); }); + +describe("legacyApplySchemaFiles", () => { + it.effect( + "reports a read failure with the workdir-relative path, not the absolute path used to read it (Go open supabase/... parity)", + () => { + // Go opens the workdir-relative `fp` from `schema_paths` directly (its process + // cwd is always the workdir, `ChangeWorkDir`), so a read failure reports + // `open supabase/unreadable.sql: ...`. This module never `process.chdir`s, so + // the real read needs an absolute path — but the wrapped read-failure message + // must still show the relative `supabase/...` form, not that absolute path. An + // unreadable file (a real permission failure, not a missing-path one) reproduces + // a genuine read failure while still passing the glob's own stat/type check — + // `stat` only needs directory execute permission, not read permission on the + // file itself, so this still resolves as a `"File"` match, unlike a directory + // (which the glob would instead expand via `legacyWalkSqlFiles`). + const dir = mkdtempSync(join(tmpdir(), "legacy-schema-files-read-fail-")); + const file = join(dir, "supabase", "unreadable.sql"); + mkdirSync(join(dir, "supabase"), { recursive: true }); + writeFileSync(file, "select 1;"); + chmodSync(file, 0o000); + const { session } = fakeSession(); + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const exit = yield* legacyApplySchemaFiles( + session, + fs, + path, + dir, + ["supabase/unreadable.sql"], + (message, suggestion) => + new TestError({ message: suggestion ? `${message} (${suggestion})` : message }), + ).pipe(Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const msg = JSON.stringify(exit.cause); + expect(msg).toContain("failed to open migration file: "); + expect(msg).toContain("supabase/unreadable.sql"); + expect(msg).not.toContain(dir); + } + chmodSync(file, 0o644); + rmSync(dir, { recursive: true, force: true }); + }).pipe(Effect.provide(BunServices.layer)); + }, + ); +}); From 17f2960bd7ef6b8715f92b0b646b2a86a5889950 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Wed, 5 Aug 2026 20:30:50 +0100 Subject: [PATCH 22/47] docs(cli): document SUPABASE_EXPERIMENTAL_PGDELTA_ENABLED reset gate (review: CLI-1958) reset.handler.ts's useSchemaFiles gate is experimental && resolvedVersion === "" && !toml.pgDelta.enabled, and legacyReadDbToml lets SUPABASE_EXPERIMENTAL_PGDELTA_ENABLED override [experimental.pgdelta].enabled. That env var therefore switches a --experimental remote reset between two different destructive code paths (schema-files vs timestamped migrations), which the side-effects table omitted. --- .../legacy/commands/db/reset/SIDE_EFFECTS.md | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md b/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md index 5c409c20a3..02b89e588a 100644 --- a/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md +++ b/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md @@ -82,14 +82,15 @@ races a restarting gateway. ## Environment Variables -| Variable | Purpose | Required? | -| ------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------- | -| `SUPABASE_ACCESS_TOKEN` | auth token for the `--linked` resolver path | no (falls back to keyring → `~/.supabase/access-token`) | -| `SUPABASE_DB_PASSWORD` | password for the linked/remote connection | no | -| `SUPABASE_YES` | auto-confirm the reset prompt | no (also `--yes`) | -| `SUPABASE_EXPERIMENTAL` | selects the remote schema-files apply branch | no (also `--experimental`) | -| `SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS` | overrides `[db.migrations].schema_paths` (viper `AutomaticEnv`, beats the config-file value) for the schema-files apply branch | no (no dedicated flag — config-file-only otherwise) | -| `SUPABASE_PROJECT_ID` | overrides the local container id (`utils.DbId`) | no | +| Variable | Purpose | Required? | +| --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------- | +| `SUPABASE_ACCESS_TOKEN` | auth token for the `--linked` resolver path | no (falls back to keyring → `~/.supabase/access-token`) | +| `SUPABASE_DB_PASSWORD` | password for the linked/remote connection | no | +| `SUPABASE_YES` | auto-confirm the reset prompt | no (also `--yes`) | +| `SUPABASE_EXPERIMENTAL` | selects the remote schema-files apply branch | no (also `--experimental`) | +| `SUPABASE_EXPERIMENTAL_PGDELTA_ENABLED` | overrides `[experimental.pgdelta].enabled`; a truthy value flips the reset gate (`experimental && resolvedVersion === "" && !toml.pgDelta.enabled`) back to timestamped migrations even with `--experimental` set — switches between two different destructive code paths | no | +| `SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS` | overrides `[db.migrations].schema_paths` (viper `AutomaticEnv`, beats the config-file value) for the schema-files apply branch | no (no dedicated flag — config-file-only otherwise) | +| `SUPABASE_PROJECT_ID` | overrides the local container id (`utils.DbId`) | no | ## Exit Codes From e37fc5c1896a9deb88ea61b5ad1a662392f05726 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Wed, 5 Aug 2026 20:31:00 +0100 Subject: [PATCH 23/47] fix(cli): sort SQL glob matches by UTF-8 byte order like Go (review: CLI-1958) Go's sort.Strings (Glob.SQLFiles and walkMatchedDir) orders raw UTF-8 bytes. JS's default Array.prototype.sort() compares UTF-16 code units instead, which disagrees with UTF-8 byte order for characters outside the Basic Multilingual Plane: a 4-byte-encoded supplementary character (lead byte 0xF0-0xF4) always sorts after a 3-byte-encoded BMP character (lead byte 0xE0-0xEF) in Go, but can sort before it under UTF-16 code unit comparison. Verified empirically against a real Go sort.Strings call. Add a shared utf8Compare comparator and use it for both the direct-match sort and the directory-walk sort, since both mirror Go call sites with the same byte-order contract. --- .../legacy/shared/legacy-sql-files-glob.ts | 29 +++++++++++++++++-- .../shared/legacy-sql-files-glob.unit.test.ts | 29 +++++++++++++++++++ 2 files changed, 56 insertions(+), 2 deletions(-) diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts index b272539991..ef72c9be41 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts @@ -18,6 +18,31 @@ const GLOB_META_CHARS = /[*?[]/u; // that escape — see `legacyPathMatch`'s escape handling below. const toSlash = (p: string): string => (process.platform === "win32" ? p.replaceAll("\\", "/") : p); +// Go's `sort.Strings` (used by both `Glob.SQLFiles`, `config.go:155`, and +// `walkMatchedDir`, `config.go:207`) orders strings by their raw UTF-8 BYTES — +// Go strings are just byte slices, so `strings.Compare` never decodes runes. JS's +// default `Array.prototype.sort()` instead compares UTF-16 CODE UNITS, which +// disagrees with UTF-8 byte order for any character outside the Basic Multilingual +// Plane: a supplementary-plane code point (`U+10000`+, a UTF-16 surrogate PAIR +// starting `0xD800`-`0xDBFF`) always UTF-8-encodes to 4 bytes leading `0xF0`-`0xF4`, +// while every 3-byte-encoded BMP character (`U+0800`-`U+FFFF`, UTF-8 lead byte +// `0xE0`-`0xEF`) is numerically SMALLER as a lead byte but can have a LARGER lone +// UTF-16 code unit than the surrogate pair's lead unit — so the two orderings can +// disagree. Verified empirically: sorting a 4-byte emoji filename against a +// 3-byte fullwidth-exclamation filename, Go's `sort.Strings` places the fullwidth +// exclamation FIRST, while JS's default `.sort()` places the emoji first. +const UTF8_ENCODER = new TextEncoder(); +const utf8Compare = (a: string, b: string): number => { + const bytesA = UTF8_ENCODER.encode(a); + const bytesB = UTF8_ENCODER.encode(b); + const len = Math.min(bytesA.length, bytesB.length); + for (let i = 0; i < len; i++) { + const diff = bytesA[i]! - bytesB[i]!; + if (diff !== 0) return diff; + } + return bytesA.length - bytesB.length; +}; + // Joins a matched directory (or glob-split directory prefix) with a child/entry name, // delegating to the injected `Path.Path` service for Go's `path.Join`-equivalent // cleaning. Two Go call sites build a path exactly this way, and both need the same @@ -227,7 +252,7 @@ const legacyWalkSqlFiles = ( } }); yield* walk(dir); - return collected.sort(); + return collected.sort(utf8Compare); }); /** Result of resolving SQL-file glob patterns against the workspace. */ @@ -327,7 +352,7 @@ export const legacySqlFilesGlob = Effect.fnUntraced(function* ( } continue; } - for (const match of [...matches].sort()) { + for (const match of [...matches].sort(utf8Compare)) { const fp = toSlash(match); // A directory match is expanded to its regular `.sql` files recursively // (`walkMatchedDir`, sorted); a file match is kept verbatim (`config.go:157-183`). diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts index 9e327b2654..c9a4720bf7 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts @@ -502,4 +502,33 @@ describe("legacySqlFilesGlob", () => { ); }, ); + + it.effect( + "sorts direct wildcard matches by UTF-8 byte order, not UTF-16 code units (Go sort.Strings parity)", + () => { + // Go's `sort.Strings` (`Glob.SQLFiles`, `config.go:155`) orders the raw UTF-8 bytes + // of each match. A supplementary-plane character (here, an emoji — 4-byte UTF-8, + // lead byte 0xF0) always sorts AFTER a 3-byte-encoded BMP character (here, a + // fullwidth exclamation mark — lead byte 0xEF) in Go, because 0xF0 > 0xEF. JS's + // default `Array.prototype.sort()` instead compares UTF-16 code units, under which + // the emoji's surrogate-pair lead unit (0xD83D) sorts BEFORE the fullwidth + // exclamation mark's code unit (0xFF01) — the opposite order. Verified empirically + // against a real Go `sort.Strings` call: it places the fullwidth-exclamation file + // first. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-utf8-sort-")); + const schemasDir = join(dir, "schemas"); + mkdirSync(schemasDir); + writeFileSync(join(schemasDir, "\u{1F600}.sql"), "select 1;"); // 😀 + writeFileSync(join(schemasDir, "！.sql"), "select 2;"); // ！ + return run(["schemas/*.sql"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["schemas/！.sql", "schemas/\u{1F600}.sql"]); + expect(result.warnings).toEqual([]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); }); From cf49af6ef15ee8fbde6ccf3fbfdfa8572a32923d Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Wed, 5 Aug 2026 21:26:46 +0100 Subject: [PATCH 24/47] fix(cli): fail walk on a raced disappearing subdirectory like Go's WalkDir (review: CLI-1958) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Go's fs.WalkDir types each walked child from its parent's cached ReadDir DirEntry, so a directory child that vanishes before WalkDir recurses into it makes the second ReadDir fail and propagates as a hard "failed to walk matched directory" error, discarding every file already collected (apps/cli-go/pkg/config/config.go:194-209) — verified empirically with a scratch fs.WalkDir probe. legacyWalkSqlFiles instead silently skipped any non-'.sql' child whose stat failed after listing, since this port's stat is a second syscall (unlike Go's free DirEntry type) and can't recover the lost entry's type. Fail the same way an unreadable still-present directory does when the vanished entry could have been a subdirectory, so a raced db reset --experimental-remote-schema-files-path schema tree can no longer silently drop nested schemas and report success. --- .../legacy/shared/legacy-sql-files-glob.ts | 34 +++++++++-- .../shared/legacy-sql-files-glob.unit.test.ts | 56 +++++++++++++++++++ 2 files changed, 86 insertions(+), 4 deletions(-) diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts index ef72c9be41..0fd058c27b 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts @@ -225,10 +225,10 @@ const legacyWalkSqlFiles = ( } const statResult = yield* fs.stat(childAbs).pipe(Effect.result); if (Result.isFailure(statResult)) { - // TOCTOU: this `.sql` child existed a moment ago in `names` (this directory's + // TOCTOU: this child existed a moment ago in `names` (this directory's // `readDirectory` snapshot) but is gone by the time we stat it here — e.g. removed - // by a concurrent process. Go never hits this window: `fs.WalkDir` decides - // file-vs-directory from the SAME `DirEntry` its parent `ReadDir` already + // by a concurrent process. Go never hits this window for the child's TYPE: `fs.WalkDir` + // decides file-vs-directory from the SAME `DirEntry` its parent `ReadDir` already // returned and never re-`Stat`s a child, so a `.sql` file that disappears here // stays in Go's declared file list, and only the later, real file-open fails // loudly. Verified empirically: a scratch `filepath.WalkDir` probe that deletes a @@ -240,8 +240,34 @@ const legacyWalkSqlFiles = ( // outcome, and let the real downstream read surface the failure. if (childRel.endsWith(".sql")) { collected.push(toSlash(childRel)); + continue; } - continue; + // TOCTOU, directory variant: the vanished entry could equally have been a + // SUBDIRECTORY of the matched tree, not a harmless non-`.sql` file — and Go's + // outcome for those two cases is NOT the same. For a directory `DirEntry`, + // `fs.WalkDir` unconditionally attempts a second `ReadDir` to recurse into it + // (`io/fs/walk.go`'s `walkDir`); when the child has since been removed, that + // second `ReadDir` fails, and `walkMatchedDir`'s callback propagates the error + // unchanged (`if err != nil { return err }`, `config.go:198-199`) — `fs.WalkDir` + // returns it, and `walkMatchedDir` wraps it as `failed to walk matched + // directory: `, discarding every file already collected + // (`config.go:205-206`). Verified empirically: a scratch `fs.WalkDir` probe that + // removes a nested subdirectory between the parent's `ReadDir` and the + // subdirectory's own `ReadDir` reproduces exactly this — zero files, error + // `failed to walk matched directory: open .../nested: no such file or + // directory` — never a silent skip, unlike the vanished-`.sql`-file case above + // (round 6). This FileSystem service can't recover the lost entry's type after + // the fact — Go's `DirEntry` type came for free from the same `ReadDir` syscall + // as the listing, whereas this port's `stat` is a second, separate syscall, so a + // raced disappearance here always loses the type along with the entry. Treat any + // non-`.sql` disappearance as a potential directory and fail the whole walk the + // same way an unreadable still-present directory does (the `readDirectory` + // failure above) — matching Go's fail-loud design intent (never silently apply a + // partial schema/seed set) rather than risk silently dropping an entire nested + // subtree of schema files. + return yield* Effect.fail( + `failed to walk matched directory: ${legacyErrorMessage(statResult.failure)}`, + ); } const childType = statResult.success.type; if (childType === "Directory") { diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts index c9a4720bf7..094bb78c7f 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts @@ -419,6 +419,62 @@ describe("legacySqlFilesGlob", () => { }, ); + it.effect( + "fails the whole walk when a non-'.sql' child whose stat fails could have been a subdirectory (Go WalkDir parity)", + () => { + // Round 6 (test above) established that a `.sql` FILE child racing away between + // `readDirectory` and this port's own `stat` call is best-effort included, matching + // Go's cached-`DirEntry` behaviour for regular files. But Go's `fs.WalkDir` does NOT + // treat every vanished child the same way: for a DIRECTORY `DirEntry`, it unconditionally + // attempts a second `ReadDir` to recurse into it; when that child is gone, the second + // `ReadDir` fails, and `walkMatchedDir`'s callback propagates the error unchanged + // (`if err != nil { return err }`, `config.go:198-199`) — `fs.WalkDir` returns it, and + // `walkMatchedDir` wraps it as `failed to walk matched directory: `, discarding + // every file already collected (`config.go:205-206`). Verified empirically with a + // scratch `fs.WalkDir` probe against `apps/cli-go`'s real `walkMatchedDir`: removing a + // nested subdirectory between the parent's `ReadDir` and the subdirectory's own `ReadDir` + // reproduces exactly this — zero files, `failed to walk matched directory: open + // .../nested: no such file or directory` — never a silent skip. This port's `stat` call + // is a second, separate syscall from the parent's `readDirectory` (unlike Go, which gets + // the child's type for free from the SAME syscall as the listing), so a raced + // disappearance here always loses the type along with the entry — a non-`.sql` name + // could equally have been the now-missing subdirectory, and must fail the same way an + // unreadable still-present directory does, not silently vanish along with the files it + // may have held. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-dir-race-")); + const schemasDir = join(dir, "schemas"); + mkdirSync(schemasDir); + writeFileSync(join(schemasDir, "a.sql"), "select 1;"); + const nestedDir = join(schemasDir, "nested"); + mkdirSync(nestedDir); + writeFileSync(join(nestedDir, "b.sql"), "select 2;"); + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const racyFs: FileSystem.FileSystem = { + ...fs, + stat: (p: string) => + p === nestedDir + ? Effect.sync(() => rmSync(nestedDir, { recursive: true })).pipe( + Effect.andThen(fs.stat(p)), + ) + : fs.stat(p), + }; + return yield* legacySqlFilesGlob(racyFs, path, ["schemas"], dir); + }).pipe( + Effect.provide(BunServices.layer), + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual([]); + expect(result.warnings).toHaveLength(1); + expect(result.warnings[0]).toMatch(/^failed to walk matched directory: /); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + it.effect("still expands a real (non-symlinked) nested directory recursively", () => { const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-nested-")); const schemasDir = join(dir, "schemas"); From 08b4dd86339eb0d381d2f906acccfe1cf86f463d Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Wed, 5 Aug 2026 22:20:03 +0100 Subject: [PATCH 25/47] fix(cli): preserve raw Windows glob pattern in "no files matched" warnings (review: CLI-1958) Go's config.Glob.files (config.go:145,155) passes filepath.ToSlash(pattern) only as an argument to fs.Glob for matching; the loop's own range variable `pattern` is never reassigned, so the "no files matched pattern" warning (and the skipEmptyGlobs skip-list) still reports the original pattern. This port had overwritten `pattern` with its slashed form and used that same variable for both matching and the warning text, so an absolute Windows pattern with backslashes (e.g. C:\schemas\*.sql) reported the slashed form in the warning instead of Go's backslash form. Match by keeping the raw pattern for display while only the slashed form feeds the actual glob. --- .../legacy/shared/legacy-sql-files-glob.ts | 13 +++++-- .../shared/legacy-sql-files-glob.unit.test.ts | 36 +++++++++++++++++++ 2 files changed, 46 insertions(+), 3 deletions(-) diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts index 0fd058c27b..40bc6eb23c 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts @@ -361,6 +361,13 @@ export const legacySqlFilesGlob = Effect.fnUntraced(function* ( const skipped: Array = []; for (const rawPattern of patterns) { + // Go's `filepath.ToSlash(pattern)` (`config.go:145`) is passed only as an ARGUMENT + // to `fs.Glob` — the loop's own `pattern` variable (Go's range variable) is never + // reassigned, so every later reference to it in THIS iteration (`hasGlobMeta`, the + // skipped-pattern list, and the "no files matched pattern" warning, all below) + // still reports the ORIGINAL, un-slashed pattern. On Windows, an absolute pattern + // with backslashes (`C:\schemas\*.sql`) must therefore warn with that raw backslash + // form, even though matching itself runs against the slashed form. const pattern = toSlash(rawPattern); // Go's `fs.Glob` validates the whole pattern up front (`Match(pattern, "")`); a // malformed glob is reported as `failed to glob files: ` and @@ -371,10 +378,10 @@ export const legacySqlFilesGlob = Effect.fnUntraced(function* ( } const matches = yield* globOne(fs, path, workdir, pattern); if (matches.length === 0) { - if (skipEmptyGlobs && GLOB_META_CHARS.test(pattern)) { - skipped.push(pattern); + if (skipEmptyGlobs && GLOB_META_CHARS.test(rawPattern)) { + skipped.push(rawPattern); } else { - warnings.push(`no files matched pattern: ${pattern}`); + warnings.push(`no files matched pattern: ${rawPattern}`); } continue; } diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts index 094bb78c7f..76136c4340 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts @@ -214,6 +214,42 @@ describe("legacySqlFilesGlob", () => { }, ); + it.effect( + "reports the raw backslash pattern in a 'no files matched' warning on Windows, not the slashed form used for matching (Go filepath.ToSlash parity)", + () => { + // Go passes `filepath.ToSlash(pattern)` only as an ARGUMENT to `fs.Glob` + // (`config.go:145`) — the loop's own `pattern` variable (Go's range variable) is + // never reassigned, so the "no files matched pattern: %s" warning (`config.go:155`) + // still reports the ORIGINAL backslash form. An absolute Windows pattern with + // backslashes that matches nothing must therefore warn with that backslash form, + // not the slashed one used internally to glob. Force win32 path semantics + // (`BunPath.layerWin32`) and this module's own `process.platform` gate, same as + // the drive-root test above. + const originalPlatform = process.platform; + Object.defineProperty(process, "platform", { value: "win32" }); + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-win-warn-")); + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + return yield* legacySqlFilesGlob(fs, path, ["C:\\schemas\\*.sql"], dir); + }).pipe( + Effect.provide(Layer.mergeAll(BunFileSystem.layer, BunPath.layerWin32)), + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual([]); + expect(result.warnings).toEqual(["no files matched pattern: C:\\schemas\\*.sql"]); + }), + ), + Effect.ensuring( + Effect.sync(() => { + Object.defineProperty(process, "platform", { value: originalPlatform }); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + it.effect( "normalizes a doubled slash when the matched directory itself has a trailing slash (Go path.Join parity)", () => { From 873f12a1a7a10e16af9c4b3dfe17285750d9c861 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Wed, 5 Aug 2026 22:20:41 +0100 Subject: [PATCH 26/47] fix(cli): report workdir-relative path in SQL glob stat-failure warnings (review: CLI-1958) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Go's fs.Stat(fsys, fp) (config.go:157) runs with its process cwd already the workdir (ChangeWorkDir, cmd/root.go), and fsys is always a real afero.OsFs (afero.OsFs.Stat delegates straight to os.Stat), so the resulting error's path is the workdir-relative fp Go passed in, not an absolute one. This module never process.chdir's, so the real stat needs an absolute path to work — but the wrapped "failed to stat matched file" warning was reporting that absolute path verbatim, leaking a local temp/workdir path a relative glob match (e.g. schemas/broken.sql) would never show in Go. Substitute the absolute path back to the relative fp in the message, matching the same display-path pattern legacyApplySchemaFiles already uses for read failures. --- .../src/legacy/shared/legacy-sql-files-glob.ts | 17 +++++++++++++---- .../shared/legacy-sql-files-glob.unit.test.ts | 10 ++++++++++ 2 files changed, 23 insertions(+), 4 deletions(-) diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts index 40bc6eb23c..9b43ac7f49 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts @@ -396,11 +396,20 @@ export const legacySqlFilesGlob = Effect.fnUntraced(function* ( // to treating it as a regular file (the previous behaviour here) would instead // hand a nonexistent path to the caller's later read, turning a warned-but- // otherwise-successful reset into a hard apply error. - const statResult = yield* fs - .stat(path.isAbsolute(fp) ? fp : path.join(workdir, fp)) - .pipe(Effect.result); + // + // Go's `fs.Stat(fsys, fp)` (`config.go:157`) runs with its process cwd already + // the workdir (`ChangeWorkDir`, `cmd/root.go`), so `fp` IS the exact string the + // real syscall sees and the resulting error's path is that workdir-relative + // `fp`. This module never `process.chdir`s, so the real stat needs an absolute + // path here — but the wrapped message must still report the relative `fp`, not + // the absolute path used to make the syscall work. Same substitution pattern as + // `legacyApplySchemaFiles`'s read-error display path (`legacy-migration-apply.ts`). + const absoluteFp = path.isAbsolute(fp) ? fp : path.join(workdir, fp); + const statResult = yield* fs.stat(absoluteFp).pipe(Effect.result); if (Result.isFailure(statResult)) { - warnings.push(`failed to stat matched file: ${legacyErrorMessage(statResult.failure)}`); + const rawMessage = legacyErrorMessage(statResult.failure); + const message = absoluteFp === fp ? rawMessage : rawMessage.split(absoluteFp).join(fp); + warnings.push(`failed to stat matched file: ${message}`); continue; } const matchType = statResult.success.type; diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts index 76136c4340..a468c134aa 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts @@ -91,6 +91,14 @@ describe("legacySqlFilesGlob", () => { // errors.Errorf("failed to stat matched file: %w", err)); continue }` (config.go:157-161) // — a match that disappears (or is a broken symlink) between the glob and this stat // becomes a warning and is skipped entirely, never silently treated as a regular file. + // + // Go's `fsys` here is always `afero.NewOsFs()` with the process cwd already the + // workdir (`ChangeWorkDir`, `cmd/root.go`), so `fs.Stat(fsys, fp)`'s embedded path + // in the resulting error is the workdir-RELATIVE `fp` (verified directly against + // `os.Stat`/`afero.OsFs.Stat`, which pass the name through to `os.Stat` unchanged). + // This module never `process.chdir`s, so the real stat needs an absolute path — but + // the warning must still report the relative form, not that absolute (temp-dir) + // path, or it would leak a local filesystem path Go never would. const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-stat-fail-")); const schemasDir = join(dir, "schemas"); mkdirSync(schemasDir); @@ -102,6 +110,8 @@ describe("legacySqlFilesGlob", () => { expect(result.files).toEqual(["schemas/good.sql"]); expect(result.warnings).toHaveLength(1); expect(result.warnings[0]).toMatch(/^failed to stat matched file: /); + expect(result.warnings[0]).toContain("schemas/broken.sql"); + expect(result.warnings[0]).not.toContain(dir); rmSync(dir, { recursive: true, force: true }); }), ), From 95072d7ae9b83fe32f41c256e61199e6f851d061 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Wed, 5 Aug 2026 23:40:29 +0100 Subject: [PATCH 27/47] fix(cli): keep SQL glob walk/stat errors and Windows match order Go-faithful (review: CLI-1958) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two Go-parity gaps in the experimental db reset remote schema-files path, both raised on the same review round: - legacyWalkSqlFiles wrapped a readDirectory failure (on the matched root directory OR any nested directory the walk descends into) with the raw absolute path the real syscall needed, instead of Go's workdir-relative display path. Go's fs.WalkDir runs against afero.OsFs with the process cwd already chdir'ed to the workdir (ChangeWorkDir, cmd/root.go), so its own ReadDir error always embeds the relative path. The same leak existed in the TOCTOU disappeared-subdirectory branch (a stat standing in for Go's second ReadDir). Hoisted the existing ad hoc absolute-to-relative substitution (already used for the "failed to stat matched file" warning and legacyApplySchemaFiles's read errors) into a shared legacyRelativizeErrorMessage helper and applied it at all four call sites, so none of them can drift independently again. - globOne slashed each direct glob match before returning it, but Go's config.Glob.SQLFiles sorts the RAW backslash-joined matches from fs.Glob/afero.Glob (built via filepath.Join) and only converts to forward slash AFTER that sort (config.go:145-156). Slashing first can reorder Windows matches whose directory components sort differently before/after separator normalization (verified against apps/cli-go's real Glob.Files/afero.Glob source directly: e.g. "a*/x.sql" matching "a\x.sql" and "a0\x.sql" sorts a0 first on raw bytes, but "a" first once slashed) — this could apply dependent schema/seed files in the wrong order after side effects. globOne now returns the raw join; the existing post-sort toSlash in legacySqlFilesGlob's caller is the only place slashing happens. Added regression coverage: a nested (not just matched-root) directory read-failure case, and a fully faked Windows FileSystem/Path exercising the raw-sort-before-slash ordering. --- .../src/legacy/shared/legacy-error-message.ts | 20 ++++ .../legacy/shared/legacy-migration-apply.ts | 12 +- .../legacy/shared/legacy-sql-files-glob.ts | 41 +++++-- .../shared/legacy-sql-files-glob.unit.test.ts | 108 ++++++++++++++++++ 4 files changed, 168 insertions(+), 13 deletions(-) diff --git a/apps/cli/src/legacy/shared/legacy-error-message.ts b/apps/cli/src/legacy/shared/legacy-error-message.ts index d0b4a72bb7..f13d8c64fe 100644 --- a/apps/cli/src/legacy/shared/legacy-error-message.ts +++ b/apps/cli/src/legacy/shared/legacy-error-message.ts @@ -9,3 +9,23 @@ export const legacyErrorMessage = (e: unknown): string => typeof e === "object" && e !== null && "message" in e && typeof e.message === "string" ? e.message : String(e); + +/** + * Substitutes an absolute path a real syscall needed back to its Go-equivalent + * display path inside an already-rendered error message. Go's own `fsys` is always a + * real `afero.OsFs` with the process cwd already `chdir`'ed into the workdir + * (`ChangeWorkDir`, `cmd/root.go`), so every Go error message embeds the + * workdir-relative (or verbatim-absolute) path it was actually called with. This shell + * deliberately never `process.chdir`s, so its own syscalls need a real absolute path to + * work — but the wrapped message must still report the Go-equivalent path, not the + * local temp/workdir absolute path the syscall needed, or it leaks a path Go would + * never show. Shared by every legacy module that wraps a raw filesystem failure this + * way (`legacy-sql-files-glob.ts`'s matched-file/matched-directory warnings, + * `legacy-migration-apply.ts`'s migration-file read errors). + */ +export const legacyRelativizeErrorMessage = ( + rawMessage: string, + absolutePath: string, + displayPath: string, +): string => + absolutePath === displayPath ? rawMessage : rawMessage.split(absolutePath).join(displayPath); diff --git a/apps/cli/src/legacy/shared/legacy-migration-apply.ts b/apps/cli/src/legacy/shared/legacy-migration-apply.ts index 7ed9e3b949..717de2b5cf 100644 --- a/apps/cli/src/legacy/shared/legacy-migration-apply.ts +++ b/apps/cli/src/legacy/shared/legacy-migration-apply.ts @@ -4,7 +4,7 @@ import { Output } from "../../shared/output/output.service.ts"; import { legacyBold } from "./legacy-colors.ts"; import type { LegacyDbExecError } from "./legacy-db-connection.errors.ts"; import type { LegacyDbSession } from "./legacy-db-connection.service.ts"; -import { legacyErrorMessage } from "./legacy-error-message.ts"; +import { legacyErrorMessage, legacyRelativizeErrorMessage } from "./legacy-error-message.ts"; import { INSERT_MIGRATION_VERSION, MIGRATE_FILE_PATTERN, @@ -180,11 +180,11 @@ const execMigrationBatch = ( // relative form Go would, not a leaked local temp/absolute path. const content = yield* fs.readFileString(migrationPath).pipe( Effect.mapError((error) => { - const rawMessage = legacyErrorMessage(error); - const message = - displayPath === migrationPath - ? rawMessage - : rawMessage.split(migrationPath).join(displayPath); + const message = legacyRelativizeErrorMessage( + legacyErrorMessage(error), + migrationPath, + displayPath, + ); return mapError(`failed to open migration file: ${message}`, "read"); }), ); diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts index 9b43ac7f49..18910dd3b0 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts @@ -1,6 +1,6 @@ import { Effect, type FileSystem, type Path, Result } from "effect"; -import { legacyErrorMessage } from "./legacy-error-message.ts"; +import { legacyErrorMessage, legacyRelativizeErrorMessage } from "./legacy-error-message.ts"; import { LEGACY_BAD_PATTERN_MESSAGE, legacyPathMatch } from "./legacy-path-match.ts"; const META_CHARS = /[*?[\\]/u; @@ -165,7 +165,19 @@ const globOne = ( // isn't enough here (Go's `afero.Glob` cleans via `filepath.Join(dir, n)`, so // `d === "/"`, `d === ""`, and a `dir` containing `.`/`..`/doubled-slash // segments must all clean the same way as the walked-child case does). - result.push(toSlash(joinRelChild(path, d, name))); + // + // Deliberately NOT `toSlash`'d here, on Windows: Go's `config.Glob.SQLFiles` + // sorts the RAW backslash-joined matches from `fs.Glob`/`afero.Glob` (built via + // `filepath.Join`, `config.go:145-155`) and only converts each surviving match + // to forward slash AFTER that sort (`fp := filepath.ToSlash(item)`, + // `config.go:156`). This function's own caller (`legacySqlFilesGlob`) already + // sorts the array `globOne` returns and slashes each item only after — slashing + // here too would sort forward-slash-joined strings instead of the raw + // backslash-joined ones, which can disagree: comparing `a\x.sql` vs `a0\x.sql` + // byte-for-byte puts `a0\x.sql` first (`\` is `0x5C`, greater than `0`'s + // `0x30`), while `a/x.sql` vs `a0/x.sql` puts `a/x.sql` first (`/` is `0x2F`, + // less than `0x30`) — a different order for the same two matches. + result.push(joinRelChild(path, d, name)); } } } @@ -198,11 +210,19 @@ const legacyWalkSqlFiles = ( const walk = (rel: string): Effect.Effect => Effect.gen(function* () { const absDir = path.isAbsolute(rel) ? rel : path.join(workdir, rel); + // Go's `fs.WalkDir` runs against `afero.OsFs` with the process cwd already the + // workdir (`ChangeWorkDir`, `cmd/root.go`), so a `ReadDir` failure — on the + // matched root `dir` itself, or on any nested directory the walk descends into — + // embeds the workdir-relative `rel` in its error text, never an absolute path. + // This module never `process.chdir`s, so the real read needs `absDir`, but the + // wrapped warning must still report `rel` (same substitution pattern as the + // matched-file stat failure below and `legacyApplySchemaFiles`'s read errors). const names = yield* fs .readDirectory(absDir) .pipe( Effect.mapError( - (error) => `failed to walk matched directory: ${legacyErrorMessage(error)}`, + (error) => + `failed to walk matched directory: ${legacyRelativizeErrorMessage(legacyErrorMessage(error), absDir, rel)}`, ), ); for (const name of names) { @@ -264,9 +284,13 @@ const legacyWalkSqlFiles = ( // same way an unreadable still-present directory does (the `readDirectory` // failure above) — matching Go's fail-loud design intent (never silently apply a // partial schema/seed set) rather than risk silently dropping an entire nested - // subtree of schema files. + // subtree of schema files. This `stat` stands in for the second `ReadDir` Go + // itself would issue on the vanished directory, so its display path needs the + // same absolute-to-relative substitution as the `readDirectory` failure above — + // `childAbs` is what the real (stand-in) syscall needed, `childRel` is what Go's + // own `ReadDir` error would embed. return yield* Effect.fail( - `failed to walk matched directory: ${legacyErrorMessage(statResult.failure)}`, + `failed to walk matched directory: ${legacyRelativizeErrorMessage(legacyErrorMessage(statResult.failure), childAbs, childRel)}`, ); } const childType = statResult.success.type; @@ -407,8 +431,11 @@ export const legacySqlFilesGlob = Effect.fnUntraced(function* ( const absoluteFp = path.isAbsolute(fp) ? fp : path.join(workdir, fp); const statResult = yield* fs.stat(absoluteFp).pipe(Effect.result); if (Result.isFailure(statResult)) { - const rawMessage = legacyErrorMessage(statResult.failure); - const message = absoluteFp === fp ? rawMessage : rawMessage.split(absoluteFp).join(fp); + const message = legacyRelativizeErrorMessage( + legacyErrorMessage(statResult.failure), + absoluteFp, + fp, + ); warnings.push(`failed to stat matched file: ${message}`); continue; } diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts index a468c134aa..f2c97fe336 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts @@ -260,6 +260,63 @@ describe("legacySqlFilesGlob", () => { }, ); + it.effect( + "sorts raw backslash-joined Windows matches BEFORE slashing, not after (Go afero.Glob/filepath.ToSlash ordering parity)", + () => { + // Go's `config.Glob.SQLFiles` (`config.go:145-156`) calls `fs.Glob`, which resolves + // to `afero.Glob`'s `glob()` helper — it builds each match with `filepath.Join(dir, + // n)` (OS-separator-joined, backslash on Windows) and NEVER slashes it. Only back in + // `SQLFiles`, AFTER `sort.Strings(matches)` sorts those raw backslash matches, does + // each surviving item get `filepath.ToSlash`'d. For a pattern like `a*/x.sql` + // matching both `a\x.sql` and `a0\x.sql`, sorting the RAW backslash strings byte-for + // -byte puts `a0\x.sql` first (`\` is `0x5C`, greater than `0`'s `0x30`) — but + // sorting the SLASHED strings instead would put `a/x.sql` first (`/` is `0x2F`, less + // than `0x30`), a different order for the same two matches. `globOne` must therefore + // push the raw joined match (slashing only happens in the caller's post-sort loop), + // matching Go's real order exactly. Fully faked filesystem (no real Windows host + // available): `readDirectory`/`stat` return canned results keyed by the exact + // backslash-joined paths `BunPath.layerWin32`'s `path.join` computes. + const originalPlatform = process.platform; + Object.defineProperty(process, "platform", { value: "win32" }); + const scratchDir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-win-sort-")); + const canaryFile = join(scratchDir, "canary.sql"); + writeFileSync(canaryFile, "select 1;"); + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const fileInfo = yield* fs.stat(canaryFile); + const workdir = "/workdir"; + const winFs: FileSystem.FileSystem = { + ...fs, + readDirectory: (p: string) => { + if (p === workdir) return Effect.succeed(["a", "a0"]); + if (p === "\\workdir\\a" || p === "\\workdir\\a0") return Effect.succeed(["x.sql"]); + return fs.readDirectory(p); + }, + stat: (p: string) => + p === "\\workdir\\a\\x.sql" || p === "\\workdir\\a0\\x.sql" + ? Effect.succeed(fileInfo) + : fs.stat(p), + }; + return yield* legacySqlFilesGlob(winFs, path, ["a*/x.sql"], workdir); + }).pipe( + Effect.provide(Layer.mergeAll(BunFileSystem.layer, BunPath.layerWin32)), + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["a0/x.sql", "a/x.sql"]); + expect(result.warnings).toEqual([]); + }), + ), + Effect.ensuring( + Effect.sync(() => { + Object.defineProperty(process, "platform", { value: originalPlatform }); + rmSync(scratchDir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + it.effect( "normalizes a doubled slash when the matched directory itself has a trailing slash (Go path.Join parity)", () => { @@ -514,6 +571,12 @@ describe("legacySqlFilesGlob", () => { expect(result.files).toEqual([]); expect(result.warnings).toHaveLength(1); expect(result.warnings[0]).toMatch(/^failed to walk matched directory: /); + // This `stat` failure stands in for the second `ReadDir` Go's own `fs.WalkDir` + // would issue on the vanished directory — whose error, like every other Go + // filesystem error here, embeds the workdir-relative path, not this port's + // absolute stand-in syscall path. + expect(result.warnings[0]).toContain("schemas/nested"); + expect(result.warnings[0]).not.toContain(dir); rmSync(dir, { recursive: true, force: true }); }), ), @@ -549,6 +612,12 @@ describe("legacySqlFilesGlob", () => { // matched directory: ...` and discards every file already found — never an empty // (successful) match. Verified empirically against `apps/cli-go`: an unreadable // matched directory makes `Glob.SQLFiles` return that error with zero files. + // + // Go's `fsys` here is always `afero.OsFs` with the process cwd already the workdir + // (`ChangeWorkDir`, `cmd/root.go`), so the `ReadDir` error's embedded path is the + // workdir-relative matched directory (`schemas`), never an absolute one. This + // module never `process.chdir`s, so the real read needs an absolute path, but the + // warning must still report the relative form. const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-walk-fail-")); const schemasDir = join(dir, "schemas"); mkdirSync(schemasDir); @@ -560,6 +629,8 @@ describe("legacySqlFilesGlob", () => { expect(result.files).toEqual([]); expect(result.warnings).toHaveLength(1); expect(result.warnings[0]).toMatch(/^failed to walk matched directory: /); + expect(result.warnings[0]).toContain("schemas"); + expect(result.warnings[0]).not.toContain(dir); }), ), Effect.ensuring( @@ -572,6 +643,43 @@ describe("legacySqlFilesGlob", () => { }, ); + it.effect.skipIf(isRoot)( + "surfaces a NESTED directory-read failure during walk with a workdir-relative path, not the matched root's (Go WalkDir parity)", + () => { + // Same leak as the matched-root-directory case above, but for a failure during the + // RECURSIVE walk of an already-descended subdirectory — a distinct code path inside + // Go's `fs.WalkDir` callback (it recurses via the SAME `ReadDir` call the matched + // root used, `io/fs/walk.go`), and this port's `walk()` closure recurses the same + // way. The matched root ("schemas") itself is readable; only "schemas/nested" is + // not, so the warning must report "schemas/nested", never the workdir's absolute + // temp-dir path. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-walk-fail-nested-")); + const schemasDir = join(dir, "schemas"); + const nestedDir = join(schemasDir, "nested"); + mkdirSync(nestedDir, { recursive: true }); + writeFileSync(join(schemasDir, "a.sql"), "select 1;"); + writeFileSync(join(nestedDir, "b.sql"), "select 2;"); + chmodSync(nestedDir, 0o000); + return run(["schemas"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual([]); + expect(result.warnings).toHaveLength(1); + expect(result.warnings[0]).toMatch(/^failed to walk matched directory: /); + expect(result.warnings[0]).toContain("schemas/nested"); + expect(result.warnings[0]).not.toContain(dir); + }), + ), + Effect.ensuring( + Effect.sync(() => { + chmodSync(nestedDir, 0o755); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + it.effect.skipIf(isRoot)( "keeps files from a sibling pattern when only one matched directory fails to walk", () => { From e61a58c01acb18062bba35f968759ca313cd94c8 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Wed, 5 Aug 2026 23:40:41 +0100 Subject: [PATCH 28/47] fix(cli): match Go's filepath.IsAbs for Windows-rooted schema/seed paths (review: CLI-1958) Go's resolve() gates the supabase/-join for [db.migrations].schema_paths and [db.seed].sql_paths on the stdlib filepath.IsAbs (config.go:970-980), and on Windows that requires a volume name -- a drive letter (C:\) or a UNC prefix (\\server\share) -- before a path counts as absolute (internal/filepathlite/path_windows.go's IsAbs/volumeNameLen, read directly since there is no Windows host available here). A bare leading separator like "/schemas/*.sql" has no volume name, so Go treats it as RELATIVE and joins it to "supabase/schemas/*.sql". legacyResolveSeedSqlPath instead deferred to the injected Path.Path service's isAbsolute, which selects Node's path.win32 on an actual Windows host; Node's win32 isAbsolute treats a leading separator as rooted at the current drive -- i.e. absolute -- skipping Go's supabase/-join entirely. Verified empirically: path.win32.isAbsolute( "/schemas/*.sql") is true, while Go's real filepath.IsAbs on the same input is false. Added legacyGoIsAbs to apply Go's exact Windows rule only at this config-resolution call site (real filesystem calls elsewhere still need the platform's own isAbsolute). legacyResolveSeedSqlPath is the single function both schema_paths and sql_paths resolve through, so this fixes both surfaces. --- .../shared/legacy-db-config.toml-read.ts | 34 ++++++++++++++++- .../legacy-db-config.toml-read.unit.test.ts | 38 ++++++++++++++++++- 2 files changed, 70 insertions(+), 2 deletions(-) diff --git a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts index d0fdaf0f59..923389dcb5 100644 --- a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts +++ b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts @@ -524,6 +524,38 @@ function legacyJoinSupabaseSeedPath(pattern: string): string { return out.length === 0 ? "." : out.join("/"); } +/** + * Go's `filepath.IsAbs` on Windows (`internal/filepathlite/path_windows.go`'s + * `IsAbs`/`volumeNameLen`) requires a volume name — a drive letter (`C:\`) or a UNC + * prefix (`\\server\share`) — before a path counts as absolute; a bare leading + * separator (`/schemas`, `\schemas`) has no volume name, so Go treats it as RELATIVE + * and joins it under `supabase/`. `pathSvc.isAbsolute` is backed by `node:path`, which + * selects `path.win32` on an actual Windows host, and Node's win32 `isAbsolute` treats + * a bare leading separator as rooted at the *current drive* — i.e. absolute — so it + * disagrees with Go on exactly this shape. Verified empirically: Node's + * `path.win32.isAbsolute("/schemas/*.sql")` is `true`, while Go's `filepath.IsAbs` on + * the same input is `false` (`volumeNameLen` returns `0` — none of its drive-letter, + * UNC, or device-path cases match a path with no volume component). Only the resolve + * step below (`config.go:970-980`'s literal `!filepath.IsAbs(pattern)` gate for + * `[db.seed].sql_paths`/`[db.migrations].schema_paths`) needs this Go-exact rule — + * real filesystem calls elsewhere in this shell still need the platform's own + * `isAbsolute` to resolve an actual path on disk. + */ +const legacyGoIsAbs = (pathSvc: Path.Path, pattern: string): boolean => { + if (process.platform !== "win32") { + return pathSvc.isAbsolute(pattern); + } + const isSeparator = (c: string | undefined): boolean => c === "/" || c === "\\"; + // Drive-letter volume (`C:\`, `c:/`): Go's `volumeNameLen` accepts any byte before + // `:` (case 2, `path[1] === ':'`), then `IsAbs` requires a separator right after. + if (pattern.length >= 3 && pattern[1] === ":" && isSeparator(pattern[2])) { + return true; + } + // UNC volume (`\\server\share`, `//server/share`): Go's `IsAbs` treats a + // double-separator-prefixed volume as absolute unconditionally. + return pattern.length >= 2 && isSeparator(pattern[0]) && isSeparator(pattern[1]); +}; + /** * Resolves a single seed `sql_paths` entry to Go's config-load form: a relative * pattern is joined under `supabase/` (Go's `path.Join`, `config.go:918-921`); an @@ -532,7 +564,7 @@ function legacyJoinSupabaseSeedPath(pattern: string): string { * `resolveSeedSqlPaths`, `cmd/db.go`) so both feed the glob the same resolved paths. */ export const legacyResolveSeedSqlPath = (pathSvc: Path.Path, pattern: string): string => - pattern.length === 0 || pathSvc.isAbsolute(pattern) + pattern.length === 0 || legacyGoIsAbs(pathSvc, pattern) ? pattern : legacyJoinSupabaseSeedPath(pattern); diff --git a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts index 0d345ba87b..2ae1f41fb0 100644 --- a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts @@ -1,7 +1,7 @@ import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; -import { BunServices } from "@effect/platform-bun"; +import { BunPath, BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { Effect, Exit, FileSystem, Option, Path } from "effect"; @@ -11,6 +11,7 @@ import { legacyLoadProjectEnv, legacyReadDbToml, legacyResolveDeclarativeDir, + legacyResolveSeedSqlPath, } from "./legacy-db-config.toml-read.ts"; function withConfig(content: string | undefined, poolerUrl?: string) { @@ -349,6 +350,41 @@ describe("legacyReadDbToml", () => { }, ); + it.effect( + "on Windows, resolves a leading-slash schema/seed path pattern under supabase/ instead of treating it as absolute (Go filepath.IsAbs parity)", + () => { + // Go's `resolve()` gates the `supabase/`-join on `!filepath.IsAbs(pattern)` + // (`config.go:976-980`), and `filepath.IsAbs` on Windows requires a volume name — + // a drive letter (`C:\`) or UNC prefix (`\\server\share`) — before a path counts as + // absolute (`internal/filepathlite/path_windows.go`'s `IsAbs`/`volumeNameLen`). A + // bare leading `/` has no volume name, so Go treats `/schemas/*.sql` as RELATIVE and + // joins it to `supabase/schemas/*.sql`. Node's `path.win32.isAbsolute`, backing the + // injected `Path.Path` service on an actual Windows host, instead treats a leading + // separator as rooted at the current drive — i.e. absolute — which would otherwise + // skip Go's `supabase/`-join entirely. Exercises `legacyResolveSeedSqlPath` (the + // single function `[db.migrations].schema_paths` and `[db.seed].sql_paths` both + // resolve through) directly with `BunPath.layerWin32`, rather than through the full + // `legacyReadDbToml` pipeline: that pipeline's OWN config-file lookup also runs + // through the same injected `Path.Path` service to open the real (POSIX-pathed, + // since this test host isn't Windows) temp config file on disk, so forcing win32 + // path semantics there breaks the read itself rather than exercising the fix. + const originalPlatform = process.platform; + Object.defineProperty(process, "platform", { value: "win32" }); + return Effect.gen(function* () { + const path = yield* Path.Path; + const resolved = legacyResolveSeedSqlPath(path, "/schemas/*.sql"); + expect(resolved).toBe("supabase/schemas/*.sql"); + }).pipe( + Effect.provide(BunPath.layerWin32), + Effect.ensuring( + Effect.sync(() => { + Object.defineProperty(process, "platform", { value: originalPlatform }); + }), + ), + ); + }, + ); + it.effect( "weakly coerces non-string db.migrations.schema_paths array elements (Go mapstructure parity)", () => { From 30bf58ad420c3d118aea8da075f527ba778f6e77 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Thu, 6 Aug 2026 00:38:39 +0100 Subject: [PATCH 29/47] fix(cli): match Go's Lstat glob fast path, scanner buffer limit, and byte-offset path.Match retries (review: CLI-1958) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - legacy-sql-files-glob.ts: a literal (no-metacharacter) schema_paths/sql_paths entry naming a broken symlink now surfaces "failed to stat matched file: ..." instead of "no files matched pattern" — Go's afero.Glob fast path probes via Lstat (which doesn't follow the link), while fs.exists here followed it. - legacy-migration-apply.ts: honor SUPABASE_SCANNER_BUFFER_SIZE in execMigrationBatch (shared by ApplyMigrations/SeedGlobals/applySchemaFiles), matching Go's bufio.Scanner: token too long failure instead of silently applying an oversized statement when the limit is explicitly configured. - legacy-path-match.ts: rewrite legacyPathMatch to operate on UTF-8 bytes instead of JS code points, matching Go's byte-offset `*`-retry loop and unicode/utf8.DecodeRuneInString semantics for multibyte filenames. All three verified against apps/cli-go with scratch Go probes before fixing. --- .../legacy/shared/legacy-migration-apply.ts | 114 +++++++- .../legacy-migration-apply.unit.test.ts | 86 ++++++ .../src/legacy/shared/legacy-path-match.ts | 244 +++++++++++++----- .../shared/legacy-path-match.unit.test.ts | 17 ++ .../legacy/shared/legacy-sql-files-glob.ts | 22 +- .../shared/legacy-sql-files-glob.unit.test.ts | 30 +++ .../cli/src/legacy/shared/legacy-sql-split.ts | 69 +++-- 7 files changed, 491 insertions(+), 91 deletions(-) diff --git a/apps/cli/src/legacy/shared/legacy-migration-apply.ts b/apps/cli/src/legacy/shared/legacy-migration-apply.ts index 717de2b5cf..d28237dbb4 100644 --- a/apps/cli/src/legacy/shared/legacy-migration-apply.ts +++ b/apps/cli/src/legacy/shared/legacy-migration-apply.ts @@ -11,7 +11,7 @@ import { legacyCreateMigrationTable, } from "./legacy-migration-history.ts"; import { legacySqlFilesGlob } from "./legacy-sql-files-glob.ts"; -import { legacySplitAndTrim } from "./legacy-sql-split.ts"; +import { legacySplitAndTrim, legacySplitSqlTokens } from "./legacy-sql-split.ts"; /** * Applying a migration file failed (Go's `ApplyMigrations` / `ExecBatch` error). @@ -102,6 +102,110 @@ type LegacyBatchItem = const utf8ByteLength = (value: string): number => new TextEncoder().encode(value).length; +// Go's `startBufSize` (`pkg/parser/token.go:15`) — the fixed initial `bufio.Scanner` +// buffer `parser.Split` pre-allocates before applying the configured/default max +// (`scanner.Buffer(buf, maxbuf)` where `buf := make([]byte, startBufSize)`). The +// scanner's buffer therefore starts at exactly this size regardless of how small +// `SUPABASE_SCANNER_BUFFER_SIZE` is set, and `bufio.Scanner`'s too-long check +// (`len(s.buf) >= s.maxTokenSize`, `$GOROOT/src/bufio/scan.go:200`) only fires once +// the buffer is full — so a statement must reach at least this many raw bytes +// before Go can ever raise `bufio.ErrTooLong`, no matter how small the override. +// Verified empirically against `apps/cli-go/pkg/parser` (`parser.SplitAndTrim`): a +// single-statement probe of exactly 4096 raw bytes always succeeds — even with +// `SUPABASE_SCANNER_BUFFER_SIZE` set to 10 bytes — while 4097 bytes always fails; +// with the override set above this floor (e.g. 5000 bytes), the exact same +// pattern repeats at the override's own value (5000 succeeds, 5001 fails). +const GO_SCANNER_START_BUF_SIZE = 4096; + +/** + * Go's `viper.GetSizeInBytes("SCANNER_BUFFER_SIZE")` (env-prefixed + * `SUPABASE_SCANNER_BUFFER_SIZE`, `pkg/parser/token.go:87`): an integer byte count, + * optionally suffixed `k`/`K`/`m`/`M`/`g`/`G` (× 1024/1024²/1024³) plus a trailing + * `b`/`B` (e.g. `"5MB"`, `"256KB"`, or a bare byte count). Ported 1:1 from viper's + * own parser (`parseSizeInBytes`, `github.com/spf13/viper@v1.21.0/util.go:151-179`): + * an unparseable or non-positive result is treated as unset (`0`). + */ +const legacyParseScannerBufferSize = (raw: string): number => { + let value = raw.trim(); + let multiplier = 1; + const lastIndex = value.length - 1; + if (lastIndex > 1 && (value[lastIndex] === "b" || value[lastIndex] === "B")) { + switch (value[lastIndex - 1]!.toLowerCase()) { + case "k": + multiplier = 1 << 10; + value = value.slice(0, lastIndex - 1).trim(); + break; + case "m": + multiplier = 1 << 20; + value = value.slice(0, lastIndex - 1).trim(); + break; + case "g": + multiplier = 1 << 30; + value = value.slice(0, lastIndex - 1).trim(); + break; + default: + value = value.slice(0, lastIndex).trim(); + break; + } + } + const size = Number.parseInt(value, 10); + return Number.isFinite(size) && size > 0 ? size * multiplier : 0; +}; + +/** + * Go's `parser.Split`/`SplitAndTrim` (`pkg/parser/token.go:81-119`) enforces + * `SUPABASE_SCANNER_BUFFER_SIZE` as the `bufio.Scanner`'s max token size — but only + * when the env var is actually SET: `parseFile` (`pkg/migration/file.go:55-70`) + * otherwise grows the package-level `parser.MaxScannerCapacity` to the real file's + * byte length before the scan even starts (`viper.IsSet("SCANNER_BUFFER_SIZE")` + * gates the auto-growth), so the DEFAULT (unset) path can never hit + * `bufio.ErrTooLong` for a file read this way — no single statement can be bigger + * than the whole file. Every caller of `execMigrationBatch` mirrors exactly this + * Go call site (`ApplyMigrations`, `SeedGlobals`, `applySchemaFiles` all read their + * file via `NewMigrationFromFile`/`parseFile`), so this is the correct single home + * for the check (CLI-1958 review) rather than duplicating it per caller. + * + * Fails the same way `parser.Split` does on the FIRST raw (pre-trim) statement + * whose byte length exceeds the effective limit (`Math.max(configured, + * GO_SCANNER_START_BUF_SIZE)` — see that constant's comment). `"After statement + * : …"` reports the count and RAW text of the last statement successfully + * scanned BEFORE the oversized one: Go's loop body (`token = scanner.Text()`) + * never runs for the failing `Scan()` call, so `token` still holds whatever the + * previous iteration left it as (`""` if the very first statement is already + * oversized) — verified empirically against the same `apps/cli-go/pkg/parser` + * probe. This is a "read"-phase failure (`NewMigrationFromFile`/`parseFile` + * returns before `apply.go`'s `CmdSuggestion` is ever set), so it carries no + * suggestion, same as the file-open failure above. + */ +const checkScannerBufferSize = ( + content: string, + mapError: (message: string, phase: "read" | "exec") => E, +): Effect.Effect => { + const raw = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + if (raw === undefined) return Effect.void; + const configuredLimit = legacyParseScannerBufferSize(raw); + if (configuredLimit <= 0) return Effect.void; + const limit = Math.max(configuredLimit, GO_SCANNER_START_BUF_SIZE); + let emitted = 0; + let lastRaw = ""; + for (const token of legacySplitSqlTokens(content)) { + if (utf8ByteLength(token.raw) > limit) { + const suggestion = `Try setting SUPABASE_SCANNER_BUFFER_SIZE=5MB (current size is ${Math.floor(configuredLimit / 1024)}KB)`; + return Effect.fail( + mapError( + `bufio.Scanner: token too long\nAfter statement ${emitted}: ${lastRaw}\n${suggestion}`, + "read", + ), + ); + } + if (token.trimmed.length > 0) { + emitted += 1; + lastRaw = token.raw; + } + } + return Effect.void; +}; + /** * Port of Go's `markError` (`pkg/migration/file.go:117-132`): renders a `^` caret * line under the error position of the failing statement. `pos` is the server's @@ -189,6 +293,14 @@ const execMigrationBatch = ( }), ); + // Still `NewMigrationFromFile`/`parseFile`'s territory (`pkg/migration/file.go:55-70`) — + // `parser.SplitAndTrim` runs INSIDE `parseFile`, before `ExecBatch` ever sees the + // statements, so a `SUPABASE_SCANNER_BUFFER_SIZE` violation is a "read"-phase + // failure like the open failure above, not an "exec"-phase one. See + // `checkScannerBufferSize`'s comment for why this is a no-op unless the env var + // is explicitly set. + yield* checkScannerBufferSize(content, mapError); + // Everything below mirrors Go's `(*MigrationFile).ExecBatch` (`pkg/migration/file.go`), // which runs against an already-read file — so every failure from here on is an // execution failure, tagged "exec" (as opposed to the "read" failure above, which diff --git a/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts b/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts index 032636bed6..10eba8032a 100644 --- a/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts @@ -510,4 +510,90 @@ describe("legacyApplySchemaFiles", () => { }).pipe(Effect.provide(BunServices.layer)); }, ); + + it.effect( + "rejects an oversized statement when SUPABASE_SCANNER_BUFFER_SIZE is configured (Go bufio.Scanner: token too long parity)", + () => { + // Go's `parser.Split` (`pkg/parser/token.go:81-119`) only enforces + // `SUPABASE_SCANNER_BUFFER_SIZE` when it's explicitly set — `parseFile` + // otherwise auto-grows the scanner to the real file's byte length, so the + // DEFAULT path can never hit `bufio.ErrTooLong`. With it set below a single + // statement's raw byte length, Go fails with `bufio.Scanner: token too long` + // instead of silently applying the oversized statement — verified empirically + // against `apps/cli-go/pkg/parser` (a `parser.SplitAndTrim` scratch probe). + const dir = mkdtempSync(join(tmpdir(), "legacy-schema-files-scanner-")); + mkdirSync(join(dir, "supabase"), { recursive: true }); + const file = join(dir, "supabase", "big.sql"); + // A single, un-splittable statement whose raw text exceeds the 4096-byte floor + // (Go's `bufio.Scanner` starts at that size regardless of the configured limit). + writeFileSync(file, `SELECT 1;\nSELECT '${"a".repeat(5000)}';\n`); + const { session } = fakeSession(); + const previous = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = "100b"; + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const exit = yield* legacyApplySchemaFiles( + session, + fs, + path, + dir, + ["supabase/big.sql"], + (message, suggestion) => + new TestError({ message: suggestion ? `${message} (${suggestion})` : message }), + ).pipe(Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const msg = JSON.stringify(exit.cause); + expect(msg).toContain("bufio.Scanner: token too long"); + expect(msg).toContain("After statement 1: SELECT 1;"); + expect(msg).toContain("Try setting SUPABASE_SCANNER_BUFFER_SIZE=5MB"); + } + }).pipe( + Effect.ensuring( + Effect.sync(() => { + if (previous === undefined) delete process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + else process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = previous; + rmSync(dir, { recursive: true, force: true }); + }), + ), + Effect.provide(BunServices.layer), + ); + }, + ); + + it.effect( + "applies an oversized statement fine when SUPABASE_SCANNER_BUFFER_SIZE is unset (Go's default auto-grows to file size)", + () => { + const dir = mkdtempSync(join(tmpdir(), "legacy-schema-files-scanner-default-")); + mkdirSync(join(dir, "supabase"), { recursive: true }); + const file = join(dir, "supabase", "big.sql"); + writeFileSync(file, `SELECT '${"a".repeat(5000)}';\n`); + const { session, calls } = fakeSession(); + const previous = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + delete process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + yield* legacyApplySchemaFiles( + session, + fs, + path, + dir, + ["supabase/big.sql"], + (message, suggestion) => + new TestError({ message: suggestion ? `${message} (${suggestion})` : message }), + ); + expect(calls.some((c) => c.kind === "exec" && c.sql.startsWith("SELECT 'a"))).toBe(true); + }).pipe( + Effect.ensuring( + Effect.sync(() => { + if (previous !== undefined) process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = previous; + rmSync(dir, { recursive: true, force: true }); + }), + ), + Effect.provide(BunServices.layer), + ); + }, + ); }); diff --git a/apps/cli/src/legacy/shared/legacy-path-match.ts b/apps/cli/src/legacy/shared/legacy-path-match.ts index b7dadac49b..f57f2acd75 100644 --- a/apps/cli/src/legacy/shared/legacy-path-match.ts +++ b/apps/cli/src/legacy/shared/legacy-path-match.ts @@ -1,19 +1,30 @@ /** - * Faithful port of Go's stdlib `path.Match` (`$GOROOT/src/path/match.go`), used - * by the seed-file globber to expand `[db.seed] sql_paths` exactly like the Go + * Faithful, BYTE-level port of Go's stdlib `path.Match` (`$GOROOT/src/path/match.go`), + * used by the seed-file globber to expand `[db.seed] sql_paths` exactly like the Go * CLI's `config.Glob.Files` → `io/fs.Glob` → `path.Match` chain. * - * Why a hand port instead of a JS `RegExp`: Go's glob grammar and JS regex - * character classes diverge — POSIX classes (`[[:alpha:]]`), `\d`/`\w`, and a - * leading `^` mean different things, and Go reports a malformed class as an - * error (`path.ErrBadPattern`) where JS would silently reinterpret it. Compiling - * each segment to a `RegExp` leaked those JS-only semantics; porting the - * algorithm keeps seed globbing byte-compatible with Go, including the - * malformed-pattern handling. + * Why byte-level, not code-point-level: Go strings are raw byte slices — every index, + * slice, and length in `path.Match` operates on UTF-8 BYTES, not decoded characters. + * This matters most in the `*`-retry loop (`legacyPathMatch`'s inner `for` below): + * Go retries the starred chunk at every BYTE offset of `name`, including offsets that + * land in the middle of a multibyte UTF-8 character. When that happens, Go's + * `unicode/utf8.DecodeRuneInString` decodes the LEADING (invalid, mid-character) + * continuation byte as a single-byte `U+FFFD` "rune" — it never throws and never + * consumes more than one byte for invalid input — so a `?` operator in the retried + * chunk can advance past exactly one such byte and let the retry succeed where a + * code-point-stepping port would not. Verified empirically against `apps/cli-go` + * (a `path.Match` scratch probe): `Match("*??.sql", "！.sql")` — a single fullwidth + * exclamation mark, U+FF01, 3 UTF-8 bytes — returns `true`: the second `?` in the + * retried chunk lands on the fullwidth character's 2nd and 3rd bytes (both mid-character + * continuation bytes, each decoded as one `U+FFFD` "rune"), not on a real code point. A + * prior code-point-based port of this file returned `false` for that same case. * - * Pure — no Effect / service dependencies. Operates on code points; Go mixes - * byte and rune indexing, which is equivalent for the BMP characters that occur - * in real seed paths. + * Why a hand port instead of a JS `RegExp`: Go's glob grammar and JS regex character + * classes diverge — POSIX classes (`[[:alpha:]]`), `\d`/`\w`, and a leading `^` mean + * different things, and Go reports a malformed class as an error (`path.ErrBadPattern`) + * where JS would silently reinterpret it. Compiling each segment to a `RegExp` leaked + * those JS-only semantics; porting the algorithm keeps seed globbing byte-compatible + * with Go, including the malformed-pattern handling and the byte-offset retry above. */ /** Mirrors Go's `path.Match` return `(matched bool, err error)`; `badPattern` ↔ `path.ErrBadPattern`. */ @@ -27,106 +38,198 @@ export const LEGACY_BAD_PATTERN_MESSAGE = "syntax error in pattern"; const BAD_PATTERN: LegacyPathMatchResult = { matched: false, badPattern: true }; -/** UTF-16 width (1 or 2 code units) of a code point. */ -const runeWidth = (cp: number): number => (cp > 0xffff ? 2 : 1); +const UTF8_ENCODER = new TextEncoder(); + +/** + * `TextEncoder.encode` returns `Uint8Array` (never a + * `SharedArrayBuffer`-backed view) — naming that explicitly so every `.subarray()` + * slice threaded through this module's helpers keeps that narrower type instead of + * widening to the generic `Uint8Array` default. + */ +type Bytes = Uint8Array; + +const RUNE_ERROR = 0xfffd; +const SLASH = 0x2f; +const STAR = 0x2a; +const QUESTION = 0x3f; +const LBRACKET = 0x5b; +const RBRACKET = 0x5d; +const CARET = 0x5e; +const HYPHEN = 0x2d; +const BACKSLASH = 0x5c; + +interface DecodedRune { + readonly r: number; + readonly size: number; +} + +/** + * Port of Go's `unicode/utf8.DecodeRuneInString`, decoding the rune starting at byte + * offset `i` of `b`. Any invalid or truncated sequence decodes as `(RuneError, 1)` — + * never throws, never consumes more than the single invalid lead byte — matching Go's + * documented behaviour exactly (`$GOROOT/src/unicode/utf8/utf8.go`'s `first` table and + * `acceptRanges`, transcribed here as explicit range checks per lead byte rather than + * the table itself, for readability; verified to agree with the table for every lead + * byte class, including the overlong/surrogate/out-of-range exclusions on `0xE0`, + * `0xED`, `0xF0`, and `0xF4`). + */ +const decodeRune = (b: Bytes, i: number): DecodedRune => { + const n = b.length - i; + if (n <= 0) return { r: RUNE_ERROR, size: 0 }; + const b0 = b[i]!; + if (b0 < 0x80) return { r: b0, size: 1 }; + let size: number; + let lo: number; + let hi: number; + if (b0 >= 0xc2 && b0 <= 0xdf) { + size = 2; + lo = 0x80; + hi = 0xbf; + } else if (b0 === 0xe0) { + size = 3; // Excludes the overlong 3-byte encoding. + lo = 0xa0; + hi = 0xbf; + } else if ((b0 >= 0xe1 && b0 <= 0xec) || b0 === 0xee || b0 === 0xef) { + size = 3; + lo = 0x80; + hi = 0xbf; + } else if (b0 === 0xed) { + size = 3; // Excludes the UTF-16 surrogate range U+D800-U+DFFF. + lo = 0x80; + hi = 0x9f; + } else if (b0 === 0xf0) { + size = 4; // Excludes the overlong 4-byte encoding. + lo = 0x90; + hi = 0xbf; + } else if (b0 >= 0xf1 && b0 <= 0xf3) { + size = 4; + lo = 0x80; + hi = 0xbf; + } else if (b0 === 0xf4) { + size = 4; // Caps the range at U+10FFFF. + lo = 0x80; + hi = 0x8f; + } else { + // 0x80-0xC1: a bare continuation byte or an overlong 2-byte lead. 0xF5-0xFF: past + // the max valid lead byte. Both are invalid lead bytes. + return { r: RUNE_ERROR, size: 1 }; + } + if (n < size) return { r: RUNE_ERROR, size: 1 }; + const b1 = b[i + 1]!; + if (b1 < lo || b1 > hi) return { r: RUNE_ERROR, size: 1 }; + if (size === 2) return { r: ((b0 & 0x1f) << 6) | (b1 & 0x3f), size: 2 }; + const b2 = b[i + 2]!; + if (b2 < 0x80 || b2 > 0xbf) return { r: RUNE_ERROR, size: 1 }; + if (size === 3) return { r: ((b0 & 0x0f) << 12) | ((b1 & 0x3f) << 6) | (b2 & 0x3f), size: 3 }; + const b3 = b[i + 3]!; + if (b3 < 0x80 || b3 > 0xbf) return { r: RUNE_ERROR, size: 1 }; + return { + r: ((b0 & 0x07) << 18) | ((b1 & 0x3f) << 12) | ((b2 & 0x3f) << 6) | (b3 & 0x3f), + size: 4, + }; +}; interface ScanChunk { readonly star: boolean; - readonly chunk: string; - readonly rest: string; + readonly chunk: Bytes; + readonly rest: Bytes; } /** Go's `scanChunk`: the next non-`*` segment, possibly preceded by a `*`. */ -const scanChunk = (pattern: string): ScanChunk => { +const scanChunk = (pattern: Bytes): ScanChunk => { let star = false; let p = pattern; - while (p.length > 0 && p[0] === "*") { - p = p.slice(1); + while (p.length > 0 && p[0] === STAR) { + p = p.subarray(1); star = true; } let inrange = false; for (let i = 0; i < p.length; i++) { - const c = p[i]; - if (c === "\\") { + const c = p[i]!; + if (c === BACKSLASH) { if (i + 1 < p.length) i++; - } else if (c === "[") { + } else if (c === LBRACKET) { inrange = true; - } else if (c === "]") { + } else if (c === RBRACKET) { inrange = false; - } else if (c === "*" && !inrange) { - return { star, chunk: p.slice(0, i), rest: p.slice(i) }; + } else if (c === STAR && !inrange) { + return { star, chunk: p.subarray(0, i), rest: p.subarray(i) }; } } - return { star, chunk: p, rest: "" }; + return { star, chunk: p, rest: p.subarray(p.length) }; }; interface GetEsc { readonly r: number; - readonly rest: string; + readonly rest: Bytes; readonly bad: boolean; } /** Go's `getEsc`: a possibly-escaped character from inside a class. */ -const getEsc = (chunk: string): GetEsc => { - if (chunk.length === 0 || chunk[0] === "-" || chunk[0] === "]") { +const getEsc = (chunk: Bytes): GetEsc => { + if (chunk.length === 0 || chunk[0] === HYPHEN || chunk[0] === RBRACKET) { return { r: 0, rest: chunk, bad: true }; } let c = chunk; - if (c[0] === "\\") { - c = c.slice(1); + if (c[0] === BACKSLASH) { + c = c.subarray(1); if (c.length === 0) return { r: 0, rest: c, bad: true }; } - const r = c.codePointAt(0)!; - const rest = c.slice(runeWidth(r)); - // Go errors when the class has no closing `]` after this character. + const { r, size } = decodeRune(c, 0); + // Go: `if r == utf8.RuneError && n == 1 { err = ErrBadPattern }` — a genuinely + // invalid byte, not a literal (valid, 3-byte-encoded) U+FFFD character. + if (r === RUNE_ERROR && size === 1) return { r, rest: c.subarray(1), bad: true }; + const rest = c.subarray(size); return { r, rest, bad: rest.length === 0 }; }; interface MatchChunk { - readonly rest: string; + readonly rest: Bytes; readonly ok: boolean; readonly bad: boolean; } -const BAD_CHUNK: MatchChunk = { rest: "", ok: false, bad: true }; +const EMPTY_BYTES = new Uint8Array(0); +const BAD_CHUNK: MatchChunk = { rest: EMPTY_BYTES, ok: false, bad: true }; /** * Go's `matchChunk`: match the all-single-char-operators `chunk` against the * start of `s`. Once the match fails the loop keeps walking `chunk` (no longer * reading `s`) so a malformed pattern is still reported. */ -const matchChunk = (chunkIn: string, sIn: string): MatchChunk => { +const matchChunk = (chunkIn: Bytes, sIn: Bytes): MatchChunk => { let chunk = chunkIn; let s = sIn; let failed = false; while (chunk.length > 0) { if (!failed && s.length === 0) failed = true; - const op = chunk[0]; - if (op === "[") { + const op = chunk[0]!; + if (op === LBRACKET) { let r = 0; if (!failed) { - r = s.codePointAt(0)!; - s = s.slice(runeWidth(r)); + const decoded = decodeRune(s, 0); + r = decoded.r; + s = s.subarray(decoded.size); } - chunk = chunk.slice(1); + chunk = chunk.subarray(1); let negated = false; - if (chunk.length > 0 && chunk[0] === "^") { + if (chunk.length > 0 && chunk[0] === CARET) { negated = true; - chunk = chunk.slice(1); + chunk = chunk.subarray(1); } let match = false; let nrange = 0; for (;;) { - if (chunk.length > 0 && chunk[0] === "]" && nrange > 0) { - chunk = chunk.slice(1); + if (chunk.length > 0 && chunk[0] === RBRACKET && nrange > 0) { + chunk = chunk.subarray(1); break; } const lo = getEsc(chunk); if (lo.bad) return BAD_CHUNK; chunk = lo.rest; let hi = lo.r; - if (chunk[0] === "-") { - const hiEsc = getEsc(chunk.slice(1)); + if (chunk.length > 0 && chunk[0] === HYPHEN) { + const hiEsc = getEsc(chunk.subarray(1)); if (hiEsc.bad) return BAD_CHUNK; chunk = hiEsc.rest; hi = hiEsc.r; @@ -135,30 +238,30 @@ const matchChunk = (chunkIn: string, sIn: string): MatchChunk => { nrange++; } if (match === negated) failed = true; - } else if (op === "?") { + } else if (op === QUESTION) { if (!failed) { - const cp = s.codePointAt(0)!; - if (cp === 0x2f) failed = true; // '/' - s = s.slice(runeWidth(cp)); + if (s[0] === SLASH) failed = true; + const { size } = decodeRune(s, 0); + s = s.subarray(size); } - chunk = chunk.slice(1); - } else if (op === "\\") { - chunk = chunk.slice(1); + chunk = chunk.subarray(1); + } else if (op === BACKSLASH) { + chunk = chunk.subarray(1); if (chunk.length === 0) return BAD_CHUNK; if (!failed) { if (chunk[0] !== s[0]) failed = true; - s = s.slice(1); + s = s.subarray(1); } - chunk = chunk.slice(1); + chunk = chunk.subarray(1); } else { if (!failed) { if (chunk[0] !== s[0]) failed = true; - s = s.slice(1); + s = s.subarray(1); } - chunk = chunk.slice(1); + chunk = chunk.subarray(1); } } - return failed ? { rest: "", ok: false, bad: false } : { rest: s, ok: true, bad: false }; + return failed ? { rest: EMPTY_BYTES, ok: false, bad: false } : { rest: s, ok: true, bad: false }; }; /** @@ -167,14 +270,16 @@ const matchChunk = (chunkIn: string, sIn: string): MatchChunk => { * pattern is malformed, mirroring Go's `path.ErrBadPattern`. */ export const legacyPathMatch = (pattern: string, name: string): LegacyPathMatchResult => { - let pat = pattern; - let nm = name; + let pat = UTF8_ENCODER.encode(pattern); + let nm = UTF8_ENCODER.encode(name); while (pat.length > 0) { const scan = scanChunk(pat); pat = scan.rest; - if (scan.star && scan.chunk === "") { - // Trailing `*` matches the rest of the name unless it contains a `/`. - return { matched: !nm.includes("/"), badPattern: false }; + if (scan.star && scan.chunk.length === 0) { + // Trailing `*` matches the rest of the name unless it contains a `/`. `/` is + // never a UTF-8 continuation byte, so a raw byte scan is safe here regardless + // of any multibyte characters elsewhere in `nm`. + return { matched: !nm.includes(SLASH), badPattern: false }; } const m = matchChunk(scan.chunk, nm); if (m.bad) return BAD_PATTERN; @@ -185,10 +290,11 @@ export const legacyPathMatch = (pattern: string, name: string): LegacyPathMatchR continue; } if (scan.star) { - // Look for a match skipping one code point at a time; `*` cannot cross `/`. + // Look for a match skipping one BYTE at a time (see this file's top comment + // for why byte-, not code-point-, stepping matters here); `*` cannot cross `/`. let advanced = false; - for (let i = 0; i < nm.length && nm[i] !== "/"; i++) { - const skip = matchChunk(scan.chunk, nm.slice(i + 1)); + for (let i = 0; i < nm.length && nm[i] !== SLASH; i++) { + const skip = matchChunk(scan.chunk, nm.subarray(i + 1)); if (skip.bad) return BAD_PATTERN; if (skip.ok) { if (pat.length === 0 && skip.rest.length > 0) continue; @@ -203,7 +309,7 @@ export const legacyPathMatch = (pattern: string, name: string): LegacyPathMatchR while (pat.length > 0) { const tail = scanChunk(pat); pat = tail.rest; - if (matchChunk(tail.chunk, "").bad) return BAD_PATTERN; + if (matchChunk(tail.chunk, EMPTY_BYTES).bad) return BAD_PATTERN; } return { matched: false, badPattern: false }; } diff --git a/apps/cli/src/legacy/shared/legacy-path-match.unit.test.ts b/apps/cli/src/legacy/shared/legacy-path-match.unit.test.ts index 66a849d48b..7285f5c3e4 100644 --- a/apps/cli/src/legacy/shared/legacy-path-match.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-path-match.unit.test.ts @@ -49,6 +49,23 @@ describe("legacyPathMatch", () => { }); }); + describe("byte-offset `*` retry against multibyte characters (Go path.Match parity)", () => { + it.each([ + // Go's byte-offset `*`-retry loop can land mid-multibyte-character and have a `?` + // consume the resulting invalid continuation byte as a single-byte `U+FFFD` "rune" + // — producing matches a code-point-stepping port would miss. Verified empirically + // against `apps/cli-go`'s `path.Match` (a fullwidth exclamation mark, U+FF01, is a + // single character but 3 UTF-8 bytes; an emoji, U+1F600, is 4 UTF-8 bytes): + ["*??.sql", "！.sql", true], + ["*??.sql", "😀.sql", true], + ["*?.sql", "！.sql", true], + ["*???.sql", "！.sql", false], + ["schemas/*??.sql", "schemas/！.sql", true], + ] as const)("%s ~ %s => %s", (pattern, name, expected) => { + expect(legacyPathMatch(pattern, name).matched).toBe(expected); + }); + }); + describe("escapes", () => { it.each([ ["\\*.sql", "*.sql", true], diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts index 18910dd3b0..f0bb8eb1c4 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts @@ -143,9 +143,27 @@ const globOne = ( // Absolute patterns resolve against the filesystem root; relative ones are // rooted at the workdir. const resolve = (p: string): string => (path.isAbsolute(p) ? p : path.join(workdir, p)); - // No metacharacters: a direct existence check (Go's `fs.Glob` fast path). + // No metacharacters: Go's `fs.Glob`/`afero.Glob` fast path (`match.go:34-40`) probes + // via `Lstat` (`OsFs.LstatIfPossible` → `os.Lstat`, verified empirically against + // `afero@v1.15.0`), which does NOT follow a symlink — so a literal pattern naming a + // BROKEN symlink still Lstat-succeeds (the link itself exists) and is reported as a + // match; the follow-up `fs.Stat(fsys, fp)` in `legacySqlFilesGlob` below (which DOES + // follow it) is what fails, with `failed to stat matched file: ...`. `fs.exists` here + // is Effect's `access`-based check (Node's `fs.access`), which follows the symlink + // like a normal `Stat` and would wrongly report "no files matched pattern" instead — + // verified empirically: `fs.access` on a broken symlink resolves ENOENT while + // `fs.lstat` on the same path succeeds. Probe for the entry itself the same + // no-follow way `legacyWalkSqlFiles` below already does for a walked child: `readLink` + // succeeds only for a symlink (broken or not), so treat that as an Lstat success + // before falling back to the normal existence check for everything else. if (!META_CHARS.test(pattern)) { - const exists = yield* fs.exists(resolve(pattern)).pipe(Effect.orElseSucceed(() => false)); + const resolved = resolve(pattern); + const isSymlink = yield* fs.readLink(resolved).pipe( + Effect.map(() => true), + Effect.orElseSucceed(() => false), + ); + const exists = + isSymlink || (yield* fs.exists(resolved).pipe(Effect.orElseSucceed(() => false))); return exists ? [pattern] : []; } const { dir, file } = splitPath(pattern); diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts index f2c97fe336..bbd995b5f0 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts @@ -119,6 +119,36 @@ describe("legacySqlFilesGlob", () => { }, ); + it.effect( + "surfaces a stat failure for a LITERAL (no-metacharacter) pattern naming a broken symlink, instead of reporting no match (Go afero.Glob Lstat parity)", + () => { + // Go's `fs.Glob`/`afero.Glob` no-metacharacter fast path (`match.go:34-40`) probes + // via `Lstat` (`OsFs.LstatIfPossible` → `os.Lstat`), which does NOT follow a + // symlink — so a LITERAL pattern naming a broken symlink still Lstat-succeeds (the + // link itself exists) and is reported as a match; the follow-up `fs.Stat` above is + // what then fails with `failed to stat matched file: ...`, exactly like the + // wildcard-pattern case the previous test covers. Verified empirically against + // `apps/cli-go` (`afero.Glob`/`fs.Stat` scratch probe): a literal broken-symlink + // pattern always Globs to a match and always fails the follow-up Stat — never + // "no files matched pattern". + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-literal-symlink-")); + const schemasDir = join(dir, "schemas"); + mkdirSync(schemasDir); + symlinkSync(join(schemasDir, "does-not-exist.sql"), join(schemasDir, "broken.sql")); + return run(["schemas/broken.sql"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual([]); + expect(result.warnings).toHaveLength(1); + expect(result.warnings[0]).toMatch(/^failed to stat matched file: /); + expect(result.warnings[0]).toContain("schemas/broken.sql"); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + it.effect( "keeps a bare root ('/') as the directory when a glob pattern's meta character is in the first path component (Go afero.Glob parity)", () => { diff --git a/apps/cli/src/legacy/shared/legacy-sql-split.ts b/apps/cli/src/legacy/shared/legacy-sql-split.ts index 4eec9072a3..ebb17d7d2c 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-split.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-split.ts @@ -143,24 +143,16 @@ class AtomicState implements State { } } -/** - * Splits `sql` into raw statements (comments/whitespace preserved), then applies - * the optional transforms to each. Mirrors Go's `parser.Split`. - */ -export function legacySplitSql( - sql: string, - ...transform: ReadonlyArray<(s: string) => string> -): string[] { +/** The FSM traversal shared by every `legacySplitSql*` entry point below. */ +function splitRaw(sql: string): string[] { let state: State = new ReadyState(); - const statements: string[] = []; + const tokens: string[] = []; let acc = ""; for (const rune of Array.from(sql)) { acc += rune; const next = state.next(rune, acc); if (next === null) { - let token = acc; - for (const apply of transform) token = apply(token); - if (token.length > 0) statements.push(token); + tokens.push(acc); acc = ""; state = new ReadyState(); } else { @@ -168,21 +160,60 @@ export function legacySplitSql( } } // Trailing non-terminated statement at EOF. - if (acc.length > 0) { - let token = acc; + if (acc.length > 0) tokens.push(acc); + return tokens; +} + +/** + * Splits `sql` into raw statements (comments/whitespace preserved), then applies + * the optional transforms to each. Mirrors Go's `parser.Split`. + */ +export function legacySplitSql( + sql: string, + ...transform: ReadonlyArray<(s: string) => string> +): string[] { + const statements: string[] = []; + for (const raw of splitRaw(sql)) { + let token = raw; for (const apply of transform) token = apply(token); if (token.length > 0) statements.push(token); } return statements; } +/** Go's `parser.SplitAndTrim`'s per-token transform: trim trailing `;` then surrounding whitespace. */ +const legacyTrimStatement = (token: string): string => token.replace(/;+$/u, "").trim(); + /** Mirrors Go's `parser.SplitAndTrim`: trim trailing `;` then surrounding whitespace. */ export function legacySplitAndTrim(sql: string): string[] { - return legacySplitSql( - sql, - (token) => token.replace(/;+$/u, ""), - (token) => token.trim(), - ); + return legacySplitSql(sql, legacyTrimStatement); +} + +/** One statement, paired with both its RAW and trimmed forms. */ +export interface LegacySplitSqlToken { + /** The exact text `legacySplitSql(sql)` (no transforms) would emit for this statement. */ + readonly raw: string; + /** `legacyTrimStatement(raw)` — what `legacySplitAndTrim` emits, including when empty. */ + readonly trimmed: string; +} + +/** + * Same FSM traversal as {@link legacySplitAndTrim}, but pairs each statement's RAW + * (pre-trim) text with its trimmed form instead of discarding the raw text once + * emitted. Go's `bufio.Scanner`-based `parser.Split` (`pkg/parser/token.go:81-119`) + * enforces `SUPABASE_SCANNER_BUFFER_SIZE` against the untransformed + * `scanner.Text()` — the RAW form — and its `bufio.ErrTooLong` message reports + * that same raw text for the LAST successfully scanned statement, so a caller + * replicating that check (`legacy-migration-apply.ts`'s `execMigrationBatch`) + * needs both forms, not just the trimmed one `legacySplitAndTrim` returns. + * + * Unlike `legacySplitSql`/`legacySplitAndTrim`, this does NOT drop a statement + * whose trimmed form is empty — callers that replicate Go's `len(stats)` counter + * (which only increments for a non-empty trimmed statement) need to see every raw + * token, including the ones `legacySplitAndTrim` itself would filter out. + */ +export function legacySplitSqlTokens(sql: string): ReadonlyArray { + return splitRaw(sql).map((raw) => ({ raw, trimmed: legacyTrimStatement(raw) })); } // `(?i)drop\s+` — Go's `dropStatementPattern` (`internal/db/diff/diff.go:100`, From 525d855b60d775877cec203a47b178626b487506 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Thu, 6 Aug 2026 01:41:13 +0100 Subject: [PATCH 30/47] fix(cli): strictly parse SUPABASE_SCANNER_BUFFER_SIZE and honor project-env overrides (review: CLI-1958) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Go's strconv.ParseInt fails the whole parse on trailing garbage ("5M" is NOT 5 MiB and NOT 5 bytes — viper only recognizes a k/m/g multiplier immediately before a trailing "b"/"B"), then falls back to parser's hardcoded 256KiB default cap, not to "no limit". The scanner-buffer check now reproduces that strict parse + default-cap fallback, and threads the reset handler's already-loaded legacyLoadProjectEnv map into legacyApplySchemaFiles so a SUPABASE_SCANNER_BUFFER_SIZE set only in supabase/.env is honored, matching Go's loadNestedEnv (which os.Setenvs project-.env values before any command body runs). --- .../legacy/commands/db/reset/reset.handler.ts | 14 +- .../legacy/shared/legacy-migration-apply.ts | 109 +++++++++++++- .../legacy-migration-apply.unit.test.ts | 141 ++++++++++++++++++ 3 files changed, 256 insertions(+), 8 deletions(-) diff --git a/apps/cli/src/legacy/commands/db/reset/reset.handler.ts b/apps/cli/src/legacy/commands/db/reset/reset.handler.ts index f701aa5539..1473eb058e 100644 --- a/apps/cli/src/legacy/commands/db/reset/reset.handler.ts +++ b/apps/cli/src/legacy/commands/db/reset/reset.handler.ts @@ -362,7 +362,19 @@ export const legacyDbReset = Effect.fn("legacy.db.reset")(function* (flags: Lega // NOTHING rather than falling back to migrations — CLI-1958). const useSchemaFiles = experimental && resolvedVersion === "" && !toml.pgDelta.enabled; if (useSchemaFiles) { - yield* legacyApplySchemaFiles(session, fs, path, workdir, toml.schemaPaths, applyError); + // `projectEnv` (loaded above, before `experimental`/`yes` resolve) is threaded + // through so a `SUPABASE_SCANNER_BUFFER_SIZE` set only in `supabase/.env` is + // honored here exactly like Go's `loadNestedEnv` (see + // `checkScannerBufferSize`'s doc comment, `legacy-migration-apply.ts`). + yield* legacyApplySchemaFiles( + session, + fs, + path, + workdir, + toml.schemaPaths, + applyError, + projectEnv, + ); } else if (toml.migrationsEnabled) { const locals = yield* legacyListLocalMigrations(fs, path, migrationsDir); // LoadPartialMigrations filter: version === "" || v <= version. diff --git a/apps/cli/src/legacy/shared/legacy-migration-apply.ts b/apps/cli/src/legacy/shared/legacy-migration-apply.ts index d28237dbb4..f6cd2a6f3a 100644 --- a/apps/cli/src/legacy/shared/legacy-migration-apply.ts +++ b/apps/cli/src/legacy/shared/legacy-migration-apply.ts @@ -117,6 +117,24 @@ const utf8ByteLength = (value: string): number => new TextEncoder().encode(value // pattern repeats at the override's own value (5000 succeeds, 5001 fails). const GO_SCANNER_START_BUF_SIZE = 4096; +// Go's `parser.MaxScannerCapacity` (`pkg/parser/token.go:19`) — the hardcoded default +// `parser.Split` falls back to when `viper.GetSizeInBytes("SCANNER_BUFFER_SIZE")` +// returns `0`. Reached whenever the env var is SET but resolves to a non-positive size +// — including a value `legacyParseScannerBufferSize` can't parse at all. Verified +// empirically against `apps/cli-go/pkg/parser` + vendored `viper@v1.21.0`: +// `SUPABASE_SCANNER_BUFFER_SIZE=5M` (a bare multiplier suffix with NO trailing `b`/`B`) +// behaves byte-for-byte identically to the var being completely unset from the default +// cap's own first-failure point on — because `parseSizeInBytes` only ever recognizes a +// `k`/`m`/`g` multiplier when it immediately precedes a trailing `b`/`B` +// (`util.go:156-174`); "5M" never strips a suffix, so it falls through to +// `cast.ToInt("5M")`, which fails whole (not a leading-digits prefix parse — unlike +// JS's lenient `Number.parseInt`) and returns `0`. This is NOT the same as truly unset, +// though: `viper.IsSet("SCANNER_BUFFER_SIZE")` is still `true` (the var IS present, just +// unparseable), so `parseFile`'s file-size auto-growth (see `checkScannerBufferSize`'s +// doc comment) never runs — the cap stays pinned at this hardcoded default regardless of +// the real file's size, unlike the genuinely-unset case where it grows to match. +const GO_DEFAULT_MAX_SCANNER_CAPACITY = 256 * 1024; + /** * Go's `viper.GetSizeInBytes("SCANNER_BUFFER_SIZE")` (env-prefixed * `SUPABASE_SCANNER_BUFFER_SIZE`, `pkg/parser/token.go:87`): an integer byte count, @@ -124,6 +142,31 @@ const GO_SCANNER_START_BUF_SIZE = 4096; * `b`/`B` (e.g. `"5MB"`, `"256KB"`, or a bare byte count). Ported 1:1 from viper's * own parser (`parseSizeInBytes`, `github.com/spf13/viper@v1.21.0/util.go:151-179`): * an unparseable or non-positive result is treated as unset (`0`). + * + * The multiplier is recognized ONLY when the string's LAST character is literally + * `b`/`B` — a bare `"5M"` (no trailing `B`) is NOT 5 MiB in real Go: `sizeStr[lastChar]` + * isn't `b`/`B`, so the multiplier branch never runs and the whole (unstripped) string + * is handed to `cast.ToInt`, which fails on the trailing letter and yields `0`. Do NOT + * special-case a bare `k`/`m`/`g` suffix here; that would make this port accept a value + * real Go rejects. + * + * Go's inner `lastChar > 1` gate (`util.go:158`) means the trailing-`B`-strip ALSO never + * runs for a 2-character value like `"5B"`/`"5b"` — only 3+ characters (an actual + * multiplier letter, or at least one digit, before the `B`) reach the switch at all — so + * `"5B"` is unstripped, `cast.ToInt("5B")` fails, and the whole thing is `0` too, same as + * `"5M"`. `cast.ToInt` (`strconv.ParseInt`) also requires the ENTIRE remaining string to + * be a clean integer — trailing garbage fails the WHOLE parse, unlike JS's lenient + * `Number.parseInt`, which stops at the first non-digit and returns whatever numeric + * prefix it found (`Number.parseInt("5M", 10) === 5`, silently discarding the "M", where + * Go's parse rejects the string outright). `cast.ToInt` does tolerate one decimal point + * via its own `trimDecimal` (keeps only the integer part, e.g. `"5.5"` → `"5"`), so allow + * exactly that one exception. All of the above verified empirically against + * `apps/cli-go/pkg/parser` + vendored `viper@v1.21.0` + * (`"5"→5, "5B"→0, "50B"→50, "5KB"→5120, "5M"→0, "0"→0, "5.5"→5, "5.5MB"→5242880`). + * Known residual delta: Go's `strconv.ParseInt(s, 0, 0)` uses base `0`, so it also + * accepts a `0x`/`0o`/`0b`-prefixed literal (`"0x5"` → `5`) — not reproduced here as a + * realistic byte-size override would never use one; flagging so a future parity sweep + * doesn't rediscover it. */ const legacyParseScannerBufferSize = (raw: string): number => { let value = raw.trim(); @@ -148,6 +191,7 @@ const legacyParseScannerBufferSize = (raw: string): number => { break; } } + if (!/^[+-]?\d+(?:\.\d*)?$/.test(value)) return 0; const size = Number.parseInt(value, 10); return Number.isFinite(size) && size > 0 ? size * multiplier : 0; }; @@ -176,21 +220,43 @@ const legacyParseScannerBufferSize = (raw: string): number => { * probe. This is a "read"-phase failure (`NewMigrationFromFile`/`parseFile` * returns before `apply.go`'s `CmdSuggestion` is ever set), so it carries no * suggestion, same as the file-open failure above. + * + * `projectEnv`, when given, is the caller's already-loaded `legacyLoadProjectEnv` map: + * Go's `loadNestedEnv` (`pkg/config/config.go:1220`) `os.Setenv`s every project-`.env` + * key that isn't already in the shell env BEFORE `ParseDatabaseConfig` returns — i.e. + * before ANY command body (including this scan) runs — so `viper.AutomaticEnv()` sees a + * `supabase/.env`-only `SUPABASE_SCANNER_BUFFER_SIZE` exactly like a real shell-exported + * one. Defaults to `{}` for callers that haven't threaded a project-env map through + * (shell-only, same as before this parameter existed). */ const checkScannerBufferSize = ( content: string, mapError: (message: string, phase: "read" | "exec") => E, + projectEnv: Readonly> = {}, ): Effect.Effect => { - const raw = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + const raw = + process.env["SUPABASE_SCANNER_BUFFER_SIZE"] ?? projectEnv["SUPABASE_SCANNER_BUFFER_SIZE"]; if (raw === undefined) return Effect.void; const configuredLimit = legacyParseScannerBufferSize(raw); - if (configuredLimit <= 0) return Effect.void; - const limit = Math.max(configuredLimit, GO_SCANNER_START_BUF_SIZE); + // `configuredLimit <= 0` covers both an explicit non-positive size and an unparseable + // value (e.g. a bare "5M", see `GO_DEFAULT_MAX_SCANNER_CAPACITY`'s comment) — Go's + // `viper.GetSizeInBytes` collapses all of these to `0` too, and `parser.Split` then + // falls back to its OWN hardcoded default cap, not to "no limit". + const limit = + configuredLimit > 0 + ? Math.max(configuredLimit, GO_SCANNER_START_BUF_SIZE) + : GO_DEFAULT_MAX_SCANNER_CAPACITY; + // Go's suggestion reports `maxbuf>>10` — the EFFECTIVE cap actually passed to + // `scanner.Buffer` (`pkg/parser/token.go:110`), which is the raw configured value + // (even below the `GO_SCANNER_START_BUF_SIZE` floor — the floor only affects when + // `bufio.ErrTooLong` can fire, never the number Go prints) when positive, or the + // hardcoded default once Go has fallen back to it. + const reportedLimit = configuredLimit > 0 ? configuredLimit : GO_DEFAULT_MAX_SCANNER_CAPACITY; let emitted = 0; let lastRaw = ""; for (const token of legacySplitSqlTokens(content)) { if (utf8ByteLength(token.raw) > limit) { - const suggestion = `Try setting SUPABASE_SCANNER_BUFFER_SIZE=5MB (current size is ${Math.floor(configuredLimit / 1024)}KB)`; + const suggestion = `Try setting SUPABASE_SCANNER_BUFFER_SIZE=5MB (current size is ${Math.floor(reportedLimit / 1024)}KB)`; return Effect.fail( mapError( `bufio.Scanner: token too long\nAfter statement ${emitted}: ${lastRaw}\n${suggestion}`, @@ -254,6 +320,9 @@ const TYPE_NAME_PATTERN = /type "([^"]+)" does not exist/; * apply.go:65-69), so role/globals files (`legacySeedGlobals`) stay reset-free like Go. * When `forceNoVersion` is set the history insert is skipped regardless of filename * (Go's `SeedGlobals` clears `Version`). + * + * `projectEnv` is forwarded to {@link checkScannerBufferSize} — see its own doc comment + * for why a project-`.env`-only `SUPABASE_SCANNER_BUFFER_SIZE` must be visible here too. */ const execMigrationBatch = ( session: LegacyDbSession, @@ -263,6 +332,7 @@ const execMigrationBatch = ( mapError: (message: string, phase: "read" | "exec") => E, forceNoVersion: boolean, displayPath: string = migrationPath, + projectEnv: Readonly> = {}, ): Effect.Effect => Effect.gen(function* () { // Go's `MigrationFile.ExecBatch` receives an already-read/parsed file (the read @@ -282,6 +352,21 @@ const execMigrationBatch = ( // absolute too. When it differs from `displayPath` (the caller's Go-equivalent // relative path), substitute it in so the wrapped message still reports the // relative form Go would, not a leaked local temp/absolute path. + // + // Known residual delta (CLI-1958 review): `readFileString` decodes via `TextDecoder` + // with `fatal: false` (the Effect `FileSystem` default), so an invalid-UTF-8 byte + // sequence in the file is lossily replaced with U+FFFD before it ever reaches + // `legacySplitAndTrim`/`session.exec`. Go's `parseFile` instead scans the raw byte + // stream and preserves those bytes verbatim into the statement strings it sends to + // PostgreSQL. Reading raw bytes here (`fs.readFile`) and mapping them 1:1 into a + // "binary string" would fix the split/parse stage, but the fix dies at the wire: the + // shared `pg`/`pg-protocol` layer this session is built on unconditionally UTF-8- + // encodes query text before writing it (`pg-protocol/dist/serializer.js` — + // `buff.write(string, offset, 'utf-8')`, no raw-byte send API), so ANY string + // representation still gets re-mangled at that boundary, just differently. Faithful + // byte parity would require patching that shared wire-serializer — infrastructure + // every legacy DB command's `session.exec` funnels through, not something scoped to + // this file's read path — so it's flagged here rather than "fixed" underneath it. const content = yield* fs.readFileString(migrationPath).pipe( Effect.mapError((error) => { const message = legacyRelativizeErrorMessage( @@ -299,7 +384,7 @@ const execMigrationBatch = ( // failure like the open failure above, not an "exec"-phase one. See // `checkScannerBufferSize`'s comment for why this is a no-op unless the env var // is explicitly set. - yield* checkScannerBufferSize(content, mapError); + yield* checkScannerBufferSize(content, mapError, projectEnv); // Everything below mirrors Go's `(*MigrationFile).ExecBatch` (`pkg/migration/file.go`), // which runs against an already-read file — so every failure from here on is an @@ -496,7 +581,9 @@ export const legacySeedGlobals = ( * `displayPath`, when given, is the path a read-failure's wrapped message should * report instead of `filePath` — see `execMigrationBatch`'s comment on why the two * can differ (an absolute path is required for the real read, but Go's equivalent - * error names the workdir-relative form). + * error names the workdir-relative form). `projectEnv`, when given, is forwarded to + * {@link checkScannerBufferSize} via `execMigrationBatch` — see that helper's doc + * comment. */ export const legacyExecSqlFile = ( session: LegacyDbSession, @@ -505,8 +592,9 @@ export const legacyExecSqlFile = ( filePath: string, mapError: (message: string, phase: "read" | "exec") => E, displayPath?: string, + projectEnv?: Readonly>, ): Effect.Effect => - execMigrationBatch(session, fs, path, filePath, mapError, true, displayPath); + execMigrationBatch(session, fs, path, filePath, mapError, true, displayPath, projectEnv); /** * Applies Go's EXPERIMENTAL declarative schema-files branch of `apply.MigrateAndSeed` @@ -540,6 +628,11 @@ export const legacyExecSqlFile = ( * failure (Go's `NewMigrationFromFile`, `apply.go:57-59`) returns before `CmdSuggestion` is * ever set, so it must NOT carry the suggestion — {@link legacyExecSqlFile}'s `mapError` * receives the `"read"`/`"exec"` phase precisely so this call site can tell them apart. + * + * `projectEnv` is the caller's already-loaded `legacyLoadProjectEnv` map, forwarded to + * {@link checkScannerBufferSize} (via `legacyExecSqlFile`/`execMigrationBatch`) so a + * `SUPABASE_SCANNER_BUFFER_SIZE` set only in `supabase/.env` is honored here exactly like + * a real Go run — see that helper's doc comment. */ export const legacyApplySchemaFiles = ( session: LegacyDbSession, @@ -548,6 +641,7 @@ export const legacyApplySchemaFiles = ( workdir: string, schemaPaths: ReadonlyArray, mapError: (message: string, suggestion?: string) => E, + projectEnv: Readonly> = {}, ): Effect.Effect => Effect.gen(function* () { const { files, warnings } = yield* legacySqlFilesGlob(fs, path, schemaPaths, workdir); @@ -574,6 +668,7 @@ export const legacyApplySchemaFiles = ( ? mapError(message, `See schema file: ${legacyBold(file)}`) : mapError(message), file, + projectEnv, ); } }); diff --git a/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts b/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts index 10eba8032a..082e9e5472 100644 --- a/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts @@ -596,4 +596,145 @@ describe("legacyApplySchemaFiles", () => { ); }, ); + + it.effect( + "falls back to Go's hardcoded default cap when SUPABASE_SCANNER_BUFFER_SIZE is set but unparseable (viper parity, not '5M' == 5MiB)", + () => { + // Verified empirically against `apps/cli-go/pkg/parser` + vendored + // `viper@v1.21.0`: a bare multiplier suffix with NO trailing "b"/"B" (e.g. "5M") + // is NOT 5 MiB in real Go — `parseSizeInBytes` only recognizes a multiplier when + // the string's LAST character is literally "b"/"B", so "5M" never strips a + // suffix and `cast.ToInt("5M")` fails whole, yielding 0. `viper.IsSet` is still + // true (the var IS present), so `parseFile`'s file-size auto-growth never runs + // — `parser.Split` falls back to its OWN hardcoded default cap + // (`MaxScannerCapacity`, 256KiB), not to "no limit" and not to a tiny 5-byte + // limit either. A statement past that hardcoded default must still fail. + const dir = mkdtempSync(join(tmpdir(), "legacy-schema-files-scanner-garbage-")); + mkdirSync(join(dir, "supabase"), { recursive: true }); + const file = join(dir, "supabase", "big.sql"); + writeFileSync(file, `SELECT 1;\nSELECT '${"a".repeat(300_000)}';\n`); + const { session } = fakeSession(); + const previous = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = "5M"; + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const exit = yield* legacyApplySchemaFiles( + session, + fs, + path, + dir, + ["supabase/big.sql"], + (message, suggestion) => + new TestError({ message: suggestion ? `${message} (${suggestion})` : message }), + ).pipe(Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const msg = JSON.stringify(exit.cause); + expect(msg).toContain("bufio.Scanner: token too long"); + // 256KiB (Go's `parser.MaxScannerCapacity` default), not "5MB" and not ~0KB. + expect(msg).toContain( + "Try setting SUPABASE_SCANNER_BUFFER_SIZE=5MB (current size is 256KB)", + ); + } + }).pipe( + Effect.ensuring( + Effect.sync(() => { + if (previous === undefined) delete process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + else process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = previous; + rmSync(dir, { recursive: true, force: true }); + }), + ), + Effect.provide(BunServices.layer), + ); + }, + ); + + it.effect( + "rejects an oversized statement when SUPABASE_SCANNER_BUFFER_SIZE is set only in the project env (Go loadNestedEnv parity)", + () => { + // Go's `loadNestedEnv` (`pkg/config/config.go:1220`) `os.Setenv`s every + // project-`.env` key that isn't already in the shell env BEFORE the command body + // runs, so `viper.AutomaticEnv()` sees a `supabase/.env`-only + // `SUPABASE_SCANNER_BUFFER_SIZE` exactly like a real shell-exported one. + // `legacyApplySchemaFiles`'s `projectEnv` parameter threads the caller's already + // -loaded `legacyLoadProjectEnv` map through to the same check. + const dir = mkdtempSync(join(tmpdir(), "legacy-schema-files-scanner-projectenv-")); + mkdirSync(join(dir, "supabase"), { recursive: true }); + const file = join(dir, "supabase", "big.sql"); + writeFileSync(file, `SELECT 1;\nSELECT '${"a".repeat(5000)}';\n`); + const { session } = fakeSession(); + const previous = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + delete process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const exit = yield* legacyApplySchemaFiles( + session, + fs, + path, + dir, + ["supabase/big.sql"], + (message, suggestion) => + new TestError({ message: suggestion ? `${message} (${suggestion})` : message }), + { SUPABASE_SCANNER_BUFFER_SIZE: "100b" }, + ).pipe(Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const msg = JSON.stringify(exit.cause); + expect(msg).toContain("bufio.Scanner: token too long"); + } + }).pipe( + Effect.ensuring( + Effect.sync(() => { + if (previous !== undefined) process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = previous; + rmSync(dir, { recursive: true, force: true }); + }), + ), + Effect.provide(BunServices.layer), + ); + }, + ); + + it.effect( + "shell env still wins over the project env for SUPABASE_SCANNER_BUFFER_SIZE (Go godotenv 'never overrides' parity)", + () => { + // `godotenv.Load`'s `overload=false` never sets a key already present in + // `os.Environ()` (`godotenv@v1.5.1/godotenv.go:184-200`) — the shell value must + // win even when a (different) project-env value is also threaded through. + const dir = mkdtempSync(join(tmpdir(), "legacy-schema-files-scanner-shellwins-")); + mkdirSync(join(dir, "supabase"), { recursive: true }); + const file = join(dir, "supabase", "big.sql"); + writeFileSync(file, `SELECT '${"a".repeat(5000)}';\n`); + const { session, calls } = fakeSession(); + const previous = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + // Shell explicitly unsets enforcement (0 → treated as unset, no check) while the + // project env sets a tiny limit — the shell value must win. + process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = "0"; + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + yield* legacyApplySchemaFiles( + session, + fs, + path, + dir, + ["supabase/big.sql"], + (message, suggestion) => + new TestError({ message: suggestion ? `${message} (${suggestion})` : message }), + { SUPABASE_SCANNER_BUFFER_SIZE: "100b" }, + ); + expect(calls.some((c) => c.kind === "exec" && c.sql.startsWith("SELECT 'a"))).toBe(true); + }).pipe( + Effect.ensuring( + Effect.sync(() => { + if (previous === undefined) delete process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + else process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = previous; + rmSync(dir, { recursive: true, force: true }); + }), + ), + Effect.provide(BunServices.layer), + ); + }, + ); }); From c405fc9eb8088a7b87c6f39d32e37d70cd0319b4 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Thu, 6 Aug 2026 01:41:23 +0100 Subject: [PATCH 31/47] fix(cli): format TOML inf/-inf/nan glob entries as Go's strconv.FormatFloat does (review: CLI-1958) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit strconv.FormatFloat special-cases the three non-finite float values before the format verb is consulted, rendering "+Inf"/"-Inf"/"NaN" — never JS's own "Infinity"/"-Infinity". A TOML schema_paths/sql_paths array entry can realistically hit this via the bare inf/-inf/nan float literals TOML v1.0 supports, which the weak mapstructure-style glob coercion must render identically to Go. --- .../shared/legacy-db-config.toml-read.ts | 12 ++++++++++ .../legacy-db-config.toml-read.unit.test.ts | 23 +++++++++++++++++++ 2 files changed, 35 insertions(+) diff --git a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts index 923389dcb5..e99be9e6ce 100644 --- a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts +++ b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts @@ -1894,7 +1894,19 @@ const readDbTomlCore = Effect.fnUntraced(function* ( // since `Number.prototype.toString()`/`toExponential()` already computed the same // shortest round-tripping digit sequence Go's algorithm would — only the notation // differs. + // + // `strconv.FormatFloat` special-cases the three non-finite values BEFORE it ever + // looks at the format verb, so `'f'` never applies to them: verified empirically + // (`apps/cli-go` probe against a real `schema_paths = [inf, -inf, nan]` config load) — + // `+Inf`/`-Inf`/`NaN` (note the "+Inf" sign Go always prints, and the short "Inf"/"NaN" + // spelling) — never JS's own `Infinity`/`-Infinity`/`NaN` (which happens to already + // match the "NaN" case, but not the two `Infinity` ones). TOML v1.0's bare `inf`/ + // `+inf`/`-inf`/`nan` float literals (smol-toml) parse to exactly these JS values, so + // a `schema_paths`/`sql_paths` array entry can realistically hit this branch. const legacyFormatGoWeakFloat = (value: number): string => { + if (Number.isNaN(value)) return "NaN"; + if (value === Number.POSITIVE_INFINITY) return "+Inf"; + if (value === Number.NEGATIVE_INFINITY) return "-Inf"; const str = value.toString(); const match = /^(-?)(\d+)(?:\.(\d+))?e([+-]\d+)$/.exec(str); if (match === null) return str; diff --git a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts index 2ae1f41fb0..8311043a23 100644 --- a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts @@ -430,6 +430,29 @@ describe("legacyReadDbToml", () => { }, ); + it.effect( + "formats TOML special-float db.migrations.schema_paths entries like Go's strconv.FormatFloat, not JS's toString (Go parity)", + () => { + // `strconv.FormatFloat` special-cases the three non-finite values BEFORE the + // format verb is even consulted, so `'f'` never applies to them — it renders + // `+Inf` / `-Inf` / `NaN` (verified empirically against `apps/cli-go`: a real + // `schema_paths = [inf, -inf, nan]` config load resolves to exactly + // `supabase/{+Inf,-Inf,NaN}`). JS's own `Number.prototype.toString()` renders + // the two infinities as `"Infinity"`/`"-Infinity"` instead — a naive port would + // record (and later glob) the wrong path. TOML v1.0's bare `inf`/`-inf`/`nan` + // float literals parse to exactly these JS values (smol-toml). + const dir = withConfig(["[db.migrations]", "schema_paths = [inf, -inf, nan]", ""].join("\n")); + return read(dir).pipe( + Effect.tap((v) => + Effect.sync(() => { + expect(v.schemaPaths).toEqual(["supabase/+Inf", "supabase/-Inf", "supabase/NaN"]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + it.effect( "weakly coerces a TOP-LEVEL scalar db.migrations.schema_paths (Go mapstructure weak-decode of a []string field)", () => { From f8a544d4aae71d94826ccfd1ee30b33b2a53d8c1 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Thu, 6 Aug 2026 02:40:35 +0100 Subject: [PATCH 32/47] fix(cli): reject an unterminated scanner token exactly at the buffer limit (review: CLI-1958) Go's bufio.Scanner can only raise "token too long" once it needs more data and the buffer is already full; a delimiter-terminated token is found in the same Scan() call that fills the buffer, before that check is reached, so it succeeds at exactly the limit. A trailing token with no delimiter never gets that chance, so it fails at exactly the limit. Track whether each split token was delimiter-terminated or emitted at EOF and compare with `>=` only for the latter. --- .../legacy/shared/legacy-migration-apply.ts | 13 ++++- .../cli/src/legacy/shared/legacy-sql-split.ts | 50 ++++++++++++++++--- 2 files changed, 56 insertions(+), 7 deletions(-) diff --git a/apps/cli/src/legacy/shared/legacy-migration-apply.ts b/apps/cli/src/legacy/shared/legacy-migration-apply.ts index f6cd2a6f3a..2616c0fe87 100644 --- a/apps/cli/src/legacy/shared/legacy-migration-apply.ts +++ b/apps/cli/src/legacy/shared/legacy-migration-apply.ts @@ -255,7 +255,18 @@ const checkScannerBufferSize = ( let emitted = 0; let lastRaw = ""; for (const token of legacySplitSqlTokens(content)) { - if (utf8ByteLength(token.raw) > limit) { + // A delimiter-terminated token is found (and emitted) by `parser.Split`'s scan in + // the SAME `Scan()` call that fills the buffer to capacity — before Go's too-long + // check is ever reached — so a token exactly AT `limit` still succeeds; only + // strictly-over fails (`>`). The trailing, unterminated token (only ever the LAST + // one `legacySplitSqlTokens` returns, if any — see `LegacySplitSqlToken.terminated`) + // has no delimiter to find: once the buffer fills to `limit` bytes without one, the + // too-long check fires immediately, without Go ever attempting the extra `Read()` + // that would reveal real EOF — so a trailing token AT `limit` already fails (`>=`). + const tooLong = token.terminated + ? utf8ByteLength(token.raw) > limit + : utf8ByteLength(token.raw) >= limit; + if (tooLong) { const suggestion = `Try setting SUPABASE_SCANNER_BUFFER_SIZE=5MB (current size is ${Math.floor(reportedLimit / 1024)}KB)`; return Effect.fail( mapError( diff --git a/apps/cli/src/legacy/shared/legacy-sql-split.ts b/apps/cli/src/legacy/shared/legacy-sql-split.ts index ebb17d7d2c..ca89362511 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-split.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-split.ts @@ -143,16 +143,29 @@ class AtomicState implements State { } } +/** + * One raw token from {@link splitRaw}. `terminated` is `false` only for a + * trailing statement emitted at EOF with no closing delimiter (the + * `acc.length > 0` fallback below) — every other token was emitted because the + * FSM itself found a boundary (a bare `;` in `ReadyState`, or `AtomicState` + * closing). Only ever `false` on the LAST element `splitRaw` returns, since + * that fallback fires at most once, after the main loop. + */ +interface RawToken { + readonly text: string; + readonly terminated: boolean; +} + /** The FSM traversal shared by every `legacySplitSql*` entry point below. */ -function splitRaw(sql: string): string[] { +function splitRaw(sql: string): RawToken[] { let state: State = new ReadyState(); - const tokens: string[] = []; + const tokens: RawToken[] = []; let acc = ""; for (const rune of Array.from(sql)) { acc += rune; const next = state.next(rune, acc); if (next === null) { - tokens.push(acc); + tokens.push({ text: acc, terminated: true }); acc = ""; state = new ReadyState(); } else { @@ -160,7 +173,7 @@ function splitRaw(sql: string): string[] { } } // Trailing non-terminated statement at EOF. - if (acc.length > 0) tokens.push(acc); + if (acc.length > 0) tokens.push({ text: acc, terminated: false }); return tokens; } @@ -173,7 +186,7 @@ export function legacySplitSql( ...transform: ReadonlyArray<(s: string) => string> ): string[] { const statements: string[] = []; - for (const raw of splitRaw(sql)) { + for (const { text: raw } of splitRaw(sql)) { let token = raw; for (const apply of transform) token = apply(token); if (token.length > 0) statements.push(token); @@ -195,6 +208,27 @@ export interface LegacySplitSqlToken { readonly raw: string; /** `legacyTrimStatement(raw)` — what `legacySplitAndTrim` emits, including when empty. */ readonly trimmed: string; + /** + * `false` only for a trailing statement with no closing delimiter, emitted at + * real EOF (`splitRaw`'s `acc.length > 0` fallback) — see {@link RawToken}. + * `checkScannerBufferSize` (`legacy-migration-apply.ts`) needs this to decide + * `>` vs `>=` against the effective buffer limit: Go's `bufio.Scanner` can + * only apply its too-long check (`len(s.buf) >= s.maxTokenSize`) once it has + * given up looking for a delimiter and still needs more data — for a + * delimiter-terminated token the delimiter is found (and the token emitted) + * in the SAME `Scan()` call that fills the buffer to capacity, before that + * check is ever reached, so a token exactly AT the limit still succeeds. An + * unterminated trailing token has no delimiter to find: once the buffer + * fills to the effective limit without one, the too-long check fires + * immediately — Go never gets to attempt the extra `Read()` that would + * reveal real EOF and let the split function emit the trailing token + * instead. Verified empirically against `apps/cli-go/pkg/parser.Split`: a + * single terminated statement of exactly `maxbuf` bytes always succeeds, + * while an unterminated one of exactly `maxbuf` bytes always fails with + * `bufio.ErrTooLong` (one byte under still succeeds; one byte over always + * fails either way). + */ + readonly terminated: boolean; } /** @@ -213,7 +247,11 @@ export interface LegacySplitSqlToken { * token, including the ones `legacySplitAndTrim` itself would filter out. */ export function legacySplitSqlTokens(sql: string): ReadonlyArray { - return splitRaw(sql).map((raw) => ({ raw, trimmed: legacyTrimStatement(raw) })); + return splitRaw(sql).map(({ text: raw, terminated }) => ({ + raw, + trimmed: legacyTrimStatement(raw), + terminated, + })); } // `(?i)drop\s+` — Go's `dropStatementPattern` (`internal/db/diff/diff.go:100`, From 3fce53007fe21ee36d418286e193909848bbefb3 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Thu, 6 Aug 2026 02:40:44 +0100 Subject: [PATCH 33/47] fix(cli): preserve negative zero when weakly formatting glob entries (review: CLI-1958) Go's strconv.FormatFloat preserves the IEEE754 sign bit on zero, so a weakly-decoded schema_paths/sql_paths entry of -0.0 renders as "-0". JS's (-0).toString() drops the sign and returns "0", so schema_paths = [-0.0] resolved to the wrong path. Detect Object.is(value, -0) before falling through to the generic toString() path. --- .../src/legacy/shared/legacy-db-config.toml-read.ts | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts index e99be9e6ce..12dc190bfb 100644 --- a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts +++ b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts @@ -1903,10 +1903,20 @@ const readDbTomlCore = Effect.fnUntraced(function* ( // match the "NaN" case, but not the two `Infinity` ones). TOML v1.0's bare `inf`/ // `+inf`/`-inf`/`nan` float literals (smol-toml) parse to exactly these JS values, so // a `schema_paths`/`sql_paths` array entry can realistically hit this branch. + // + // `strconv.FormatFloat` also preserves the IEEE754 sign bit on zero: a genuine + // negative-zero float64 formats as `"-0"`, never `"0"`. Verified empirically — + // `strconv.FormatFloat(math.Copysign(0, -1), 'f', -1, 64)` returns `"-0"` — and + // end-to-end through the real decode pipeline this weak-decode mirrors + // (`BurntSushi/toml` + `go-viper/mapstructure`'s `WeaklyTypedInput`): a + // `schema_paths = [-0.0]` config decodes its glob entry to the literal string + // `"-0"`. JS's own `(-0).toString()` is `"0"` (the sign is dropped), by spec — + // `Object.is(value, -0)` is JS's only way to detect it, since `-0 === 0`. const legacyFormatGoWeakFloat = (value: number): string => { if (Number.isNaN(value)) return "NaN"; if (value === Number.POSITIVE_INFINITY) return "+Inf"; if (value === Number.NEGATIVE_INFINITY) return "-Inf"; + if (Object.is(value, -0)) return "-0"; const str = value.toString(); const match = /^(-?)(\d+)(?:\.(\d+))?e([+-]\d+)$/.exec(str); if (match === null) return str; From 2b232d080568d7336d9089ccabb3c79517ac27f5 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Thu, 6 Aug 2026 02:40:54 +0100 Subject: [PATCH 34/47] fix(cli): cache the pg-delta migrations catalog after a remote reset (review: CLI-1958) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Go's down.ResetAll (resetRemote's delegate) best-effort caches the pg-delta migrations catalog right after apply.MigrateAndSeed succeeds, warning on failure rather than failing the reset. The native remote reset finished without calling the equivalent — already-ported legacyTryCacheMigrationsCatalog (wired into db push) — so downstream pg-delta tooling missed the refreshed cache and the warning line never appeared. Wire it in after the seed step, gated the same way Go gates it: skip for a versioned reset (--version/--last), since TryCacheMigrationsCatalog no-ops on any non-empty version. --- .../legacy/commands/db/reset/reset.handler.ts | 57 ++++++++- .../db/reset/reset.integration.test.ts | 121 +++++++++++++++++- 2 files changed, 174 insertions(+), 4 deletions(-) diff --git a/apps/cli/src/legacy/commands/db/reset/reset.handler.ts b/apps/cli/src/legacy/commands/db/reset/reset.handler.ts index 1473eb058e..75d37e746a 100644 --- a/apps/cli/src/legacy/commands/db/reset/reset.handler.ts +++ b/apps/cli/src/legacy/commands/db/reset/reset.handler.ts @@ -1,4 +1,4 @@ -import { Effect, FileSystem, Option, Path } from "effect"; +import { Clock, Effect, FileSystem, Option, Path } from "effect"; import { CliArgs } from "../../../../shared/cli/cli-args.service.ts"; import { detectGitBranch } from "../../../../shared/git/git-branch.ts"; @@ -12,6 +12,7 @@ import { Output } from "../../../../shared/output/output.service.ts"; import { legacyAqua, legacyYellow } from "../../../shared/legacy-colors.ts"; import { LegacyCliConfig } from "../../../config/legacy-cli-config.service.ts"; import { LegacyProjectRefResolver } from "../../../config/legacy-project-ref.service.ts"; +import { redactLegacyConnectionString } from "../../../shared/legacy-db-config.parse.ts"; import { LegacyDbConfigResolver } from "../../../shared/legacy-db-config.service.ts"; import { legacyCheckDbToml, @@ -19,18 +20,28 @@ import { legacyResolveSeedSqlPath, } from "../../../shared/legacy-db-config.toml-read.ts"; import { LegacyDbConnection } from "../../../shared/legacy-db-connection.service.ts"; +import { + legacyResolveLocalProjectId, + legacySanitizeProjectId, +} from "../../../shared/legacy-docker-ids.ts"; import { legacyApplyMigrations, legacyApplySchemaFiles, } from "../../../shared/legacy-migration-apply.ts"; import { legacyParseMigrationVersion } from "../../../shared/legacy-migration-timestamp.format.ts"; +import { legacyToPostgresURL } from "../../../shared/legacy-postgres-url.ts"; import { legacyPromptYesNo } from "../../../../shared/legacy/legacy-prompt-yes-no.ts"; import { resolveLegacyDbTargetFlags } from "../../../shared/legacy-db-target-flags.ts"; import { LegacyLinkedProjectCache } from "../../../telemetry/legacy-linked-project-cache.service.ts"; import { LegacyTelemetryState } from "../../../telemetry/legacy-telemetry-state.service.ts"; import { legacyDropUserSchemas } from "../shared/legacy-drop-schemas.ts"; import { LegacyDbBootstrapSeam } from "../shared/legacy-db-bootstrap.seam.service.ts"; -import { legacyListLocalMigrations } from "../shared/legacy-pgdelta.cache.ts"; +import { legacyParseBoolEnv } from "../shared/legacy-diff-engine.ts"; +import { + legacyListLocalMigrations, + legacyTryCacheMigrationsCatalog, +} from "../shared/legacy-pgdelta.cache.ts"; +import { type LegacyPgDeltaContext } from "../shared/legacy-pgdelta.ts"; import { legacyPathMatch } from "../../../shared/legacy-path-match.ts"; import { legacyGetPendingSeeds, legacySeedData } from "../../../shared/legacy-seed-ops.ts"; import { legacyUpsertVaultSecrets } from "../../../shared/legacy-vault.ts"; @@ -403,7 +414,47 @@ export const legacyDbReset = Effect.fn("legacy.db.reset")(function* (flags: Lega const seeds = yield* legacyGetPendingSeeds(session, fs, path, seedPaths, workdir); yield* legacySeedData(session, fs, workdir, path, seeds, applyError); } - // Go's best-effort pgcache catalog warning is not ported (no output impact). + + // Go's `down.ResetAll` (`internal/migration/down/down.go:48-61`) — the function + // `resetRemote` delegates to — best-effort caches the migrations catalog for + // pg-delta right after `apply.MigrateAndSeed` succeeds, warning (never failing + // the reset) on error. `pgcache.TryCacheMigrationsCatalog` itself no-ops when + // `resolvedVersion` is non-empty (`len(version) > 0`, `pgcache/cache.go:73`) — + // a versioned reset (`--version`/`--last`) never refreshes the cache — so gate + // the call the same way rather than threading that check into the shared + // native helper (already used by `db push`, which has no version concept). + const cacheEnabled = + resolvedVersion === "" && + (toml.pgDelta.enabled || + legacyParseBoolEnv(toml.envLookup("SUPABASE_EXPERIMENTAL_PG_DELTA"))); + const pgDeltaCtx: LegacyPgDeltaContext = { + projectId: legacySanitizeProjectId( + legacyResolveLocalProjectId( + Option.getOrUndefined(cliConfig.projectId), + Option.getOrUndefined(toml.projectId) ?? + (linkedRef !== undefined && linkedRef !== "" ? linkedRef : undefined), + workdir, + ), + ), + cwd: workdir, + npmVersion: Option.getOrUndefined(toml.pgDelta.npmVersion), + denoVersion: toml.denoVersion, + }; + yield* legacyTryCacheMigrationsCatalog(fs, path, pgDeltaCtx, { + enabled: cacheEnabled, + targetUrl: legacyToPostgresURL(cfg.conn), + conn: cfg.conn, + isLocal: false, + migrationsDir, + nowMillis: yield* Clock.currentTimeMillis, + }).pipe( + Effect.catch((error) => + output.raw( + `Warning: failed to cache migrations catalog: ${redactLegacyConnectionString(error.message)}\n`, + "stderr", + ), + ), + ); }), ); diff --git a/apps/cli/src/legacy/commands/db/reset/reset.integration.test.ts b/apps/cli/src/legacy/commands/db/reset/reset.integration.test.ts index a8f5d19182..20241cb722 100644 --- a/apps/cli/src/legacy/commands/db/reset/reset.integration.test.ts +++ b/apps/cli/src/legacy/commands/db/reset/reset.integration.test.ts @@ -43,6 +43,12 @@ import { LegacyDbConnection, type LegacyPgConnInput, } from "../../../shared/legacy-db-connection.service.ts"; +import { LegacyEdgeRuntimeScriptError } from "../../../shared/legacy-edge-runtime-script.errors.ts"; +import { + LegacyEdgeRuntimeScript, + type LegacyEdgeRuntimeRunOpts, +} from "../../../shared/legacy-edge-runtime-script.service.ts"; +import { LegacyPgDeltaSslProbe } from "../../../shared/legacy-pgdelta-ssl-probe.service.ts"; import { LegacyDbBootstrapSeam } from "../shared/legacy-db-bootstrap.seam.service.ts"; import { legacyDbReset } from "./reset.handler.ts"; import type { LegacyDbResetFlags } from "./reset.command.ts"; @@ -251,6 +257,10 @@ function setup( running?: boolean; storageReady?: boolean; awaitStorageReadyExitCode?: number; + // pg-delta migrations-catalog cache (Go's `down.ResetAll` → `pgcache.TryCacheMigrationsCatalog`, + // wired into the remote-reset path after a successful migrate/schema-files + seed). + catalogStdout?: string; + catalogExportFailWith?: string; }, ) { if (opts.toml !== undefined) { @@ -282,9 +292,28 @@ function setup( resolveFails: opts.resolveFails, }); + const edgeRunCalls: Array = []; + const edge = Layer.succeed(LegacyEdgeRuntimeScript, { + run: (runOpts: LegacyEdgeRuntimeRunOpts) => { + edgeRunCalls.push(runOpts); + if (opts.catalogExportFailWith !== undefined) { + return Effect.fail( + new LegacyEdgeRuntimeScriptError({ message: opts.catalogExportFailWith }), + ); + } + return Effect.succeed({ stdout: opts.catalogStdout ?? '{"version":1}', stderr: "" }); + }, + }); + const sslProbe = Layer.succeed(LegacyPgDeltaSslProbe, { + requireSsl: () => Effect.succeed(false), + requireSslForHost: () => Effect.succeed(false), + }); + const layer = Layer.mergeAll( out.layer, conn.layer, + edge, + sslProbe, seam.layer, resolver.layer, mockLegacyCliConfig({ workdir }), @@ -316,7 +345,7 @@ function setup( telemetry.layer, linkedCache.layer, ); - return { layer, out, conn, seam, telemetry, linkedCache, resolver }; + return { layer, out, conn, seam, telemetry, linkedCache, resolver, edgeRunCalls }; } const migrationFile = (version: string, body = "create table t ();") => ({ @@ -755,6 +784,96 @@ describe("legacy db reset", () => { }); }); + it.live( + "caches the migrations catalog after a successful remote reset with SUPABASE_EXPERIMENTAL_PG_DELTA set", + () => { + // Go's `down.ResetAll` (`internal/migration/down/down.go:48-61`, the function + // `resetRemote` delegates to) best-effort caches the pg-delta migrations + // catalog right after `apply.MigrateAndSeed` succeeds — gated on + // `pgcache.ShouldCacheMigrationsCatalog()` (`experimental.pgdelta.enabled` OR + // the legacy `SUPABASE_EXPERIMENTAL_PG_DELTA` env switch), independent of + // `--experimental`'s own schema-files gate. + const { layer, out, edgeRunCalls } = setup(tmp.current, { + toml: 'project_id = "test"\n', + files: { + ...migrationFile("20240101000000"), + "supabase/.env": "SUPABASE_EXPERIMENTAL_PG_DELTA=true\n", + }, + confirm: [true], + }); + return Effect.gen(function* () { + yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe(Effect.provide(layer)); + expect(out.stderrText).not.toContain("failed to cache migrations catalog"); + expect(edgeRunCalls).toHaveLength(1); + }); + }, + ); + + it.live("warns without failing the reset when the migrations-catalog cache write fails", () => { + const { layer, out } = setup(tmp.current, { + toml: 'project_id = "test"\n[experimental.pgdelta]\nenabled = true\n', + files: migrationFile("20240101000000"), + confirm: [true], + catalogExportFailWith: "edge-runtime script produced no output", + }); + return Effect.gen(function* () { + const exit = yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe( + Effect.provide(layer), + Effect.exit, + ); + expect(Exit.isSuccess(exit)).toBe(true); + expect(out.stderrText).toContain( + "Warning: failed to cache migrations catalog: edge-runtime script produced no output", + ); + }); + }); + + it.live( + "falls back to the linked project ref for the pg-delta cache when config.toml has no project_id", + () => { + // Go's `flags.LoadConfig` seeds `Config.ProjectId = ProjectRef` BEFORE + // `Config.Load` runs, so on the linked remote path an absent `project_id` + // retains the linked ref rather than falling to the workdir basename. + const { layer, out, edgeRunCalls } = setup(tmp.current, { + toml: "[experimental.pgdelta]\nenabled = true\n", + ref: LEGACY_VALID_REF, + files: migrationFile("20240101000000"), + confirm: [true], + }); + return Effect.gen(function* () { + yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe(Effect.provide(layer)); + expect(out.stderrText).not.toContain("failed to cache migrations catalog"); + expect(edgeRunCalls).toHaveLength(1); + }); + }, + ); + + it.live( + "skips the migrations-catalog cache for a versioned remote reset even with pg-delta caching enabled", + () => { + // `pgcache.TryCacheMigrationsCatalog` no-ops on any non-empty `version` + // (`pgcache/cache.go:73`, `len(version) > 0`) — a `--version`/`--last` reset + // never refreshes the cache, unlike a full (versionless) reset. + const { layer, out, edgeRunCalls } = setup(tmp.current, { + toml: 'project_id = "test"\n[experimental.pgdelta]\nenabled = true\n', + files: { + ...migrationFile("20240101000000"), + ...migrationFile("20240202000000"), + }, + confirm: [true], + }); + return Effect.gen(function* () { + yield* legacyDbReset({ + ...DEFAULT_FLAGS, + linked: true, + version: Option.some("20240101000000"), + }).pipe(Effect.provide(layer)); + expect(out.stderrText).not.toContain("failed to cache migrations catalog"); + expect(edgeRunCalls).toHaveLength(0); + }); + }, + ); + it.live("drops schemas and applies migrations + seed on a confirmed remote reset", () => { const { layer, out, conn, linkedCache } = setup(tmp.current, { toml: 'project_id = "test"\n', From c06bc55f076e4b8a16682519f600cff22c505e1c Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Thu, 6 Aug 2026 03:37:19 +0100 Subject: [PATCH 35/47] fix(cli): parse SUPABASE_SCANNER_BUFFER_SIZE with Go's base-0 int grammar (review: CLI-1958) viper.GetSizeInBytes -> cast.ToInt -> strconv.ParseInt(s, 0, 0) parses the post-multiplier-strip remainder with base 0, so hex/octal/binary literals (e.g. 0x100000) are valid Go byte counts that the previous decimal-only regex silently rejected, falling back to the 256KiB default instead of Go's parsed value. Verified against real vendored viper@v1.21.0 + cast@v1.10.0. --- .../legacy/shared/legacy-migration-apply.ts | 75 +++++++++++++++++-- .../legacy-migration-apply.unit.test.ts | 53 +++++++++++++ 2 files changed, 121 insertions(+), 7 deletions(-) diff --git a/apps/cli/src/legacy/shared/legacy-migration-apply.ts b/apps/cli/src/legacy/shared/legacy-migration-apply.ts index 2616c0fe87..89d36dcc5f 100644 --- a/apps/cli/src/legacy/shared/legacy-migration-apply.ts +++ b/apps/cli/src/legacy/shared/legacy-migration-apply.ts @@ -163,11 +163,73 @@ const GO_DEFAULT_MAX_SCANNER_CAPACITY = 256 * 1024; * exactly that one exception. All of the above verified empirically against * `apps/cli-go/pkg/parser` + vendored `viper@v1.21.0` * (`"5"→5, "5B"→0, "50B"→50, "5KB"→5120, "5M"→0, "0"→0, "5.5"→5, "5.5MB"→5242880`). - * Known residual delta: Go's `strconv.ParseInt(s, 0, 0)` uses base `0`, so it also - * accepts a `0x`/`0o`/`0b`-prefixed literal (`"0x5"` → `5`) — not reproduced here as a - * realistic byte-size override would never use one; flagging so a future parity sweep - * doesn't rediscover it. + * + * `cast.ToInt`'s underlying `strconv.ParseInt(s, 0, 0)` (review CLI-1958) uses base + * `0`, so the remaining (post multiplier-strip) string is ALSO accepted as a + * `0x`/`0X`-prefixed hex literal, an `0o`/`0O`-prefixed OR bare-leading-zero octal + * literal, or a `0b`/`0B`-prefixed binary literal — handled below by + * {@link parseGoBaseZeroInt}. Verified empirically against vendored `viper@v1.21.0` + * (via `viper.GetSizeInBytes`, since `cast.ToInt` is itself unexported call + * plumbing): `"0x100000"→1048576`, `"0o40000"→16384`, + * `"0b100000000000000000000"→2097152`, `"0755"` (legacy octal, no `"o"`)`→493`, + * `"0x"/"garbage"/"0x1g"→unparseable (0)`. A hex/octal/binary value ending in a + * literal `b`/`B` digit (e.g. `"0x1B"`) still hits the multiplier-strip switch + * ABOVE first, same as any other value — that consumes the trailing `B` before + * base-0 parsing ever sees it (`"0x1B"→1`, not `27`), which is a genuine Go quirk + * this port reproduces automatically by keeping the same two-step order, not a bug. + * Known residual delta: Go's base-0 grammar also permits `_` digit separators + * (e.g. `"1_048_576"`, verified empirically to equal `1048576`) — not reproduced + * here as a realistic byte-size override would never use one; flagging so a future + * parity sweep doesn't rediscover it. */ +const parseGoBaseZeroInt = (value: string): number | undefined => { + const negative = value.startsWith("-"); + const unsigned = negative || value.startsWith("+") ? value.slice(1) : value; + if (unsigned.length === 0) return undefined; + + let base = 10; + let digits = unsigned; + const prefix = unsigned.slice(0, 2).toLowerCase(); + if (prefix === "0x") { + base = 16; + digits = unsigned.slice(2); + } else if (prefix === "0o") { + base = 8; + digits = unsigned.slice(2); + } else if (prefix === "0b") { + base = 2; + digits = unsigned.slice(2); + } else if (unsigned.length > 1 && unsigned[0] === "0") { + // Legacy (no "o") leading-zero octal, e.g. "0755". + base = 8; + digits = unsigned.slice(1); + } + if (digits.length === 0) return undefined; + + const validDigits = + base === 16 ? /^[0-9a-fA-F]+$/ : base === 8 ? /^[0-7]+$/ : base === 2 ? /^[01]+$/ : /^[0-9]+$/; + if (!validDigits.test(digits)) return undefined; + + const n = Number.parseInt(digits, base); + return negative ? -n : n; +}; + +// `cast.ToInt`'s `trimDecimal` (`spf13/cast@v1.10.0/number.go:507-525`) runs BEFORE +// `strconv.ParseInt`: when the whole string is a sign + plain decimal digits + an +// optional ".digits" tail (`stringNumberRe`, `^([-+]?\d*)(\.\d*)?$` — never matches +// a `0x`/`0o`/`0b` literal, which contains letters), it drops the fractional part +// outright rather than rounding (`"5.5"` → `"5"`). Anything else (including a +// non-decimal-looking string that merely contains a ".") passes through unchanged +// and is left for {@link parseGoBaseZeroInt} to accept or reject. +const trimGoDecimal = (value: string): string => { + if (!value.includes(".")) return value; + const match = /^([+-]?\d*)(?:\.\d*)?$/.exec(value); + if (!match) return value; + const intPart = match[1] ?? ""; + if (intPart === "+" || intPart === "-") return `${intPart}0`; + return intPart === "" ? "0" : intPart; +}; + const legacyParseScannerBufferSize = (raw: string): number => { let value = raw.trim(); let multiplier = 1; @@ -191,9 +253,8 @@ const legacyParseScannerBufferSize = (raw: string): number => { break; } } - if (!/^[+-]?\d+(?:\.\d*)?$/.test(value)) return 0; - const size = Number.parseInt(value, 10); - return Number.isFinite(size) && size > 0 ? size * multiplier : 0; + const size = parseGoBaseZeroInt(trimGoDecimal(value)); + return size !== undefined && Number.isFinite(size) && size > 0 ? size * multiplier : 0; }; /** diff --git a/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts b/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts index 082e9e5472..a5c9022a18 100644 --- a/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts @@ -650,6 +650,59 @@ describe("legacyApplySchemaFiles", () => { }, ); + it.effect( + "accepts a hex-literal SUPABASE_SCANNER_BUFFER_SIZE (Go strconv.ParseInt base-0 parity, review CLI-1958)", + () => { + // `viper.GetSizeInBytes` → `cast.ToInt` → `strconv.ParseInt(s, 0, 0)` parses + // with base 0, so a `0x`-prefixed literal is a valid byte count in real Go: + // "0x1400" is 5120 (5KiB) — verified empirically against vendored + // `viper@v1.21.0` (`viper.GetSizeInBytes("SCANNER_BUFFER_SIZE")` with the env + // var set to "0x1400" returns 5120). A decimal-only parser would reject this + // string outright and silently fall back to the 256KiB default instead, so a + // statement between 5120 and 262144 bytes would apply in TS but Go would + // already have failed with "bufio.Scanner: token too long" at 5121 bytes. + const dir = mkdtempSync(join(tmpdir(), "legacy-schema-files-scanner-hex-")); + mkdirSync(join(dir, "supabase"), { recursive: true }); + const file = join(dir, "supabase", "big.sql"); + writeFileSync(file, `SELECT 1;\nSELECT '${"a".repeat(5116)}';\n`); + const { session } = fakeSession(); + const previous = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = "0x1400"; + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const exit = yield* legacyApplySchemaFiles( + session, + fs, + path, + dir, + ["supabase/big.sql"], + (message, suggestion) => + new TestError({ message: suggestion ? `${message} (${suggestion})` : message }), + ).pipe(Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const msg = JSON.stringify(exit.cause); + expect(msg).toContain("bufio.Scanner: token too long"); + // 5KiB (0x1400 bytes), not the 256KiB hardcoded fallback a decimal-only + // parser would have silently used instead. + expect(msg).toContain( + "Try setting SUPABASE_SCANNER_BUFFER_SIZE=5MB (current size is 5KB)", + ); + } + }).pipe( + Effect.ensuring( + Effect.sync(() => { + if (previous === undefined) delete process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + else process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = previous; + rmSync(dir, { recursive: true, force: true }); + }), + ), + Effect.provide(BunServices.layer), + ); + }, + ); + it.effect( "rejects an oversized statement when SUPABASE_SCANNER_BUFFER_SIZE is set only in the project env (Go loadNestedEnv parity)", () => { From 560ff8a5577921e85952d1a7c998925340c39c89 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Thu, 6 Aug 2026 03:37:30 +0100 Subject: [PATCH 36/47] docs(cli): document the ported pg-delta catalog cache in db reset (review: CLI-1958) SIDE_EFFECTS.md still said the best-effort catalog-cache warning was "not ported", but reset.handler.ts already wires legacyTryCacheMigrationsCatalog unconditionally after either apply branch. Documents the cache file, its gating (no resolved version + pg-delta enabled), SUPABASE_EXPERIMENTAL_PG_DELTA, and the warn-never-fail behavior, matching db push's existing SIDE_EFFECTS wording. --- .../legacy/commands/db/reset/SIDE_EFFECTS.md | 27 ++++++++++++++----- 1 file changed, 20 insertions(+), 7 deletions(-) diff --git a/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md b/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md index 02b89e588a..361f52bcef 100644 --- a/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md +++ b/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md @@ -28,10 +28,11 @@ is still handled by Go (CLI-1955 scope, unaffected by CLI-1958). ## Files Written -| Path | Format | When | -| ------------------------------------------------ | ------ | --------------------------------- | -| `~/.supabase//linked-project.json` | JSON | `--linked` (post-run cache) | -| `~/.supabase/telemetry.json` | JSON | always (post-run telemetry flush) | +| Path | Format | When | +| ------------------------------------------------------------------------------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `~/.supabase//linked-project.json` | JSON | `--linked` (post-run cache) | +| `~/.supabase/telemetry.json` | JSON | always (post-run telemetry flush) | +| `/supabase/.temp/pgdelta/catalog--migrations--.json` | JSON | remote path only, best-effort, after either branch (schema-files or migrations) + seeding succeed, when no `--version`/`--last` resolved a version AND pg-delta is enabled (`[experimental.pgdelta].enabled` or `SUPABASE_EXPERIMENTAL_PG_DELTA`); a failure only warns on stderr and never fails the reset (Go's `down.ResetAll` → `pgcache.TryCacheMigrationsCatalog`, `down.go:58-59`) — see Notes | On the local path the Go seam additionally recreates the `supabase_db_` container/volume and applies the initial schema (`SetupLocalDatabase`). @@ -91,6 +92,7 @@ races a restarting gateway. | `SUPABASE_EXPERIMENTAL_PGDELTA_ENABLED` | overrides `[experimental.pgdelta].enabled`; a truthy value flips the reset gate (`experimental && resolvedVersion === "" && !toml.pgDelta.enabled`) back to timestamped migrations even with `--experimental` set — switches between two different destructive code paths | no | | `SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS` | overrides `[db.migrations].schema_paths` (viper `AutomaticEnv`, beats the config-file value) for the schema-files apply branch | no (no dedicated flag — config-file-only otherwise) | | `SUPABASE_PROJECT_ID` | overrides the local container id (`utils.DbId`) | no | +| `SUPABASE_EXPERIMENTAL_PG_DELTA` | enables the post-reset migrations-catalog cache (see Files Written) when `[experimental.pgdelta].enabled` is unset — distinct from `SUPABASE_EXPERIMENTAL_PGDELTA_ENABLED` above, which switches the reset's own apply branch instead | no (project `.env` or shell) | ## Exit Codes @@ -176,12 +178,23 @@ path has no confirmation prompt. failure attaches Go's `See schema file: ` suggestion. No progress line is printed per file (Go's `applySchemaFiles` has no output), no migration-history row is inserted, and no `RESET ALL` runs between files. Seeding still runs afterward, - unconditionally, exactly as on the migrations branch. The best-effort pg-delta - catalog-cache warning (`down.go:58-59`, gated on `SUPABASE_EXPERIMENTAL_PG_DELTA`) - is not ported (no output impact) — same known gap as the migrations branch. + unconditionally, exactly as on the migrations branch. `encrypted:` vault secrets are NOT skipped on the remote path — `legacyCheckDbToml` decrypts them into `toml.vault`, and `legacyUpsertVaultSecrets` upserts the decrypted values unconditionally, before either branch (schema-files or migrations) runs. +- **Migrations catalog cache**: ported (Go's best-effort `down.ResetAll` → + `pgcache.TryCacheMigrationsCatalog`, `down.go:48-61`). Runs on the remote path only, + after either apply branch (schema-files or migrations) and seeding complete, gated on + BOTH no `--version`/`--last` having resolved a version AND pg-delta being enabled + (`[experimental.pgdelta].enabled` or `SUPABASE_EXPERIMENTAL_PG_DELTA` — see + Environment Variables); a versioned reset never refreshes the cache, matching Go's + own `len(version) > 0` no-op inside `TryCacheMigrationsCatalog` itself. Exports the + target's pg-delta catalog via the edge-runtime stack and writes it under + `supabase/.temp/pgdelta/` (see Files Written), pruning older snapshots for the same + prefix (retains 2). A failure only warns on stderr and never fails the reset, matching + Go exactly. Reuses `legacyExportCatalogPgDelta` and `legacyTryCacheMigrationsCatalog` + — the same helpers `db push` uses for its own post-apply cache (see that command's + SIDE_EFFECTS Notes) — rather than a second copy. - The local path's own `--experimental` schema-files branch is still handled by the Go child behind the `db __db-bootstrap` seam (CLI-1955 scope) — this command's handler forwards `--experimental` to that seam but does not implement the branch From 8e8393d5261d429ae4518186d7b8d051464a66e8 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Thu, 6 Aug 2026 03:37:40 +0100 Subject: [PATCH 37/47] fix(cli): aggregate glob decode issues across sql_paths and schema_paths (review: CLI-1958) Go's UnmarshalExact decodes the whole config in one mapstructure pass and joins every field's decode error together (decodeStructFromMap never stops at the first field), so a config invalid in both db.seed.sql_paths and db.migrations.schema_paths reports both in one combined error. The reader was failing on the first field and never evaluating the second. Verified against the real apps/cli-go/pkg/config package (scratch probe via a local replace directive) with both fields invalid simultaneously. --- .../shared/legacy-db-config.toml-read.ts | 164 ++++++++++++------ .../legacy-db-config.toml-read.unit.test.ts | 43 +++++ 2 files changed, 151 insertions(+), 56 deletions(-) diff --git a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts index 12dc190bfb..80709315d9 100644 --- a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts +++ b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts @@ -1959,22 +1959,45 @@ const readDbTomlCore = Effect.fnUntraced(function* ( : typeof value === "object" && value !== null ? "map[string]interface {}" : undefined; - const legacyFailOnUnconvertibleGlobEntries = ( + // Pure — returns the mapstructure-style issue strings for a real `Glob` array's + // unconvertible elements WITHOUT failing. Go's `UnmarshalExact` decodes the WHOLE + // config in a SINGLE mapstructure pass: `decodeStructFromMap`'s per-field loop + // (`mapstructure.go:1657-1724`) appends each field's decode error to a shared `errs` + // slice and keeps going — it never stops at the first field's error — then + // `errors.Join(errs...)`-s everything together at the very end + // (`mapstructure.go:1777`). So an invalid `db.migrations.schema_paths` does NOT + // prevent `db.seed.sql_paths` from ALSO being decoded (and erroring) in the same + // pass; both surface together in ONE combined error. Verified empirically against + // `apps/cli-go` (`config.Load` with both fields containing an unconvertible entry, + // e.g. `sql_paths = [[]]` + `schema_paths = [[]]`): the single returned error + // contains BOTH lines, `db.migrations.schema_paths[0]` BEFORE `db.seed.sql_paths[0]` + // — Go's `db` struct declares `Migrations` before `Seed` (`pkg/config/db.go:90-91`), + // and mapstructure iterates struct fields in declaration order, not alphabetically, + // so callers below must combine in that same order before failing once (see + // `legacyFailOnGlobIssues`). + const legacyGlobArrayIssues = ( keyPath: string, values: ReadonlyArray, - ): Effect.Effect => { - const issues = values.flatMap((value, index) => { + ): ReadonlyArray => + values.flatMap((value, index) => { const goType = legacyGoUnconvertibleType(value); return goType === undefined ? [] : [`'${keyPath}[${index}]' expected type 'string', got unconvertible type '${goType}'`]; }); - return issues.length === 0 + // Fails ONCE with every issue collected across BOTH `Glob` fields (see + // `legacyGlobArrayIssues`'s doc comment) — never called per-field, so a config + // invalid in both `db.seed.sql_paths` and `db.migrations.schema_paths` reports both, + // matching Go's single combined `UnmarshalExact` error instead of only the first + // field checked. + const legacyFailOnGlobIssues = ( + issues: ReadonlyArray, + ): Effect.Effect => + issues.length === 0 ? Effect.void : fail( `failed to parse config: decoding failed due to the following error(s):\n\n${issues.join("\n")}`, ); - }; // A TOP-LEVEL raw value that is neither an array nor a string (e.g. // `schema_paths = 42`/`true`, or a stray inline table) still reaches // mapstructure's `decodeSlice`, which is weakly typed the same way an array @@ -1991,46 +2014,78 @@ const readDbTomlCore = Effect.fnUntraced(function* ( // 'string', got unconvertible type 'map[string]interface {}'`. Never // called with `undefined` — an absent key has its own Go-matching default // per caller below, so callers guard that case before reaching here. + // Pure — the TOP-LEVEL scalar fallback (see doc comment above), returning either the + // one resolved pattern or the one issue it would raise, WITHOUT failing (same reason + // as `legacyGlobArrayIssues`: the caller combines issues across both `Glob` fields + // before deciding whether to fail). const legacyResolveScalarGlobFallback = ( keyPath: string, value: unknown, - ): Effect.Effect, LegacyDbConfigLoadError> => - Effect.gen(function* () { - if (typeof value === "object" && value !== null && Object.keys(value).length === 0) { - return []; - } - const coerced = legacyWeakCoerceGlobEntry(value); - if (coerced !== undefined) { - return [coerced]; - } - yield* legacyFailOnUnconvertibleGlobEntries(keyPath, [value]); - return []; - }); + ): { readonly resolved: ReadonlyArray; readonly issues: ReadonlyArray } => { + if (typeof value === "object" && value !== null && Object.keys(value).length === 0) { + return { resolved: [], issues: [] }; + } + const coerced = legacyWeakCoerceGlobEntry(value); + if (coerced !== undefined) { + return { resolved: [coerced], issues: [] }; + } + return { resolved: [], issues: legacyGlobArrayIssues(keyPath, [value]) }; + }; + /** + * Resolves ONE `Glob`-typed field (`[db.seed].sql_paths` / `[db.migrations]. + * schema_paths`) into its pre-supabase-join patterns, covering every decode branch + * in one place: override env var, real array, bare string, absent key (caller's own + * Go-matching default), or the top-level scalar fallback. Returns any issues + * alongside the best-effort patterns rather than failing here — see + * `legacyFailOnGlobIssues`'s doc comment for why the two `Glob` fields must combine + * their issues into ONE error before failing, matching Go's single `UnmarshalExact` + * pass, instead of each field failing independently on its own first bad entry. + */ + const legacyResolveGlobField = ( + keyPath: string, + raw: unknown, + override: string | undefined, + absentDefault: ReadonlyArray, + ): { readonly patterns: ReadonlyArray; readonly issues: ReadonlyArray } => { + if (override !== undefined) { + return { patterns: splitGoSeedPaths(override), issues: [] }; + } + if (Array.isArray(raw)) { + return { + patterns: raw + .map((pattern) => legacyWeakCoerceGlobEntry(pattern)) + .filter((pattern): pattern is string => pattern !== undefined) + .map((pattern) => legacyExpandEnv(pattern, lookup)), + issues: legacyGlobArrayIssues(keyPath, raw), + }; + } + if (typeof raw === "string") { + return { patterns: splitGoSeedPaths(raw), issues: [] }; + } + if (raw === undefined) { + return { patterns: absentDefault, issues: [] }; + } + const fallback = legacyResolveScalarGlobFallback(keyPath, raw); + return { + patterns: fallback.resolved.map((pattern) => legacyExpandEnv(pattern, lookup)), + issues: fallback.issues, + }; + }; const rawSqlPaths = seedRaw?.["sql_paths"]; const sqlPathsOverride = remoteOverrideKeys.has("db.seed.sql_paths") ? undefined : envOverride("SUPABASE_DB_SEED_SQL_PATHS"); - if (sqlPathsOverride === undefined && Array.isArray(rawSqlPaths)) { - yield* legacyFailOnUnconvertibleGlobEntries("db.seed.sql_paths", rawSqlPaths); - } - const sqlPathPatterns = - sqlPathsOverride !== undefined - ? splitGoSeedPaths(sqlPathsOverride) - : Array.isArray(rawSqlPaths) - ? rawSqlPaths - .map((pattern) => legacyWeakCoerceGlobEntry(pattern)) - .filter((pattern): pattern is string => pattern !== undefined) - .map((pattern) => legacyExpandEnv(pattern, lookup)) - : typeof rawSqlPaths === "string" - ? splitGoSeedPaths(rawSqlPaths) - : rawSqlPaths === undefined - ? ["seed.sql"] - : (yield* legacyResolveScalarGlobFallback("db.seed.sql_paths", rawSqlPaths)).map( - (pattern) => legacyExpandEnv(pattern, lookup), - ); + const sqlPathsResolved = legacyResolveGlobField( + "db.seed.sql_paths", + rawSqlPaths, + sqlPathsOverride, + ["seed.sql"], + ); // Patterns are already env-expanded above (Go's LoadEnvHook runs before the split); // resolve each to Go's config-load form (absolute verbatim, relative supabase-joined). - const seedSqlPaths = sqlPathPatterns.map((pattern) => legacyResolveSeedSqlPath(path, pattern)); + const seedSqlPaths = sqlPathsResolved.patterns.map((pattern) => + legacyResolveSeedSqlPath(path, pattern), + ); // `[db.migrations] schema_paths` — Go default `[]` (`pkg/config/templates/config.toml:64`), // resolved through the exact same decode + env-expand + supabase-join pipeline as @@ -2041,26 +2096,23 @@ const readDbTomlCore = Effect.fnUntraced(function* ( const schemaPathsOverride = remoteOverrideKeys.has("db.migrations.schema_paths") ? undefined : envOverride("SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS"); - if (schemaPathsOverride === undefined && Array.isArray(rawSchemaPaths)) { - yield* legacyFailOnUnconvertibleGlobEntries("db.migrations.schema_paths", rawSchemaPaths); - } - const schemaPathPatterns = - schemaPathsOverride !== undefined - ? splitGoSeedPaths(schemaPathsOverride) - : Array.isArray(rawSchemaPaths) - ? rawSchemaPaths - .map((pattern) => legacyWeakCoerceGlobEntry(pattern)) - .filter((pattern): pattern is string => pattern !== undefined) - .map((pattern) => legacyExpandEnv(pattern, lookup)) - : typeof rawSchemaPaths === "string" - ? splitGoSeedPaths(rawSchemaPaths) - : rawSchemaPaths === undefined - ? [] - : (yield* legacyResolveScalarGlobFallback( - "db.migrations.schema_paths", - rawSchemaPaths, - )).map((pattern) => legacyExpandEnv(pattern, lookup)); - const schemaPaths = schemaPathPatterns.map((pattern) => legacyResolveSeedSqlPath(path, pattern)); + const schemaPathsResolved = legacyResolveGlobField( + "db.migrations.schema_paths", + rawSchemaPaths, + schemaPathsOverride, + [], + ); + + // Go's `UnmarshalExact` decodes the whole config in ONE mapstructure pass (see + // `legacyGlobArrayIssues`'s doc comment) — combine BOTH `Glob` fields' issues, in + // Go's struct-declaration order (`Migrations` before `Seed`), before failing once, + // so a config invalid in both surfaces both, matching Go's single combined error + // instead of only the first field checked. + yield* legacyFailOnGlobIssues([...schemaPathsResolved.issues, ...sqlPathsResolved.issues]); + + const schemaPaths = schemaPathsResolved.patterns.map((pattern) => + legacyResolveSeedSqlPath(path, pattern), + ); // `[db.vault]` secrets: env-expand each value, then decrypt dotenvx `encrypted:` // ciphertext. `resolved` mirrors Go's `len(SHA256) > 0` gate (Go sets SHA256 only diff --git a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts index 8311043a23..3fd861d780 100644 --- a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts @@ -611,6 +611,49 @@ describe("legacyReadDbToml", () => { }, ); + it.effect( + "aggregates unconvertible-entry issues from BOTH db.seed.sql_paths and db.migrations.schema_paths in one error (Go UnmarshalExact single-pass parity, review CLI-1958)", + () => { + // Go's `UnmarshalExact` decodes the WHOLE config in a SINGLE mapstructure pass: + // `decodeStructFromMap`'s per-field loop never stops at the first field's error + // — it visits every field, collects every error, then joins them all together + // at the end. So a config invalid in BOTH `Glob` fields reports BOTH, not just + // whichever field is checked first. Verified empirically against `apps/cli-go` + // (`config.Load` with `sql_paths = [[]]` + `schema_paths = [[]]`): the single + // returned error contains both lines, `db.migrations.schema_paths[0]` BEFORE + // `db.seed.sql_paths[0]` — Go's `db` struct declares `Migrations` before `Seed` + // (`pkg/config/db.go:90-91`), so mapstructure visits (and therefore reports) + // `schema_paths` first regardless of which field this reader happens to resolve + // first internally. + const dir = withConfig( + ["[db.seed]", "sql_paths = [[]]", "", "[db.migrations]", "schema_paths = [[]]", ""].join( + "\n", + ), + ); + return read(dir).pipe( + Effect.exit, + Effect.tap((exit) => + Effect.sync(() => { + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const message = JSON.stringify(exit.cause); + const schemaIssue = + "'db.migrations.schema_paths[0]' expected type 'string', got unconvertible type '[]interface {}'"; + const seedIssue = + "'db.seed.sql_paths[0]' expected type 'string', got unconvertible type '[]interface {}'"; + expect(message).toContain(schemaIssue); + expect(message).toContain(seedIssue); + // Both issues in ONE combined error, schema_paths first (Go's struct + // field declaration order), not two separate failures. + expect(message.indexOf(schemaIssue)).toBeLessThan(message.indexOf(seedIssue)); + } + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + it.effect( "an explicit remote db.migrations.schema_paths beats SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS", () => { From 8e886cea0a55211a64b9531b7b786fcee25b79dc Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Thu, 6 Aug 2026 04:50:59 +0100 Subject: [PATCH 38/47] fix(cli): provide pg-delta runtime services to db reset (review: CLI-1958) legacyDbResetRuntimeLayer omitted LegacyPgDeltaSslProbe, LegacyEdgeRuntimeScript, and LegacyDockerRun, unlike legacyDbPushRuntimeLayer. When pg-delta caching is enabled, the post-reset catalog cache reaches those services via legacyExportCatalogPgDelta; missing them is an untyped missing-service defect the handler's Effect.catch cannot recover from, crashing the process after the remote database has already been reset instead of writing the catalog or emitting Go's best-effort warning. Compose the same three layers db push uses. Added a regression test that builds the real legacyDbResetRuntimeLayer (not a mocked service) and asserts both services are actually exposed. --- .../legacy/commands/db/reset/reset.layers.ts | 22 +++ .../db/reset/reset.layers.unit.test.ts | 146 ++++++++++++++++++ 2 files changed, 168 insertions(+) create mode 100644 apps/cli/src/legacy/commands/db/reset/reset.layers.unit.test.ts diff --git a/apps/cli/src/legacy/commands/db/reset/reset.layers.ts b/apps/cli/src/legacy/commands/db/reset/reset.layers.ts index 23e0d88514..6bfb86e9af 100644 --- a/apps/cli/src/legacy/commands/db/reset/reset.layers.ts +++ b/apps/cli/src/legacy/commands/db/reset/reset.layers.ts @@ -9,8 +9,11 @@ import { legacyProjectRefLayer } from "../../../config/legacy-project-ref.layer. import { legacyDbConfigLayer } from "../../../shared/legacy-db-config.layer.ts"; import { legacyDbConnectionLayer } from "../../../shared/legacy-db-connection.layer.ts"; import { legacyDebugLoggerLayer } from "../../../shared/legacy-debug-logger.layer.ts"; +import { legacyDockerRunLayer } from "../../../shared/legacy-docker-run.layer.ts"; +import { legacyEdgeRuntimeScriptLayer } from "../../../shared/legacy-edge-runtime-script.layer.ts"; import { stdinLayer } from "../../../../shared/runtime/stdin.layer.ts"; import { legacyIdentityStitchLayer } from "../../../shared/legacy-identity-stitch.ts"; +import { legacyPgDeltaSslProbeLayer } from "../../../shared/legacy-pgdelta-ssl-probe.layer.ts"; import { legacyLinkedProjectCacheLayer } from "../../../telemetry/legacy-linked-project-cache.layer.ts"; import { legacyTelemetryStateLayer } from "../../../telemetry/legacy-telemetry-state.layer.ts"; import { legacyDbBootstrapSeamLayer } from "../shared/legacy-db-bootstrap.seam.layer.ts"; @@ -24,6 +27,17 @@ import { legacyDbBootstrapSeamLayer } from "../shared/legacy-db-bootstrap.seam.l * is fully native. The local path's container primitives still reach `LegacyGoProxy` * through the bootstrap seam below (`legacyDbBootstrapSeamLayer`, ambient from the * root) — that native port is CLI-1955's scope, not this one. + * + * `legacyDockerRunLayer` + `edgeRuntime` + `legacyPgDeltaSslProbeLayer` (same shape + * as `db push`'s `legacyDbPushRuntimeLayer`) are required by the post-reset + * best-effort pg-delta catalog cache below (`legacyTryCacheMigrationsCatalog` → + * `legacyExportCatalogPgDelta` → `LegacyEdgeRuntimeScript`/`LegacyPgDeltaSslProbe`, + * `legacy-pgdelta.ts`/`legacy-pgdelta-ssl.ts`). Without them, a versionless remote + * reset with pg-delta enabled (`[experimental.pgdelta].enabled` or + * `SUPABASE_EXPERIMENTAL_PG_DELTA`) would hit an unhandled missing-service defect + * — not caught by the handler's typed `Effect.catch` — AFTER the remote database + * has already been reset, instead of writing the catalog or emitting Go's + * best-effort warning (review CLI-1958). */ const cliConfig = legacyCliConfigLayer.pipe(Layer.provide(legacyDebugLoggerLayer)); const httpClient = legacyHttpClientLayer.pipe(Layer.provide(legacyDebugLoggerLayer)); @@ -58,9 +72,17 @@ const dbConfig = legacyDbConfigLayer.pipe( Layer.provide(legacyIdentityStitchLayer), ); +const edgeRuntime = legacyEdgeRuntimeScriptLayer.pipe( + Layer.provide(legacyDockerRunLayer), + Layer.provide(cliConfig), +); + export const legacyDbResetRuntimeLayer = Layer.mergeAll( dbConfig, legacyDbConnectionLayer, + legacyDockerRunLayer, + edgeRuntime, + legacyPgDeltaSslProbeLayer, cliConfig, httpClient, credentials, diff --git a/apps/cli/src/legacy/commands/db/reset/reset.layers.unit.test.ts b/apps/cli/src/legacy/commands/db/reset/reset.layers.unit.test.ts new file mode 100644 index 0000000000..9f7fe90e68 --- /dev/null +++ b/apps/cli/src/legacy/commands/db/reset/reset.layers.unit.test.ts @@ -0,0 +1,146 @@ +/** + * Layer-exposure test for `legacyDbResetRuntimeLayer`. + * + * Regression guard (review CLI-1958): the post-reset best-effort pg-delta + * catalog cache (`legacyTryCacheMigrationsCatalog` in `reset.handler.ts`, gated + * on `[experimental.pgdelta].enabled` / `SUPABASE_EXPERIMENTAL_PG_DELTA`) reaches + * `LegacyEdgeRuntimeScript` and `LegacyPgDeltaSslProbe` via + * `legacyExportCatalogPgDelta` (`legacy-pgdelta.ts`). `legacyDbResetRuntimeLayer` + * previously omitted both services (and the `LegacyDockerRun` layer the real + * edge-runtime implementation needs) — unlike `legacyDbPushRuntimeLayer`, which + * already composes all three. That gap was invisible to `reset.integration.test.ts` + * because that suite drives `legacyDbReset` directly with its own hand-built layer + * (which mocks `LegacyEdgeRuntimeScript`/`LegacyPgDeltaSslProbe` in), bypassing + * `reset.layers.ts` entirely — so a versionless remote reset with pg-delta enabled + * would crash on a missing-service defect (uncaught by the handler's typed + * `Effect.catch`) AFTER the remote database was already reset. This test builds + * the REAL `legacyDbResetRuntimeLayer` (not a mock of the pg-delta services) and + * asserts both are actually present in its context. + * + * See `db/lint/lint.layers.unit.test.ts` for the canonical ambient-stub pattern. + */ + +import { describe, expect, it } from "@effect/vitest"; +import { BunServices } from "@effect/platform-bun"; +import { Effect, Layer, Option } from "effect"; + +import { + mockAnalytics, + mockOutput, + mockProcessControl, + mockRuntimeInfo, + mockStdin, + mockTelemetryRuntime, + mockTty, +} from "../../../../../tests/helpers/mocks.ts"; +import { + mockLegacyCliConfig, + mockLegacyCredentialsLayer, + mockLegacyLinkedProjectCacheLayer, + mockLegacyTelemetryStateLayer, +} from "../../../../../tests/helpers/legacy-mocks.ts"; + +import { CliArgs } from "../../../../shared/cli/cli-args.service.ts"; +import { + LegacyDebugFlag, + LegacyDnsResolverFlag, + LegacyExperimentalFlag, + LegacyNetworkIdFlag, + LegacyOutputFlag, + LegacyProfileFlag, + LegacyWorkdirFlag, +} from "../../../../shared/legacy/global-flags.ts"; + +import { LegacyPlatformApiFactory } from "../../../auth/legacy-platform-api-factory.service.ts"; +import { LegacyProjectRefResolver } from "../../../config/legacy-project-ref.service.ts"; +import { LegacyDbConfigResolver } from "../../../shared/legacy-db-config.service.ts"; +import { LegacyDbConnection } from "../../../shared/legacy-db-connection.service.ts"; +import { LegacyEdgeRuntimeScript } from "../../../shared/legacy-edge-runtime-script.service.ts"; +import { LegacyPgDeltaSslProbe } from "../../../shared/legacy-pgdelta-ssl-probe.service.ts"; + +import { legacyDbResetRuntimeLayer } from "./reset.layers.ts"; + +/** + * Builds a stub ambient layer that satisfies every external service required by + * `legacyDbResetRuntimeLayer` from the root runtime. Services whose logic is not + * under test are no-op stubs; `LegacyEdgeRuntimeScript` and `LegacyPgDeltaSslProbe` + * are deliberately NOT stubbed here — the point of this test is to prove the real + * `legacyDbResetRuntimeLayer` provides them itself. + */ +function ambientStubs() { + const analytics = mockAnalytics(); + const out = mockOutput(); + + const flagLayers = Layer.mergeAll( + Layer.succeed(LegacyDebugFlag, false), + Layer.succeed(LegacyProfileFlag, "supabase"), + Layer.succeed(LegacyWorkdirFlag, Option.none()), + Layer.succeed(LegacyOutputFlag, Option.none()), + Layer.succeed(LegacyDnsResolverFlag, "native"), + Layer.succeed(LegacyNetworkIdFlag, Option.none()), + Layer.succeed(LegacyExperimentalFlag, false), + Layer.succeed(CliArgs, { args: ["db", "reset"] }), + ); + + // Stub out the heavy service layers so layer construction doesn't require a + // real DB, real API, or real credentials. + const heavyServiceStubs = Layer.mergeAll( + Layer.succeed(LegacyDbConnection, { + connect: () => Effect.die("db-connection not needed for layer-exposure test"), + }), + Layer.succeed(LegacyDbConfigResolver, { + resolve: () => Effect.die("db-config-resolver not needed for layer-exposure test"), + resolvePoolerFallback: () => + Effect.die("db-config-resolver not needed for layer-exposure test"), + }), + Layer.succeed(LegacyProjectRefResolver, { + resolve: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), + resolveForLink: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), + resolveOptional: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), + loadProjectRef: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), + promptProjectRef: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), + }), + Layer.succeed(LegacyPlatformApiFactory, { + make: Effect.die("platform-api-factory not needed for layer-exposure test"), + }), + ); + + return Layer.mergeAll( + BunServices.layer, + mockRuntimeInfo(), + mockTty(), + mockProcessControl().layer, + mockStdin(false), + analytics.layer, + mockTelemetryRuntime(), + out.layer, + flagLayers, + mockLegacyCliConfig({ workdir: "/tmp/reset-layers-test" }), + mockLegacyCredentialsLayer, + mockLegacyLinkedProjectCacheLayer, + mockLegacyTelemetryStateLayer, + heavyServiceStubs, + ); +} + +describe("legacyDbResetRuntimeLayer — pg-delta service exposure (regression guard, review CLI-1958)", () => { + it.live( + "exposes LegacyEdgeRuntimeScript so the post-reset pg-delta catalog cache does not crash on a missing-service defect", + () => { + return Effect.gen(function* () { + const edgeRuntime = yield* Effect.serviceOption(LegacyEdgeRuntimeScript); + expect(Option.isSome(edgeRuntime)).toBe(true); + }).pipe(Effect.provide(legacyDbResetRuntimeLayer), Effect.provide(ambientStubs())); + }, + ); + + it.live( + "exposes LegacyPgDeltaSslProbe so the post-reset pg-delta catalog cache does not crash on a missing-service defect", + () => { + return Effect.gen(function* () { + const sslProbe = yield* Effect.serviceOption(LegacyPgDeltaSslProbe); + expect(Option.isSome(sslProbe)).toBe(true); + }).pipe(Effect.provide(legacyDbResetRuntimeLayer), Effect.provide(ambientStubs())); + }, + ); +}); From c9ecd1550d3f05e53ba431273d46e9996254bb5b Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Thu, 6 Aug 2026 04:51:10 +0100 Subject: [PATCH 39/47] fix(cli): accept Go's underscore digit separators in scanner-size integers (review: CLI-1958) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit parseGoBaseZeroInt rejected underscore digit separators outright (e.g. "1_048_576"), silently falling back to the 256KiB default even though Go's strconv.ParseInt(s, 0, 64) accepts them per its integer-literal grammar. A statement between the two limits would apply in TS but Go would already have failed with "bufio.Scanner: token too long". Implemented Go's exact underscore-placement grammar, verified empirically against a real Go strconv.ParseInt(s, 0, 64): a single underscore may sit immediately after a base prefix (0x/0o/0b, or the bare leading "0" of legacy octal) or between two digits — never doubled, never leading a plain decimal literal, never trailing. --- .../legacy/shared/legacy-migration-apply.ts | 35 +++++-- .../legacy-migration-apply.unit.test.ts | 97 +++++++++++++++++++ 2 files changed, 124 insertions(+), 8 deletions(-) diff --git a/apps/cli/src/legacy/shared/legacy-migration-apply.ts b/apps/cli/src/legacy/shared/legacy-migration-apply.ts index 89d36dcc5f..28f31df390 100644 --- a/apps/cli/src/legacy/shared/legacy-migration-apply.ts +++ b/apps/cli/src/legacy/shared/legacy-migration-apply.ts @@ -177,10 +177,23 @@ const GO_DEFAULT_MAX_SCANNER_CAPACITY = 256 * 1024; * ABOVE first, same as any other value — that consumes the trailing `B` before * base-0 parsing ever sees it (`"0x1B"→1`, not `27`), which is a genuine Go quirk * this port reproduces automatically by keeping the same two-step order, not a bug. - * Known residual delta: Go's base-0 grammar also permits `_` digit separators - * (e.g. `"1_048_576"`, verified empirically to equal `1048576`) — not reproduced - * here as a realistic byte-size override would never use one; flagging so a future - * parity sweep doesn't rediscover it. + * Go's base-0 grammar also permits `_` digit separators (e.g. `"1_048_576"`) — see + * {@link parseGoBaseZeroInt}'s own doc comment for the exact placement grammar. + */ +/** + * Go's underscore digit-separator grammar (`go.dev/ref/spec#Integer_literals`, + * reproduced by `strconv.ParseInt`'s base-0 mode, review CLI-1958): a SINGLE `_` + * may sit immediately after a base prefix (explicit `0x`/`0o`/`0b`, or the bare + * leading `"0"` of legacy octal) or between two digits of the same base — never + * doubled, never leading a plain (no-prefix) decimal literal, and never trailing. + * Verified empirically against the real `strconv.ParseInt(s, 0, 64)`: + * `"1_048_576"→1048576`, `"0x_100000"/"0x10_0000"→1048576`, + * `"0o_40000"/"0o4_0000"→16384`, `"0b_100000000000000000000"→1048576`, + * `"0_755"/"07_55"→493` (legacy octal, underscore right after the leading `"0"` + * or between later octal digits); while `"_1048576"`, `"1048576_"`, + * `"1__048576"`, `"0x100000_"`, and `"0_x100000"` (underscore splitting the + * leading `"0"` from the `"x"` — not a real prefix, so it's parsed as legacy + * octal digits `"x100000"`) all fail, matching Go exactly. */ const parseGoBaseZeroInt = (value: string): number | undefined => { const negative = value.startsWith("-"); @@ -206,11 +219,17 @@ const parseGoBaseZeroInt = (value: string): number | undefined => { } if (digits.length === 0) return undefined; - const validDigits = - base === 16 ? /^[0-9a-fA-F]+$/ : base === 8 ? /^[0-7]+$/ : base === 2 ? /^[01]+$/ : /^[0-9]+$/; - if (!validDigits.test(digits)) return undefined; + // Only a real base prefix (or the legacy-octal leading "0") may be followed + // immediately by an underscore; a plain decimal literal has no prefix to + // follow, so a leading underscore there is always invalid (matches Go). + const hadPrefix = base !== 10; + const digitClass = base === 16 ? "0-9a-fA-F" : base === 8 ? "0-7" : base === 2 ? "01" : "0-9"; + const validPattern = new RegExp( + `^${hadPrefix ? "_?" : ""}[${digitClass}](?:_?[${digitClass}])*$`, + ); + if (!validPattern.test(digits)) return undefined; - const n = Number.parseInt(digits, base); + const n = Number.parseInt(digits.replace(/_/g, ""), base); return negative ? -n : n; }; diff --git a/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts b/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts index a5c9022a18..74b06eb762 100644 --- a/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts @@ -703,6 +703,103 @@ describe("legacyApplySchemaFiles", () => { }, ); + it.effect( + "accepts underscore digit separators in a decimal SUPABASE_SCANNER_BUFFER_SIZE (Go strconv.ParseInt base-0 underscore-literal parity, review CLI-1958)", + () => { + // Go's base-0 integer grammar (`go.dev/ref/spec#Integer_literals`, reproduced + // by `strconv.ParseInt`) permits a single `_` between digits: "5_120" is the + // same 5120 (5KiB) byte count as the hex-literal test above's "0x1400" — + // verified empirically against the real `strconv.ParseInt("5_120", 0, 64)`. + // A parser that rejects underscores outright would silently fall back to the + // 256KiB default instead, so a statement between 5120 and 262144 bytes would + // apply in TS but Go would already have failed with "bufio.Scanner: token too + // long" at 5121 bytes. + const dir = mkdtempSync(join(tmpdir(), "legacy-schema-files-scanner-underscore-")); + mkdirSync(join(dir, "supabase"), { recursive: true }); + const file = join(dir, "supabase", "big.sql"); + writeFileSync(file, `SELECT 1;\nSELECT '${"a".repeat(5116)}';\n`); + const { session } = fakeSession(); + const previous = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = "5_120"; + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const exit = yield* legacyApplySchemaFiles( + session, + fs, + path, + dir, + ["supabase/big.sql"], + (message, suggestion) => + new TestError({ message: suggestion ? `${message} (${suggestion})` : message }), + ).pipe(Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const msg = JSON.stringify(exit.cause); + expect(msg).toContain("bufio.Scanner: token too long"); + // 5KiB (5_120 bytes), not the 256KiB hardcoded fallback an + // underscore-rejecting parser would have silently used instead. + expect(msg).toContain( + "Try setting SUPABASE_SCANNER_BUFFER_SIZE=5MB (current size is 5KB)", + ); + } + }).pipe( + Effect.ensuring( + Effect.sync(() => { + if (previous === undefined) delete process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + else process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = previous; + rmSync(dir, { recursive: true, force: true }); + }), + ), + Effect.provide(BunServices.layer), + ); + }, + ); + + it.effect( + "rejects an invalid underscore placement in SUPABASE_SCANNER_BUFFER_SIZE, unlike a valid digit separator (Go strconv.ParseInt underscore-grammar parity, review CLI-1958)", + () => { + // Go only permits a SINGLE underscore immediately after a base prefix or + // between two digits — never leading a plain (no-prefix) decimal literal, + // never doubled, never trailing. "_5120" (leading underscore, no prefix) is + // invalid in real Go (`strconv.ParseInt("_5120", 0, 64)` errors), so it falls + // back to the same 256KiB default as a genuinely unset/unparseable value — + // verified empirically against the real Go `strconv.ParseInt`. + const dir = mkdtempSync(join(tmpdir(), "legacy-schema-files-scanner-bad-underscore-")); + mkdirSync(join(dir, "supabase"), { recursive: true }); + const file = join(dir, "supabase", "big.sql"); + writeFileSync(file, `SELECT 1;\nSELECT '${"a".repeat(5116)}';\n`); + const { session } = fakeSession(); + const previous = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = "_5120"; + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const exit = yield* legacyApplySchemaFiles( + session, + fs, + path, + dir, + ["supabase/big.sql"], + (message, suggestion) => + new TestError({ message: suggestion ? `${message} (${suggestion})` : message }), + ).pipe(Effect.exit); + // The 5116-byte statement fits comfortably under the 256KiB default + // fallback, so an invalid underscore placement must NOT fail the apply. + expect(Exit.isSuccess(exit)).toBe(true); + }).pipe( + Effect.ensuring( + Effect.sync(() => { + if (previous === undefined) delete process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + else process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = previous; + rmSync(dir, { recursive: true, force: true }); + }), + ), + Effect.provide(BunServices.layer), + ); + }, + ); + it.effect( "rejects an oversized statement when SUPABASE_SCANNER_BUFFER_SIZE is set only in the project env (Go loadNestedEnv parity)", () => { From f119a1c08a305e3e58a5566111a0075e9aec36f2 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Thu, 6 Aug 2026 04:51:23 +0100 Subject: [PATCH 40/47] fix(cli): read the pg-delta catalog timestamp after the export, not before (review: CLI-1958) Both db reset and db push captured the snapshot's Clock.currentTimeMillis before calling legacyTryCacheMigrationsCatalog, i.e. before the hash and the pg-delta export (a network round-trip) resolved. Go's real WriteMigrationCatalogSnapshot reads time.Now().UTC() internally, after TryCacheMigrationsCatalog has already resolved hash and snapshot, immediately before the write. The early capture could make a concurrent cache write from another process sort in the wrong order during catalog resolution/retention. Moved the clock read inside legacyTryCacheMigrationsCatalog itself, right before the write, fixing both callers at their shared root. Added a regression test that proves the timestamp reflects a real time gap the mocked export takes before resolving. --- .../legacy/commands/db/reset/reset.handler.ts | 3 +- .../db/shared/legacy-pgdelta.cache.ts | 20 +++++- .../shared/legacy-pgdelta.cache.unit.test.ts | 69 +++++++++++++++++++ .../src/legacy/shared/legacy-db-push-core.ts | 3 +- 4 files changed, 88 insertions(+), 7 deletions(-) diff --git a/apps/cli/src/legacy/commands/db/reset/reset.handler.ts b/apps/cli/src/legacy/commands/db/reset/reset.handler.ts index 75d37e746a..4fa784f607 100644 --- a/apps/cli/src/legacy/commands/db/reset/reset.handler.ts +++ b/apps/cli/src/legacy/commands/db/reset/reset.handler.ts @@ -1,4 +1,4 @@ -import { Clock, Effect, FileSystem, Option, Path } from "effect"; +import { Effect, FileSystem, Option, Path } from "effect"; import { CliArgs } from "../../../../shared/cli/cli-args.service.ts"; import { detectGitBranch } from "../../../../shared/git/git-branch.ts"; @@ -446,7 +446,6 @@ export const legacyDbReset = Effect.fn("legacy.db.reset")(function* (flags: Lega conn: cfg.conn, isLocal: false, migrationsDir, - nowMillis: yield* Clock.currentTimeMillis, }).pipe( Effect.catch((error) => output.raw( diff --git a/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta.cache.ts b/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta.cache.ts index f06b687000..6fa9eee5e7 100644 --- a/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta.cache.ts +++ b/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta.cache.ts @@ -1,5 +1,5 @@ import { createHash } from "node:crypto"; -import { Effect, type FileSystem, Option, type Path } from "effect"; +import { Clock, Effect, type FileSystem, Option, type Path } from "effect"; import { Output } from "../../../../shared/output/output.service.ts"; import { LegacyMigrationsReadError } from "../../../shared/legacy-migration.errors.ts"; @@ -398,6 +398,20 @@ export const legacyWriteMigrationCatalogSnapshot = Effect.fnUntraced(function* ( * `diff/pgdelta.go` `ExportCatalogPgDelta`) rather than porting a second copy, * so this can't reintroduce the `/workspace` mount bug `pgcache/cache.go` had * (supabase/cli#5921). + * + * The snapshot's timestamp is read from `Clock` HERE — after `legacyHashMigrations` + * and `legacyExportCatalogPgDelta` (the network round-trip) have both resolved, + * immediately before the write — never accepted as a caller-supplied parameter. + * This mirrors Go's own call order exactly: `TryCacheMigrationsCatalog` + * (`pgcache/cache.go:71-91`) resolves `hash` and `snapshot` FIRST, and only THEN + * calls `WriteMigrationCatalogSnapshot`, which itself reads `time.Now().UTC()` + * (`pgcache/cache.go:151-163`) — i.e. Go's clock read happens LAST, right before + * the file write, not before the export. A caller capturing the timestamp before + * calling this function (review CLI-1958) would race a concurrent cache write + * from another process: Go would order the two snapshots by real write-time, but + * the early-captured timestamp could sort the wrong one as "latest" during + * catalog resolution/retention (`legacyResolveMigrationCatalogPath`, + * `legacyCleanupOldMigrationCatalogs`). */ export const legacyTryCacheMigrationsCatalog = Effect.fnUntraced(function* ( fs: FileSystem.FileSystem, @@ -414,7 +428,6 @@ export const legacyTryCacheMigrationsCatalog = Effect.fnUntraced(function* ( }; readonly isLocal: boolean; readonly migrationsDir: string; - readonly nowMillis: number; }, ) { if (!params.enabled) return; @@ -424,6 +437,7 @@ export const legacyTryCacheMigrationsCatalog = Effect.fnUntraced(function* ( targetRef: params.targetUrl, role: "postgres", }); + const nowMillis = yield* Clock.currentTimeMillis; yield* legacyWriteMigrationCatalogSnapshot( fs, path, @@ -431,6 +445,6 @@ export const legacyTryCacheMigrationsCatalog = Effect.fnUntraced(function* ( prefix, hash, snapshot, - params.nowMillis, + nowMillis, ); }); diff --git a/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta.cache.unit.test.ts b/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta.cache.unit.test.ts index 83535b91c0..9469c6a9b7 100644 --- a/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta.cache.unit.test.ts +++ b/apps/cli/src/legacy/commands/db/shared/legacy-pgdelta.cache.unit.test.ts @@ -8,6 +8,9 @@ import { Effect, FileSystem, Layer, Option, Path } from "effect"; import { Output } from "../../../../shared/output/output.service.ts"; import { mockOutput } from "../../../../../tests/helpers/mocks.ts"; +import { LegacyEdgeRuntimeScript } from "../../../shared/legacy-edge-runtime-script.service.ts"; +import { LegacyPgDeltaSslProbe } from "../../../shared/legacy-pgdelta-ssl-probe.service.ts"; +import { type LegacyPgDeltaContext } from "./legacy-pgdelta.ts"; import { type LegacySetupInputs, legacyBaselineCatalogFileName, @@ -25,6 +28,7 @@ import { legacyResolveDeclarativeCatalogPath, legacySanitizedCatalogPrefix, legacySetupInputsToken, + legacyTryCacheMigrationsCatalog, legacyWriteMigrationCatalogSnapshot, } from "./legacy-pgdelta.cache.ts"; @@ -419,6 +423,71 @@ describe("legacyWriteMigrationCatalogSnapshot + cleanup", () => { }); }); +describe("legacyTryCacheMigrationsCatalog — timestamp ordering (review CLI-1958)", () => { + // `it.live` (not `it.effect`): the mocked export below uses a real `Effect.sleep` + // to create a measurable time gap, which needs the real wall clock, not + // `it.effect`'s virtual `TestClock` (which never auto-advances and would hang). + it.live( + "reads the clock AFTER the pg-delta export resolves, matching Go's WriteMigrationCatalogSnapshot ordering", + () => { + // Go's `TryCacheMigrationsCatalog` (`pgcache/cache.go:71-91`) resolves `hash` + // and `snapshot` FIRST and only THEN calls `WriteMigrationCatalogSnapshot`, + // which itself reads `time.Now().UTC()` (`pgcache/cache.go:151-163`) — i.e. + // Go's clock read happens LAST, right before the file write. The mocked + // edge-runtime export below sleeps for a real, measurable interval before + // resolving; the written snapshot's embedded timestamp must reflect a moment + // AFTER that sleep, proving the clock was read after the export — not + // captured up front by a caller before this function even started (the + // pre-fix bug). + const dir = withTemp(); + const migrationsDir = join(dir, "supabase", "migrations"); + mkdirSync(migrationsDir, { recursive: true }); + // Mirrors `legacyPgDeltaTempPath` (`/supabase/.temp/pgdelta`). + const tempDir = join(dir, "supabase", ".temp", "pgdelta"); + const beforeCallMillis = Date.now(); + const edge = Layer.succeed(LegacyEdgeRuntimeScript, { + run: () => + Effect.gen(function* () { + yield* Effect.sleep("30 millis"); + return { stdout: "{}", stderr: "" }; + }), + }); + const sslProbe = Layer.succeed(LegacyPgDeltaSslProbe, { + requireSsl: () => Effect.succeed(false), + requireSslForHost: () => Effect.succeed(false), + }); + const ctx: LegacyPgDeltaContext = { + projectId: "test", + cwd: dir, + npmVersion: undefined, + denoVersion: 1, + }; + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + yield* legacyTryCacheMigrationsCatalog(fs, path, ctx, { + enabled: true, + targetUrl: "postgresql://postgres:postgres@127.0.0.1:5432/postgres", + conn: { host: "127.0.0.1", port: 5432, user: "postgres", database: "postgres" }, + isLocal: true, + migrationsDir, + }); + const names = (yield* fs.readDirectory(tempDir)).filter((n) => + n.startsWith("catalog-local-migrations-"), + ); + expect(names.length).toBe(1); + const match = /-(\d+)\.json$/.exec(names[0]!); + expect(match).not.toBeNull(); + const embeddedMillis = Number(match![1]); + expect(embeddedMillis).toBeGreaterThanOrEqual(beforeCallMillis + 25); + }).pipe( + Effect.provide(Layer.mergeAll(BunServices.layer, mockOutput().layer, edge, sslProbe)), + Effect.tap(() => Effect.sync(() => rmSync(dir, { recursive: true, force: true }))), + ); + }, + ); +}); + describe("legacyCleanupOldMigrationCatalogs", () => { it.effect("only prunes files matching the given prefix's family", () => { const dir = withTemp(); diff --git a/apps/cli/src/legacy/shared/legacy-db-push-core.ts b/apps/cli/src/legacy/shared/legacy-db-push-core.ts index 31b855bdd4..685dd53742 100644 --- a/apps/cli/src/legacy/shared/legacy-db-push-core.ts +++ b/apps/cli/src/legacy/shared/legacy-db-push-core.ts @@ -1,4 +1,4 @@ -import { Clock, Effect, FileSystem, Option, Path } from "effect"; +import { Effect, FileSystem, Option, Path } from "effect"; import { legacyPromptYesNo } from "../../shared/legacy/legacy-prompt-yes-no.ts"; import { CONTEXT_CANCELED_MESSAGE } from "../../shared/output/errors.ts"; @@ -346,7 +346,6 @@ export const legacyDbPushCore = Effect.fnUntraced(function* (input: LegacyDbPush conn, isLocal, migrationsDir: path.join(workdir, "supabase", "migrations"), - nowMillis: yield* Clock.currentTimeMillis, }).pipe( Effect.catch((error) => output.raw( From 319e84e63028935dc43874dc5fbfea8a1d5d5e5f Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Thu, 6 Aug 2026 04:51:33 +0100 Subject: [PATCH 41/47] docs(cli): document the local reset path's pg-delta catalog write (review: CLI-1958) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit SIDE_EFFECTS.md labeled the migrations-catalog cache write as remote-path only. Go's start.SetupLocalDatabase (called by the local reset's PG15 recreate branch, behind this port's db __db-bootstrap seam) also calls pgcache.TryCacheMigrationsCatalog after MigrateAndSeed, with a "local" prefix — inherited automatically since the local path delegates to the real Go binary rather than being reimplemented in TS. The PG<=14 branch never calls it at all. Documented both paths and the PG14/PG15 split. --- .../legacy/commands/db/reset/SIDE_EFFECTS.md | 48 ++++++++++++------- 1 file changed, 30 insertions(+), 18 deletions(-) diff --git a/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md b/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md index 361f52bcef..fde2825a5b 100644 --- a/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md +++ b/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md @@ -28,11 +28,11 @@ is still handled by Go (CLI-1955 scope, unaffected by CLI-1958). ## Files Written -| Path | Format | When | -| ------------------------------------------------------------------------------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `~/.supabase//linked-project.json` | JSON | `--linked` (post-run cache) | -| `~/.supabase/telemetry.json` | JSON | always (post-run telemetry flush) | -| `/supabase/.temp/pgdelta/catalog--migrations--.json` | JSON | remote path only, best-effort, after either branch (schema-files or migrations) + seeding succeed, when no `--version`/`--last` resolved a version AND pg-delta is enabled (`[experimental.pgdelta].enabled` or `SUPABASE_EXPERIMENTAL_PG_DELTA`); a failure only warns on stderr and never fails the reset (Go's `down.ResetAll` → `pgcache.TryCacheMigrationsCatalog`, `down.go:58-59`) — see Notes | +| Path | Format | When | +| ------------------------------------------------------------------------------- | ------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `~/.supabase//linked-project.json` | JSON | `--linked` (post-run cache) | +| `~/.supabase/telemetry.json` | JSON | always (post-run telemetry flush) | +| `/supabase/.temp/pgdelta/catalog--migrations--.json` | JSON | best-effort, after migrations/seeding succeed, when no `--version`/`--last` resolved a version AND pg-delta is enabled (`[experimental.pgdelta].enabled` or `SUPABASE_EXPERIMENTAL_PG_DELTA`); a failure only warns on stderr and never fails the reset — see Notes. **Remote path** (native TS, `` = the project ref/URL hash): after either apply branch (schema-files or migrations) (Go's `down.ResetAll` → `pgcache.TryCacheMigrationsCatalog`, `down.go:58-59`). **Local path** (inherited from the real Go binary via the `db __db-bootstrap` seam, `` = `"local"`): PG15 only, after `MigrateAndSeed` inside `start.SetupLocalDatabase` (`start.go:359-381`) — the PG≤14 branch (`resetDatabase14`) never calls this at all, so a PG≤14 local project never writes this file regardless of pg-delta config | On the local path the Go seam additionally recreates the `supabase_db_` container/volume and applies the initial schema (`SetupLocalDatabase`). @@ -182,19 +182,31 @@ path has no confirmation prompt. `encrypted:` vault secrets are NOT skipped on the remote path — `legacyCheckDbToml` decrypts them into `toml.vault`, and `legacyUpsertVaultSecrets` upserts the decrypted values unconditionally, before either branch (schema-files or migrations) runs. -- **Migrations catalog cache**: ported (Go's best-effort `down.ResetAll` → - `pgcache.TryCacheMigrationsCatalog`, `down.go:48-61`). Runs on the remote path only, - after either apply branch (schema-files or migrations) and seeding complete, gated on - BOTH no `--version`/`--last` having resolved a version AND pg-delta being enabled - (`[experimental.pgdelta].enabled` or `SUPABASE_EXPERIMENTAL_PG_DELTA` — see - Environment Variables); a versioned reset never refreshes the cache, matching Go's - own `len(version) > 0` no-op inside `TryCacheMigrationsCatalog` itself. Exports the - target's pg-delta catalog via the edge-runtime stack and writes it under - `supabase/.temp/pgdelta/` (see Files Written), pruning older snapshots for the same - prefix (retains 2). A failure only warns on stderr and never fails the reset, matching - Go exactly. Reuses `legacyExportCatalogPgDelta` and `legacyTryCacheMigrationsCatalog` - — the same helpers `db push` uses for its own post-apply cache (see that command's - SIDE_EFFECTS Notes) — rather than a second copy. +- **Migrations catalog cache**: gated on BOTH no `--version`/`--last` having resolved + a version AND pg-delta being enabled (`[experimental.pgdelta].enabled` or + `SUPABASE_EXPERIMENTAL_PG_DELTA` — see Environment Variables); a versioned reset + never refreshes the cache, matching Go's own `len(version) > 0` no-op inside + `TryCacheMigrationsCatalog` itself. A failure only warns on stderr and never fails + the reset, matching Go exactly. Writes under `supabase/.temp/pgdelta/` (see Files + Written), pruning older snapshots for the same prefix (retains 2). This is NOT + remote-only — both paths write it, on different call chains: + - **Remote path** (native TS, ported): Go's best-effort `down.ResetAll` → + `pgcache.TryCacheMigrationsCatalog` (`down.go:48-61`), after either apply branch + (schema-files or migrations) and seeding complete. Exports the target's pg-delta + catalog via the edge-runtime stack. Reuses `legacyExportCatalogPgDelta` and + `legacyTryCacheMigrationsCatalog` — the same helpers `db push` uses for its own + post-apply cache (see that command's SIDE_EFFECTS Notes) — rather than a second + copy. + - **Local path** (inherited automatically from the real Go binary behind the + `db __db-bootstrap --mode recreate` seam — not reimplemented in TS, CLI-1955 + scope): `start.SetupLocalDatabase` (`start.go:359-381`) calls the same + `pgcache.TryCacheMigrationsCatalog` (with prefix `"local"`) right after + `apply.MigrateAndSeed` succeeds, warning the same way on failure. This only + happens on the **PG15** recreate branch (`resetDatabase15` → `SetupLocalDatabase`, + `reset.go:146-174`) — the **PG≤14** branch (`resetDatabase14`, `reset.go:128-144`) + returns immediately after `apply.MigrateAndSeed` and never calls + `pgcache.TryCacheMigrationsCatalog` at all, so a PG≤14 local project never writes + this file, no matter how pg-delta is configured. - The local path's own `--experimental` schema-files branch is still handled by the Go child behind the `db __db-bootstrap` seam (CLI-1955 scope) — this command's handler forwards `--experimental` to that seam but does not implement the branch From 89acd6c011e92bdc4e67bdf4ebe79dd492caf446 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Thu, 6 Aug 2026 04:51:46 +0100 Subject: [PATCH 42/47] fix(cli): sort directory entries before walking to match Go's fs.WalkDir order (review: CLI-1958) legacyWalkSqlFiles iterated a directory's readDirectory() result in raw filesystem enumeration order. Go's fs.WalkDir visits entries in lexical byte order (os.ReadDir's own "sorted by filename" contract), so when a matched directory has multiple problematic children (e.g. two unreadable subdirectories), Go deterministically fails on the lexically-first one. This port's unsorted iteration could pick a different one depending on filesystem enumeration order, surfacing a different fatal/WARN message than Go. Sorted directory entries with the existing UTF-8 byte-order comparator before recursing. Added a regression test using a fake FileSystem that deliberately returns entries in reverse order to prove the fix, independent of what the real OS happens to return. --- .../legacy/shared/legacy-sql-files-glob.ts | 13 ++++- .../shared/legacy-sql-files-glob.unit.test.ts | 54 +++++++++++++++++++ 2 files changed, 66 insertions(+), 1 deletion(-) diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts index f0bb8eb1c4..dba95222a6 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts @@ -243,7 +243,18 @@ const legacyWalkSqlFiles = ( `failed to walk matched directory: ${legacyRelativizeErrorMessage(legacyErrorMessage(error), absDir, rel)}`, ), ); - for (const name of names) { + // Go's `fs.WalkDir` visits directory entries in lexical byte order — its own + // `ReadDir` (`os.ReadDir`/`afero.OsFs`) contract guarantees results "sorted by + // filename" before `walkDir` ever iterates them. This FileSystem service's + // `readDirectory` makes no such promise (raw OS enumeration order), so when a + // directory contains MULTIPLE problematic children (e.g. two unreadable + // subdirectories), which one's failure short-circuits this loop — and therefore + // which single error message this walk fails with, or which `WARN:`/fatal text a + // caller (the experimental schema branch, or seed-path globbing) surfaces — + // would otherwise depend on filesystem enumeration order instead of matching + // Go's deterministic choice (review CLI-1958). Sort with the same UTF-8 + // byte-order comparator `globOne`/the top-level match list already use above. + for (const name of [...names].sort(utf8Compare)) { const childRel = joinRelChild(path, rel, name); const childAbs = path.isAbsolute(childRel) ? childRel : path.join(workdir, childRel); // Go's `fs.WalkDir` types each child from the parent's `ReadDir` entry diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts index bbd995b5f0..a40acc4cc5 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts @@ -743,6 +743,60 @@ describe("legacySqlFilesGlob", () => { }, ); + it.effect.skipIf(isRoot)( + "picks the lexically-first failing subdirectory as the fatal error, matching Go's fs.WalkDir sorted-visit order (review CLI-1958)", + () => { + // Go's `fs.WalkDir` visits directory entries in lexical byte order — its + // `ReadDir` (`os.ReadDir`/`afero.OsFs`) contract guarantees results "sorted by + // filename" before `walkDir` ever iterates them, so when a matched directory + // has MULTIPLE unreadable subdirectories, Go deterministically fails on the + // FIRST one lexically ("aaa" before "bbb") and never even attempts the second. + // This module's own `readDirectory` makes no such ordering promise, so this + // test provides a fake `FileSystem` whose `readDirectory` deliberately returns + // "schemas"'s children in REVERSE order ("bbb" before "aaa") — the opposite of + // Go's guaranteed order — to prove the walk sorts them back (`utf8Compare`) + // before iterating, rather than trusting raw (here: adversarial) enumeration + // order. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-walk-order-")); + const schemasDir = join(dir, "schemas"); + const aaaDir = join(schemasDir, "aaa"); + const bbbDir = join(schemasDir, "bbb"); + mkdirSync(aaaDir, { recursive: true }); + mkdirSync(bbbDir, { recursive: true }); + chmodSync(aaaDir, 0o000); + chmodSync(bbbDir, 0o000); + return Effect.gen(function* () { + const realFs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const reorderedFs: FileSystem.FileSystem = { + ...realFs, + readDirectory: (p, opts) => + realFs + .readDirectory(p, opts) + .pipe( + Effect.map((names) => (p === schemasDir ? [...names].sort().reverse() : names)), + ), + }; + const result = yield* legacySqlFilesGlob(reorderedFs, path, ["schemas"], dir); + expect(result.files).toEqual([]); + expect(result.warnings).toHaveLength(1); + // Go descends into "aaa" first (lexical order), fails reading it, and stops — + // "bbb" is never even attempted. + expect(result.warnings[0]).toContain("schemas/aaa"); + expect(result.warnings[0]).not.toContain("schemas/bbb"); + }).pipe( + Effect.provide(BunServices.layer), + Effect.ensuring( + Effect.sync(() => { + chmodSync(aaaDir, 0o755); + chmodSync(bbbDir, 0o755); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + it.effect( "sorts direct wildcard matches by UTF-8 byte order, not UTF-16 code units (Go sort.Strings parity)", () => { From ed712c6e6395fe44ddcd80e50b6c5216f6fe162c Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Thu, 6 Aug 2026 06:03:10 +0100 Subject: [PATCH 43/47] fix(cli): scope project-env registry overrides around db reset's catalog export (review: CLI-1958) db push scopes legacyApplyProjectEnv(projectEnv) around its whole run so a SUPABASE_INTERNAL_IMAGE_REGISTRY/PGDELTA_NPM_REGISTRY set only in supabase/.env reaches the pg-delta edge-runtime helpers, which read process.env directly. db reset loaded the same projectEnv but never applied it, so the same override was silently ignored for its post-reset migrations-catalog export, falling back to the default registries. --- .../legacy/commands/db/reset/SIDE_EFFECTS.md | 2 + .../legacy/commands/db/reset/reset.handler.ts | 14 ++++++ .../db/reset/reset.integration.test.ts | 50 ++++++++++++++++++- 3 files changed, 65 insertions(+), 1 deletion(-) diff --git a/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md b/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md index fde2825a5b..794fd53e46 100644 --- a/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md +++ b/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md @@ -93,6 +93,8 @@ races a restarting gateway. | `SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS` | overrides `[db.migrations].schema_paths` (viper `AutomaticEnv`, beats the config-file value) for the schema-files apply branch | no (no dedicated flag — config-file-only otherwise) | | `SUPABASE_PROJECT_ID` | overrides the local container id (`utils.DbId`) | no | | `SUPABASE_EXPERIMENTAL_PG_DELTA` | enables the post-reset migrations-catalog cache (see Files Written) when `[experimental.pgdelta].enabled` is unset — distinct from `SUPABASE_EXPERIMENTAL_PGDELTA_ENABLED` above, which switches the reset's own apply branch instead | no (project `.env` or shell) | +| `SUPABASE_INTERNAL_IMAGE_REGISTRY` | overrides the pg-delta edge-runtime image registry for the migrations-catalog cache export (scoped for the whole run via `legacyApplyProjectEnv`, matching `db push`) | no (project `.env` or shell) | +| `PGDELTA_NPM_REGISTRY` | overrides the pg-delta edge-runtime npm registry (`.npmrc` + `NPM_CONFIG_REGISTRY` forward) for the migrations-catalog cache export (scoped for the whole run via `legacyApplyProjectEnv`, matching `db push`) | no (project `.env` or shell) | ## Exit Codes diff --git a/apps/cli/src/legacy/commands/db/reset/reset.handler.ts b/apps/cli/src/legacy/commands/db/reset/reset.handler.ts index 4fa784f607..9d8641d171 100644 --- a/apps/cli/src/legacy/commands/db/reset/reset.handler.ts +++ b/apps/cli/src/legacy/commands/db/reset/reset.handler.ts @@ -15,6 +15,7 @@ import { LegacyProjectRefResolver } from "../../../config/legacy-project-ref.ser import { redactLegacyConnectionString } from "../../../shared/legacy-db-config.parse.ts"; import { LegacyDbConfigResolver } from "../../../shared/legacy-db-config.service.ts"; import { + legacyApplyProjectEnv, legacyCheckDbToml, legacyLoadProjectEnv, legacyResolveSeedSqlPath, @@ -107,6 +108,16 @@ export const legacyDbReset = Effect.fn("legacy.db.reset")(function* (flags: Lega let linkedRefForCache: string | undefined; const body = Effect.gen(function* () { + // Go's `loadNestedEnv` (`os.Setenv`) makes every project-`.env` key visible to the + // WHOLE reset run, not just the flag-gate reads above — in particular + // `legacyGetRegistryImageUrl` / `legacyPgDeltaNpmRegistryOption` read + // `SUPABASE_INTERNAL_IMAGE_REGISTRY` / `PGDELTA_NPM_REGISTRY` straight from + // `process.env` for the pg-delta catalog export below (review CLI-1958). `db push` + // (`push.handler.ts`) scopes this the same way, as the first statement of its own + // `body` — mirror that exactly so a private/air-gapped registry configured only in + // `supabase/.env` reaches the catalog export instead of silently falling back to the + // default registries. + yield* legacyApplyProjectEnv(projectEnv); const target = resolveLegacyDbTargetFlags(cliArgs.args); // cobra MarkFlagsMutuallyExclusive("db-url", "linked", "local"). if (target.setFlags.length > 1) { @@ -474,5 +485,8 @@ export const legacyDbReset = Effect.fn("legacy.db.reset")(function* (flags: Lega ), ), Effect.ensuring(telemetryState.flush), + // Closes the `Scope` `legacyApplyProjectEnv` (above) acquires its `process.env` + // reverts against — mirrors `push.handler.ts`'s own `body.pipe(..., Effect.scoped)`. + Effect.scoped, ); }); diff --git a/apps/cli/src/legacy/commands/db/reset/reset.integration.test.ts b/apps/cli/src/legacy/commands/db/reset/reset.integration.test.ts index 20241cb722..885c1b96f0 100644 --- a/apps/cli/src/legacy/commands/db/reset/reset.integration.test.ts +++ b/apps/cli/src/legacy/commands/db/reset/reset.integration.test.ts @@ -293,9 +293,11 @@ function setup( }); const edgeRunCalls: Array = []; + const registryEnvAtRunTime: Array = []; const edge = Layer.succeed(LegacyEdgeRuntimeScript, { run: (runOpts: LegacyEdgeRuntimeRunOpts) => { edgeRunCalls.push(runOpts); + registryEnvAtRunTime.push(process.env["SUPABASE_INTERNAL_IMAGE_REGISTRY"]); if (opts.catalogExportFailWith !== undefined) { return Effect.fail( new LegacyEdgeRuntimeScriptError({ message: opts.catalogExportFailWith }), @@ -345,7 +347,17 @@ function setup( telemetry.layer, linkedCache.layer, ); - return { layer, out, conn, seam, telemetry, linkedCache, resolver, edgeRunCalls }; + return { + layer, + out, + conn, + seam, + telemetry, + linkedCache, + resolver, + edgeRunCalls, + registryEnvAtRunTime, + }; } const migrationFile = (version: string, body = "create table t ();") => ({ @@ -809,6 +821,42 @@ describe("legacy db reset", () => { }, ); + it.live( + "resolves the pg-delta cache export image via SUPABASE_INTERNAL_IMAGE_REGISTRY from supabase/.env", + () => { + // Go's `loadNestedEnv` (`os.Setenv`) makes a `supabase/.env`-only + // `SUPABASE_INTERNAL_IMAGE_REGISTRY` visible to the WHOLE reset run, including + // the pg-delta catalog export the reset handler triggers after a successful + // remote reset (review CLI-1958 round 18) — mirroring `db push`'s own + // `legacyApplyProjectEnv(projectEnv)` scoping (same-named test in + // `push.integration.test.ts`). Without that scoping, this reads only real + // `process.env` and falls back to the default registry instead. + const prev = process.env["SUPABASE_INTERNAL_IMAGE_REGISTRY"]; + delete process.env["SUPABASE_INTERNAL_IMAGE_REGISTRY"]; + const { layer, registryEnvAtRunTime } = setup(tmp.current, { + toml: 'project_id = "test"\n[experimental.pgdelta]\nenabled = true\n', + files: { + ...migrationFile("20240101000000"), + "supabase/.env": "SUPABASE_INTERNAL_IMAGE_REGISTRY=my-mirror.example.com\n", + }, + confirm: [true], + }); + return Effect.gen(function* () { + yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe(Effect.provide(layer)); + expect(registryEnvAtRunTime).toEqual(["my-mirror.example.com"]); + // The finalizer reverted it — never leaks into the surrounding process. + expect(process.env["SUPABASE_INTERNAL_IMAGE_REGISTRY"]).toBeUndefined(); + }).pipe( + Effect.ensuring( + Effect.sync(() => { + if (prev === undefined) delete process.env["SUPABASE_INTERNAL_IMAGE_REGISTRY"]; + else process.env["SUPABASE_INTERNAL_IMAGE_REGISTRY"] = prev; + }), + ), + ); + }, + ); + it.live("warns without failing the reset when the migrations-catalog cache write fails", () => { const { layer, out } = setup(tmp.current, { toml: 'project_id = "test"\n[experimental.pgdelta]\nenabled = true\n', From 1ac66092de0da21d84deef201b4696eaab1410a2 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Thu, 6 Aug 2026 06:03:39 +0100 Subject: [PATCH 44/47] fix(cli): match Go's scanner-size range rejection and last-token error text (review: CLI-1958) Two follow-ups on the SUPABASE_SCANNER_BUFFER_SIZE parsing work: - Reject a magnitude outside Go's signed int64 range (e.g. "9223372036854775808", one over math.MaxInt64) the same way cast.ToInt does: strconv.ParseInt(s, 0, 0) returns a range error, and cast.ToInt discards ANY parseFn error and returns exactly 0 (falls back to the 256KiB default), not the huge value Number.parseInt would silently round to. Verified against the pinned spf13/cast@v1.10.0 (cast.ToInt("9223372036854775808") -> 0). - Track the last RAW scanned token unconditionally, matching Go's `token = scanner.Text()` (runs on every successful Scan(), before the len(trim) > 0 append gate). A statement that trims to empty right before an oversized one (e.g. a lone ";") must still show in the "After statement N: ..." error text, not a blank token. --- .../legacy/shared/legacy-migration-apply.ts | 34 +++- .../legacy-migration-apply.unit.test.ts | 148 ++++++++++++++++++ 2 files changed, 180 insertions(+), 2 deletions(-) diff --git a/apps/cli/src/legacy/shared/legacy-migration-apply.ts b/apps/cli/src/legacy/shared/legacy-migration-apply.ts index 28f31df390..35c2479bc1 100644 --- a/apps/cli/src/legacy/shared/legacy-migration-apply.ts +++ b/apps/cli/src/legacy/shared/legacy-migration-apply.ts @@ -195,6 +195,20 @@ const GO_DEFAULT_MAX_SCANNER_CAPACITY = 256 * 1024; * leading `"0"` from the `"x"` — not a real prefix, so it's parsed as legacy * octal digits `"x100000"`) all fail, matching Go exactly. */ +// `strconv.ParseInt(s, 0, 0)`'s bitSize-0 mode requires the result to fit in Go's `int` +// — 64 bits on every platform this CLI ships for (amd64/arm64). A magnitude outside +// this range is a range error (`strconv.ErrRange`), and `cast.ToInt` (`spf13/cast@v1.10.0 +// /number.go:407-414`'s `parseInt[T]`) discards ANY `parseFn` error — range or +// syntax — and returns exactly `0`, not the (possibly huge, saturated-to-max-magnitude) +// value `strconv.ParseInt` itself returns alongside that error. Verified empirically +// against the pinned `spf13/cast@v1.10.0`: `cast.ToInt("9223372036854775808")` (one over +// `math.MaxInt64`) → `0`. `Number.parseInt` has no such range check (it silently rounds +// via IEEE-754 double precision instead), so this port must reject the same magnitudes +// Go does, or it would treat an out-of-range override as an enormous-but-finite limit +// instead of falling back to the 256KiB default like Go (review CLI-1958 round 18). +const GO_MAX_INT64 = 9223372036854775807n; +const GO_MIN_INT64 = -9223372036854775808n; + const parseGoBaseZeroInt = (value: string): number | undefined => { const negative = value.startsWith("-"); const unsigned = negative || value.startsWith("+") ? value.slice(1) : value; @@ -229,7 +243,15 @@ const parseGoBaseZeroInt = (value: string): number | undefined => { ); if (!validPattern.test(digits)) return undefined; - const n = Number.parseInt(digits.replace(/_/g, ""), base); + const cleanDigits = digits.replace(/_/g, ""); + // Exact-magnitude range check via BigInt — `Number.parseInt` below loses precision + // past 2^53 and never errors, so the int64 bound must be checked independently of it. + const bigPrefix = base === 16 ? "0x" : base === 8 ? "0o" : base === 2 ? "0b" : ""; + const magnitude = BigInt(`${bigPrefix}${cleanDigits}`); + const signedMagnitude = negative ? -magnitude : magnitude; + if (signedMagnitude > GO_MAX_INT64 || signedMagnitude < GO_MIN_INT64) return undefined; + + const n = Number.parseInt(cleanDigits, base); return negative ? -n : n; }; @@ -355,9 +377,17 @@ const checkScannerBufferSize = ( ), ); } + // Go's `token = scanner.Text()` (`pkg/parser/token.go:96`) runs on EVERY successful + // `Scan()` — unconditionally, before the `len(trim) > 0` gate that decides whether to + // `append` to `stats` — so `token` (and therefore the eventual `bufio.ErrTooLong` + // message) reflects the last RAW text scanned even when that statement trimmed to + // empty and was never appended (e.g. a lone `;` immediately before an oversized + // statement reports "After statement N: ;", not a blank token). `emitted` mirrors + // Go's `len(stats)` (append-gated); `lastRaw` must NOT share that gate — verified + // against the Go source directly (review CLI-1958 round 18). + lastRaw = token.raw; if (token.trimmed.length > 0) { emitted += 1; - lastRaw = token.raw; } } return Effect.void; diff --git a/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts b/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts index 74b06eb762..3a5ed138b3 100644 --- a/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts @@ -562,6 +562,57 @@ describe("legacyApplySchemaFiles", () => { }, ); + it.effect( + "reports the last scanned RAW token in the too-long error even when it trimmed to empty (Go scanner.Text() parity, review CLI-1958)", + () => { + // Go's `token = scanner.Text()` (`pkg/parser/token.go:96`) runs on EVERY + // successful `Scan()`, unconditionally — BEFORE the `len(trim) > 0` gate that + // decides whether to append to `stats`. So when a statement trims to empty + // (a lone ";") immediately before an oversized one, Go's `bufio.ErrTooLong` + // message still reports that lone ";" as the last-scanned text, not a blank + // token — `len(stats)` (this port's `emitted`) stays gated on non-empty trim, + // but the reported RAW text must not share that gate. + const dir = mkdtempSync(join(tmpdir(), "legacy-schema-files-scanner-empty-token-")); + mkdirSync(join(dir, "supabase"), { recursive: true }); + const file = join(dir, "supabase", "big.sql"); + writeFileSync(file, `;\nSELECT '${"a".repeat(5000)}';\n`); + const { session } = fakeSession(); + const previous = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = "100b"; + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const exit = yield* legacyApplySchemaFiles( + session, + fs, + path, + dir, + ["supabase/big.sql"], + (message, suggestion) => + new TestError({ message: suggestion ? `${message} (${suggestion})` : message }), + ).pipe(Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const msg = JSON.stringify(exit.cause); + expect(msg).toContain("bufio.Scanner: token too long"); + // 0 statements were EMITTED (the lone ";" trimmed to empty and was never + // appended), but the last scanned RAW token (";") must still show — not a + // blank token, which a trim-gated tracker would wrongly report instead. + expect(msg).toContain("After statement 0: ;"); + } + }).pipe( + Effect.ensuring( + Effect.sync(() => { + if (previous === undefined) delete process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + else process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = previous; + rmSync(dir, { recursive: true, force: true }); + }), + ), + Effect.provide(BunServices.layer), + ); + }, + ); + it.effect( "applies an oversized statement fine when SUPABASE_SCANNER_BUFFER_SIZE is unset (Go's default auto-grows to file size)", () => { @@ -800,6 +851,103 @@ describe("legacyApplySchemaFiles", () => { }, ); + it.effect( + "falls back to Go's hardcoded default cap when SUPABASE_SCANNER_BUFFER_SIZE overflows Go's signed int range (strconv.ParseInt/cast.ToInt range-error parity, review CLI-1958)", + () => { + // "9223372036854775808" is one more than `math.MaxInt64`. Go's + // `strconv.ParseInt(s, 0, 0)` rejects it with a range error, and `cast.ToInt` + // (`spf13/cast@v1.10.0/number.go:407-414`) discards ANY `parseFn` error — + // range or syntax — returning exactly `0`, never the huge (if imprecise) + // magnitude `Number.parseInt` would otherwise accept. `viper.IsSet` is still + // true, so this falls back to the 256KiB hardcoded default, same as a + // genuinely unparseable value ("5M" above) — NOT to "no limit". Verified + // empirically against the pinned `spf13/cast@v1.10.0` + // (`cast.ToInt("9223372036854775808")` → `0`). + const dir = mkdtempSync(join(tmpdir(), "legacy-schema-files-scanner-int64-overflow-")); + mkdirSync(join(dir, "supabase"), { recursive: true }); + const file = join(dir, "supabase", "big.sql"); + writeFileSync(file, `SELECT 1;\nSELECT '${"a".repeat(300_000)}';\n`); + const { session } = fakeSession(); + const previous = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = "9223372036854775808"; + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const exit = yield* legacyApplySchemaFiles( + session, + fs, + path, + dir, + ["supabase/big.sql"], + (message, suggestion) => + new TestError({ message: suggestion ? `${message} (${suggestion})` : message }), + ).pipe(Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const msg = JSON.stringify(exit.cause); + expect(msg).toContain("bufio.Scanner: token too long"); + // 256KiB (Go's hardcoded default), not "no limit" — a treat-as-unbounded + // bug would let this 300_000-byte statement apply successfully instead. + expect(msg).toContain( + "Try setting SUPABASE_SCANNER_BUFFER_SIZE=5MB (current size is 256KB)", + ); + } + }).pipe( + Effect.ensuring( + Effect.sync(() => { + if (previous === undefined) delete process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + else process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = previous; + rmSync(dir, { recursive: true, force: true }); + }), + ), + Effect.provide(BunServices.layer), + ); + }, + ); + + it.effect( + "still accepts the exact int64 boundary magnitudes for SUPABASE_SCANNER_BUFFER_SIZE (Go strconv.ParseInt range-boundary parity, review CLI-1958)", + () => { + // `math.MaxInt64` itself ("9223372036854775807", one less than the overflow + // test above) is NOT a range error in Go — only magnitudes strictly beyond it + // are. A range check that's off-by-one in the strict direction would wrongly + // reject this legitimate (if enormous) configured size and fall back to the + // 256KiB default instead of the requested cap. + const dir = mkdtempSync(join(tmpdir(), "legacy-schema-files-scanner-int64-boundary-")); + mkdirSync(join(dir, "supabase"), { recursive: true }); + const file = join(dir, "supabase", "big.sql"); + writeFileSync(file, `SELECT 1;\nSELECT '${"a".repeat(5116)}';\n`); + const { session } = fakeSession(); + const previous = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = "9223372036854775807"; + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const exit = yield* legacyApplySchemaFiles( + session, + fs, + path, + dir, + ["supabase/big.sql"], + (message, suggestion) => + new TestError({ message: suggestion ? `${message} (${suggestion})` : message }), + ).pipe(Effect.exit); + // The 5116-byte statement fits comfortably under the (enormous) configured + // limit, so this must succeed, not fall back to the 256KiB default. + expect(Exit.isSuccess(exit)).toBe(true); + }).pipe( + Effect.ensuring( + Effect.sync(() => { + if (previous === undefined) delete process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + else process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = previous; + rmSync(dir, { recursive: true, force: true }); + }), + ), + Effect.provide(BunServices.layer), + ); + }, + ); + it.effect( "rejects an oversized statement when SUPABASE_SCANNER_BUFFER_SIZE is set only in the project env (Go loadNestedEnv parity)", () => { From da39dd06607e08d8e4b63a546a66a9c6f8f306b7 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Thu, 6 Aug 2026 06:43:45 +0100 Subject: [PATCH 45/47] fix(cli): reject bare TOML datetime schema_paths/sql_paths instead of treating as empty (review: CLI-1958) smol-toml parses every TOML datetime variant to a TomlDate (a Date subclass) that stores its value internally, not as an enumerable own property, so it satisfies the same zero-enumerable-key test this reader used to detect an empty inline table (`schema_paths = {}`). A bare datetime therefore silently resolved to an empty pattern list instead of failing config load. Verified empirically against the real apps/cli-go config.Load: Go's mapstructure decoder reports a bare datetime as unconvertible and aborts the whole load, with a distinct Go type per TOML datetime variant (time.Time for offset date-time, toml.LocalDateTime/LocalDate/LocalTime for the three zone-less "local" variants). Exclude TomlDate from the empty-table special case and teach legacyGoUnconvertibleType to name the correct per-variant Go type, matching Go's error text exactly whether the datetime is a top-level scalar or an array element. --- .../shared/legacy-db-config.toml-read.ts | 47 ++++++++-- .../legacy-db-config.toml-read.unit.test.ts | 87 +++++++++++++++++++ 2 files changed, 128 insertions(+), 6 deletions(-) diff --git a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts index 80709315d9..d8f840d1a2 100644 --- a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts +++ b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts @@ -1953,12 +1953,33 @@ const readDbTomlCore = Effect.fnUntraced(function* ( // migrations.schema_paths[0]' expected type 'string', got unconvertible type // '[]interface {}'` / `'map[string]interface {}'` respectively — never silently // dropping the element and continuing with an empty/partial glob list. + // + // A bare TOML datetime (e.g. `schema_paths = 1979-05-27T07:32:00Z`) hits this same + // `UnconvertibleTypeError` path in Go, but as a DIFFERENT Go type per TOML datetime + // variant — `BurntSushi/toml` decodes an offset date-time to stdlib `time.Time` and + // each of the 3 zone-less "local" variants to its own `toml.Local*` wrapper type. + // Verified empirically against the real `apps/cli-go` `config.Load` (review + // CLI-1958): `schema_paths = 1979-05-27T07:32:00Z` → `unconvertible type + // 'time.Time'`; `= 1979-05-27T07:32:00` (no zone) → `'toml.LocalDateTime'`; + // `= 1979-05-27` → `'toml.LocalDate'`; `= 07:32:00` → `'toml.LocalTime'` — same four + // messages whether the datetime is this top-level scalar or an array element. + // `smol-toml` parses every TOML datetime to a `TomlDate` (a `Date` subclass, so + // `typeof`/`Array.isArray` alone can't tell it apart from an inline table) exposing + // exactly the `isDate`/`isTime`/`isDateTime`/`isLocal` discriminators needed to + // reproduce Go's per-variant type name. + const legacyGoTomlDateType = (value: SmolToml.TomlDate): string => { + if (value.isDate()) return "toml.LocalDate"; + if (value.isTime()) return "toml.LocalTime"; + return value.isLocal() ? "toml.LocalDateTime" : "time.Time"; + }; const legacyGoUnconvertibleType = (value: unknown): string | undefined => - Array.isArray(value) - ? "[]interface {}" - : typeof value === "object" && value !== null - ? "map[string]interface {}" - : undefined; + value instanceof SmolToml.TomlDate + ? legacyGoTomlDateType(value) + : Array.isArray(value) + ? "[]interface {}" + : typeof value === "object" && value !== null + ? "map[string]interface {}" + : undefined; // Pure — returns the mapstructure-style issue strings for a real `Glob` array's // unconvertible elements WITHOUT failing. Go's `UnmarshalExact` decodes the WHOLE // config in a SINGLE mapstructure pass: `decodeStructFromMap`'s per-field loop @@ -2014,6 +2035,15 @@ const readDbTomlCore = Effect.fnUntraced(function* ( // 'string', got unconvertible type 'map[string]interface {}'`. Never // called with `undefined` — an absent key has its own Go-matching default // per caller below, so callers guard that case before reaching here. + // + // The zero-length-map special case must NOT match a `TomlDate` (e.g. `schema_paths = + // 1979-05-27T07:32:00Z`): a `TomlDate` stores its value internally, not as an + // enumerable own property, so `Object.keys(tomlDate).length === 0` is ALSO true for + // it — but Go does not treat a bare datetime as an empty map; mapstructure reports it + // unconvertible and aborts the whole load (see `legacyGoUnconvertibleType` above). + // Without this exclusion, a `TomlDate` would silently resolve to `[]` here instead of + // falling through to the unconvertible-type issue below, turning Go's hard config-load + // failure into a silently-empty schema/seed path list (review CLI-1958). // Pure — the TOP-LEVEL scalar fallback (see doc comment above), returning either the // one resolved pattern or the one issue it would raise, WITHOUT failing (same reason // as `legacyGlobArrayIssues`: the caller combines issues across both `Glob` fields @@ -2022,7 +2052,12 @@ const readDbTomlCore = Effect.fnUntraced(function* ( keyPath: string, value: unknown, ): { readonly resolved: ReadonlyArray; readonly issues: ReadonlyArray } => { - if (typeof value === "object" && value !== null && Object.keys(value).length === 0) { + if ( + typeof value === "object" && + value !== null && + !(value instanceof SmolToml.TomlDate) && + Object.keys(value).length === 0 + ) { return { resolved: [], issues: [] }; } const coerced = legacyWeakCoerceGlobEntry(value); diff --git a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts index 3fd861d780..cfbb8e7e0b 100644 --- a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts @@ -494,6 +494,93 @@ describe("legacyReadDbToml", () => { }, ); + it.effect.each([ + { name: "offset date-time", literal: "1979-05-27T07:32:00Z", goType: "time.Time" }, + { name: "local date-time", literal: "1979-05-27T07:32:00", goType: "toml.LocalDateTime" }, + { name: "local date", literal: "1979-05-27", goType: "toml.LocalDate" }, + { name: "local time", literal: "07:32:00", goType: "toml.LocalTime" }, + ])( + "aborts the whole config load on a TOP-LEVEL bare $name db.migrations.schema_paths instead of silently treating it as empty (Go mapstructure UnconvertibleTypeError, review CLI-1958)", + ({ literal, goType }) => { + // `smol-toml` parses every TOML datetime variant to a `TomlDate` (a `Date` + // subclass) that stores its value internally, not as an enumerable own + // property — so `Object.keys(tomlDate).length === 0`, same as a genuine empty + // inline table (`schema_paths = {}`, tested above). Without excluding `TomlDate` + // from that zero-length-map special case, this would silently resolve to `[]` + // instead of aborting. Verified empirically against the real `apps/cli-go` + // `config.Load`: a bare datetime literal here fails with exactly this message, + // never resolving to an empty/partial glob list — one distinct Go type per TOML + // datetime variant (`time.Time` for the offset form, `toml.Local*` wrappers for + // the 3 zone-less "local" forms). + const dir = withConfig(["[db.migrations]", `schema_paths = ${literal}`, ""].join("\n")); + return read(dir).pipe( + Effect.exit, + Effect.tap((exit) => + Effect.sync(() => { + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + expect(JSON.stringify(exit.cause)).toContain( + `'db.migrations.schema_paths[0]' expected type 'string', got unconvertible type '${goType}'`, + ); + } + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "aborts the whole config load on a bare datetime db.migrations.schema_paths ARRAY element (Go mapstructure UnconvertibleTypeError, review CLI-1958)", + () => { + // Same `TomlDate`-vs-generic-object collision as the top-level scalar case above, + // but reached through the real-array branch (`legacyGoUnconvertibleType`) instead + // of the scalar fallback. Verified empirically against `apps/cli-go`: + // `schema_paths = ["schemas/*.sql", 1979-05-27T07:32:00Z]` fails config load with + // exactly this message — the valid glob entry never masks the datetime's failure. + const dir = withConfig( + ["[db.migrations]", 'schema_paths = ["schemas/*.sql", 1979-05-27T07:32:00Z]', ""].join( + "\n", + ), + ); + return read(dir).pipe( + Effect.exit, + Effect.tap((exit) => + Effect.sync(() => { + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + expect(JSON.stringify(exit.cause)).toContain( + "'db.migrations.schema_paths[1]' expected type 'string', got unconvertible type 'time.Time'", + ); + } + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "aborts the whole config load on a TOP-LEVEL bare datetime db.seed.sql_paths (same UnmarshalExact call as schema_paths, review CLI-1958)", + () => { + const dir = withConfig(["[db.seed]", "sql_paths = 1979-05-27T07:32:00Z", ""].join("\n")); + return read(dir).pipe( + Effect.exit, + Effect.tap((exit) => + Effect.sync(() => { + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + expect(JSON.stringify(exit.cause)).toContain( + "'db.seed.sql_paths[0]' expected type 'string', got unconvertible type 'time.Time'", + ); + } + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + it.effect( "aborts the whole config load on a TOP-LEVEL table db.migrations.schema_paths (Go mapstructure UnconvertibleTypeError, synthetic index 0)", () => { From 6d7e1f8ab16ac259690edc28fb11900b6806609b Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Thu, 6 Aug 2026 11:28:38 +0100 Subject: [PATCH 46/47] chore(cli): fix markdown table formatting after develop merge --- apps/cli/docs/go-cli-porting-status.md | 89 +++++++++++++------------- 1 file changed, 45 insertions(+), 44 deletions(-) diff --git a/apps/cli/docs/go-cli-porting-status.md b/apps/cli/docs/go-cli-porting-status.md index 39e14ac3fd..fb6404b5eb 100644 --- a/apps/cli/docs/go-cli-porting-status.md +++ b/apps/cli/docs/go-cli-porting-status.md @@ -80,51 +80,52 @@ These commands exist in the TS CLI today but have no direct top-level equivalent ## Database -| Old command | TS status | TS command path or `missing` | Missing flags/params | Extra TS flags/params | Notes | -| --------------------------------- | --------- | -------------------------------------------------- | -------------------- | --------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `db diff` | `ported` | `legacy/commands/db/diff/` | `n/a` | `n/a` | Native TS port. Native pg-delta / migra diff via edge-runtime against a Go-seam-provisioned live shadow (`db __shadow`); `--use-pgadmin` / `--use-pg-schema` delegate to the Go binary. `--use-pg-schema` is deprecated (CLI-1960: TS-only stderr warning + `--help` note) in favor of the pg-delta engine or the default migra engine — it wraps the in-process `stripe/pg-schema-diff` Go library, which has no TS/container equivalent, so it is a documented keep-in-Go exception, not a pending port. It will be the sole remaining Go delegation once `--use-pgadmin`, the `db __shadow`/`db __db-bootstrap` seams, and the other in-flight M9 issues are done. | -| `db dump` | `ported` | `legacy/commands/db/dump/` | `n/a` | `n/a` | Native TS port. Streams `pg_dump`/`pg_dumpall` via a Docker container (`LegacyDockerRun`); schema/data/role modes, `--dry-run` script print, IPv4 transaction-pooler fallback. | -| `db lint` | `ported` | `legacy/commands/db/lint/` | `n/a` | `n/a` | Native TS port. Runs `plpgsql_check` in a rolled-back transaction via LegacyDbConnection; emits Go-parity pretty JSON. | -| `db pull` | `ported` | `legacy/commands/db/pull/` | `n/a` | `n/a` | Native TS port. Native pg-delta / migra migration + `--declarative` pg-delta export; reconciles `schema_migrations`. The initial-migra pull dumps the remote schema natively (`pg_dump`) then appends the migra diff. Only `--experimental` (structured dump) still delegates to Go, deprecated in favor of `--declarative` (CLI-1957) — it needs a TS PostgreSQL DDL parser for Go's `format.WriteStructuredSchemas` that has no equivalent in this repo, and `--declarative` already delivers the same per-object schema split via pg-delta catalog introspection. | -| `db push` | `ported` | `legacy/commands/db/push/` | `n/a` | `n/a` | Native TS port. Connects local/linked/`--db-url`; pushes pending migrations, `--include-seed` seeds (`seed_files` hash tracking), `--include-roles`, `[db.vault]` secrets including decrypted `encrypted:` values; `--dry-run`; best-effort pg-delta migrations-catalog cache (warning-only on failure). Pipeline-incompatible statements (`CREATE INDEX CONCURRENTLY`, `VACUUM`, …) run standalone outside the batch transaction — from the closed Go PR supabase/cli#5156, also ported into `apps/cli-go` (CLI-1989 ruling). | +| Old command | TS status | TS command path or `missing` | Missing flags/params | Extra TS flags/params | Notes | +| --------------------------------- | --------- | -------------------------------------------------- | -------------------- | --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `db diff` | `ported` | `legacy/commands/db/diff/` | `n/a` | `n/a` | Native TS port. Native pg-delta / migra diff via edge-runtime against a Go-seam-provisioned live shadow (`db __shadow`); `--use-pgadmin` / `--use-pg-schema` delegate to the Go binary. `--use-pg-schema` is deprecated (CLI-1960: TS-only stderr warning + `--help` note) in favor of the pg-delta engine or the default migra engine — it wraps the in-process `stripe/pg-schema-diff` Go library, which has no TS/container equivalent, so it is a documented keep-in-Go exception, not a pending port. It will be the sole remaining Go delegation once `--use-pgadmin`, the `db __shadow`/`db __db-bootstrap` seams, and the other in-flight M9 issues are done. | +| `db dump` | `ported` | `legacy/commands/db/dump/` | `n/a` | `n/a` | Native TS port. Streams `pg_dump`/`pg_dumpall` via a Docker container (`LegacyDockerRun`); schema/data/role modes, `--dry-run` script print, IPv4 transaction-pooler fallback. | +| `db lint` | `ported` | `legacy/commands/db/lint/` | `n/a` | `n/a` | Native TS port. Runs `plpgsql_check` in a rolled-back transaction via LegacyDbConnection; emits Go-parity pretty JSON. | +| `db pull` | `ported` | `legacy/commands/db/pull/` | `n/a` | `n/a` | Native TS port. Native pg-delta / migra migration + `--declarative` pg-delta export; reconciles `schema_migrations`. The initial-migra pull dumps the remote schema natively (`pg_dump`) then appends the migra diff. Only `--experimental` (structured dump) still delegates to Go, deprecated in favor of `--declarative` (CLI-1957) — it needs a TS PostgreSQL DDL parser for Go's `format.WriteStructuredSchemas` that has no equivalent in this repo, and `--declarative` already delivers the same per-object schema split via pg-delta catalog introspection. | +| `db push` | `ported` | `legacy/commands/db/push/` | `n/a` | `n/a` | Native TS port. Connects local/linked/`--db-url`; pushes pending migrations, `--include-seed` seeds (`seed_files` hash tracking), `--include-roles`, `[db.vault]` secrets including decrypted `encrypted:` values; `--dry-run`; best-effort pg-delta migrations-catalog cache (warning-only on failure). Pipeline-incompatible statements (`CREATE INDEX CONCURRENTLY`, `VACUUM`, …) run standalone outside the batch transaction — from the closed Go PR supabase/cli#5156, also ported into `apps/cli-go` (CLI-1989 ruling). | | `db reset` | `ported` | `legacy/commands/db/reset/` | `n/a` | `n/a` | Remote path native (drop user schemas, vault upsert, MigrateAndSeed — including the `--experimental` declarative `[db.migrations].schema_paths` apply branch, CLI-1958 — `--version`/`--last`, `--sql-paths` seed override). Local path native: running check, recreate + migrate + seed via the hidden Go `db __db-bootstrap` seam (that seam still forwards `--experimental` for its own schema-files branch, CLI-1955 scope), storage-gated bucket seeding (reuses `seed buckets`), git-branch `Finished…` line. Pipeline-incompatible statements run standalone outside the batch transaction, same as `db push` (closed Go PR supabase/cli#5156, CLI-1989 ruling). | -| `db start` | `ported` | `legacy/commands/db/start/` | `n/a` | `n/a` | Native TS port. Validates config, checks "already running" (prints Go's line), else delegates the container bootstrap (create + health + initial schema/roles/migrations/seed + `_current_branch`) to the hidden Go `db __db-bootstrap --mode start` seam. No status table / `cli_stack_started` (those are `supabase start`). `--from-backup` supported. | -| `inspect report` | `ported` | `legacy/commands/inspect/report/` | `n/a` | `n/a` | Native TS port. Runs every inspect query via server-side `COPY ... CSV`, writes 14 CSVs under `//`, then renders a Go-parity Glamour rules summary (bounded csvq-subset evaluator; custom `[experimental.inspect.rules]` supported). | -| `inspect db db-stats` | `ported` | `legacy/commands/inspect/db/db-stats/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | -| `inspect db replication-slots` | `ported` | `legacy/commands/inspect/db/replication-slots/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | -| `inspect db locks` | `ported` | `legacy/commands/inspect/db/locks/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | -| `inspect db blocking` | `ported` | `legacy/commands/inspect/db/blocking/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | -| `inspect db outliers` | `ported` | `legacy/commands/inspect/db/outliers/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | -| `inspect db calls` | `ported` | `legacy/commands/inspect/db/calls/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | -| `inspect db index-stats` | `ported` | `legacy/commands/inspect/db/index-stats/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | -| `inspect db long-running-queries` | `ported` | `legacy/commands/inspect/db/long-running-queries/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | -| `inspect db bloat` | `ported` | `legacy/commands/inspect/db/bloat/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | -| `inspect db role-stats` | `ported` | `legacy/commands/inspect/db/role-stats/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | -| `inspect db vacuum-stats` | `ported` | `legacy/commands/inspect/db/vacuum-stats/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | -| `inspect db table-stats` | `ported` | `legacy/commands/inspect/db/table-stats/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | -| `inspect db traffic-profile` | `ported` | `legacy/commands/inspect/db/traffic-profile/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | -| `inspect db cache-hit` | `ported` | `legacy/commands/inspect/db/cache-hit/` | `n/a` | `n/a` | Native TS port. Deprecated (use db-stats); routes to the active query. | -| `inspect db index-usage` | `ported` | `legacy/commands/inspect/db/index-usage/` | `n/a` | `n/a` | Native TS port. Deprecated (use index-stats); routes to the active query. | -| `inspect db total-index-size` | `ported` | `legacy/commands/inspect/db/total-index-size/` | `n/a` | `n/a` | Native TS port. Deprecated (use index-stats); routes to the active query. | -| `inspect db index-sizes` | `ported` | `legacy/commands/inspect/db/index-sizes/` | `n/a` | `n/a` | Native TS port. Deprecated (use index-stats); routes to the active query. | -| `inspect db table-sizes` | `ported` | `legacy/commands/inspect/db/table-sizes/` | `n/a` | `n/a` | Native TS port. Deprecated (use table-stats); routes to the active query. | -| `inspect db table-index-sizes` | `ported` | `legacy/commands/inspect/db/table-index-sizes/` | `n/a` | `n/a` | Native TS port. Deprecated (use table-stats); routes to the active query. | -| `inspect db total-table-sizes` | `ported` | `legacy/commands/inspect/db/total-table-sizes/` | `n/a` | `n/a` | Native TS port. Deprecated (use table-stats); routes to the active query. | -| `inspect db unused-indexes` | `ported` | `legacy/commands/inspect/db/unused-indexes/` | `n/a` | `n/a` | Native TS port. Deprecated (use index-stats); routes to the active query. | -| `inspect db table-record-counts` | `ported` | `legacy/commands/inspect/db/table-record-counts/` | `n/a` | `n/a` | Native TS port. Deprecated (use table-stats); routes to the active query. | -| `inspect db seq-scans` | `ported` | `legacy/commands/inspect/db/seq-scans/` | `n/a` | `n/a` | Native TS port. Deprecated (use index-stats); routes to the active query. | -| `inspect db role-configs` | `ported` | `legacy/commands/inspect/db/role-configs/` | `n/a` | `n/a` | Native TS port. Deprecated (use role-stats); routes to the active query. | -| `inspect db role-connections` | `ported` | `legacy/commands/inspect/db/role-connections/` | `n/a` | `n/a` | Native TS port. Deprecated (use role-stats); routes to the active query. | -| `migration down` | `ported` | `legacy/commands/migration/down/` | `n/a` | `n/a` | Native TS port. Revert prompt → drop user schemas → vault upsert → migrate&seed to the target version; defaults to `--local`. Skips Go's pgcache catalog write. | -| `migration fetch` | `ported` | `legacy/commands/migration/fetch/` | `n/a` | `n/a` | Native TS port. Reads `schema_migrations` and writes `supabase/migrations/_.sql`; overwrite prompt for a non-empty dir. | -| `migration list` | `ported` | `legacy/commands/migration/list/` | `n/a` | `n/a` | Native TS port. Merges remote `schema_migrations` with local files into a Glamour ASCII table (Local / Remote / Time-UTC columns); defaults to `--linked`. | -| `migration new` | `ported` | `legacy/commands/migration/new/` | `n/a` | `n/a` | Native TS port. Writes `supabase/migrations/_.sql` (mode 0644) from piped stdin; no DB/API. | -| `migration repair` | `ported` | `legacy/commands/migration/repair/` | `n/a` | `n/a` | Native TS port. Transactional create-table + TRUNCATE/UPSERT/DELETE; applied mode reads local files; repair-all prompt; defaults to `--linked`. | -| `migration squash` | `missing` | `missing` | `n/a` | `n/a` | No native TS implementation yet. Wrapped in legacy shell. | -| `migration up` | `ported` | `legacy/commands/migration/up/` | `n/a` | `n/a` | Native TS port. Computes pending migrations, upserts `[db.vault]`, applies each transactionally (pipeline-incompatible statements run standalone — closed Go PR supabase/cli#5156, ported into `apps/cli-go`, CLI-1989 ruling); `--include-all` for out-of-order; defaults to `--local`. Does not seed (matches Go). | -| `seed buckets` | `ported` | `legacy/commands/seed/buckets/` | `n/a` | `n/a` | Native TS port. Local-only (Go's `seed` defines no `--project-ref`, so the ref is always empty): seeds `[storage.buckets]` + `[storage.vector]` against the local Storage service gateway; remote/analytics paths are unreachable and omitted. `--linked`/`--local` accepted for surface parity (both seed local). Vector graceful-skip WARNINGs ported. | -| `test db` | `ported` | `legacy/commands/test/db/` | `n/a` | `n/a` | Native TS port. `--db-url`/`--local`/`--linked` + variadic paths; runs `supabase/pg_prove:3.36` via `docker run`; pgTAP enable/disable via `@effect/sql-pg`. `--network-id` override and `[images]` config override not modeled (documented divergences). | -| `test new` | `ported` | `legacy/commands/test/new/` | `n/a` | `n/a` | Native TS port. Writes `supabase/tests/_test.sql` from the embedded pgtap template; `--template` (pgtap). | +| `db start` | `ported` | `legacy/commands/db/start/` | `n/a` | `n/a` | Native TS port. Validates config, checks "already running" (prints Go's line), else delegates the container bootstrap (create + health + initial schema/roles/migrations/seed + `_current_branch`) to the hidden Go `db __db-bootstrap --mode start` seam. No status table / `cli_stack_started` (those are `supabase start`). `--from-backup` supported. | +| `inspect report` | `ported` | `legacy/commands/inspect/report/` | `n/a` | `n/a` | Native TS port. Runs every inspect query via server-side `COPY ... CSV`, writes 14 CSVs under `//`, then renders a Go-parity Glamour rules summary (bounded csvq-subset evaluator; custom `[experimental.inspect.rules]` supported). | +| `inspect db db-stats` | `ported` | `legacy/commands/inspect/db/db-stats/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | +| `inspect db replication-slots` | `ported` | `legacy/commands/inspect/db/replication-slots/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | +| `inspect db locks` | `ported` | `legacy/commands/inspect/db/locks/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | +| `inspect db blocking` | `ported` | `legacy/commands/inspect/db/blocking/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | +| `inspect db outliers` | `ported` | `legacy/commands/inspect/db/outliers/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | +| `inspect db calls` | `ported` | `legacy/commands/inspect/db/calls/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | +| `inspect db index-stats` | `ported` | `legacy/commands/inspect/db/index-stats/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | +| `inspect db long-running-queries` | `ported` | `legacy/commands/inspect/db/long-running-queries/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | +| `inspect db bloat` | `ported` | `legacy/commands/inspect/db/bloat/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | +| `inspect db role-stats` | `ported` | `legacy/commands/inspect/db/role-stats/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | +| `inspect db vacuum-stats` | `ported` | `legacy/commands/inspect/db/vacuum-stats/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | +| `inspect db table-stats` | `ported` | `legacy/commands/inspect/db/table-stats/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | +| `inspect db traffic-profile` | `ported` | `legacy/commands/inspect/db/traffic-profile/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | +| `inspect db cache-hit` | `ported` | `legacy/commands/inspect/db/cache-hit/` | `n/a` | `n/a` | Native TS port. Deprecated (use db-stats); routes to the active query. | +| `inspect db index-usage` | `ported` | `legacy/commands/inspect/db/index-usage/` | `n/a` | `n/a` | Native TS port. Deprecated (use index-stats); routes to the active query. | +| `inspect db total-index-size` | `ported` | `legacy/commands/inspect/db/total-index-size/` | `n/a` | `n/a` | Native TS port. Deprecated (use index-stats); routes to the active query. | +| `inspect db index-sizes` | `ported` | `legacy/commands/inspect/db/index-sizes/` | `n/a` | `n/a` | Native TS port. Deprecated (use index-stats); routes to the active query. | +| `inspect db table-sizes` | `ported` | `legacy/commands/inspect/db/table-sizes/` | `n/a` | `n/a` | Native TS port. Deprecated (use table-stats); routes to the active query. | +| `inspect db table-index-sizes` | `ported` | `legacy/commands/inspect/db/table-index-sizes/` | `n/a` | `n/a` | Native TS port. Deprecated (use table-stats); routes to the active query. | +| `inspect db total-table-sizes` | `ported` | `legacy/commands/inspect/db/total-table-sizes/` | `n/a` | `n/a` | Native TS port. Deprecated (use table-stats); routes to the active query. | +| `inspect db unused-indexes` | `ported` | `legacy/commands/inspect/db/unused-indexes/` | `n/a` | `n/a` | Native TS port. Deprecated (use index-stats); routes to the active query. | +| `inspect db table-record-counts` | `ported` | `legacy/commands/inspect/db/table-record-counts/` | `n/a` | `n/a` | Native TS port. Deprecated (use table-stats); routes to the active query. | +| `inspect db seq-scans` | `ported` | `legacy/commands/inspect/db/seq-scans/` | `n/a` | `n/a` | Native TS port. Deprecated (use index-stats); routes to the active query. | +| `inspect db role-configs` | `ported` | `legacy/commands/inspect/db/role-configs/` | `n/a` | `n/a` | Native TS port. Deprecated (use role-stats); routes to the active query. | +| `inspect db role-connections` | `ported` | `legacy/commands/inspect/db/role-connections/` | `n/a` | `n/a` | Native TS port. Deprecated (use role-stats); routes to the active query. | +| `migration down` | `ported` | `legacy/commands/migration/down/` | `n/a` | `n/a` | Native TS port. Revert prompt → drop user schemas → vault upsert → migrate&seed to the target version; defaults to `--local`. Skips Go's pgcache catalog write. | +| `migration fetch` | `ported` | `legacy/commands/migration/fetch/` | `n/a` | `n/a` | Native TS port. Reads `schema_migrations` and writes `supabase/migrations/_.sql`; overwrite prompt for a non-empty dir. | +| `migration list` | `ported` | `legacy/commands/migration/list/` | `n/a` | `n/a` | Native TS port. Merges remote `schema_migrations` with local files into a Glamour ASCII table (Local / Remote / Time-UTC columns); defaults to `--linked`. | +| `migration new` | `ported` | `legacy/commands/migration/new/` | `n/a` | `n/a` | Native TS port. Writes `supabase/migrations/_.sql` (mode 0644) from piped stdin; no DB/API. | +| `migration repair` | `ported` | `legacy/commands/migration/repair/` | `n/a` | `n/a` | Native TS port. Transactional create-table + TRUNCATE/UPSERT/DELETE; applied mode reads local files; repair-all prompt; defaults to `--linked`. | +| `migration squash` | `missing` | `missing` | `n/a` | `n/a` | No native TS implementation yet. Wrapped in legacy shell. | +| `migration up` | `ported` | `legacy/commands/migration/up/` | `n/a` | `n/a` | Native TS port. Computes pending migrations, upserts `[db.vault]`, applies each transactionally (pipeline-incompatible statements run standalone — closed Go PR supabase/cli#5156, ported into `apps/cli-go`, CLI-1989 ruling); `--include-all` for out-of-order; defaults to `--local`. Does not seed (matches Go). | +| `seed buckets` | `ported` | `legacy/commands/seed/buckets/` | `n/a` | `n/a` | Native TS port. Local-only (Go's `seed` defines no `--project-ref`, so the ref is always empty): seeds `[storage.buckets]` + `[storage.vector]` against the local Storage service gateway; remote/analytics paths are unreachable and omitted. `--linked`/`--local` accepted for surface parity (both seed local). Vector graceful-skip WARNINGs ported. | +| `test db` | `ported` | `legacy/commands/test/db/` | `n/a` | `n/a` | Native TS port. `--db-url`/`--local`/`--linked` + variadic paths; runs `supabase/pg_prove:3.36` via `docker run`; pgTAP enable/disable via `@effect/sql-pg`. `--network-id` override and `[images]` config override not modeled (documented divergences). | +| `test new` | `ported` | `legacy/commands/test/new/` | `n/a` | `n/a` | Native TS port. Writes `supabase/tests/_test.sql` from the embedded pgtap template; `--template` (pgtap). | + ## Code Generation | Old command | TS status | TS command path or `missing` | Missing flags/params | Extra TS flags/params | Notes | From 9d1eb7a2b473c41acd33c3d7a22b2dcfb63fd309 Mon Sep 17 00:00:00 2001 From: Colum Ferry Date: Thu, 6 Aug 2026 17:40:16 +0100 Subject: [PATCH 47/47] fix(cli): close 3 db reset --experimental Go-parity gaps found in review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - legacy-sql-split.ts: match Go's unicode.IsDigit (decimal digits only, \p{Nd}) for dollar-quote tag/identifier characters instead of \p{N} (all Unicode numbers), which wrongly accepted non-decimal digit runes like superscript-2 and could split schema SQL differently than Go. - legacy-seed-ops.ts / legacy-seed.ts: enforce SUPABASE_SCANNER_BUFFER_SIZE on seed file parsing via the shared checkScannerBufferSize, matching Go's SeedFile.ExecBatchWithCache, which parses through the same parseFile every other file type does. - legacy-pgdelta.cache.ts: listJsonEntries now only swallows a genuinely missing catalog directory and propagates every other read failure (e.g. permission denied), matching Go's ReadDir usage after ensureTempDir — previously every failure was treated as empty, silently defeating catalog retention and cache resolution. --- .../legacy/shared/legacy-migration-apply.ts | 2 +- .../src/legacy/shared/legacy-pgdelta.cache.ts | 28 ++++++++++++--- .../shared/legacy-pgdelta.cache.unit.test.ts | 32 +++++++++++++++-- apps/cli/src/legacy/shared/legacy-seed-ops.ts | 15 +++++--- .../shared/legacy-seed-ops.unit.test.ts | 29 ++++++++++++++++ apps/cli/src/legacy/shared/legacy-seed.ts | 34 ++++++++++++------- .../legacy/shared/legacy-seed.unit.test.ts | 34 ++++++++++++++++++- .../cli/src/legacy/shared/legacy-sql-split.ts | 11 ++++-- .../shared/legacy-sql-split.unit.test.ts | 11 ++++++ 9 files changed, 166 insertions(+), 30 deletions(-) diff --git a/apps/cli/src/legacy/shared/legacy-migration-apply.ts b/apps/cli/src/legacy/shared/legacy-migration-apply.ts index 720cedb03d..dc8aec9b24 100644 --- a/apps/cli/src/legacy/shared/legacy-migration-apply.ts +++ b/apps/cli/src/legacy/shared/legacy-migration-apply.ts @@ -336,7 +336,7 @@ const legacyParseScannerBufferSize = (raw: string): number => { * one. Defaults to `{}` for callers that haven't threaded a project-env map through * (shell-only, same as before this parameter existed). */ -const checkScannerBufferSize = ( +export const checkScannerBufferSize = ( content: string, mapError: (message: string, phase: "read" | "exec") => E, projectEnv: Readonly> = {}, diff --git a/apps/cli/src/legacy/shared/legacy-pgdelta.cache.ts b/apps/cli/src/legacy/shared/legacy-pgdelta.cache.ts index d7c12ae127..d999a0ea7a 100644 --- a/apps/cli/src/legacy/shared/legacy-pgdelta.cache.ts +++ b/apps/cli/src/legacy/shared/legacy-pgdelta.cache.ts @@ -332,12 +332,30 @@ const parseCatalogTimestamp = (name: string): Option.Option => { return Number.isInteger(ts) ? Option.some(ts) : Option.none(); }; +/** + * Mirrors Go's `ensureTempDir` + `ReadDir` pairing (`pgcache/cache.go`, + * `declarative.go`): the temp dir's existence is already guaranteed by the + * `MkdirAll` that runs before every write into it, so Go's `ReadDir` only ever + * needs to tolerate a genuinely missing directory (a cache that was never + * written to) — every OTHER read failure (e.g. permission denied) propagates, + * same as {@link legacyListLocalMigrations} above. Swallowing every failure + * (as an earlier version of this did) let a real read error silently look like + * "no cached catalogs", which both bypasses catalog resolution's cache HIT and + * — for cleanup's caller — bypasses the retention limit indefinitely, since + * the caller's own warning path never fires without a propagated failure. + */ const listJsonEntries = Effect.fnUntraced(function* (fs: FileSystem.FileSystem, tempDir: string) { - const exists = yield* fs.exists(tempDir).pipe(Effect.orElseSucceed(() => false)); - if (!exists) return [] as ReadonlyArray; - return yield* fs - .readDirectory(tempDir) - .pipe(Effect.orElseSucceed(() => [] as ReadonlyArray)); + return yield* fs.readDirectory(tempDir).pipe( + Effect.catchTag("PlatformError", (error) => + error.reason._tag === "NotFound" + ? Effect.succeed([] as ReadonlyArray) + : Effect.fail( + new LegacyMigrationsReadError({ + message: `failed to read directory: ${error.message}`, + }), + ), + ), + ); }); /** diff --git a/apps/cli/src/legacy/shared/legacy-pgdelta.cache.unit.test.ts b/apps/cli/src/legacy/shared/legacy-pgdelta.cache.unit.test.ts index 4d370b18a8..55ccfa61c5 100644 --- a/apps/cli/src/legacy/shared/legacy-pgdelta.cache.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-pgdelta.cache.unit.test.ts @@ -1,10 +1,10 @@ import { createHash } from "node:crypto"; -import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { chmodSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; -import { Effect, FileSystem, Layer, Option, Path } from "effect"; +import { Effect, Exit, FileSystem, Layer, Option, Path } from "effect"; import { Output } from "../../shared/output/output.service.ts"; import { mockOutput } from "../../../tests/helpers/mocks.ts"; @@ -608,4 +608,32 @@ describe("legacyCleanupOldMigrationCatalogs", () => { }), ).pipe(Effect.tap(() => Effect.sync(() => rmSync(dir, { recursive: true, force: true })))); }); + + it.effect( + "propagates a permission-denied directory read instead of treating it as empty (Go ReadDir parity)", + () => { + // Go's CleanupOldMigrationCatalogs only tolerates a genuinely MISSING temp dir + // (ensureTempDir already created it before ReadDir runs) — any other ReadDir + // failure propagates, so a permission-denied listing must fail here too rather + // than silently look like "no cached catalogs" (which would bypass retention + // indefinitely, since the caller's own best-effort warning never fires without + // a propagated failure). + const dir = withTemp(); + const tempDir = join(dir, "pgdelta"); + mkdirSync(tempDir, { recursive: true }); + writeFileSync(join(tempDir, "catalog-local-migrations-h-100.json"), "{}"); + chmodSync(tempDir, 0o000); + return withServices((fs, path) => + legacyCleanupOldMigrationCatalogs(fs, path, tempDir, "local").pipe(Effect.exit), + ).pipe( + Effect.tap((exit) => + Effect.sync(() => { + chmodSync(tempDir, 0o755); + expect(Exit.isFailure(exit)).toBe(true); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); }); diff --git a/apps/cli/src/legacy/shared/legacy-seed-ops.ts b/apps/cli/src/legacy/shared/legacy-seed-ops.ts index d95bdba572..2690f5fcf9 100644 --- a/apps/cli/src/legacy/shared/legacy-seed-ops.ts +++ b/apps/cli/src/legacy/shared/legacy-seed-ops.ts @@ -4,6 +4,7 @@ import { Effect, type FileSystem, type Path } from "effect"; import { Output } from "../../shared/output/output.service.ts"; import type { LegacyDbExecError } from "./legacy-db-connection.errors.ts"; import type { LegacyDbSession } from "./legacy-db-connection.service.ts"; +import { checkScannerBufferSize } from "./legacy-migration-apply.ts"; import { legacyCreateSeedTable } from "./legacy-migration-history.ts"; import { legacySqlFilesGlob } from "./legacy-sql-files-glob.ts"; import { legacySplitAndTrim } from "./legacy-sql-split.ts"; @@ -126,12 +127,16 @@ export const legacySeedData = ( // Go's `ExecBatchWithCache` parses the file (read + `SplitAndTrim`) // UNCONDITIONALLY before the dirty check (`file.go:198-211`), so a dirty seed // that is unreadable or contains malformed SQL still fails and leaves the - // previous hash — only the queueing of statements is gated on `Dirty`. - const lines = legacySplitAndTrim( - yield* fs.readFileString( - path.isAbsolute(seed.path) ? seed.path : path.join(workdir, seed.path), - ), + // previous hash — only the queueing of statements is gated on `Dirty`. Parsing + // includes the same `SUPABASE_SCANNER_BUFFER_SIZE` enforcement every other + // `parseFile` caller gets (`checkScannerBufferSize`'s own doc comment) — Go's + // `SeedFile.ExecBatchWithCache` runs through the identical `parseFile`, so an + // oversized seed statement must fail here too, not execute silently. + const content = yield* fs.readFileString( + path.isAbsolute(seed.path) ? seed.path : path.join(workdir, seed.path), ); + yield* checkScannerBufferSize(content, (message) => new Error(message)); + const lines = legacySplitAndTrim(content); const statements = seed.dirty ? [] : lines; yield* session.exec("BEGIN"); const body = Effect.gen(function* () { diff --git a/apps/cli/src/legacy/shared/legacy-seed-ops.unit.test.ts b/apps/cli/src/legacy/shared/legacy-seed-ops.unit.test.ts index 2282ea54ef..72fd1800f6 100644 --- a/apps/cli/src/legacy/shared/legacy-seed-ops.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-seed-ops.unit.test.ts @@ -122,6 +122,35 @@ describe("legacySeedData (dirty parse)", () => { ); }); + it.effect( + "rejects an oversized seed statement when SUPABASE_SCANNER_BUFFER_SIZE is configured (Go SeedFile.ExecBatchWithCache parity)", + () => { + // Go's SeedFile.ExecBatchWithCache parses through the same parseFile every + // other file type does, so an oversized statement must abort the seed run — + // same as legacy-migration-apply.unit.test.ts's equivalent case for migrations. + const dir = mkdtempSync(join(tmpdir(), "legacy-seed-scanner-")); + // Raw text must exceed the 4096-byte floor Go's bufio.Scanner starts at + // regardless of the configured limit (see legacy-migration-apply.unit.test.ts's + // equivalent case for the exact same 4096-byte floor). + writeFileSync(join(dir, "big.sql"), `select '${"x".repeat(5000)}';`); + const { session, calls } = fakeSeedSession(); + const previous = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = "100b"; + return runSeed(session, dir, [{ path: "big.sql", hash: "newhash", dirty: false }]).pipe( + Effect.exit, + Effect.tap((exit) => + Effect.sync(() => { + expect(Exit.isFailure(exit)).toBe(true); + expect(calls.some((c) => c.sql.includes("select"))).toBe(false); + rmSync(dir, { recursive: true, force: true }); + if (previous === undefined) delete process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + else process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = previous; + }), + ), + ); + }, + ); + it.effect("refreshes the hash for a dirty seed that parses, without running statements", () => { const dir = mkdtempSync(join(tmpdir(), "legacy-seed-")); writeFileSync(join(dir, "data.sql"), "insert into t values (1);"); diff --git a/apps/cli/src/legacy/shared/legacy-seed.ts b/apps/cli/src/legacy/shared/legacy-seed.ts index dcdbaa8499..c023e7f159 100644 --- a/apps/cli/src/legacy/shared/legacy-seed.ts +++ b/apps/cli/src/legacy/shared/legacy-seed.ts @@ -4,6 +4,7 @@ import { Data, Effect, FileSystem, Path } from "effect"; import { Output } from "../../shared/output/output.service.ts"; import type { LegacyDbSession } from "./legacy-db-connection.service.ts"; import { legacyResolveUnderWorkdir } from "./legacy-glob.ts"; +import { checkScannerBufferSize } from "./legacy-migration-apply.ts"; import { legacyCreateSeedTable, legacyReadSeedTable, @@ -129,20 +130,27 @@ export const legacyApplySeedFiles = ( // statements are in memory at a time, matching Go's `ExecBatchWithCache` → // `parseFile` inside the apply loop (`file.go:198-203`). A dirty seed only // updates its recorded hash, so Go never re-reads it — skip the read. - const statements = seed.dirty - ? [] - : legacySplitAndTrim( - new TextDecoder().decode( - yield* fs.readFile(legacyResolveUnderWorkdir(path, workdir, seed.path)).pipe( - Effect.mapError( - (cause) => - new LegacyMigrationSeedError({ - message: `failed to open seed file: ${cause.message}`, - }), - ), - ), + let statements: ReadonlyArray = []; + if (!seed.dirty) { + const content = new TextDecoder().decode( + yield* fs.readFile(legacyResolveUnderWorkdir(path, workdir, seed.path)).pipe( + Effect.mapError( + (cause) => + new LegacyMigrationSeedError({ + message: `failed to open seed file: ${cause.message}`, + }), ), - ); + ), + ); + // Go's `SeedFile.ExecBatchWithCache` parses through the same `parseFile` every + // other caller does, so it enforces `SUPABASE_SCANNER_BUFFER_SIZE` here too — + // see `checkScannerBufferSize`'s own doc comment. + yield* checkScannerBufferSize( + content, + (message) => new LegacyMigrationSeedError({ message }), + ); + statements = legacySplitAndTrim(content); + } const txn = Effect.gen(function* () { yield* session.exec("BEGIN"); if (!seed.dirty) { diff --git a/apps/cli/src/legacy/shared/legacy-seed.unit.test.ts b/apps/cli/src/legacy/shared/legacy-seed.unit.test.ts index e3f672ef51..177084c845 100644 --- a/apps/cli/src/legacy/shared/legacy-seed.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-seed.unit.test.ts @@ -3,7 +3,7 @@ import { tmpdir } from "node:os"; import { join } from "node:path"; import { BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; -import { Effect, FileSystem, Layer, Path } from "effect"; +import { Effect, Exit, FileSystem, Layer, Path } from "effect"; import { mockOutput } from "../../../tests/helpers/mocks.ts"; import type { LegacyDbSession } from "./legacy-db-connection.service.ts"; @@ -111,3 +111,35 @@ describe("legacyApplySeedFiles seed glob", () => { }, ); }); + +describe("legacyApplySeedFiles scanner buffer size", () => { + it.effect( + "rejects an oversized seed statement when SUPABASE_SCANNER_BUFFER_SIZE is configured (Go SeedFile.ExecBatchWithCache parity)", + () => { + // Ports the same `parseFile` every migration/globals/schema-file caller goes + // through (see `checkScannerBufferSize`'s doc comment), so an oversized + // statement must abort here too, not execute silently. + const dir = mkdtempSync(join(tmpdir(), "legacy-seed-scanner-")); + // Raw text must exceed the 4096-byte floor Go's bufio.Scanner starts at + // regardless of the configured limit (see legacy-migration-apply.unit.test.ts's + // equivalent case for the exact same 4096-byte floor). + writeFileSync(join(dir, "big.sql"), `insert into t values ('${"x".repeat(5000)}');`); + const { session, queries } = fakeSession(); + const out = mockOutput(); + const previous = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = "100b"; + return run(session, dir, ["big.sql"], out).pipe( + Effect.exit, + Effect.tap((exit) => + Effect.sync(() => { + expect(Exit.isFailure(exit)).toBe(true); + expect(queries.some((q) => q.sql.includes("insert into t"))).toBe(false); + rmSync(dir, { recursive: true, force: true }); + if (previous === undefined) delete process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + else process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = previous; + }), + ), + ); + }, + ); +}); diff --git a/apps/cli/src/legacy/shared/legacy-sql-split.ts b/apps/cli/src/legacy/shared/legacy-sql-split.ts index ca89362511..a2205847f0 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-split.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-split.ts @@ -19,7 +19,11 @@ interface State { const BEGIN_ATOMIC = "ATOMIC"; const END_ATOMIC = "END"; -const isIdentifierRune = (rune: string): boolean => /[\p{L}\p{N}_$]/u.test(rune); +// `\p{Nd}` (decimal digits only), not `\p{N}` (all Unicode numbers): Go's +// `unicode.IsDigit` — what `isIdentifierRune`/`TagState.next` port — is an alias for +// category `Nd` alone, so it rejects `No`/`Nl` runes like superscript-2 (`²`) that +// `\p{N}` would wrongly accept as a valid identifier/dollar-tag character. +const isIdentifierRune = (rune: string): boolean => /[\p{L}\p{Nd}_$]/u.test(rune); function isBeginAtomic(data: string): boolean { let offset = data.length - BEGIN_ATOMIC.length; @@ -114,8 +118,9 @@ class TagState implements State { constructor(private readonly offset: number) {} next(rune: string, data: string): State | null { if (rune === "$") return new DollarState(data.slice(this.offset)); - // Valid dollar-tag characters. - if (/[\p{L}\p{N}_]/u.test(rune)) return this; + // Valid dollar-tag characters — see `isIdentifierRune`'s comment on why `\p{Nd}`, + // not `\p{N}`. + if (/[\p{L}\p{Nd}_]/u.test(rune)) return this; return new ReadyState().next(rune, data); } } diff --git a/apps/cli/src/legacy/shared/legacy-sql-split.unit.test.ts b/apps/cli/src/legacy/shared/legacy-sql-split.unit.test.ts index b04a4793a0..6acf7cc5b1 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-split.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-split.unit.test.ts @@ -39,6 +39,17 @@ describe("legacySplitAndTrim", () => { ]); }); + it("treats a non-decimal Unicode digit as an invalid dollar-tag character, like Go's unicode.IsDigit", () => { + // Go's TagState.Next gates on unicode.IsDigit (category Nd only), which is false + // for superscript-2 (U+00B2, category No) — the tag "a²" is therefore invalid, + // Go falls back out of the tag and the embedded `;` becomes a real boundary. + const sql = "CREATE FUNCTION f() AS $a²$foo; bar$a²$ LANGUAGE sql;"; + expect(legacySplitAndTrim(sql)).toEqual([ + "CREATE FUNCTION f() AS $a²$foo", + "bar$a²$ LANGUAGE sql", + ]); + }); + it("respects named dollar tags", () => { const sql = "CREATE FUNCTION f() AS $body$ SELECT ';'; $body$ LANGUAGE sql; SELECT 2;"; expect(legacySplitAndTrim(sql)).toEqual([