diff --git a/apps/cli/docs/go-cli-porting-status.md b/apps/cli/docs/go-cli-porting-status.md index defaffc3e2..db0a790e41 100644 --- a/apps/cli/docs/go-cli-porting-status.md +++ b/apps/cli/docs/go-cli-porting-status.md @@ -87,7 +87,7 @@ These commands exist in the TS CLI today but have no direct top-level equivalent | `db lint` | `ported` | `legacy/commands/db/lint/` | `n/a` | `n/a` | Native TS port. Runs `plpgsql_check` in a rolled-back transaction via LegacyDbConnection; emits Go-parity pretty JSON. | | `db pull` | `ported` | `legacy/commands/db/pull/` | `n/a` | `n/a` | Native TS port. Native pg-delta / migra migration + `--declarative` pg-delta export; reconciles `schema_migrations`. The initial-migra pull dumps the remote schema natively (`pg_dump`) then appends the migra diff. Only `--experimental` (structured dump) still delegates to Go, deprecated in favor of `--declarative` (CLI-1957) — it needs a TS PostgreSQL DDL parser for Go's `format.WriteStructuredSchemas` that has no equivalent in this repo, and `--declarative` already delivers the same per-object schema split via pg-delta catalog introspection. | | `db push` | `ported` | `legacy/commands/db/push/` | `n/a` | `n/a` | Native TS port. Connects local/linked/`--db-url`; pushes pending migrations, `--include-seed` seeds (`seed_files` hash tracking), `--include-roles`, `[db.vault]` secrets including decrypted `encrypted:` values; `--dry-run`; best-effort pg-delta migrations-catalog cache (warning-only on failure). Pipeline-incompatible statements (`CREATE INDEX CONCURRENTLY`, `VACUUM`, …) run standalone outside the batch transaction — from the closed Go PR supabase/cli#5156, also ported into `apps/cli-go` (CLI-1989 ruling). | -| `db reset` | `ported` | `legacy/commands/db/reset/` | `n/a` | `n/a` | Remote path native (drop user schemas, vault upsert, MigrateAndSeed, `--version`/`--last`, `--sql-paths` seed override). Local path native: running check, recreate + migrate + seed via the hidden Go `db __db-bootstrap` seam, storage-gated bucket seeding (reuses `seed buckets`), git-branch `Finished…` line. Only the niche `--experimental` remote schema-files path still delegates to the Go binary (telemetry-disabled). Pipeline-incompatible statements run standalone outside the batch transaction, same as `db push` (closed Go PR supabase/cli#5156, CLI-1989 ruling). | +| `db reset` | `ported` | `legacy/commands/db/reset/` | `n/a` | `n/a` | Remote path native (drop user schemas, vault upsert, MigrateAndSeed — including the `--experimental` declarative `[db.migrations].schema_paths` apply branch, CLI-1958 — `--version`/`--last`, `--sql-paths` seed override). Local path native: running check, recreate + migrate + seed via the hidden Go `db __db-bootstrap` seam (that seam still forwards `--experimental` for its own schema-files branch, CLI-1955 scope), storage-gated bucket seeding (reuses `seed buckets`), git-branch `Finished…` line. Pipeline-incompatible statements run standalone outside the batch transaction, same as `db push` (closed Go PR supabase/cli#5156, CLI-1989 ruling). | | `db start` | `ported` | `legacy/commands/db/start/` | `n/a` | `n/a` | Fully native TS port (CLI-1954 removed the last Go delegation). Validates config, checks "already running" (prints Go's line, native `docker container inspect`), else natively brings up the Postgres container (network/volume/create/start via `legacy/shared/db-bootstrap/`'s shared primitives), waits for health, runs the fresh-volume `SetupLocalDatabase`-equivalent pipeline (including its best-effort pg-delta migrations-catalog warmup), and writes `_current_branch`. `--from-backup` is fully native too: a third entrypoint variant (schema.sql + a ported `restore.sh`, no `webhook.sql`), a `backup volume already exists` guard when the volume isn't fresh, and a swallowed (not failed) health-check timeout. No status table / `cli_stack_started` (those are `supabase start`). | | `inspect report` | `ported` | `legacy/commands/inspect/report/` | `n/a` | `n/a` | Native TS port. Runs every inspect query via server-side `COPY ... CSV`, writes 14 CSVs under `//`, then renders a Go-parity Glamour rules summary (bounded csvq-subset evaluator; custom `[experimental.inspect.rules]` supported). | | `inspect db db-stats` | `ported` | `legacy/commands/inspect/db/db-stats/` | `n/a` | `n/a` | Native TS port. Queries Postgres directly via LegacyDbConnection; renders Go-parity Glamour tables. | diff --git a/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md b/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md index fd5cf2f93b..794fd53e46 100644 --- a/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md +++ b/apps/cli/src/legacy/commands/db/reset/SIDE_EFFECTS.md @@ -2,12 +2,16 @@ Native TypeScript port of `apps/cli-go/internal/db/reset/reset.go`. Reinitialises a database from local migrations (plus seed). The **remote** path (`--linked`, or a -remote `--db-url`) is native: drop all user schemas, upsert vault secrets, then -re-apply migrations and seed. The **local** path (`--local`/default, or a `--db-url` +remote `--db-url`) is native: drop all user schemas, upsert vault secrets, then either +re-apply migrations (the default) or, on a versionless `--experimental`/ +`SUPABASE_EXPERIMENTAL` reset with pg-delta not enabled, apply the declarative +`[db.migrations].schema_paths` files instead (Go's `apply.MigrateAndSeed` EXPERIMENTAL +branch, CLI-1958), then seed. The **local** path (`--local`/default, or a `--db-url` pointing at the local stack) is also native: TS orchestrates the running check, messages, bucket seeding, and git-branch line, while the container-recreate -primitives run behind the hidden Go `db __db-bootstrap` seam. Only the niche -**`--experimental`** remote schema-files path still delegates to the Go binary. +primitives run behind the hidden Go `db __db-bootstrap` seam — that seam forwards +`--experimental` to the Go child itself, so the local path's own schema-files branch +is still handled by Go (CLI-1955 scope, unaffected by CLI-1958). ## Files Read @@ -19,41 +23,44 @@ primitives run behind the hidden Go `db __db-bootstrap` seam. Only the niche | `~/.supabase//project-ref` | plain text | `--linked`, to resolve the ref | | `~/.supabase/access-token` | plain text | `--linked`, when `SUPABASE_ACCESS_TOKEN` unset and a temp role is minted | | seed files from `--sql-paths` or `[db.seed].sql_paths` | SQL | when seeding is enabled (not `--no-seed`); `--sql-paths` overrides config | +| schema files from `[db.migrations].schema_paths` | SQL | remote path only, when the `--experimental` schema-files branch is taken (see Notes) | | `/supabase/buckets/` | files | local path, when storage is up and `[storage.buckets]` configure objects | ## Files Written -| Path | Format | When | -| ------------------------------------------------ | ------ | --------------------------------- | -| `~/.supabase//linked-project.json` | JSON | `--linked` (post-run cache) | -| `~/.supabase/telemetry.json` | JSON | always (post-run telemetry flush) | +| Path | Format | When | +| ------------------------------------------------------------------------------- | ------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `~/.supabase//linked-project.json` | JSON | `--linked` (post-run cache) | +| `~/.supabase/telemetry.json` | JSON | always (post-run telemetry flush) | +| `/supabase/.temp/pgdelta/catalog--migrations--.json` | JSON | best-effort, after migrations/seeding succeed, when no `--version`/`--last` resolved a version AND pg-delta is enabled (`[experimental.pgdelta].enabled` or `SUPABASE_EXPERIMENTAL_PG_DELTA`); a failure only warns on stderr and never fails the reset — see Notes. **Remote path** (native TS, `` = the project ref/URL hash): after either apply branch (schema-files or migrations) (Go's `down.ResetAll` → `pgcache.TryCacheMigrationsCatalog`, `down.go:58-59`). **Local path** (inherited from the real Go binary via the `db __db-bootstrap` seam, `` = `"local"`): PG15 only, after `MigrateAndSeed` inside `start.SetupLocalDatabase` (`start.go:359-381`) — the PG≤14 branch (`resetDatabase14`) never calls this at all, so a PG≤14 local project never writes this file regardless of pg-delta config | On the local path the Go seam additionally recreates the `supabase_db_` -container/volume and applies the initial schema (`SetupLocalDatabase`); the -`--experimental` remote path produces whatever the delegated Go binary writes. +container/volume and applies the initial schema (`SetupLocalDatabase`). ## Subprocesses -| Command | When | Purpose | -| --------------------------------------------------------------------------- | ----------------------------------- | ----------------------------------------------------------------------- | -| `docker container inspect supabase_db_` | local path | `AssertSupabaseDbIsRunning` probe (Podman fallback) | -| `supabase-go db __db-bootstrap --mode recreate [--version ] [--no-seed]` | local path | recreate container + init schema + migrate + seed + restart services | -| `supabase-go db __db-bootstrap --mode await-storage` | local path | storage health gate before bucket seeding (`ready` / `absent`) | -| `supabase-go db reset --linked\|--db-url … [--no-seed]` | `--experimental` remote, no version | the un-ported experimental schema-files apply path (telemetry disabled) | +| Command | When | Purpose | +| --------------------------------------------------------------------------- | ---------- | -------------------------------------------------------------------- | +| `docker container inspect supabase_db_` | local path | `AssertSupabaseDbIsRunning` probe (Podman fallback) | +| `supabase-go db __db-bootstrap --mode recreate [--version ] [--no-seed]` | local path | recreate container + init schema + migrate + seed + restart services | +| `supabase-go db __db-bootstrap --mode await-storage` | local path | storage health gate before bucket seeding (`ready` / `absent`) | The seam subprocesses run with `SUPABASE_TELEMETRY_DISABLED=1`, stderr inherited; -`--network-id` / a flag-selected `--profile` are forwarded. +`--network-id` / a flag-selected `--profile` (plus `--experimental`, so the seam's own +`MigrateAndSeed` takes Go's schema-files branch on a versionless local reset) are +forwarded. ## Database Mutations ### Remote path (native, in TS) -| Statement | When | -| ------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------ | -| `drop.sql` `DO` block (drops user schemas/extensions/public objects, truncates auth/migrations) | always, first | -| `SELECT vault.update_secret(...)` / `vault.create_secret(...)` | when `[db.vault]` has syncable secrets | -| migration statements + `schema_migrations` history insert (per file, transactional; pipeline-incompatible statements run standalone — see Notes) | when `[db.migrations].enabled`, for migrations `≤ --version` | -| seed statements + `seed_files` hash upsert | when `[db.seed].enabled` and not `--no-seed` | +| Statement | When | +| ------------------------------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------------------------------------------- | +| `drop.sql` `DO` block (drops user schemas/extensions/public objects, truncates auth/migrations) | always, first | +| `SELECT vault.update_secret(...)` / `vault.create_secret(...)` | when `[db.vault]` has syncable secrets | +| schema-file statements (no history bookkeeping, no `RESET ALL` between files) | `--experimental` + no resolved version + pg-delta not enabled (see Notes) | +| migration statements + `schema_migrations` history insert (per file, transactional; pipeline-incompatible statements run standalone — see Notes) | otherwise, when `[db.migrations].enabled`, for migrations `≤ --version` | +| seed statements + `seed_files` hash upsert | when `[db.seed].enabled` and not `--no-seed` (runs after either branch above) | ### Local path (inside the Go seam) @@ -76,40 +83,46 @@ races a restarting gateway. ## Environment Variables -| Variable | Purpose | Required? | -| ----------------------- | ----------------------------------------------- | ------------------------------------------------------- | -| `SUPABASE_ACCESS_TOKEN` | auth token for the `--linked` resolver path | no (falls back to keyring → `~/.supabase/access-token`) | -| `SUPABASE_DB_PASSWORD` | password for the linked/remote connection | no | -| `SUPABASE_YES` | auto-confirm the reset prompt | no (also `--yes`) | -| `SUPABASE_EXPERIMENTAL` | routes the experimental schema-files path to Go | no (also `--experimental`) | -| `SUPABASE_PROJECT_ID` | overrides the local container id (`utils.DbId`) | no | +| Variable | Purpose | Required? | +| --------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------- | +| `SUPABASE_ACCESS_TOKEN` | auth token for the `--linked` resolver path | no (falls back to keyring → `~/.supabase/access-token`) | +| `SUPABASE_DB_PASSWORD` | password for the linked/remote connection | no | +| `SUPABASE_YES` | auto-confirm the reset prompt | no (also `--yes`) | +| `SUPABASE_EXPERIMENTAL` | selects the remote schema-files apply branch | no (also `--experimental`) | +| `SUPABASE_EXPERIMENTAL_PGDELTA_ENABLED` | overrides `[experimental.pgdelta].enabled`; a truthy value flips the reset gate (`experimental && resolvedVersion === "" && !toml.pgDelta.enabled`) back to timestamped migrations even with `--experimental` set — switches between two different destructive code paths | no | +| `SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS` | overrides `[db.migrations].schema_paths` (viper `AutomaticEnv`, beats the config-file value) for the schema-files apply branch | no (no dedicated flag — config-file-only otherwise) | +| `SUPABASE_PROJECT_ID` | overrides the local container id (`utils.DbId`) | no | +| `SUPABASE_EXPERIMENTAL_PG_DELTA` | enables the post-reset migrations-catalog cache (see Files Written) when `[experimental.pgdelta].enabled` is unset — distinct from `SUPABASE_EXPERIMENTAL_PGDELTA_ENABLED` above, which switches the reset's own apply branch instead | no (project `.env` or shell) | +| `SUPABASE_INTERNAL_IMAGE_REGISTRY` | overrides the pg-delta edge-runtime image registry for the migrations-catalog cache export (scoped for the whole run via `legacyApplyProjectEnv`, matching `db push`) | no (project `.env` or shell) | +| `PGDELTA_NPM_REGISTRY` | overrides the pg-delta edge-runtime npm registry (`.npmrc` + `NPM_CONFIG_REGISTRY` forward) for the migrations-catalog cache export (scoped for the whole run via `legacyApplyProjectEnv`, matching `db push`) | no (project `.env` or shell) | ## Exit Codes -| Code | Condition | -| -------------------- | -------------------------------------------------------------------------------------------------------------------------- | -| `0` | success | -| `1` | mutually exclusive target flags (`[db-url linked local]`) | -| `1` | `--version` + `--last` together (`[last version]`) | -| `1` | `--version` not an integer (`invalid version number`) | -| `1` | `--version` has no matching migration file | -| `1` | local: database not running (`supabase start is not running.`) | -| `1` | user declined the reset confirmation (`context canceled`) | -| `1` | `config.toml` parse failure | -| `1` | drop / migrate / seed / vault apply failure, or connection error | -| child's exact code\* | local: container recreate / storage health-gate failure (seam), or `--experimental`/`--linked` delegate (proxy) child exit | - -\* The `db __db-bootstrap` seam and the `--experimental` remote delegate both -propagate the spawned `supabase-go` child's real exit code (e.g. `130` after a -Ctrl-C mid-recreate) instead of collapsing every failure to `1` — in every -`--output-format` (CLI-1879). +| Code | Condition | +| -------------------- | ----------------------------------------------------------------------------------------- | +| `0` | success | +| `1` | mutually exclusive target flags (`[db-url linked local]`) | +| `1` | `--version` + `--last` together (`[last version]`) | +| `1` | `--version` not an integer (`invalid version number`) | +| `1` | `--version` has no matching migration file | +| `1` | local: database not running (`supabase start is not running.`) | +| `1` | user declined the reset confirmation (`context canceled`) | +| `1` | `config.toml` parse failure | +| `1` | drop / migrate / seed / vault apply failure, or connection error | +| `1` | no `[db.migrations].schema_paths` pattern matched anything on the `--experimental` branch | +| child's exact code\* | local: container recreate / storage health-gate failure (seam) | + +\* The `db __db-bootstrap` seam propagates the spawned `supabase-go` child's real exit +code (e.g. `130` after a Ctrl-C mid-recreate) instead of collapsing every failure to +`1` — in every `--output-format` (CLI-1879). ## Output -The remote path prints `Resetting remote database…` to **stderr**, then the -drop/migrate/seed progress (`Applying migration …`, `Seeding data from …`). Go -connects with `io.Discard`, so there is **no** `Connecting to … database…` line and -**no** `Finished …` line on the remote path. +The remote path prints `Resetting remote database…` to **stderr**, then either the +schema-files branch's apply (no per-file progress — Go's `applySchemaFiles` prints +nothing, CLI-1958) or the migrate/seed progress (`Applying migration …`, `Seeding +data from …`). Go connects with `io.Discard`, so there is **no** `Connecting to … +database…` line and **no** `Finished …` line on the remote path. The local path prints `Resetting local database…` to **stderr**, then the seam's `Recreating database...` / `Restarting containers...` progress, and finally @@ -118,9 +131,8 @@ in Aqua). ### `--output-format text` (Go CLI compatible) -Byte-matches Go's stderr progress for both the remote and local paths. The -`--experimental` remote path passes the delegated Go binary's output through -unchanged. +Byte-matches Go's stderr progress for both the remote and local paths, including the +silent (no-progress-line) `--experimental` schema-files apply. ### `--output-format json` / `stream-json` @@ -152,6 +164,52 @@ path has no confirmation prompt. - `--last n` reverts the most recent `n` migrations; if `n ≥ total`, the reset target version becomes `-` (revert everything). Mutually exclusive with `--version`. - `--db-url`, `--linked`, and `--local` (default true) are mutually exclusive. -- **Known interim**: only `--experimental` remote resets run via the Go binary; the - best-effort pg-delta catalog cache (inside the seam) is not surfaced (no output - impact). `encrypted:` vault secrets are skipped on the remote path. +- **`--experimental` remote schema-files apply** (Go's `apply.MigrateAndSeed` + EXPERIMENTAL branch, `apps/cli-go/internal/migration/apply/apply.go:19,51-68`; + ported natively CLI-1958): taken on the remote path when `--experimental` / + `SUPABASE_EXPERIMENTAL` is set, no `--version`/`--last` resolved a version, AND + `[experimental.pgdelta].enabled` is NOT set. A hard `if`/`else if` in Go — taking + this branch means timestamped migrations never run at all, even when + `[db.migrations].schema_paths` matches nothing. Faithfully reproduces two + undocumented Go quirks: (1) the `schema_paths` default is `[]`, so a stock project + running an experimental remote reset silently applies NOTHING (drops schemas, + seeds, but replays no SQL) rather than falling back to migrations; (2) a partial + glob failure (some patterns match, others don't) is silently dropped — only a + TOTAL failure (no pattern matches anything) aborts the reset, with Go's joined + `no files matched pattern: …` text and no `CmdSuggestion`. A per-file apply + failure attaches Go's `See schema file: ` suggestion. No progress line is + printed per file (Go's `applySchemaFiles` has no output), no migration-history row + is inserted, and no `RESET ALL` runs between files. Seeding still runs afterward, + unconditionally, exactly as on the migrations branch. + `encrypted:` vault secrets are NOT skipped on the remote path — `legacyCheckDbToml` + decrypts them into `toml.vault`, and `legacyUpsertVaultSecrets` upserts the decrypted + values unconditionally, before either branch (schema-files or migrations) runs. +- **Migrations catalog cache**: gated on BOTH no `--version`/`--last` having resolved + a version AND pg-delta being enabled (`[experimental.pgdelta].enabled` or + `SUPABASE_EXPERIMENTAL_PG_DELTA` — see Environment Variables); a versioned reset + never refreshes the cache, matching Go's own `len(version) > 0` no-op inside + `TryCacheMigrationsCatalog` itself. A failure only warns on stderr and never fails + the reset, matching Go exactly. Writes under `supabase/.temp/pgdelta/` (see Files + Written), pruning older snapshots for the same prefix (retains 2). This is NOT + remote-only — both paths write it, on different call chains: + - **Remote path** (native TS, ported): Go's best-effort `down.ResetAll` → + `pgcache.TryCacheMigrationsCatalog` (`down.go:48-61`), after either apply branch + (schema-files or migrations) and seeding complete. Exports the target's pg-delta + catalog via the edge-runtime stack. Reuses `legacyExportCatalogPgDelta` and + `legacyTryCacheMigrationsCatalog` — the same helpers `db push` uses for its own + post-apply cache (see that command's SIDE_EFFECTS Notes) — rather than a second + copy. + - **Local path** (inherited automatically from the real Go binary behind the + `db __db-bootstrap --mode recreate` seam — not reimplemented in TS, CLI-1955 + scope): `start.SetupLocalDatabase` (`start.go:359-381`) calls the same + `pgcache.TryCacheMigrationsCatalog` (with prefix `"local"`) right after + `apply.MigrateAndSeed` succeeds, warning the same way on failure. This only + happens on the **PG15** recreate branch (`resetDatabase15` → `SetupLocalDatabase`, + `reset.go:146-174`) — the **PG≤14** branch (`resetDatabase14`, `reset.go:128-144`) + returns immediately after `apply.MigrateAndSeed` and never calls + `pgcache.TryCacheMigrationsCatalog` at all, so a PG≤14 local project never writes + this file, no matter how pg-delta is configured. +- The local path's own `--experimental` schema-files branch is still handled by the + Go child behind the `db __db-bootstrap` seam (CLI-1955 scope) — this command's + handler forwards `--experimental` to that seam but does not implement the branch + itself for the local target. diff --git a/apps/cli/src/legacy/commands/db/reset/reset.errors.ts b/apps/cli/src/legacy/commands/db/reset/reset.errors.ts index 80d2cba332..9ecae3c60f 100644 --- a/apps/cli/src/legacy/commands/db/reset/reset.errors.ts +++ b/apps/cli/src/legacy/commands/db/reset/reset.errors.ts @@ -51,9 +51,15 @@ export class LegacyDbResetCancelledError extends Data.TaggedError("LegacyDbReset readonly message: string; }> {} -/** A drop / migrate / seed / vault statement failed during the remote reset. */ +/** + * A drop / migrate / seed / vault statement failed during the remote reset. `suggestion` + * is Go's `CmdSuggestion` — set only by the `--experimental` schema-files apply branch + * (`"See schema file: "`, `apply.go:63`); every other apply failure on this + * command leaves it unset, matching Go. + */ export class LegacyDbResetApplyError extends Data.TaggedError("LegacyDbResetApplyError")<{ readonly message: string; + readonly suggestion?: string; }> {} /** diff --git a/apps/cli/src/legacy/commands/db/reset/reset.handler.ts b/apps/cli/src/legacy/commands/db/reset/reset.handler.ts index 29ccc17783..7d4d3f57b8 100644 --- a/apps/cli/src/legacy/commands/db/reset/reset.handler.ts +++ b/apps/cli/src/legacy/commands/db/reset/reset.handler.ts @@ -8,32 +8,43 @@ import { legacyResolveExperimentalWithProjectEnv, legacyResolveYesWithProjectEnv, } from "../../../../shared/legacy/global-flags.ts"; -import { LegacyGoProxy } from "../../../../shared/legacy/go-proxy.service.ts"; import { CONTEXT_CANCELED_MESSAGE } from "../../../../shared/output/errors.ts"; import { Output } from "../../../../shared/output/output.service.ts"; import { legacyAqua, legacyYellow } from "../../../shared/legacy-colors.ts"; import { LegacyCliConfig } from "../../../config/legacy-cli-config.service.ts"; import { LegacyProjectRefResolver } from "../../../config/legacy-project-ref.service.ts"; +import { redactLegacyConnectionString } from "../../../shared/legacy-db-config.parse.ts"; import { LegacyDbConfigResolver } from "../../../shared/legacy-db-config.service.ts"; import { + legacyApplyProjectEnv, legacyCheckDbToml, legacyLoadProjectEnv, legacyResolveSeedSqlPath, } from "../../../shared/legacy-db-config.toml-read.ts"; import { LegacyDbConnection } from "../../../shared/legacy-db-connection.service.ts"; -import { legacyApplyMigrations } from "../../../shared/legacy-migration-apply.ts"; +import { + legacyResolveLocalProjectId, + legacySanitizeProjectId, +} from "../../../shared/legacy-docker-ids.ts"; +import { + legacyApplyMigrations, + legacyApplySchemaFiles, +} from "../../../shared/legacy-migration-apply.ts"; import { legacyParseMigrationVersion } from "../../../shared/legacy-migration-timestamp.format.ts"; +import { legacyToPostgresURL } from "../../../shared/legacy-postgres-url.ts"; import { legacyPromptYesNo } from "../../../../shared/legacy/legacy-prompt-yes-no.ts"; -import { - type LegacyDbConnType, - resolveLegacyDbTargetFlags, -} from "../../../shared/legacy-db-target-flags.ts"; +import { resolveLegacyDbTargetFlags } from "../../../shared/legacy-db-target-flags.ts"; import { LegacyLinkedProjectCache } from "../../../telemetry/legacy-linked-project-cache.service.ts"; import { LegacyTelemetryState } from "../../../telemetry/legacy-telemetry-state.service.ts"; import { legacyDropUserSchemas } from "../shared/legacy-drop-schemas.ts"; import { LegacyDbBootstrapSeam } from "../shared/legacy-db-bootstrap.seam.service.ts"; import { legacyIsLocalDbRunning } from "../../../shared/db-bootstrap/local-db-running.ts"; -import { legacyListLocalMigrations } from "../../../shared/legacy-pgdelta.cache.ts"; +import { legacyParseBoolEnv } from "../../../shared/legacy-diff-engine.ts"; +import { + legacyListLocalMigrations, + legacyTryCacheMigrationsCatalog, +} from "../../../shared/legacy-pgdelta.cache.ts"; +import { type LegacyPgDeltaContext } from "../../../shared/legacy-pgdelta.ts"; import { legacyPathMatch } from "../../../shared/legacy-path-match.ts"; import { legacyGetPendingSeeds, legacySeedData } from "../../../shared/legacy-seed-ops.ts"; import { legacyUpsertVaultSecrets } from "../../../shared/legacy-vault.ts"; @@ -53,59 +64,31 @@ import { const MIGRATE_FILE_PATTERN = /^([0-9]+)_(.*)\.sql$/u; -const applyError = (message: string) => new LegacyDbResetApplyError({ message }); +const applyError = (message: string, suggestion?: string) => + new LegacyDbResetApplyError({ message, ...(suggestion !== undefined ? { suggestion } : {}) }); /** Go's `toLogMessage` (`internal/db/reset/reset.go:88-91`). */ const toLogMessage = (version: string): string => version.length > 0 ? ` to version: ${version}` : "..."; -/** - * Rebuilds the `db reset` argv for the remaining Go-delegated path: a remote - * `--experimental` reset with no resolved version. Only the flags reachable on - * that path are forwarded — `--local` always takes the native path, and a set - * `--version`/`--last` resolves a non-empty version which disables the experimental - * delegation (a degenerate `--last 0` resolves to "" and is behaviourally identical - * whether or not it is forwarded, so it is omitted). - * - * The target selector is forwarded from the RESOLVED `connType`, not the raw `--linked` - * boolean: the parent's `resolveLegacyDbTargetFlags` follows Cobra's `Changed` semantics, so - * `--linked=false` selects the linked/remote target (this path is remote-only). Forwarding - * only when `flags.linked === true` would drop the selector for `--linked=false` and let the - * Go child fall back to its local default — resetting the wrong database. - */ -const buildResetArgs = ( - flags: LegacyDbResetFlags, - connType: LegacyDbConnType, - yes: boolean, -): Array => { - const args = ["db", "reset"]; - if (Option.isSome(flags.dbUrl)) args.push("--db-url", flags.dbUrl.value); - else if (connType === "linked") args.push("--linked"); - if (flags.noSeed) args.push("--no-seed"); - for (const p of flags.sqlPaths) args.push("--sql-paths", p); - // Forward the parent's RESOLVED `yes` as a bound flag. Go's `--yes` beats `AutomaticEnv`, - // so `--yes=false` overrides an inherited `SUPABASE_YES=true` (the child no longer - // auto-confirms a reset the user protected with `--yes=false`), while `--yes=true` honors - // an explicit `--yes` / env even in machine mode where the child's stdin is ignored. - // `--yes=false` still prompts on a TTY (Go's PromptYesNo only short-circuits on true), so - // this matches the default behavior when neither flag nor env is set. - args.push(`--yes=${yes}`); - return args; -}; - /** * `supabase db reset` — reinitialise a database from local migrations (+ seed). * - * Strict 1:1 port of `apps/cli-go/internal/db/reset/reset.go`. The remote path - * (`--linked` / a remote `--db-url`) is native. The local path (and the niche - * `--experimental` schema-files path) delegate to the Go binary as a documented - * interim until the container-bootstrap seam is ported (CLI-1325 Stage 3). + * Strict 1:1 port of `apps/cli-go/internal/db/reset/reset.go`. Both the remote path + * (`--linked` / a remote `--db-url`) and the local path (`--local`/default, or a + * `--db-url` pointing at the local stack) are native TS. On the remote path, a + * versionless `--experimental` (or `SUPABASE_EXPERIMENTAL`) reset with pg-delta NOT + * enabled takes Go's EXPERIMENTAL declarative schema-files branch of + * `apply.MigrateAndSeed` (`legacyApplySchemaFiles`, CLI-1958) instead of replaying + * timestamped migrations. The local path's container-recreate primitives still run + * behind the hidden Go `db __db-bootstrap` seam (CLI-1955); that seam forwards + * `--experimental` to the Go child itself, so the local path's own schema-files + * branch is out of scope here. */ export const legacyDbReset = Effect.fn("legacy.db.reset")(function* (flags: LegacyDbResetFlags) { const output = yield* Output; const resolver = yield* LegacyDbConfigResolver; const dbConn = yield* LegacyDbConnection; - const proxy = yield* LegacyGoProxy; const seam = yield* LegacyDbBootstrapSeam; const cliConfig = yield* LegacyCliConfig; const telemetryState = yield* LegacyTelemetryState; @@ -128,6 +111,16 @@ export const legacyDbReset = Effect.fn("legacy.db.reset")(function* (flags: Lega let linkedRefForCache: string | undefined; const body = Effect.gen(function* () { + // Go's `loadNestedEnv` (`os.Setenv`) makes every project-`.env` key visible to the + // WHOLE reset run, not just the flag-gate reads above — in particular + // `legacyGetRegistryImageUrl` / `legacyPgDeltaNpmRegistryOption` read + // `SUPABASE_INTERNAL_IMAGE_REGISTRY` / `PGDELTA_NPM_REGISTRY` straight from + // `process.env` for the pg-delta catalog export below (review CLI-1958). `db push` + // (`push.handler.ts`) scopes this the same way, as the first statement of its own + // `body` — mirror that exactly so a private/air-gapped registry configured only in + // `supabase/.env` reaches the catalog export instead of silently falling back to the + // default registries. + yield* legacyApplyProjectEnv(projectEnv); const target = resolveLegacyDbTargetFlags(cliArgs.args); // cobra MarkFlagsMutuallyExclusive("db-url", "linked", "local"). if (target.setFlags.length > 1) { @@ -237,35 +230,6 @@ export const legacyDbReset = Effect.fn("legacy.db.reset")(function* (flags: Lega } const connType = target.connType ?? "local"; - // Single source of truth for "does this reset delegate to the Go child?" — - // checked at both delegation sites below (before `resolve()` for a linked - // target, after it for a `--db-url` target) so the two call sites can never - // drift apart. - const shouldDelegateExperimental = experimental && resolvedVersion === ""; - - // Delegates the remaining `--experimental` schema-files apply path - // (`apply.MigrateAndSeed`, not ported) to the Go child. In text mode inherit - // its stdio. Under a machine-output mode (`--output-format json|stream-json`) - // the Go child emits no TS envelope, so suppress its stdout (capture + discard) - // and emit the same structured success the native local and remote paths do, - // keeping the JSON contract consistent across all reset paths. - const delegateExperimentalReset = () => - Effect.gen(function* () { - const env = { SUPABASE_TELEMETRY_DISABLED: "1" }; - if (output.format === "text") { - yield* proxy.exec(buildResetArgs(flags, connType, yes), { env }); - } else { - // Machine-output mode is non-interactive: give the Go child a non-TTY stdin - // (`stdin: "ignore"`) so it can't block on (or be answered at) Go's - // destructive reset prompt — it takes the default `false`, matching the - // native reset path which suppresses prompts under json/stream-json. - yield* proxy.execCapture(buildResetArgs(flags, connType, yes), { env, stdin: "ignore" }); - yield* output.success("Reset remote database.", { - target: "remote", - version: resolvedVersion, - }); - } - }); // Go's ParseDatabaseConfig runs LoadProjectRef BEFORE the fallible linked // resolution (db_url.go:87-95), and Execute() writes the linked-project cache @@ -275,21 +239,6 @@ export const legacyDbReset = Effect.fn("legacy.db.reset")(function* (flags: Lega if (connType === "linked") { const refResolver = yield* LegacyProjectRefResolver; linkedRefForCache = yield* refResolver.loadProjectRef(Option.none()); - - // A linked target is never local (`resolver.resolve()`'s "linked" branch - // always returns `isLocal: false`), so the delegated-experimental check can - // run BEFORE calling `resolve()`. This matters: for `connType === "linked"`, - // `resolve()` mints/verifies a temporary Postgres login role over the - // Management API — and the delegated Go child re-runs that exact same - // `ParseDatabaseConfig` work itself once delegation happens. Calling - // `resolve()` here would mint the temp role twice for zero downstream use on - // this branch (Go's own reset flow mints it exactly once, as part of the code - // path being delegated to — confirmed against `apps/cli-go/internal/utils/ - // flags/db_url.go`'s `NewDbConfigWithPassword`/`initLoginRole`). CLI-1879. - if (shouldDelegateExperimental) { - yield* delegateExperimentalReset(); - return; - } } const cfg = yield* resolver.resolve({ dbUrl: flags.dbUrl, connType, dnsResolver }); @@ -395,22 +344,12 @@ export const legacyDbReset = Effect.fn("legacy.db.reset")(function* (flags: Lega } // Re-confirm `linkedRefForCache` from the now-resolved `cfg.ref` for the native - // remote linked path below (a linked+experimental+versionless target already - // delegated and returned above, before `resolve()` was ever called — see the - // `connType === "linked"` block earlier in this function). A `connType === - // "db-url"` target leaves `linkedRefForCache` as whatever the pre-load block - // set (nothing, for `db-url`), since this assignment only fires when linked. + // remote path below. A `connType === "db-url"` target leaves `linkedRefForCache` + // as whatever the pre-load block set (nothing, for `db-url`), since this + // assignment only fires when linked. const linkedRef = Option.getOrUndefined(cfg.ref ?? Option.none()); if (connType === "linked" && linkedRef !== undefined) linkedRefForCache = linkedRef; - // Remaining remote target: a `--db-url` pointing at a non-local host (the - // `connType === "linked"` case already delegated above, before `resolve()`, - // without resolving a connection at all). - if (shouldDelegateExperimental) { - yield* delegateExperimentalReset(); - return; - } - // Single Go-parity config load (`flags.LoadConfig` → `config.Load` + `Validate`): // decodes the whole config with Go's env-expansion + `strconv.ParseBool` weak typing // (so `enabled = "env(SEED_ENABLED)"` etc. load like Go), applies `SUPABASE_*` @@ -447,7 +386,28 @@ export const legacyDbReset = Effect.fn("legacy.db.reset")(function* (flags: Lega yield* legacyDropUserSchemas(session, applyError); yield* legacyUpsertVaultSecrets(session, vaultSecrets); - if (toml.migrationsEnabled) { + // Go's three-conjunct EXPERIMENTAL gate (`apply.MigrateAndSeed`, `apply.go:19`): + // `--experimental`/`SUPABASE_EXPERIMENTAL` + no resolved version + pg-delta NOT + // enabled. A hard `if`/`else if` in Go (`apply.go:19-27`) — taking the + // schema-files branch means timestamped migrations never run at all, even when + // the glob matches nothing (Go's `schema_paths = []` default silently applies + // NOTHING rather than falling back to migrations — CLI-1958). + const useSchemaFiles = experimental && resolvedVersion === "" && !toml.pgDelta.enabled; + if (useSchemaFiles) { + // `projectEnv` (loaded above, before `experimental`/`yes` resolve) is threaded + // through so a `SUPABASE_SCANNER_BUFFER_SIZE` set only in `supabase/.env` is + // honored here exactly like Go's `loadNestedEnv` (see + // `checkScannerBufferSize`'s doc comment, `legacy-migration-apply.ts`). + yield* legacyApplySchemaFiles( + session, + fs, + path, + workdir, + toml.schemaPaths, + applyError, + projectEnv, + ); + } else if (toml.migrationsEnabled) { const locals = yield* legacyListLocalMigrations(fs, path, migrationsDir); // LoadPartialMigrations filter: version === "" || v <= version. const pending = locals.filter((p) => { @@ -475,7 +435,46 @@ export const legacyDbReset = Effect.fn("legacy.db.reset")(function* (flags: Lega const seeds = yield* legacyGetPendingSeeds(session, fs, path, seedPaths, workdir); yield* legacySeedData(session, fs, workdir, path, seeds, applyError); } - // Go's best-effort pgcache catalog warning is not ported (no output impact). + + // Go's `down.ResetAll` (`internal/migration/down/down.go:48-61`) — the function + // `resetRemote` delegates to — best-effort caches the migrations catalog for + // pg-delta right after `apply.MigrateAndSeed` succeeds, warning (never failing + // the reset) on error. `pgcache.TryCacheMigrationsCatalog` itself no-ops when + // `resolvedVersion` is non-empty (`len(version) > 0`, `pgcache/cache.go:73`) — + // a versioned reset (`--version`/`--last`) never refreshes the cache — so gate + // the call the same way rather than threading that check into the shared + // native helper (already used by `db push`, which has no version concept). + const cacheEnabled = + resolvedVersion === "" && + (toml.pgDelta.enabled || + legacyParseBoolEnv(toml.envLookup("SUPABASE_EXPERIMENTAL_PG_DELTA"))); + const pgDeltaCtx: LegacyPgDeltaContext = { + projectId: legacySanitizeProjectId( + legacyResolveLocalProjectId( + Option.getOrUndefined(cliConfig.projectId), + Option.getOrUndefined(toml.projectId) ?? + (linkedRef !== undefined && linkedRef !== "" ? linkedRef : undefined), + workdir, + ), + ), + cwd: workdir, + npmVersion: Option.getOrUndefined(toml.pgDelta.npmVersion), + denoVersion: toml.denoVersion, + }; + yield* legacyTryCacheMigrationsCatalog(fs, path, pgDeltaCtx, { + enabled: cacheEnabled, + targetUrl: legacyToPostgresURL(cfg.conn), + conn: cfg.conn, + isLocal: false, + migrationsDir, + }).pipe( + Effect.catch((error) => + output.raw( + `Warning: failed to cache migrations catalog: ${redactLegacyConnectionString(error.message)}\n`, + "stderr", + ), + ), + ); }), ); @@ -496,5 +495,8 @@ export const legacyDbReset = Effect.fn("legacy.db.reset")(function* (flags: Lega ), ), Effect.ensuring(telemetryState.flush), + // Closes the `Scope` `legacyApplyProjectEnv` (above) acquires its `process.env` + // reverts against — mirrors `push.handler.ts`'s own `body.pipe(..., Effect.scoped)`. + Effect.scoped, ); }); diff --git a/apps/cli/src/legacy/commands/db/reset/reset.integration.test.ts b/apps/cli/src/legacy/commands/db/reset/reset.integration.test.ts index 79b1bc935d..08a14591c1 100644 --- a/apps/cli/src/legacy/commands/db/reset/reset.integration.test.ts +++ b/apps/cli/src/legacy/commands/db/reset/reset.integration.test.ts @@ -1,4 +1,4 @@ -import { mkdirSync, writeFileSync } from "node:fs"; +import { chmodSync, mkdirSync, writeFileSync } from "node:fs"; import { dirname, join } from "node:path"; import { BunServices } from "@effect/platform-bun"; @@ -31,7 +31,6 @@ import { LegacyExperimentalFlag, LegacyYesFlag, } from "../../../../shared/legacy/global-flags.ts"; -import { LegacyGoProxy } from "../../../../shared/legacy/go-proxy.service.ts"; import { LegacyGoChildExitError } from "../../../../shared/legacy/legacy-go-child-exit.error.ts"; import type { OutputFormat } from "../../../../shared/output/types.ts"; import { LegacyDbConfigResolver } from "../../../shared/legacy-db-config.service.ts"; @@ -45,6 +44,12 @@ import { LegacyDbConnection, type LegacyPgConnInput, } from "../../../shared/legacy-db-connection.service.ts"; +import { LegacyEdgeRuntimeScriptError } from "../../../shared/legacy-edge-runtime-script.errors.ts"; +import { + LegacyEdgeRuntimeScript, + type LegacyEdgeRuntimeRunOpts, +} from "../../../shared/legacy-edge-runtime-script.service.ts"; +import { LegacyPgDeltaSslProbe } from "../../../shared/legacy-pgdelta-ssl-probe.service.ts"; import { LegacyDbBootstrapSeam } from "../shared/legacy-db-bootstrap.seam.service.ts"; import { legacyDbReset } from "./reset.handler.ts"; import type { LegacyDbResetFlags } from "./reset.command.ts"; @@ -73,10 +78,8 @@ const DEFAULT_FLAGS: LegacyDbResetFlags = { /** * Tracks every `resolve`/`resolvePoolerFallback` invocation so tests can prove a - * connection was (or, for the delegated-experimental path, was NOT) resolved — - * `resolve()` mints/verifies a temporary Postgres login role over the Management - * API, so calling it on a path that immediately discards the result is wasted - * (and duplicated) work (CLI-1879). + * connection was resolved exactly once per reset — `resolve()` mints/verifies a + * temporary Postgres login role over the Management API for a `--linked` target. */ function mockResolver(opts: { isLocal: boolean; @@ -117,7 +120,12 @@ function mockResolver(opts: { }; } -function mockConnection(opts: { remoteSeeds?: Readonly> }) { +function mockConnection(opts: { + remoteSeeds?: Readonly>; + /** When set, an `exec` whose SQL contains this substring fails instead of succeeding. */ + execFailsOn?: string; + execFailsMessage?: string; +}) { const execs: Array = []; const queries: Array<{ sql: string; params?: ReadonlyArray }> = []; const layer = Layer.succeed(LegacyDbConnection, { @@ -127,9 +135,13 @@ function mockConnection(opts: { remoteSeeds?: Readonly> } copyToCsv: () => Effect.succeed(new Uint8Array()), queryRaw: () => Effect.succeed({ fields: [], rows: [], commandTag: "" }), exec: (sql: string): Effect.Effect => - Effect.sync(() => { - execs.push(sql); - }), + opts.execFailsOn !== undefined && sql.includes(opts.execFailsOn) + ? Effect.fail( + new LegacyDbExecError({ message: opts.execFailsMessage ?? "syntax error" }), + ) + : Effect.sync(() => { + execs.push(sql); + }), query: ( sql: string, params?: ReadonlyArray, @@ -267,42 +279,6 @@ const mockStorageHttp = Layer.succeed( ), ); -/** - * `execCaptureExitCode`, when set, makes `execCapture` fail with a - * `LegacyGoChildExitError` carrying that code instead of succeeding — simulating - * a delegated Go child exiting non-zero under a machine-output mode (CLI-1879). - */ -function mockProxy(opts: { execCaptureExitCode?: number } = {}) { - const calls: Array<{ args: ReadonlyArray; env?: Record }> = []; - const layer = Layer.succeed(LegacyGoProxy, { - exec: (args, execOpts) => - Effect.sync(() => { - calls.push({ args, env: execOpts?.env }); - }), - execCapture: (args, execOpts) => - Effect.sync(() => { - calls.push({ args, env: execOpts?.env }); - }).pipe( - Effect.flatMap(() => - opts.execCaptureExitCode !== undefined - ? Effect.fail( - new LegacyGoChildExitError({ - exitCode: opts.execCaptureExitCode, - message: `supabase-go exited with code ${opts.execCaptureExitCode}`, - }), - ) - : Effect.succeed(""), - ), - ), - }); - return { - layer, - get calls() { - return calls; - }, - }; -} - function setup( workdir: string, opts: { @@ -315,13 +291,18 @@ function setup( ref?: string; experimental?: boolean; remoteSeeds?: Readonly>; + execFailsOn?: string; + execFailsMessage?: string; yes?: boolean; omitRef?: boolean; resolveFails?: boolean; running?: boolean; storageReady?: boolean; awaitStorageReadyExitCode?: number; - execCaptureExitCode?: number; + // pg-delta migrations-catalog cache (Go's `down.ResetAll` → `pgcache.TryCacheMigrationsCatalog`, + // wired into the remote-reset path after a successful migrate/schema-files + seed). + catalogStdout?: string; + catalogExportFailWith?: string; }, ) { if (opts.toml !== undefined) { @@ -336,7 +317,6 @@ function setup( const out = mockOutput({ format: opts.format ?? "text", promptConfirmResponses: opts.confirm }); const conn = mockConnection(opts); - const proxy = mockProxy({ execCaptureExitCode: opts.execCaptureExitCode }); const seam = mockBootstrapSeam({ storageReady: opts.storageReady, awaitStorageReadyExitCode: opts.awaitStorageReadyExitCode, @@ -354,10 +334,30 @@ function setup( resolveFails: opts.resolveFails, }); + const edgeRunCalls: Array = []; + const registryEnvAtRunTime: Array = []; + const edge = Layer.succeed(LegacyEdgeRuntimeScript, { + run: (runOpts: LegacyEdgeRuntimeRunOpts) => { + edgeRunCalls.push(runOpts); + registryEnvAtRunTime.push(process.env["SUPABASE_INTERNAL_IMAGE_REGISTRY"]); + if (opts.catalogExportFailWith !== undefined) { + return Effect.fail( + new LegacyEdgeRuntimeScriptError({ message: opts.catalogExportFailWith }), + ); + } + return Effect.succeed({ stdout: opts.catalogStdout ?? '{"version":1}', stderr: "" }); + }, + }); + const sslProbe = Layer.succeed(LegacyPgDeltaSslProbe, { + requireSsl: () => Effect.succeed(false), + requireSslForHost: () => Effect.succeed(false), + }); + const layer = Layer.mergeAll( out.layer, conn.layer, - proxy.layer, + edge, + sslProbe, seam.layer, resolver.layer, mockLegacyCliConfig({ workdir }), @@ -390,7 +390,17 @@ function setup( telemetry.layer, linkedCache.layer, ); - return { layer, out, conn, proxy, seam, telemetry, linkedCache, resolver }; + return { + layer, + out, + conn, + seam, + telemetry, + linkedCache, + resolver, + edgeRunCalls, + registryEnvAtRunTime, + }; } const migrationFile = (version: string, body = "create table t ();") => ({ @@ -401,7 +411,7 @@ describe("legacy db reset", () => { const tmp = useLegacyTempWorkdir("supabase-db-reset-"); it.live("resets the local database via the bootstrap seam", () => { - const { layer, out, seam, proxy } = setup(tmp.current, { + const { layer, out, seam } = setup(tmp.current, { toml: 'project_id = "test"\n', args: ["db", "reset"], isLocal: true, @@ -409,8 +419,6 @@ describe("legacy db reset", () => { }); return Effect.gen(function* () { yield* legacyDbReset(DEFAULT_FLAGS).pipe(Effect.provide(layer)); - // Native path — no Go delegation. - expect(proxy.calls).toHaveLength(0); expect(out.stderrText).toContain("Resetting local database..."); expect(seam.recreateCalls).toEqual([{ version: "", noSeed: false, sqlPaths: [] }]); // Storage gate checked; with no buckets configured nothing is seeded. @@ -831,6 +839,132 @@ describe("legacy db reset", () => { }); }); + it.live( + "caches the migrations catalog after a successful remote reset with SUPABASE_EXPERIMENTAL_PG_DELTA set", + () => { + // Go's `down.ResetAll` (`internal/migration/down/down.go:48-61`, the function + // `resetRemote` delegates to) best-effort caches the pg-delta migrations + // catalog right after `apply.MigrateAndSeed` succeeds — gated on + // `pgcache.ShouldCacheMigrationsCatalog()` (`experimental.pgdelta.enabled` OR + // the legacy `SUPABASE_EXPERIMENTAL_PG_DELTA` env switch), independent of + // `--experimental`'s own schema-files gate. + const { layer, out, edgeRunCalls } = setup(tmp.current, { + toml: 'project_id = "test"\n', + files: { + ...migrationFile("20240101000000"), + "supabase/.env": "SUPABASE_EXPERIMENTAL_PG_DELTA=true\n", + }, + confirm: [true], + }); + return Effect.gen(function* () { + yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe(Effect.provide(layer)); + expect(out.stderrText).not.toContain("failed to cache migrations catalog"); + expect(edgeRunCalls).toHaveLength(1); + }); + }, + ); + + it.live( + "resolves the pg-delta cache export image via SUPABASE_INTERNAL_IMAGE_REGISTRY from supabase/.env", + () => { + // Go's `loadNestedEnv` (`os.Setenv`) makes a `supabase/.env`-only + // `SUPABASE_INTERNAL_IMAGE_REGISTRY` visible to the WHOLE reset run, including + // the pg-delta catalog export the reset handler triggers after a successful + // remote reset (review CLI-1958 round 18) — mirroring `db push`'s own + // `legacyApplyProjectEnv(projectEnv)` scoping (same-named test in + // `push.integration.test.ts`). Without that scoping, this reads only real + // `process.env` and falls back to the default registry instead. + const prev = process.env["SUPABASE_INTERNAL_IMAGE_REGISTRY"]; + delete process.env["SUPABASE_INTERNAL_IMAGE_REGISTRY"]; + const { layer, registryEnvAtRunTime } = setup(tmp.current, { + toml: 'project_id = "test"\n[experimental.pgdelta]\nenabled = true\n', + files: { + ...migrationFile("20240101000000"), + "supabase/.env": "SUPABASE_INTERNAL_IMAGE_REGISTRY=my-mirror.example.com\n", + }, + confirm: [true], + }); + return Effect.gen(function* () { + yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe(Effect.provide(layer)); + expect(registryEnvAtRunTime).toEqual(["my-mirror.example.com"]); + // The finalizer reverted it — never leaks into the surrounding process. + expect(process.env["SUPABASE_INTERNAL_IMAGE_REGISTRY"]).toBeUndefined(); + }).pipe( + Effect.ensuring( + Effect.sync(() => { + if (prev === undefined) delete process.env["SUPABASE_INTERNAL_IMAGE_REGISTRY"]; + else process.env["SUPABASE_INTERNAL_IMAGE_REGISTRY"] = prev; + }), + ), + ); + }, + ); + + it.live("warns without failing the reset when the migrations-catalog cache write fails", () => { + const { layer, out } = setup(tmp.current, { + toml: 'project_id = "test"\n[experimental.pgdelta]\nenabled = true\n', + files: migrationFile("20240101000000"), + confirm: [true], + catalogExportFailWith: "edge-runtime script produced no output", + }); + return Effect.gen(function* () { + const exit = yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe( + Effect.provide(layer), + Effect.exit, + ); + expect(Exit.isSuccess(exit)).toBe(true); + expect(out.stderrText).toContain( + "Warning: failed to cache migrations catalog: edge-runtime script produced no output", + ); + }); + }); + + it.live( + "falls back to the linked project ref for the pg-delta cache when config.toml has no project_id", + () => { + // Go's `flags.LoadConfig` seeds `Config.ProjectId = ProjectRef` BEFORE + // `Config.Load` runs, so on the linked remote path an absent `project_id` + // retains the linked ref rather than falling to the workdir basename. + const { layer, out, edgeRunCalls } = setup(tmp.current, { + toml: "[experimental.pgdelta]\nenabled = true\n", + ref: LEGACY_VALID_REF, + files: migrationFile("20240101000000"), + confirm: [true], + }); + return Effect.gen(function* () { + yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe(Effect.provide(layer)); + expect(out.stderrText).not.toContain("failed to cache migrations catalog"); + expect(edgeRunCalls).toHaveLength(1); + }); + }, + ); + + it.live( + "skips the migrations-catalog cache for a versioned remote reset even with pg-delta caching enabled", + () => { + // `pgcache.TryCacheMigrationsCatalog` no-ops on any non-empty `version` + // (`pgcache/cache.go:73`, `len(version) > 0`) — a `--version`/`--last` reset + // never refreshes the cache, unlike a full (versionless) reset. + const { layer, out, edgeRunCalls } = setup(tmp.current, { + toml: 'project_id = "test"\n[experimental.pgdelta]\nenabled = true\n', + files: { + ...migrationFile("20240101000000"), + ...migrationFile("20240202000000"), + }, + confirm: [true], + }); + return Effect.gen(function* () { + yield* legacyDbReset({ + ...DEFAULT_FLAGS, + linked: true, + version: Option.some("20240101000000"), + }).pipe(Effect.provide(layer)); + expect(out.stderrText).not.toContain("failed to cache migrations catalog"); + expect(edgeRunCalls).toHaveLength(0); + }); + }, + ); + it.live("drops schemas and applies migrations + seed on a confirmed remote reset", () => { const { layer, out, conn, linkedCache } = setup(tmp.current, { toml: 'project_id = "test"\n', @@ -1004,80 +1138,310 @@ describe("legacy db reset", () => { }); }); - it.live("delegates an experimental remote reset to the Go binary", () => { - const { layer, proxy } = setup(tmp.current, { - toml: 'project_id = "test"\n', - experimental: true, - }); - return Effect.gen(function* () { - yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe(Effect.provide(layer)); - expect(proxy.calls).toHaveLength(1); - expect(proxy.calls[0]!.args).toEqual(["db", "reset", "--linked", "--yes=false"]); - expect(proxy.calls[0]!.env).toEqual({ SUPABASE_TELEMETRY_DISABLED: "1" }); - }); - }); + it.live( + "applies configured schema files instead of replaying migrations on an experimental remote reset", + () => { + // `--linked=false` still selects the linked/remote target (Cobra `Changed` + // semantics) — exercised here alongside the schema-files branch itself. + const { layer, out, conn, resolver, linkedCache } = setup(tmp.current, { + toml: 'project_id = "test"\n\n[db.migrations]\nschema_paths = ["schemas/*.sql"]\n', + files: { + "supabase/schemas/01_users.sql": "create table schema_users ();", + ...migrationFile("20240101000000", "create table migrated_table ();"), + "supabase/seed.sql": "insert into t values (1);", + }, + experimental: true, + args: ["db", "reset", "--linked=false"], + confirm: [true], + ref: LEGACY_VALID_REF, + }); + return Effect.gen(function* () { + yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: false }).pipe(Effect.provide(layer)); + // The configured schema file ran... + expect(conn.execs.some((s) => s.includes("create table schema_users"))).toBe(true); + // ...but the timestamped migration did NOT — Go's `if`/`else if` is mutually + // exclusive (`apply.go:19-27`); taking the schema-files branch means migrations + // never run at all. + expect(conn.execs.some((s) => s.includes("create table migrated_table"))).toBe(false); + expect(out.stderrText).not.toContain("Applying migration"); + // Seeding still runs afterward — Go's `applySeedFiles` sits outside the + // if/else if (`apply.go:26`). + expect(out.stderrText).toContain("Seeding data from supabase/seed.sql..."); + // A real connection is resolved now — this is a fully native path, not a + // delegated one that discarded the resolve (CLI-1958 removed the delegate). + expect(resolver.calls).toBe(1); + expect(linkedCache.cached).toBe(true); + expect(linkedCache.cachedRef).toBe(LEGACY_VALID_REF); + }); + }, + ); - it.live("does not resolve a linked DB connection before delegating an experimental reset", () => { - const { layer, proxy, resolver } = setup(tmp.current, { - toml: 'project_id = "test"\n', - experimental: true, - }); - return Effect.gen(function* () { - yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe(Effect.provide(layer)); - expect(proxy.calls).toHaveLength(1); - // The delegated Go child re-runs its own connection resolution (including - // minting/verifying the temp login role) once it starts — the TS wrapper - // must not do that same Management-API work first only to discard it (CLI-1879). - expect(resolver.calls).toBe(0); - }); - }); + it.live( + "applies schema files across multiple schema_paths patterns in declaration order, sorted within each pattern", + () => { + // Go sorts matches WITHIN each pattern (`sort.Strings`, `config.go:155`) but + // preserves DECLARATION order ACROSS patterns (no global re-sort) — `zz/*.sql`'s + // files must all run before `aa/*.sql`'s, even though "aa" sorts before "zz". + const { layer, conn } = setup(tmp.current, { + toml: 'project_id = "test"\n\n[db.migrations]\nschema_paths = ["zz/*.sql", "aa/*.sql"]\n', + files: { + "supabase/zz/b.sql": "create table zz_b ();", + "supabase/zz/a.sql": "create table zz_a ();", + "supabase/aa/b.sql": "create table aa_b ();", + "supabase/aa/a.sql": "create table aa_a ();", + }, + experimental: true, + confirm: [true], + }); + return Effect.gen(function* () { + yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe(Effect.provide(layer)); + const order = conn.execs + .map((s) => /create table (\w+) \(\)/.exec(s)?.[1]) + .filter((name): name is string => name !== undefined); + expect(order).toEqual(["zz_a", "zz_b", "aa_a", "aa_b"]); + }); + }, + ); - it.live("still caches the linked ref when delegating an experimental reset", () => { - // `linkedRefForCache` is pre-loaded via `LegacyProjectRefResolver.loadProjectRef` - // separately from `resolver.resolve()`, specifically so the post-run - // linked-project-cache finalizer still fires on this path even though - // `resolve()` itself is skipped entirely (CLI-1879). - const { layer, linkedCache } = setup(tmp.current, { - toml: 'project_id = "test"\n', + it.live( + "expands a schema_paths directory entry to its nested .sql files on an experimental remote reset", + () => { + // `[db.migrations].schema_paths` resolves through Go's `Glob.SQLFiles` (not + // `Glob.Files`), which expands a directory match to its regular `.sql` files, + // recursively — unlike a plain glob pattern. + const { layer, conn } = setup(tmp.current, { + toml: 'project_id = "test"\n\n[db.migrations]\nschema_paths = ["some-dir"]\n', + files: { + "supabase/some-dir/01_top.sql": "create table dir_top ();", + "supabase/some-dir/nested/02_nested.sql": "create table dir_nested ();", + }, + experimental: true, + confirm: [true], + }); + return Effect.gen(function* () { + yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe(Effect.provide(layer)); + expect(conn.execs.some((s) => s.includes("create table dir_top"))).toBe(true); + expect(conn.execs.some((s) => s.includes("create table dir_nested"))).toBe(true); + }); + }, + ); + + it.live( + "silently applies nothing when schema_paths is unset on an experimental remote reset (Go's undocumented default-config behavior)", + () => { + // Go's `schema_paths` default is `[]` (`pkg/config/templates/config.toml:64`). + // With no patterns to glob, `SQLFiles` returns a nil error, so `applySchemaFiles` + // is a silent no-op — Go does NOT fall back to replaying migrations (`apply.go: + // 19-27` is a hard `if`/`else if`). + const { layer, out, conn } = setup(tmp.current, { + toml: 'project_id = "test"\n', + files: migrationFile("20240101000000", "create table migrated_table ();"), + experimental: true, + confirm: [true], + }); + return Effect.gen(function* () { + yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe(Effect.provide(layer)); + // Schemas are still dropped (ResetAll drops before MigrateAndSeed)... + expect(conn.execs.some((s) => s.includes("drop schema if exists"))).toBe(true); + // ...but the local migration is silently skipped, not applied. + expect(conn.execs.some((s) => s.includes("create table migrated_table"))).toBe(false); + expect(out.stderrText).not.toContain("Applying migration"); + }); + }, + ); + + it.live( + "replays migrations instead of schema files on an experimental remote reset when pg-delta is enabled", + () => { + const { layer, out, conn } = setup(tmp.current, { + toml: 'project_id = "test"\n\n[db.migrations]\nschema_paths = ["schemas/*.sql"]\n\n[experimental.pgdelta]\nenabled = true\n', + files: { + "supabase/schemas/01_users.sql": "create table schema_users ();", + ...migrationFile("20240101000000", "create table migrated_table ();"), + }, + experimental: true, + confirm: [true], + }); + return Effect.gen(function* () { + yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe(Effect.provide(layer)); + // `IsPgDeltaEnabled()` disables the schema-files branch (`apply.go:19`) even + // though `--experimental` and `schema_paths` are both set. + expect(conn.execs.some((s) => s.includes("create table migrated_table"))).toBe(true); + expect(conn.execs.some((s) => s.includes("create table schema_users"))).toBe(false); + expect(out.stderrText).toContain("Applying migration"); + }); + }, + ); + + it.live( + "replays migrations instead of schema files on an experimental remote reset with a resolved version", + () => { + const { layer, conn } = setup(tmp.current, { + toml: 'project_id = "test"\n\n[db.migrations]\nschema_paths = ["schemas/*.sql"]\n', + files: { + "supabase/schemas/01_users.sql": "create table schema_users ();", + ...migrationFile("20240101000000", "create table migrated_table ();"), + }, + experimental: true, + confirm: [true], + }); + return Effect.gen(function* () { + yield* legacyDbReset({ + ...DEFAULT_FLAGS, + linked: true, + version: Option.some("20240101000000"), + }).pipe(Effect.provide(layer)); + // A resolved --version disables the schema-files branch (`apply.go:19` requires + // `len(version) == 0`), even with `--experimental` set. + expect(conn.execs.some((s) => s.includes("create table migrated_table"))).toBe(true); + expect(conn.execs.some((s) => s.includes("create table schema_users"))).toBe(false); + }); + }, + ); + + it.live( + "fails an experimental remote reset when no schema_paths pattern matches anything", + () => { + const { layer, conn } = setup(tmp.current, { + toml: 'project_id = "test"\n\n[db.migrations]\nschema_paths = ["nomatch/*.sql"]\n', + experimental: true, + confirm: [true], + }); + return Effect.gen(function* () { + const exit = yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe( + Effect.provide(layer), + Effect.exit, + ); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const cause = JSON.stringify(exit.cause); + expect(cause).toContain("no files matched pattern: supabase/nomatch/*.sql"); + // No CmdSuggestion on this failure mode — only a per-file exec failure sets one. + expect(cause).not.toContain("See schema file"); + } + // Schemas were already dropped before the failed apply step (Go's ResetAll order). + expect(conn.execs.some((s) => s.includes("drop schema if exists"))).toBe(true); + }); + }, + ); + + it.live("ignores a partial schema_paths glob failure once at least one pattern matches", () => { + // Go's `applySchemaFiles` only surfaces the joined glob error when NO pattern + // matched anything at all (`apply.go:53-55`); a partial failure is silently dropped. + const { layer, out, conn } = setup(tmp.current, { + toml: 'project_id = "test"\n\n[db.migrations]\nschema_paths = ["schemas/*.sql", "typo/*.sql"]\n', + files: { + "supabase/schemas/01_users.sql": "create table schema_users ();", + // Present so the (unrelated) seed glob's own "no files matched" WARN line + // doesn't show up and get confused with the schema-files warning below. + "supabase/seed.sql": "insert into t values (1);", + }, experimental: true, - ref: LEGACY_VALID_REF, + confirm: [true], }); return Effect.gen(function* () { yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe(Effect.provide(layer)); - expect(linkedCache.cached).toBe(true); - expect(linkedCache.cachedRef).toBe(LEGACY_VALID_REF); + expect(conn.execs.some((s) => s.includes("create table schema_users"))).toBe(true); + expect(out.stderrText).not.toContain("no files matched pattern"); }); }); it.live( - "surfaces a delegated experimental-reset child failure as a LegacyGoChildExitError under json output", + "attaches Go's schema-file suggestion when a schema file fails to apply on an experimental remote reset", () => { const { layer } = setup(tmp.current, { - toml: 'project_id = "test"\n', + toml: 'project_id = "test"\n\n[db.migrations]\nschema_paths = ["schemas/*.sql"]\n', + files: { "supabase/schemas/01_users.sql": "not valid sql;" }, experimental: true, - format: "json", - execCaptureExitCode: 3, + confirm: [true], + execFailsOn: "not valid sql", + execFailsMessage: 'syntax error at or near "not"', }); return Effect.gen(function* () { - // Under json/stream-json, the delegated path uses `execCapture` (non-text - // branch of `delegateExperimentalReset`) — this must flow through the normal - // Effect failure channel (reachable by `withJsonErrorHandling` at the - // command-wiring layer) instead of an immediate `ProcessControl.exit()` that a - // handler-level test could never observe (CLI-1879). const exit = yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe( Effect.provide(layer), Effect.exit, ); expect(Exit.isFailure(exit)).toBe(true); if (Exit.isFailure(exit)) { - const error = Cause.squash(exit.cause); - expect(error).toBeInstanceOf(LegacyGoChildExitError); - expect((error as LegacyGoChildExitError).exitCode).toBe(3); + const cause = JSON.stringify(exit.cause); + expect(cause).toContain("syntax error at or near"); + // Go's `CmdSuggestion = "See schema file: "` (`apply.go:63`). + expect(cause).toContain("See schema file:"); + expect(cause).toContain("supabase/schemas/01_users.sql"); } }); }, ); + const isRoot = typeof process.getuid === "function" && process.getuid() === 0; + + it.live.skipIf(isRoot)( + "does not attach the schema-file suggestion when a schema file cannot be READ on an experimental remote reset", + () => { + // Go's `NewMigrationFromFile` (the file-read/parse step, `apply.go:57-59`) returns + // BEFORE `CmdSuggestion` is ever set — only a later `ExecBatch` (statement + // execution) failure attaches it (`apply.go:61-63`). A file that glob-matches but + // can't be read (permissions changed after the glob) must fail WITHOUT the + // suggestion, unlike the exec-failure case above. + const schemaFile = join(tmp.current, "supabase", "schemas", "01_users.sql"); + const { layer, conn } = setup(tmp.current, { + toml: 'project_id = "test"\n\n[db.migrations]\nschema_paths = ["schemas/*.sql"]\n', + files: { "supabase/schemas/01_users.sql": "create table schema_users ();" }, + experimental: true, + confirm: [true], + }); + chmodSync(schemaFile, 0o000); + return Effect.gen(function* () { + const exit = yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe( + Effect.provide(layer), + Effect.exit, + ); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const cause = JSON.stringify(exit.cause); + expect(cause).not.toContain("See schema file"); + } + // The statement was never reached, so it was never executed. + expect(conn.execs.some((s) => s.includes("create table schema_users"))).toBe(false); + }).pipe(Effect.ensuring(Effect.sync(() => chmodSync(schemaFile, 0o644)))); + }, + ); + + it.live.skipIf(isRoot)( + "fails an experimental remote reset (without silently succeeding) when a matched schema_paths directory cannot be walked", + () => { + // Go's `fs.WalkDir` stops on the first `ReadDir` failure and `applySchemaFiles` + // only silently drops that error when at least one OTHER file was still found + // (`apply.go:53-55`); with a single pattern matching only the unreadable + // directory, `declared` stays empty and Go aborts the command — it must not + // report success having applied nothing. Verified empirically against `apps/cli-go`. + const schemasDir = join(tmp.current, "supabase", "schemas"); + const { layer, conn } = setup(tmp.current, { + toml: 'project_id = "test"\n\n[db.migrations]\nschema_paths = ["schemas"]\n', + files: { "supabase/schemas/01_users.sql": "create table schema_users ();" }, + experimental: true, + confirm: [true], + }); + chmodSync(schemasDir, 0o000); + return Effect.gen(function* () { + const exit = yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe( + Effect.provide(layer), + Effect.exit, + ); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const cause = JSON.stringify(exit.cause); + expect(cause).toContain("failed to walk matched directory"); + expect(cause).not.toContain("See schema file"); + } + // Schemas were already dropped before the failed apply step (Go's ResetAll order). + expect(conn.execs.some((s) => s.includes("drop schema if exists"))).toBe(true); + expect(conn.execs.some((s) => s.includes("create table schema_users"))).toBe(false); + }).pipe(Effect.ensuring(Effect.sync(() => chmodSync(schemasDir, 0o755)))); + }, + ); + it.live( "propagates the storage-ready check's exact exit code and still flushes telemetry on a local reset", () => { @@ -1105,76 +1469,29 @@ describe("legacy db reset", () => { }, ); - it.live("forwards the linked selector to the delegate even for --linked=false", () => { - // Cobra `Changed` semantics: `--linked=false` still selects the linked/remote target in - // the parent, so the delegated argv must carry `--linked` — otherwise the Go child falls - // back to its local default and resets the wrong database. - const { layer, proxy } = setup(tmp.current, { - toml: 'project_id = "test"\n', - experimental: true, - args: ["db", "reset", "--linked=false"], - }); - return Effect.gen(function* () { - yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: false }).pipe(Effect.provide(layer)); - expect(proxy.calls).toHaveLength(1); - expect(proxy.calls[0]!.args).toEqual(["db", "reset", "--linked", "--yes=false"]); - }); - }); - - it.live("forwards --yes=false to the delegate even when SUPABASE_YES is set", () => { - // Explicit `--yes=false` beats `AutomaticEnv` in Go; the delegated child must receive the - // bound false flag so an inherited `SUPABASE_YES=true` doesn't auto-confirm the reset and - // drop the remote schemas the user tried to protect. - const previous = process.env["SUPABASE_YES"]; - process.env["SUPABASE_YES"] = "true"; - const { layer, proxy } = setup(tmp.current, { - toml: 'project_id = "test"\n', - experimental: true, - args: ["db", "reset", "--linked", "--yes=false"], - }); - return Effect.gen(function* () { - yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe(Effect.provide(layer)); - expect(proxy.calls[0]!.args).toContain("--yes=false"); - }).pipe( - Effect.ensuring( - Effect.sync(() => { - if (previous === undefined) delete process.env["SUPABASE_YES"]; - else process.env["SUPABASE_YES"] = previous; - }), - ), - ); - }); - - it.live("forwards --yes=true to the delegate when --yes is set", () => { - const { layer, proxy } = setup(tmp.current, { - toml: 'project_id = "test"\n', - experimental: true, - args: ["db", "reset", "--linked", "--yes"], - yes: true, - }); - return Effect.gen(function* () { - yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe(Effect.provide(layer)); - expect(proxy.calls[0]!.args).toContain("--yes=true"); - }); - }); - it.live( - "takes the experimental delegate path via SUPABASE_EXPERIMENTAL in the project .env", + "takes the native experimental schema-files path via SUPABASE_EXPERIMENTAL in the project .env", () => { - // Go loads nested env before reset.Run reads viper EXPERIMENTAL, so the versionless remote - // reset delegates to the Go binary rather than replaying migrations natively. + // Go loads nested env before `reset.Run` reads viper's EXPERIMENTAL, so a + // `SUPABASE_EXPERIMENTAL` set only in `supabase/.env` reaches the native + // three-conjunct gate the same way an explicit `--experimental` does. const previous = process.env["SUPABASE_EXPERIMENTAL"]; delete process.env["SUPABASE_EXPERIMENTAL"]; - const { layer, proxy, conn } = setup(tmp.current, { - toml: 'project_id = "test"\n', - files: { "supabase/.env": "SUPABASE_EXPERIMENTAL=true\n" }, + const { layer, out, conn } = setup(tmp.current, { + toml: 'project_id = "test"\n\n[db.migrations]\nschema_paths = ["schemas/*.sql"]\n', + files: { + "supabase/.env": "SUPABASE_EXPERIMENTAL=true\n", + "supabase/schemas/01_users.sql": "create table schema_users ();", + ...migrationFile("20240101000000", "create table migrated_table ();"), + }, + confirm: [true], // No experimental flag / shell env — only the project .env sets it. }); return Effect.gen(function* () { yield* legacyDbReset({ ...DEFAULT_FLAGS, linked: true }).pipe(Effect.provide(layer)); - expect(proxy.calls).toHaveLength(1); - // Delegated, so the native remote path never dropped schemas. - expect(conn.execs.some((s) => s.includes("drop schema if exists"))).toBe(false); + expect(conn.execs.some((s) => s.includes("create table schema_users"))).toBe(true); + expect(conn.execs.some((s) => s.includes("create table migrated_table"))).toBe(false); + expect(out.stderrText).not.toContain("Applying migration"); }).pipe( Effect.ensuring( Effect.sync(() => { @@ -1203,32 +1520,30 @@ describe("legacy db reset", () => { }); }); - it.live("forwards --db-url and --no-seed on an experimental remote db-url reset", () => { - const { layer, proxy, resolver } = setup(tmp.current, { - toml: 'project_id = "test"\n', - experimental: true, - args: ["db", "reset", "--db-url", "postgresql://db.example.com:5432/postgres"], - }); - return Effect.gen(function* () { - yield* legacyDbReset({ - ...DEFAULT_FLAGS, - dbUrl: Option.some("postgresql://db.example.com:5432/postgres"), - noSeed: true, - }).pipe(Effect.provide(layer)); - expect(proxy.calls[0]!.args).toEqual([ - "db", - "reset", - "--db-url", - "postgresql://db.example.com:5432/postgres", - "--no-seed", - "--yes=false", - ]); - // Unlike the `connType === "linked"` branch above, a `--db-url` target still - // resolves a connection before delegating — the pre-delegation skip (CLI-1879) - // is scoped to the linked branch only, not "never call resolve when delegating". - expect(resolver.calls).toBe(1); - }); - }); + it.live( + "applies configured schema files and skips seeding on an experimental remote --db-url reset", + () => { + const { layer, conn, resolver } = setup(tmp.current, { + toml: 'project_id = "test"\n\n[db.migrations]\nschema_paths = ["schemas/*.sql"]\n', + files: { "supabase/schemas/01_users.sql": "create table schema_users ();" }, + experimental: true, + args: ["db", "reset", "--db-url", "postgresql://db.example.com:5432/postgres"], + confirm: [true], + }); + return Effect.gen(function* () { + yield* legacyDbReset({ + ...DEFAULT_FLAGS, + dbUrl: Option.some("postgresql://db.example.com:5432/postgres"), + noSeed: true, + }).pipe(Effect.provide(layer)); + expect(conn.execs.some((s) => s.includes("create table schema_users"))).toBe(true); + expect(conn.execs.some((s) => s.includes("insert into"))).toBe(false); + // A `--db-url` target always resolved a real connection, same as before — + // this is no longer delegated at all (CLI-1958). + expect(resolver.calls).toBe(1); + }); + }, + ); it.live("passes --no-seed and the resolved --last version to the recreate seam", () => { const { layer, seam } = setup(tmp.current, { @@ -1463,25 +1778,30 @@ describe("legacy db reset", () => { }); }); - it.live("forwards --sql-paths to the Go binary on an experimental remote reset", () => { - const { layer, proxy } = setup(tmp.current, { - toml: 'project_id = "test"\n', - experimental: true, - }); - return Effect.gen(function* () { - yield* legacyDbReset({ - ...DEFAULT_FLAGS, - linked: true, - sqlPaths: ["custom-seed.sql"], - }).pipe(Effect.provide(layer)); - expect(proxy.calls[0]!.args).toEqual([ - "db", - "reset", - "--linked", - "--sql-paths", - "custom-seed.sql", - "--yes=false", - ]); - }); - }); + it.live( + "seeds from --sql-paths on an experimental remote reset, independently of the schema-files apply", + () => { + // `--sql-paths` overrides `[db.seed].sql_paths` regardless of which branch of + // `apply.MigrateAndSeed` ran — Go's `applySeedFiles` sits outside the if/else if + // (`apply.go:26`), and the seed override is resolved entirely upstream of it. + const { layer, out, conn } = setup(tmp.current, { + toml: 'project_id = "test"\n\n[db.migrations]\nschema_paths = ["schemas/*.sql"]\n', + files: { + "supabase/schemas/01_users.sql": "create table schema_users ();", + "supabase/custom-seed.sql": "insert into t values (2);", + }, + experimental: true, + confirm: [true], + }); + return Effect.gen(function* () { + yield* legacyDbReset({ + ...DEFAULT_FLAGS, + linked: true, + sqlPaths: ["custom-seed.sql"], + }).pipe(Effect.provide(layer)); + expect(conn.execs.some((s) => s.includes("create table schema_users"))).toBe(true); + expect(out.stderrText).toContain("Seeding data from supabase/custom-seed.sql..."); + }); + }, + ); }); diff --git a/apps/cli/src/legacy/commands/db/reset/reset.layers.ts b/apps/cli/src/legacy/commands/db/reset/reset.layers.ts index 7cf648f3fe..6bfb86e9af 100644 --- a/apps/cli/src/legacy/commands/db/reset/reset.layers.ts +++ b/apps/cli/src/legacy/commands/db/reset/reset.layers.ts @@ -9,8 +9,11 @@ import { legacyProjectRefLayer } from "../../../config/legacy-project-ref.layer. import { legacyDbConfigLayer } from "../../../shared/legacy-db-config.layer.ts"; import { legacyDbConnectionLayer } from "../../../shared/legacy-db-connection.layer.ts"; import { legacyDebugLoggerLayer } from "../../../shared/legacy-debug-logger.layer.ts"; +import { legacyDockerRunLayer } from "../../../shared/legacy-docker-run.layer.ts"; +import { legacyEdgeRuntimeScriptLayer } from "../../../shared/legacy-edge-runtime-script.layer.ts"; import { stdinLayer } from "../../../../shared/runtime/stdin.layer.ts"; import { legacyIdentityStitchLayer } from "../../../shared/legacy-identity-stitch.ts"; +import { legacyPgDeltaSslProbeLayer } from "../../../shared/legacy-pgdelta-ssl-probe.layer.ts"; import { legacyLinkedProjectCacheLayer } from "../../../telemetry/legacy-linked-project-cache.layer.ts"; import { legacyTelemetryStateLayer } from "../../../telemetry/legacy-telemetry-state.layer.ts"; import { legacyDbBootstrapSeamLayer } from "../shared/legacy-db-bootstrap.seam.layer.ts"; @@ -19,8 +22,22 @@ import { legacyDbBootstrapSeamLayer } from "../shared/legacy-db-bootstrap.seam.l * Runtime layer for `supabase db reset`. Same composition as `db push` / `db lint`: * the Postgres connection, the db-config resolver, project-ref resolution, and the * linked-project cache, all over the lazy management-API factory so the local / - * `--db-url` paths never resolve an access token at layer-build time. `LegacyGoProxy` - * (used to delegate the local / experimental reset paths) is ambient from the root. + * `--db-url` paths never resolve an access token at layer-build time. The remote + * reset path (including the `--experimental` remote schema-files apply, CLI-1958) + * is fully native. The local path's container primitives still reach `LegacyGoProxy` + * through the bootstrap seam below (`legacyDbBootstrapSeamLayer`, ambient from the + * root) — that native port is CLI-1955's scope, not this one. + * + * `legacyDockerRunLayer` + `edgeRuntime` + `legacyPgDeltaSslProbeLayer` (same shape + * as `db push`'s `legacyDbPushRuntimeLayer`) are required by the post-reset + * best-effort pg-delta catalog cache below (`legacyTryCacheMigrationsCatalog` → + * `legacyExportCatalogPgDelta` → `LegacyEdgeRuntimeScript`/`LegacyPgDeltaSslProbe`, + * `legacy-pgdelta.ts`/`legacy-pgdelta-ssl.ts`). Without them, a versionless remote + * reset with pg-delta enabled (`[experimental.pgdelta].enabled` or + * `SUPABASE_EXPERIMENTAL_PG_DELTA`) would hit an unhandled missing-service defect + * — not caught by the handler's typed `Effect.catch` — AFTER the remote database + * has already been reset, instead of writing the catalog or emitting Go's + * best-effort warning (review CLI-1958). */ const cliConfig = legacyCliConfigLayer.pipe(Layer.provide(legacyDebugLoggerLayer)); const httpClient = legacyHttpClientLayer.pipe(Layer.provide(legacyDebugLoggerLayer)); @@ -55,9 +72,17 @@ const dbConfig = legacyDbConfigLayer.pipe( Layer.provide(legacyIdentityStitchLayer), ); +const edgeRuntime = legacyEdgeRuntimeScriptLayer.pipe( + Layer.provide(legacyDockerRunLayer), + Layer.provide(cliConfig), +); + export const legacyDbResetRuntimeLayer = Layer.mergeAll( dbConfig, legacyDbConnectionLayer, + legacyDockerRunLayer, + edgeRuntime, + legacyPgDeltaSslProbeLayer, cliConfig, httpClient, credentials, diff --git a/apps/cli/src/legacy/commands/db/reset/reset.layers.unit.test.ts b/apps/cli/src/legacy/commands/db/reset/reset.layers.unit.test.ts new file mode 100644 index 0000000000..9f7fe90e68 --- /dev/null +++ b/apps/cli/src/legacy/commands/db/reset/reset.layers.unit.test.ts @@ -0,0 +1,146 @@ +/** + * Layer-exposure test for `legacyDbResetRuntimeLayer`. + * + * Regression guard (review CLI-1958): the post-reset best-effort pg-delta + * catalog cache (`legacyTryCacheMigrationsCatalog` in `reset.handler.ts`, gated + * on `[experimental.pgdelta].enabled` / `SUPABASE_EXPERIMENTAL_PG_DELTA`) reaches + * `LegacyEdgeRuntimeScript` and `LegacyPgDeltaSslProbe` via + * `legacyExportCatalogPgDelta` (`legacy-pgdelta.ts`). `legacyDbResetRuntimeLayer` + * previously omitted both services (and the `LegacyDockerRun` layer the real + * edge-runtime implementation needs) — unlike `legacyDbPushRuntimeLayer`, which + * already composes all three. That gap was invisible to `reset.integration.test.ts` + * because that suite drives `legacyDbReset` directly with its own hand-built layer + * (which mocks `LegacyEdgeRuntimeScript`/`LegacyPgDeltaSslProbe` in), bypassing + * `reset.layers.ts` entirely — so a versionless remote reset with pg-delta enabled + * would crash on a missing-service defect (uncaught by the handler's typed + * `Effect.catch`) AFTER the remote database was already reset. This test builds + * the REAL `legacyDbResetRuntimeLayer` (not a mock of the pg-delta services) and + * asserts both are actually present in its context. + * + * See `db/lint/lint.layers.unit.test.ts` for the canonical ambient-stub pattern. + */ + +import { describe, expect, it } from "@effect/vitest"; +import { BunServices } from "@effect/platform-bun"; +import { Effect, Layer, Option } from "effect"; + +import { + mockAnalytics, + mockOutput, + mockProcessControl, + mockRuntimeInfo, + mockStdin, + mockTelemetryRuntime, + mockTty, +} from "../../../../../tests/helpers/mocks.ts"; +import { + mockLegacyCliConfig, + mockLegacyCredentialsLayer, + mockLegacyLinkedProjectCacheLayer, + mockLegacyTelemetryStateLayer, +} from "../../../../../tests/helpers/legacy-mocks.ts"; + +import { CliArgs } from "../../../../shared/cli/cli-args.service.ts"; +import { + LegacyDebugFlag, + LegacyDnsResolverFlag, + LegacyExperimentalFlag, + LegacyNetworkIdFlag, + LegacyOutputFlag, + LegacyProfileFlag, + LegacyWorkdirFlag, +} from "../../../../shared/legacy/global-flags.ts"; + +import { LegacyPlatformApiFactory } from "../../../auth/legacy-platform-api-factory.service.ts"; +import { LegacyProjectRefResolver } from "../../../config/legacy-project-ref.service.ts"; +import { LegacyDbConfigResolver } from "../../../shared/legacy-db-config.service.ts"; +import { LegacyDbConnection } from "../../../shared/legacy-db-connection.service.ts"; +import { LegacyEdgeRuntimeScript } from "../../../shared/legacy-edge-runtime-script.service.ts"; +import { LegacyPgDeltaSslProbe } from "../../../shared/legacy-pgdelta-ssl-probe.service.ts"; + +import { legacyDbResetRuntimeLayer } from "./reset.layers.ts"; + +/** + * Builds a stub ambient layer that satisfies every external service required by + * `legacyDbResetRuntimeLayer` from the root runtime. Services whose logic is not + * under test are no-op stubs; `LegacyEdgeRuntimeScript` and `LegacyPgDeltaSslProbe` + * are deliberately NOT stubbed here — the point of this test is to prove the real + * `legacyDbResetRuntimeLayer` provides them itself. + */ +function ambientStubs() { + const analytics = mockAnalytics(); + const out = mockOutput(); + + const flagLayers = Layer.mergeAll( + Layer.succeed(LegacyDebugFlag, false), + Layer.succeed(LegacyProfileFlag, "supabase"), + Layer.succeed(LegacyWorkdirFlag, Option.none()), + Layer.succeed(LegacyOutputFlag, Option.none()), + Layer.succeed(LegacyDnsResolverFlag, "native"), + Layer.succeed(LegacyNetworkIdFlag, Option.none()), + Layer.succeed(LegacyExperimentalFlag, false), + Layer.succeed(CliArgs, { args: ["db", "reset"] }), + ); + + // Stub out the heavy service layers so layer construction doesn't require a + // real DB, real API, or real credentials. + const heavyServiceStubs = Layer.mergeAll( + Layer.succeed(LegacyDbConnection, { + connect: () => Effect.die("db-connection not needed for layer-exposure test"), + }), + Layer.succeed(LegacyDbConfigResolver, { + resolve: () => Effect.die("db-config-resolver not needed for layer-exposure test"), + resolvePoolerFallback: () => + Effect.die("db-config-resolver not needed for layer-exposure test"), + }), + Layer.succeed(LegacyProjectRefResolver, { + resolve: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), + resolveForLink: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), + resolveOptional: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), + loadProjectRef: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), + promptProjectRef: () => Effect.die("project-ref-resolver not needed for layer-exposure test"), + }), + Layer.succeed(LegacyPlatformApiFactory, { + make: Effect.die("platform-api-factory not needed for layer-exposure test"), + }), + ); + + return Layer.mergeAll( + BunServices.layer, + mockRuntimeInfo(), + mockTty(), + mockProcessControl().layer, + mockStdin(false), + analytics.layer, + mockTelemetryRuntime(), + out.layer, + flagLayers, + mockLegacyCliConfig({ workdir: "/tmp/reset-layers-test" }), + mockLegacyCredentialsLayer, + mockLegacyLinkedProjectCacheLayer, + mockLegacyTelemetryStateLayer, + heavyServiceStubs, + ); +} + +describe("legacyDbResetRuntimeLayer — pg-delta service exposure (regression guard, review CLI-1958)", () => { + it.live( + "exposes LegacyEdgeRuntimeScript so the post-reset pg-delta catalog cache does not crash on a missing-service defect", + () => { + return Effect.gen(function* () { + const edgeRuntime = yield* Effect.serviceOption(LegacyEdgeRuntimeScript); + expect(Option.isSome(edgeRuntime)).toBe(true); + }).pipe(Effect.provide(legacyDbResetRuntimeLayer), Effect.provide(ambientStubs())); + }, + ); + + it.live( + "exposes LegacyPgDeltaSslProbe so the post-reset pg-delta catalog cache does not crash on a missing-service defect", + () => { + return Effect.gen(function* () { + const sslProbe = yield* Effect.serviceOption(LegacyPgDeltaSslProbe); + expect(Option.isSome(sslProbe)).toBe(true); + }).pipe(Effect.provide(legacyDbResetRuntimeLayer), Effect.provide(ambientStubs())); + }, + ); +}); diff --git a/apps/cli/src/legacy/shared/db-bootstrap/db-setup.ts b/apps/cli/src/legacy/shared/db-bootstrap/db-setup.ts index d1c9bbb033..79f8489f84 100644 --- a/apps/cli/src/legacy/shared/db-bootstrap/db-setup.ts +++ b/apps/cli/src/legacy/shared/db-bootstrap/db-setup.ts @@ -95,7 +95,7 @@ */ import type { ProjectConfig } from "@supabase/config"; -import { Clock, Data, Effect, type FileSystem, Option, type Path } from "effect"; +import { Data, Effect, type FileSystem, Option, type Path } from "effect"; import type { ChildProcessSpawner } from "effect/unstable/process/ChildProcessSpawner"; import { Output } from "../../../shared/output/output.service.ts"; @@ -830,7 +830,6 @@ export const legacyStartSetupLocalDatabase = ( }, isLocal: true, migrationsDir: path.join(workdir, "supabase", "migrations"), - nowMillis: yield* Clock.currentTimeMillis, }).pipe( // Best-effort: Go's own `TryCacheMigrationsCatalog` failure only ever warns // (`fmt.Fprintln(os.Stderr, "Warning: failed to cache migrations catalog:", err)`, diff --git a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts index dc2ac7b869..1fe45cdeab 100644 --- a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts +++ b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.ts @@ -101,8 +101,11 @@ export interface LegacyDbTomlValues { /** * `[db.migrations] schema_paths`, default `[]` — resolved (supabase-prefixed when * relative, Go's `path.Join`/`path.Clean`) and `SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS` - * env-overridable exactly like `seed.sqlPaths` below. Only consumed by the - * `--experimental` declarative-schema-files branch of `legacyMigrateAndSeed`. + * env-overridable exactly like `seed.sqlPaths` below, resolved unconditionally (not + * gated on `db.migrations.enabled`). Feeds `apply.MigrateAndSeed`'s EXPERIMENTAL + * declarative branch (`legacyApplySchemaFiles`) — consumed by `legacyMigrateAndSeed` + * (`start`'s fresh-volume setup, `migration down`) and by `db reset`'s own + * `--experimental` remote path. */ readonly schemaPaths: ReadonlyArray; /** `[db.seed]` enabled + supabase-prefixed `sql_paths` globs — used by `down`. */ @@ -187,8 +190,6 @@ const DEFAULT_SHADOW_PORT = 54320; const DEFAULT_MAJOR_VERSION = 17; const DEFAULT_PASSWORD = "postgres"; const DEFAULT_API_SCHEMAS = ["public", "graphql_public"] as const; -/** `[db.migrations] schema_paths` default — Go's `Glob` zero value (`pkg/config/db.go:101`). */ -const DEFAULT_SCHEMA_PATHS: ReadonlyArray = []; /** `[edge_runtime] deno_version` default (`config.toml` template). 2 → the current edge-runtime image. */ const DEFAULT_DENO_VERSION = 2; @@ -532,6 +533,38 @@ function legacyJoinSupabaseSeedPath(pattern: string): string { return out.length === 0 ? "." : out.join("/"); } +/** + * Go's `filepath.IsAbs` on Windows (`internal/filepathlite/path_windows.go`'s + * `IsAbs`/`volumeNameLen`) requires a volume name — a drive letter (`C:\`) or a UNC + * prefix (`\\server\share`) — before a path counts as absolute; a bare leading + * separator (`/schemas`, `\schemas`) has no volume name, so Go treats it as RELATIVE + * and joins it under `supabase/`. `pathSvc.isAbsolute` is backed by `node:path`, which + * selects `path.win32` on an actual Windows host, and Node's win32 `isAbsolute` treats + * a bare leading separator as rooted at the *current drive* — i.e. absolute — so it + * disagrees with Go on exactly this shape. Verified empirically: Node's + * `path.win32.isAbsolute("/schemas/*.sql")` is `true`, while Go's `filepath.IsAbs` on + * the same input is `false` (`volumeNameLen` returns `0` — none of its drive-letter, + * UNC, or device-path cases match a path with no volume component). Only the resolve + * step below (`config.go:970-980`'s literal `!filepath.IsAbs(pattern)` gate for + * `[db.seed].sql_paths`/`[db.migrations].schema_paths`) needs this Go-exact rule — + * real filesystem calls elsewhere in this shell still need the platform's own + * `isAbsolute` to resolve an actual path on disk. + */ +const legacyGoIsAbs = (pathSvc: Path.Path, pattern: string): boolean => { + if (process.platform !== "win32") { + return pathSvc.isAbsolute(pattern); + } + const isSeparator = (c: string | undefined): boolean => c === "/" || c === "\\"; + // Drive-letter volume (`C:\`, `c:/`): Go's `volumeNameLen` accepts any byte before + // `:` (case 2, `path[1] === ':'`), then `IsAbs` requires a separator right after. + if (pattern.length >= 3 && pattern[1] === ":" && isSeparator(pattern[2])) { + return true; + } + // UNC volume (`\\server\share`, `//server/share`): Go's `IsAbs` treats a + // double-separator-prefixed volume as absolute unconditionally. + return pattern.length >= 2 && isSeparator(pattern[0]) && isSeparator(pattern[1]); +}; + /** * Resolves a single seed/schema-paths entry to Go's config-load form: a relative * pattern is joined under `supabase/` (Go's `path.Join`, `config.go:918-921` for @@ -543,7 +576,7 @@ function legacyJoinSupabaseSeedPath(pattern: string): string { * the glob the same resolved paths. */ export const legacyResolveSeedSqlPath = (pathSvc: Path.Path, pattern: string): string => - pattern.length === 0 || pathSvc.isAbsolute(pattern) + pattern.length === 0 || legacyGoIsAbs(pathSvc, pattern) ? pattern : legacyJoinSupabaseSeedPath(pattern); @@ -1917,31 +1950,6 @@ const readDbTomlCore = Effect.fnUntraced(function* ( ? undefined : envOverride("SUPABASE_DB_MIGRATIONS_ENABLED"), ); - // `[db.migrations] schema_paths` — Go default `[]`; overridable by - // `SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS` via viper AutomaticEnv (`config.go:494-498`) — EXCEPT - // when the matched remote block explicitly set it, same tiering as every other field in - // `LEGACY_ENV_OVERRIDABLE_KEYS`. A STRING value (the env override, or a TOML string) is - // env-expanded then comma-split; a TOML ARRAY is expanded element-by-element with no - // re-split (`resolveStringSlice`, shared with `api.schemas`). Each resulting pattern is then - // resolved to Go's config-load form (`path.Join(builder.SupabaseDirPath, pattern)`, - // `config.go:976-978`) via the same `legacyResolveSeedSqlPath` helper `db.seed.sql_paths` uses - // below — this is the only current TS reader of this field that needs real, Go-path-cleaned - // filesystem paths, so resolution happens here rather than in the declarative-schema-files - // consumer (`legacy-migrate-and-seed.ts`), matching where `seedSqlPaths` is resolved. - const rawSchemaPaths = - (remoteOverrideKeys.has("db.migrations.schema_paths") - ? undefined - : envOverride("SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS")) ?? migrationsRaw?.["schema_paths"]; - const schemaPathPatterns = resolveStringSlice(rawSchemaPaths, DEFAULT_SCHEMA_PATHS, lookup); - if (schemaPathPatterns === undefined) { - return yield* Effect.fail( - new LegacyDbConfigLoadError({ - message: "failed to parse config: invalid db.migrations.schema_paths.", - }), - ); - } - const schemaPaths = schemaPathPatterns.map((pattern) => legacyResolveSeedSqlPath(path, pattern)); - // `[db.seed]` — Go defaults enabled true, sql_paths ["seed.sql"]; relative // patterns are supabase-prefixed (`config.go:801-806`). `db.seed.enabled` is // overridable by `SUPABASE_DB_SEED_ENABLED` via viper AutomaticEnv — EXCEPT when a @@ -1968,23 +1976,271 @@ const readDbTomlCore = Effect.fnUntraced(function* ( const expanded = legacyExpandEnv(value, lookup); return expanded.length === 0 ? [] : expanded.split(","); }; + // Go's `decodeString` renders a weakly-converted float via `strconv.FormatFloat(v, + // 'f', -1, 64)` (`mapstructure.go:747-748`, format `'f'`) — ALWAYS fixed decimal + // notation, never scientific, regardless of magnitude. JS's `String(value)` agrees + // for ordinary magnitudes (both use the same shortest-round-trip digit sequence — + // `String()` just chooses "e" notation once `|value| >= 1e21` or `< 1e-6`, which + // `FormatFloat('f', …)` never does). Verified empirically: `strconv.FormatFloat(1e21, + // 'f', -1, 64)` returns `"1000000000000000000000"`, not `"1e+21"`. Expand JS's own + // exponential notation back into fixed notation instead of re-deriving the digits, + // since `Number.prototype.toString()`/`toExponential()` already computed the same + // shortest round-tripping digit sequence Go's algorithm would — only the notation + // differs. + // + // `strconv.FormatFloat` special-cases the three non-finite values BEFORE it ever + // looks at the format verb, so `'f'` never applies to them: verified empirically + // (`apps/cli-go` probe against a real `schema_paths = [inf, -inf, nan]` config load) — + // `+Inf`/`-Inf`/`NaN` (note the "+Inf" sign Go always prints, and the short "Inf"/"NaN" + // spelling) — never JS's own `Infinity`/`-Infinity`/`NaN` (which happens to already + // match the "NaN" case, but not the two `Infinity` ones). TOML v1.0's bare `inf`/ + // `+inf`/`-inf`/`nan` float literals (smol-toml) parse to exactly these JS values, so + // a `schema_paths`/`sql_paths` array entry can realistically hit this branch. + // + // `strconv.FormatFloat` also preserves the IEEE754 sign bit on zero: a genuine + // negative-zero float64 formats as `"-0"`, never `"0"`. Verified empirically — + // `strconv.FormatFloat(math.Copysign(0, -1), 'f', -1, 64)` returns `"-0"` — and + // end-to-end through the real decode pipeline this weak-decode mirrors + // (`BurntSushi/toml` + `go-viper/mapstructure`'s `WeaklyTypedInput`): a + // `schema_paths = [-0.0]` config decodes its glob entry to the literal string + // `"-0"`. JS's own `(-0).toString()` is `"0"` (the sign is dropped), by spec — + // `Object.is(value, -0)` is JS's only way to detect it, since `-0 === 0`. + const legacyFormatGoWeakFloat = (value: number): string => { + if (Number.isNaN(value)) return "NaN"; + if (value === Number.POSITIVE_INFINITY) return "+Inf"; + if (value === Number.NEGATIVE_INFINITY) return "-Inf"; + if (Object.is(value, -0)) return "-0"; + const str = value.toString(); + const match = /^(-?)(\d+)(?:\.(\d+))?e([+-]\d+)$/.exec(str); + if (match === null) return str; + const [, sign = "", intPart = "", fracPart = "", expStr = "0"] = match; + const digits = intPart + fracPart; + const pointPos = intPart.length + Number(expStr); + if (pointPos <= 0) return `${sign}0.${"0".repeat(-pointPos)}${digits}`; + if (pointPos >= digits.length) return `${sign}${digits}${"0".repeat(pointPos - digits.length)}`; + return `${sign}${digits.slice(0, pointPos)}.${digits.slice(pointPos)}`; + }; + // Go decodes both `[db.seed].sql_paths` and `[db.migrations].schema_paths` as + // `config.Glob` (`[]string`) through the SAME mapstructure `UnmarshalExact` call + // (`config.go:749-756`), whose decoder config never sets `WeaklyTypedInput: false` + // — viper's `defaultDecoderConfig` defaults it to `true` and nothing here overrides + // it. So a non-string array element isn't dropped: `decodeString` + // (`github.com/go-viper/mapstructure/v2@v2.5.0/mapstructure.go:729-780`) weakly + // converts a bool to `"1"`/`"0"` and a number to its decimal string, THEN the + // result flows through the same env-expand/resolve pipeline as a real string + // entry. Verified empirically against `apps/cli-go` (`schema_paths = [42]` resolves + // to `supabase/42`, `schema_paths = [true]` to `supabase/1`). + const legacyWeakCoerceGlobEntry = (value: unknown): string | undefined => { + if (typeof value === "string") return value; + if (typeof value === "boolean") return value ? "1" : "0"; + if (typeof value === "number") return legacyFormatGoWeakFloat(value); + return undefined; + }; + // A non-scalar element (nested array/table, e.g. `schema_paths = [[]]` or + // `[{path = "x.sql"}]`) is mapstructure's `UnconvertibleTypeError` instead of a weak + // conversion, and mapstructure reports every offending index from the SAME + // `UnmarshalExact` call together, joined by its own `Error.Error()` — which is what + // aborts the entire config load, not just that one array. Verified empirically + // against `apps/cli-go`: `schema_paths = [[]]` / `[{path = "x.sql"}]` both fail with + // `failed to parse config: decoding failed due to the following error(s):\n\n'db. + // migrations.schema_paths[0]' expected type 'string', got unconvertible type + // '[]interface {}'` / `'map[string]interface {}'` respectively — never silently + // dropping the element and continuing with an empty/partial glob list. + // + // A bare TOML datetime (e.g. `schema_paths = 1979-05-27T07:32:00Z`) hits this same + // `UnconvertibleTypeError` path in Go, but as a DIFFERENT Go type per TOML datetime + // variant — `BurntSushi/toml` decodes an offset date-time to stdlib `time.Time` and + // each of the 3 zone-less "local" variants to its own `toml.Local*` wrapper type. + // Verified empirically against the real `apps/cli-go` `config.Load` (review + // CLI-1958): `schema_paths = 1979-05-27T07:32:00Z` → `unconvertible type + // 'time.Time'`; `= 1979-05-27T07:32:00` (no zone) → `'toml.LocalDateTime'`; + // `= 1979-05-27` → `'toml.LocalDate'`; `= 07:32:00` → `'toml.LocalTime'` — same four + // messages whether the datetime is this top-level scalar or an array element. + // `smol-toml` parses every TOML datetime to a `TomlDate` (a `Date` subclass, so + // `typeof`/`Array.isArray` alone can't tell it apart from an inline table) exposing + // exactly the `isDate`/`isTime`/`isDateTime`/`isLocal` discriminators needed to + // reproduce Go's per-variant type name. + const legacyGoTomlDateType = (value: SmolToml.TomlDate): string => { + if (value.isDate()) return "toml.LocalDate"; + if (value.isTime()) return "toml.LocalTime"; + return value.isLocal() ? "toml.LocalDateTime" : "time.Time"; + }; + const legacyGoUnconvertibleType = (value: unknown): string | undefined => + value instanceof SmolToml.TomlDate + ? legacyGoTomlDateType(value) + : Array.isArray(value) + ? "[]interface {}" + : typeof value === "object" && value !== null + ? "map[string]interface {}" + : undefined; + // Pure — returns the mapstructure-style issue strings for a real `Glob` array's + // unconvertible elements WITHOUT failing. Go's `UnmarshalExact` decodes the WHOLE + // config in a SINGLE mapstructure pass: `decodeStructFromMap`'s per-field loop + // (`mapstructure.go:1657-1724`) appends each field's decode error to a shared `errs` + // slice and keeps going — it never stops at the first field's error — then + // `errors.Join(errs...)`-s everything together at the very end + // (`mapstructure.go:1777`). So an invalid `db.migrations.schema_paths` does NOT + // prevent `db.seed.sql_paths` from ALSO being decoded (and erroring) in the same + // pass; both surface together in ONE combined error. Verified empirically against + // `apps/cli-go` (`config.Load` with both fields containing an unconvertible entry, + // e.g. `sql_paths = [[]]` + `schema_paths = [[]]`): the single returned error + // contains BOTH lines, `db.migrations.schema_paths[0]` BEFORE `db.seed.sql_paths[0]` + // — Go's `db` struct declares `Migrations` before `Seed` (`pkg/config/db.go:90-91`), + // and mapstructure iterates struct fields in declaration order, not alphabetically, + // so callers below must combine in that same order before failing once (see + // `legacyFailOnGlobIssues`). + const legacyGlobArrayIssues = ( + keyPath: string, + values: ReadonlyArray, + ): ReadonlyArray => + values.flatMap((value, index) => { + const goType = legacyGoUnconvertibleType(value); + return goType === undefined + ? [] + : [`'${keyPath}[${index}]' expected type 'string', got unconvertible type '${goType}'`]; + }); + // Fails ONCE with every issue collected across BOTH `Glob` fields (see + // `legacyGlobArrayIssues`'s doc comment) — never called per-field, so a config + // invalid in both `db.seed.sql_paths` and `db.migrations.schema_paths` reports both, + // matching Go's single combined `UnmarshalExact` error instead of only the first + // field checked. + const legacyFailOnGlobIssues = ( + issues: ReadonlyArray, + ): Effect.Effect => + issues.length === 0 + ? Effect.void + : fail( + `failed to parse config: decoding failed due to the following error(s):\n\n${issues.join("\n")}`, + ); + // A TOP-LEVEL raw value that is neither an array nor a string (e.g. + // `schema_paths = 42`/`true`, or a stray inline table) still reaches + // mapstructure's `decodeSlice`, which is weakly typed the same way an array + // ELEMENT is (see `legacyWeakCoerceGlobEntry` above): a zero-length map + // decodes straight to an empty slice; anything else is wrapped into a + // synthetic single-element `[]any{value}` and decoded through the exact + // same per-element rules as a real array entry — a scalar weakly coerces, + // an unconvertible value (map/array) fails with `'[0]' expected + // type 'string', got unconvertible type '...'` (mapstructure always + // reports the synthetic wrapped index, which is `0`). Verified empirically + // against `apps/cli-go`: `schema_paths = 42` → `["42"]`, `= true` → + // `["1"]`, `= {}` → `[]`, `[db.migrations.schema_paths]\nfoo = "bar"` → + // `failed to parse config: … 'db.migrations.schema_paths[0]' expected type + // 'string', got unconvertible type 'map[string]interface {}'`. Never + // called with `undefined` — an absent key has its own Go-matching default + // per caller below, so callers guard that case before reaching here. + // + // The zero-length-map special case must NOT match a `TomlDate` (e.g. `schema_paths = + // 1979-05-27T07:32:00Z`): a `TomlDate` stores its value internally, not as an + // enumerable own property, so `Object.keys(tomlDate).length === 0` is ALSO true for + // it — but Go does not treat a bare datetime as an empty map; mapstructure reports it + // unconvertible and aborts the whole load (see `legacyGoUnconvertibleType` above). + // Without this exclusion, a `TomlDate` would silently resolve to `[]` here instead of + // falling through to the unconvertible-type issue below, turning Go's hard config-load + // failure into a silently-empty schema/seed path list (review CLI-1958). + // Pure — the TOP-LEVEL scalar fallback (see doc comment above), returning either the + // one resolved pattern or the one issue it would raise, WITHOUT failing (same reason + // as `legacyGlobArrayIssues`: the caller combines issues across both `Glob` fields + // before deciding whether to fail). + const legacyResolveScalarGlobFallback = ( + keyPath: string, + value: unknown, + ): { readonly resolved: ReadonlyArray; readonly issues: ReadonlyArray } => { + if ( + typeof value === "object" && + value !== null && + !(value instanceof SmolToml.TomlDate) && + Object.keys(value).length === 0 + ) { + return { resolved: [], issues: [] }; + } + const coerced = legacyWeakCoerceGlobEntry(value); + if (coerced !== undefined) { + return { resolved: [coerced], issues: [] }; + } + return { resolved: [], issues: legacyGlobArrayIssues(keyPath, [value]) }; + }; + /** + * Resolves ONE `Glob`-typed field (`[db.seed].sql_paths` / `[db.migrations]. + * schema_paths`) into its pre-supabase-join patterns, covering every decode branch + * in one place: override env var, real array, bare string, absent key (caller's own + * Go-matching default), or the top-level scalar fallback. Returns any issues + * alongside the best-effort patterns rather than failing here — see + * `legacyFailOnGlobIssues`'s doc comment for why the two `Glob` fields must combine + * their issues into ONE error before failing, matching Go's single `UnmarshalExact` + * pass, instead of each field failing independently on its own first bad entry. + */ + const legacyResolveGlobField = ( + keyPath: string, + raw: unknown, + override: string | undefined, + absentDefault: ReadonlyArray, + ): { readonly patterns: ReadonlyArray; readonly issues: ReadonlyArray } => { + if (override !== undefined) { + return { patterns: splitGoSeedPaths(override), issues: [] }; + } + if (Array.isArray(raw)) { + return { + patterns: raw + .map((pattern) => legacyWeakCoerceGlobEntry(pattern)) + .filter((pattern): pattern is string => pattern !== undefined) + .map((pattern) => legacyExpandEnv(pattern, lookup)), + issues: legacyGlobArrayIssues(keyPath, raw), + }; + } + if (typeof raw === "string") { + return { patterns: splitGoSeedPaths(raw), issues: [] }; + } + if (raw === undefined) { + return { patterns: absentDefault, issues: [] }; + } + const fallback = legacyResolveScalarGlobFallback(keyPath, raw); + return { + patterns: fallback.resolved.map((pattern) => legacyExpandEnv(pattern, lookup)), + issues: fallback.issues, + }; + }; const rawSqlPaths = seedRaw?.["sql_paths"]; const sqlPathsOverride = remoteOverrideKeys.has("db.seed.sql_paths") ? undefined : envOverride("SUPABASE_DB_SEED_SQL_PATHS"); - const sqlPathPatterns = - sqlPathsOverride !== undefined - ? splitGoSeedPaths(sqlPathsOverride) - : Array.isArray(rawSqlPaths) - ? rawSqlPaths - .filter((pattern): pattern is string => typeof pattern === "string") - .map((pattern) => legacyExpandEnv(pattern, lookup)) - : typeof rawSqlPaths === "string" - ? splitGoSeedPaths(rawSqlPaths) - : ["seed.sql"]; + const sqlPathsResolved = legacyResolveGlobField( + "db.seed.sql_paths", + rawSqlPaths, + sqlPathsOverride, + ["seed.sql"], + ); // Patterns are already env-expanded above (Go's LoadEnvHook runs before the split); // resolve each to Go's config-load form (absolute verbatim, relative supabase-joined). - const seedSqlPaths = sqlPathPatterns.map((pattern) => legacyResolveSeedSqlPath(path, pattern)); + const seedSqlPaths = sqlPathsResolved.patterns.map((pattern) => + legacyResolveSeedSqlPath(path, pattern), + ); + + // `[db.migrations] schema_paths` — Go default `[]` (`pkg/config/templates/config.toml:64`), + // resolved through the exact same decode + env-expand + supabase-join pipeline as + // `[db.seed].sql_paths` above, but UNCONDITIONALLY (Go's resolve loop for schema_paths, + // `config.go:976-980`, is not gated on `db.migrations.enabled` the way the seed loop is + // gated on `db.seed.enabled`, `config.go:968-975`). + const rawSchemaPaths = migrationsRaw?.["schema_paths"]; + const schemaPathsOverride = remoteOverrideKeys.has("db.migrations.schema_paths") + ? undefined + : envOverride("SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS"); + const schemaPathsResolved = legacyResolveGlobField( + "db.migrations.schema_paths", + rawSchemaPaths, + schemaPathsOverride, + [], + ); + + // Go's `UnmarshalExact` decodes the whole config in ONE mapstructure pass (see + // `legacyGlobArrayIssues`'s doc comment) — combine BOTH `Glob` fields' issues, in + // Go's struct-declaration order (`Migrations` before `Seed`), before failing once, + // so a config invalid in both surfaces both, matching Go's single combined error + // instead of only the first field checked. + yield* legacyFailOnGlobIssues([...schemaPathsResolved.issues, ...sqlPathsResolved.issues]); + + const schemaPaths = schemaPathsResolved.patterns.map((pattern) => + legacyResolveSeedSqlPath(path, pattern), + ); // `[db.vault]` secrets: env-expand each value, then decrypt dotenvx `encrypted:` // ciphertext. `resolved` mirrors Go's `len(SHA256) > 0` gate (Go sets SHA256 only diff --git a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts index a35e7048de..94d16cb57b 100644 --- a/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-db-config.toml-read.unit.test.ts @@ -1,7 +1,7 @@ import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; -import { BunServices } from "@effect/platform-bun"; +import { BunPath, BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; import { Effect, Exit, FileSystem, Option, Path } from "effect"; @@ -11,6 +11,7 @@ import { legacyLoadProjectEnv, legacyReadDbToml, legacyResolveDeclarativeDir, + legacyResolveSeedSqlPath, } from "./legacy-db-config.toml-read.ts"; function withConfig(content: string | undefined, poolerUrl?: string) { @@ -293,6 +294,486 @@ describe("legacyReadDbToml", () => { ); }); + it.effect( + "weakly coerces non-string db.seed.sql_paths array elements (Go mapstructure parity)", + () => { + // Same `config.Glob` decode path as schema_paths below — a bool/number element + // is coerced to its Go string form ("1"/"0" for bool, decimal for a number), + // not dropped. + const dir = withConfig(["[db.seed]", 'sql_paths = [42, true, "seed.sql"]', ""].join("\n")); + return read(dir).pipe( + Effect.tap((v) => + Effect.sync(() => { + expect(v.seed.sqlPaths).toEqual(["supabase/42", "supabase/1", "supabase/seed.sql"]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "honors SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS over the TOML array (comma split, no trim)", + () => { + const previous = process.env["SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS"]; + process.env["SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS"] = "a.sql, b.sql"; + const dir = withConfig(["[db.migrations]", 'schema_paths = ["ignored.sql"]', ""].join("\n")); + return read(dir).pipe( + Effect.tap((v) => + Effect.sync(() => { + expect(v.schemaPaths).toEqual(["supabase/a.sql", "supabase/ b.sql"]); + }), + ), + Effect.ensuring( + Effect.sync(() => { + if (previous === undefined) delete process.env["SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS"]; + else process.env["SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS"] = previous; + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "decodes a STRING db.migrations.schema_paths via StringToSliceHookFunc (comma, no trim)", + () => { + const dir = withConfig(["[db.migrations]", 'schema_paths = "a.sql,b.sql"', ""].join("\n")); + return read(dir).pipe( + Effect.tap((v) => + Effect.sync(() => { + expect(v.schemaPaths).toEqual(["supabase/a.sql", "supabase/b.sql"]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "on Windows, resolves a leading-slash schema/seed path pattern under supabase/ instead of treating it as absolute (Go filepath.IsAbs parity)", + () => { + // Go's `resolve()` gates the `supabase/`-join on `!filepath.IsAbs(pattern)` + // (`config.go:976-980`), and `filepath.IsAbs` on Windows requires a volume name — + // a drive letter (`C:\`) or UNC prefix (`\\server\share`) — before a path counts as + // absolute (`internal/filepathlite/path_windows.go`'s `IsAbs`/`volumeNameLen`). A + // bare leading `/` has no volume name, so Go treats `/schemas/*.sql` as RELATIVE and + // joins it to `supabase/schemas/*.sql`. Node's `path.win32.isAbsolute`, backing the + // injected `Path.Path` service on an actual Windows host, instead treats a leading + // separator as rooted at the current drive — i.e. absolute — which would otherwise + // skip Go's `supabase/`-join entirely. Exercises `legacyResolveSeedSqlPath` (the + // single function `[db.migrations].schema_paths` and `[db.seed].sql_paths` both + // resolve through) directly with `BunPath.layerWin32`, rather than through the full + // `legacyReadDbToml` pipeline: that pipeline's OWN config-file lookup also runs + // through the same injected `Path.Path` service to open the real (POSIX-pathed, + // since this test host isn't Windows) temp config file on disk, so forcing win32 + // path semantics there breaks the read itself rather than exercising the fix. + const originalPlatform = process.platform; + Object.defineProperty(process, "platform", { value: "win32" }); + return Effect.gen(function* () { + const path = yield* Path.Path; + const resolved = legacyResolveSeedSqlPath(path, "/schemas/*.sql"); + expect(resolved).toBe("supabase/schemas/*.sql"); + }).pipe( + Effect.provide(BunPath.layerWin32), + Effect.ensuring( + Effect.sync(() => { + Object.defineProperty(process, "platform", { value: originalPlatform }); + }), + ), + ); + }, + ); + + it.effect( + "weakly coerces non-string db.migrations.schema_paths array elements (Go mapstructure parity)", + () => { + // Go's `v.UnmarshalExact` never sets `WeaklyTypedInput: false`, so viper's + // `defaultDecoderConfig` default of `true` stands — mapstructure's `decodeString` + // coerces a bool to "1"/"0" and a number to its decimal string rather than + // erroring or dropping the element. Verified empirically against `apps/cli-go`: + // `schema_paths = [42, true, "schemas/*.sql"]` resolves to + // `supabase/{42,1,schemas/*.sql}`, not a filtered two-element list. + const dir = withConfig( + ["[db.migrations]", 'schema_paths = [42, true, "schemas/*.sql"]', ""].join("\n"), + ); + return read(dir).pipe( + Effect.tap((v) => + Effect.sync(() => { + expect(v.schemaPaths).toEqual(["supabase/42", "supabase/1", "supabase/schemas/*.sql"]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "formats a large numeric db.migrations.schema_paths entry as fixed decimal, not scientific notation (Go strconv.FormatFloat parity)", + () => { + // Go's `decodeString` renders a weakly-converted float via + // `strconv.FormatFloat(v, 'f', -1, 64)` — format `'f'` is ALWAYS fixed decimal, + // never scientific, regardless of magnitude. JS's bare `String(1e21)` switches to + // exponential notation ("1e+21") once the magnitude crosses 1e21, which would + // record (and later search for) the wrong file path. Verified empirically against + // Go's stdlib: `strconv.FormatFloat(1e21, 'f', -1, 64)` returns + // `"1000000000000000000000"`, not `"1e+21"`. + const dir = withConfig(["[db.migrations]", "schema_paths = [1e21]", ""].join("\n")); + return read(dir).pipe( + Effect.tap((v) => + Effect.sync(() => { + expect(v.schemaPaths).toEqual(["supabase/1000000000000000000000"]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "formats TOML special-float db.migrations.schema_paths entries like Go's strconv.FormatFloat, not JS's toString (Go parity)", + () => { + // `strconv.FormatFloat` special-cases the three non-finite values BEFORE the + // format verb is even consulted, so `'f'` never applies to them — it renders + // `+Inf` / `-Inf` / `NaN` (verified empirically against `apps/cli-go`: a real + // `schema_paths = [inf, -inf, nan]` config load resolves to exactly + // `supabase/{+Inf,-Inf,NaN}`). JS's own `Number.prototype.toString()` renders + // the two infinities as `"Infinity"`/`"-Infinity"` instead — a naive port would + // record (and later glob) the wrong path. TOML v1.0's bare `inf`/`-inf`/`nan` + // float literals parse to exactly these JS values (smol-toml). + const dir = withConfig(["[db.migrations]", "schema_paths = [inf, -inf, nan]", ""].join("\n")); + return read(dir).pipe( + Effect.tap((v) => + Effect.sync(() => { + expect(v.schemaPaths).toEqual(["supabase/+Inf", "supabase/-Inf", "supabase/NaN"]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "weakly coerces a TOP-LEVEL scalar db.migrations.schema_paths (Go mapstructure weak-decode of a []string field)", + () => { + // Go's `decodeSlice` wraps a non-array/non-string value into a synthetic + // single-element `[]any{value}` and decodes it through the same per-element + // rules as a real array entry — it does NOT fall back to the `[]` default the + // way an absent key does. Verified empirically against `apps/cli-go`: + // `schema_paths = 42` → `["42"]` (resolves to `supabase/42`), `= true` → `["1"]`. + const dirNumber = withConfig(["[db.migrations]", "schema_paths = 42", ""].join("\n")); + const dirBool = withConfig(["[db.migrations]", "schema_paths = true", ""].join("\n")); + return Effect.all([read(dirNumber), read(dirBool)]).pipe( + Effect.tap(([numberResult, boolResult]) => + Effect.sync(() => { + expect(numberResult.schemaPaths).toEqual(["supabase/42"]); + expect(boolResult.schemaPaths).toEqual(["supabase/1"]); + rmSync(dirNumber, { recursive: true, force: true }); + rmSync(dirBool, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "treats a TOP-LEVEL empty-table db.migrations.schema_paths as no patterns (Go mapstructure zero-length-map special case)", + () => { + // Go's `decodeSlice` special-cases a zero-length map BEFORE the generic weak-typing + // wrap above: it decodes straight to an empty slice. Verified empirically against + // `apps/cli-go`: `schema_paths = {}` → `[]`. + const dir = withConfig(["[db.migrations]", "schema_paths = {}", ""].join("\n")); + return read(dir).pipe( + Effect.tap((v) => + Effect.sync(() => { + expect(v.schemaPaths).toEqual([]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect.each([ + { name: "offset date-time", literal: "1979-05-27T07:32:00Z", goType: "time.Time" }, + { name: "local date-time", literal: "1979-05-27T07:32:00", goType: "toml.LocalDateTime" }, + { name: "local date", literal: "1979-05-27", goType: "toml.LocalDate" }, + { name: "local time", literal: "07:32:00", goType: "toml.LocalTime" }, + ])( + "aborts the whole config load on a TOP-LEVEL bare $name db.migrations.schema_paths instead of silently treating it as empty (Go mapstructure UnconvertibleTypeError, review CLI-1958)", + ({ literal, goType }) => { + // `smol-toml` parses every TOML datetime variant to a `TomlDate` (a `Date` + // subclass) that stores its value internally, not as an enumerable own + // property — so `Object.keys(tomlDate).length === 0`, same as a genuine empty + // inline table (`schema_paths = {}`, tested above). Without excluding `TomlDate` + // from that zero-length-map special case, this would silently resolve to `[]` + // instead of aborting. Verified empirically against the real `apps/cli-go` + // `config.Load`: a bare datetime literal here fails with exactly this message, + // never resolving to an empty/partial glob list — one distinct Go type per TOML + // datetime variant (`time.Time` for the offset form, `toml.Local*` wrappers for + // the 3 zone-less "local" forms). + const dir = withConfig(["[db.migrations]", `schema_paths = ${literal}`, ""].join("\n")); + return read(dir).pipe( + Effect.exit, + Effect.tap((exit) => + Effect.sync(() => { + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + expect(JSON.stringify(exit.cause)).toContain( + `'db.migrations.schema_paths[0]' expected type 'string', got unconvertible type '${goType}'`, + ); + } + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "aborts the whole config load on a bare datetime db.migrations.schema_paths ARRAY element (Go mapstructure UnconvertibleTypeError, review CLI-1958)", + () => { + // Same `TomlDate`-vs-generic-object collision as the top-level scalar case above, + // but reached through the real-array branch (`legacyGoUnconvertibleType`) instead + // of the scalar fallback. Verified empirically against `apps/cli-go`: + // `schema_paths = ["schemas/*.sql", 1979-05-27T07:32:00Z]` fails config load with + // exactly this message — the valid glob entry never masks the datetime's failure. + const dir = withConfig( + ["[db.migrations]", 'schema_paths = ["schemas/*.sql", 1979-05-27T07:32:00Z]', ""].join( + "\n", + ), + ); + return read(dir).pipe( + Effect.exit, + Effect.tap((exit) => + Effect.sync(() => { + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + expect(JSON.stringify(exit.cause)).toContain( + "'db.migrations.schema_paths[1]' expected type 'string', got unconvertible type 'time.Time'", + ); + } + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "aborts the whole config load on a TOP-LEVEL bare datetime db.seed.sql_paths (same UnmarshalExact call as schema_paths, review CLI-1958)", + () => { + const dir = withConfig(["[db.seed]", "sql_paths = 1979-05-27T07:32:00Z", ""].join("\n")); + return read(dir).pipe( + Effect.exit, + Effect.tap((exit) => + Effect.sync(() => { + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + expect(JSON.stringify(exit.cause)).toContain( + "'db.seed.sql_paths[0]' expected type 'string', got unconvertible type 'time.Time'", + ); + } + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "aborts the whole config load on a TOP-LEVEL table db.migrations.schema_paths (Go mapstructure UnconvertibleTypeError, synthetic index 0)", + () => { + // A non-empty map isn't weakly coercible, so Go's `decodeSlice` wraps it into + // `[]any{value}` and fails decoding element 0 the same way a nested-array/table + // ARRAY element does. Verified empirically against `apps/cli-go`: + // `[db.migrations.schema_paths]\nfoo = "bar"` fails with this exact message. + const dir = withConfig(["[db.migrations.schema_paths]", 'foo = "bar"', ""].join("\n")); + return read(dir).pipe( + Effect.exit, + Effect.tap((exit) => + Effect.sync(() => { + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + expect(JSON.stringify(exit.cause)).toContain( + "'db.migrations.schema_paths[0]' expected type 'string', got unconvertible type 'map[string]interface {}'", + ); + } + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "weakly coerces a TOP-LEVEL scalar db.seed.sql_paths instead of falling back to the ['seed.sql'] default", + () => { + // The absent-key default (`["seed.sql"]`) only applies when the key is missing + // entirely — a PRESENT scalar still goes through Go's weak-decode wrap, same as + // schema_paths above. Verified empirically against `apps/cli-go`: + // `[db.seed]\nenabled = true\nsql_paths = 42` → `["42"]`, not `["seed.sql"]`. + const dir = withConfig(["[db.seed]", "enabled = true", "sql_paths = 42", ""].join("\n")); + return read(dir).pipe( + Effect.tap((v) => + Effect.sync(() => { + expect(v.seed.sqlPaths).toEqual(["supabase/42"]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "aborts the whole config load on a non-scalar db.migrations.schema_paths element (Go mapstructure UnconvertibleTypeError)", + () => { + // Unlike a bool/number (weakly coerced above), a nested array/table is + // mapstructure's `UnconvertibleTypeError`, which fails `UnmarshalExact` entirely + // rather than dropping just that element. Verified empirically against + // `apps/cli-go`: `schema_paths = [[]]` fails config load with exactly this + // message, never resolving to an empty/partial glob list. + const dir = withConfig(["[db.migrations]", "schema_paths = [[]]", ""].join("\n")); + return read(dir).pipe( + Effect.exit, + Effect.tap((exit) => + Effect.sync(() => { + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + expect(JSON.stringify(exit.cause)).toContain( + "failed to parse config: decoding failed due to the following error(s):\\n\\n'db.migrations.schema_paths[0]' expected type 'string', got unconvertible type '[]interface {}'", + ); + } + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "aborts the whole config load on a table db.migrations.schema_paths element, reporting every bad index (Go mapstructure parity)", + () => { + // Verified empirically against `apps/cli-go`: a second bad entry is reported + // alongside the first (mapstructure aggregates every `UnmarshalExact` error from + // the same decode call), and an inline table decodes as `map[string]interface {}`. + const dir = withConfig( + ["[db.migrations]", 'schema_paths = ["schemas/*.sql", { path = "x.sql" }]', ""].join("\n"), + ); + return read(dir).pipe( + Effect.exit, + Effect.tap((exit) => + Effect.sync(() => { + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + expect(JSON.stringify(exit.cause)).toContain( + "'db.migrations.schema_paths[1]' expected type 'string', got unconvertible type 'map[string]interface {}'", + ); + } + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "aborts the whole config load on a non-scalar db.seed.sql_paths element (same UnmarshalExact call as schema_paths)", + () => { + const dir = withConfig(["[db.seed]", "sql_paths = [[]]", ""].join("\n")); + return read(dir).pipe( + Effect.exit, + Effect.tap((exit) => + Effect.sync(() => { + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + expect(JSON.stringify(exit.cause)).toContain( + "'db.seed.sql_paths[0]' expected type 'string', got unconvertible type '[]interface {}'", + ); + } + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "aggregates unconvertible-entry issues from BOTH db.seed.sql_paths and db.migrations.schema_paths in one error (Go UnmarshalExact single-pass parity, review CLI-1958)", + () => { + // Go's `UnmarshalExact` decodes the WHOLE config in a SINGLE mapstructure pass: + // `decodeStructFromMap`'s per-field loop never stops at the first field's error + // — it visits every field, collects every error, then joins them all together + // at the end. So a config invalid in BOTH `Glob` fields reports BOTH, not just + // whichever field is checked first. Verified empirically against `apps/cli-go` + // (`config.Load` with `sql_paths = [[]]` + `schema_paths = [[]]`): the single + // returned error contains both lines, `db.migrations.schema_paths[0]` BEFORE + // `db.seed.sql_paths[0]` — Go's `db` struct declares `Migrations` before `Seed` + // (`pkg/config/db.go:90-91`), so mapstructure visits (and therefore reports) + // `schema_paths` first regardless of which field this reader happens to resolve + // first internally. + const dir = withConfig( + ["[db.seed]", "sql_paths = [[]]", "", "[db.migrations]", "schema_paths = [[]]", ""].join( + "\n", + ), + ); + return read(dir).pipe( + Effect.exit, + Effect.tap((exit) => + Effect.sync(() => { + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const message = JSON.stringify(exit.cause); + const schemaIssue = + "'db.migrations.schema_paths[0]' expected type 'string', got unconvertible type '[]interface {}'"; + const seedIssue = + "'db.seed.sql_paths[0]' expected type 'string', got unconvertible type '[]interface {}'"; + expect(message).toContain(schemaIssue); + expect(message).toContain(seedIssue); + // Both issues in ONE combined error, schema_paths first (Go's struct + // field declaration order), not two separate failures. + expect(message.indexOf(schemaIssue)).toBeLessThan(message.indexOf(seedIssue)); + } + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "an explicit remote db.migrations.schema_paths beats SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS", + () => { + // Go applies each matched-remote key via v.Set (override tier) above AutomaticEnv + // (config.go:635-637), so an explicit remote value wins over the env var. + const ref = "schmschmschmschmschm"; + const previous = process.env["SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS"]; + process.env["SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS"] = "env-only.sql"; + const dir = withConfig( + [ + "[remotes.prod]", + `project_id = "${ref}"`, + 'db.migrations.schema_paths = ["remote-only.sql"]', + "", + ].join("\n"), + ); + return readRef(dir, ref).pipe( + Effect.tap((v) => + Effect.sync(() => { + expect(v.schemaPaths).toEqual(["supabase/remote-only.sql"]); + }), + ), + Effect.ensuring( + Effect.sync(() => { + if (previous === undefined) delete process.env["SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS"]; + else process.env["SUPABASE_DB_MIGRATIONS_SCHEMA_PATHS"] = previous; + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + it.effect("decodes a numeric db.seed.enabled = 0 as false (Go weak-bool decode)", () => { const dir = withConfig(["[db.seed]", "enabled = 0", ""].join("\n")); return read(dir).pipe( diff --git a/apps/cli/src/legacy/shared/legacy-db-push-core.ts b/apps/cli/src/legacy/shared/legacy-db-push-core.ts index be8759fda7..d328735119 100644 --- a/apps/cli/src/legacy/shared/legacy-db-push-core.ts +++ b/apps/cli/src/legacy/shared/legacy-db-push-core.ts @@ -1,4 +1,4 @@ -import { Clock, Effect, FileSystem, Option, Path } from "effect"; +import { Effect, FileSystem, Option, Path } from "effect"; import { legacyPromptYesNo } from "../../shared/legacy/legacy-prompt-yes-no.ts"; import { CONTEXT_CANCELED_MESSAGE } from "../../shared/output/errors.ts"; @@ -346,7 +346,6 @@ export const legacyDbPushCore = Effect.fnUntraced(function* (input: LegacyDbPush conn, isLocal, migrationsDir: path.join(workdir, "supabase", "migrations"), - nowMillis: yield* Clock.currentTimeMillis, }).pipe( Effect.catch((error) => output.raw( diff --git a/apps/cli/src/legacy/shared/legacy-error-message.ts b/apps/cli/src/legacy/shared/legacy-error-message.ts new file mode 100644 index 0000000000..f13d8c64fe --- /dev/null +++ b/apps/cli/src/legacy/shared/legacy-error-message.ts @@ -0,0 +1,31 @@ +/** + * Best-effort extraction of a human-readable message from an unknown thrown/failed + * value — an Effect `PlatformError`, a driver error, a plain `Error`, or anything else. + * Shared by every legacy module that wraps a raw Effect/driver failure into Go-style + * error text (Go's own `err.Error()` equivalent), so wording stays consistent across + * call sites instead of each one re-deriving its own fallback. + */ +export const legacyErrorMessage = (e: unknown): string => + typeof e === "object" && e !== null && "message" in e && typeof e.message === "string" + ? e.message + : String(e); + +/** + * Substitutes an absolute path a real syscall needed back to its Go-equivalent + * display path inside an already-rendered error message. Go's own `fsys` is always a + * real `afero.OsFs` with the process cwd already `chdir`'ed into the workdir + * (`ChangeWorkDir`, `cmd/root.go`), so every Go error message embeds the + * workdir-relative (or verbatim-absolute) path it was actually called with. This shell + * deliberately never `process.chdir`s, so its own syscalls need a real absolute path to + * work — but the wrapped message must still report the Go-equivalent path, not the + * local temp/workdir absolute path the syscall needed, or it leaks a path Go would + * never show. Shared by every legacy module that wraps a raw filesystem failure this + * way (`legacy-sql-files-glob.ts`'s matched-file/matched-directory warnings, + * `legacy-migration-apply.ts`'s migration-file read errors). + */ +export const legacyRelativizeErrorMessage = ( + rawMessage: string, + absolutePath: string, + displayPath: string, +): string => + absolutePath === displayPath ? rawMessage : rawMessage.split(absolutePath).join(displayPath); diff --git a/apps/cli/src/legacy/shared/legacy-migrate-and-seed.ts b/apps/cli/src/legacy/shared/legacy-migrate-and-seed.ts index 5f9c3c0683..724765d667 100644 --- a/apps/cli/src/legacy/shared/legacy-migrate-and-seed.ts +++ b/apps/cli/src/legacy/shared/legacy-migrate-and-seed.ts @@ -1,16 +1,13 @@ -import { Effect, type FileSystem, type Path, Result } from "effect"; +import { Effect, type FileSystem, type Path } from "effect"; import { Output } from "../../shared/output/output.service.ts"; -import { legacyBold } from "./legacy-colors.ts"; import type { LegacyDbSession } from "./legacy-db-connection.service.ts"; -import { legacyGlobPattern, legacyResolveUnderWorkdir, legacyWalkSqlFiles } from "./legacy-glob.ts"; import { LegacyMigrationApplyError, legacyApplyMigrationFile, - legacyExecSqlFile, + legacyApplySchemaFiles, } from "./legacy-migration-apply.ts"; import { legacyLoadPartialMigrations } from "./legacy-migration-history.ts"; -import { LEGACY_BAD_PATTERN_MESSAGE, legacyPathMatch } from "./legacy-path-match.ts"; import { legacyApplySeedFiles, type LegacySeedConfig } from "./legacy-seed.ts"; /** Config consumed by `legacyMigrateAndSeed`. */ @@ -33,136 +30,16 @@ export interface LegacyMigrateAndSeedConfig { readonly schemaPaths: ReadonlyArray; } -/** - * Port of Go's `Glob.SQLFiles` (`pkg/config/config.go:122-128` → `files`/`walkMatchedDir`), - * called with zero `GlobOption`s exactly like `applySchemaFiles`'s own call - * (`internal/migration/apply/apply.go:52`): each `schemaPaths` pattern is glob-matched, in - * declared order, via {@link legacyGlobPattern} — the same `fs.Glob` port `[db.seed] - * sql_paths` already uses. Patterns arrive already resolved to Go's config-load form - * (supabase-prefixed and `path.Clean`-ed when relative) — `legacyCheckDbToml` - * (`legacy-db-config.toml-read.ts`) does that once, at config-load time, the same place - * `db.seed.sql_paths` is resolved, so this function (unlike an earlier version of this - * comment) does no path-shape work of its own. A matched directory is expanded to its - * `.sql` regular files, recursively, sorted; a matched plain file is kept as-is — even a - * non-`.sql` one, since Go's `expandDir` callback only ever runs on `IsDir()` matches, - * never on an explicitly-matched file. Results are deduplicated across ALL patterns - * (first occurrence wins), preserving pattern declaration order. - * - * A pattern matching nothing, a stat failure, or a directory-walk failure is an error, - * but — mirroring `applySchemaFiles`'s `if len(declared) == 0 { return err }` (the error - * `Glob.SQLFiles` returns alongside a non-empty `declared` is joined from every - * problem, including `walkMatchedDir`'s) — every such problem is discarded outright - * whenever the combined result ends up non-empty regardless; they only surface when NO - * pattern matched anything at all. - */ -const legacyResolveSchemaPathFiles = ( - fs: FileSystem.FileSystem, - path: Path.Path, - workdir: string, - patterns: ReadonlyArray, -): Effect.Effect, LegacyMigrationApplyError> => - Effect.gen(function* () { - const seen = new Set(); - const result: Array = []; - const problems: Array = []; - - for (const pattern of patterns) { - if (legacyPathMatch(pattern, "").badPattern) { - problems.push(`failed to glob files: ${LEGACY_BAD_PATTERN_MESSAGE}`); - continue; - } - const matches = [...(yield* legacyGlobPattern(fs, path, workdir, pattern))].sort(); - if (matches.length === 0) { - problems.push(`no files matched pattern: ${pattern}`); - continue; - } - for (const match of matches) { - const absMatch = legacyResolveUnderWorkdir(path, workdir, match); - const statResult = yield* fs.stat(absMatch).pipe(Effect.result); - if (Result.isFailure(statResult)) { - problems.push(`failed to stat matched file: ${match}`); - continue; - } - if (statResult.success.type !== "Directory") { - if (!seen.has(match)) { - seen.add(match); - result.push(match); - } - continue; - } - // Go's `walkMatchedDir`: recursively list the matched directory, keep only regular - // `.sql` files, sorted (a global sort over the full relative-to-fsys-root path, not - // per-directory — matches `sort.Strings(files)` running once after the whole walk). - // A read/walk failure is Go's `failed to walk matched directory: %w` — recorded as a - // problem (not silently treated as an empty directory) so it surfaces exactly like - // Go's joined error does whenever nothing else matched anything either. - const namesResult = yield* legacyWalkSqlFiles(fs, absMatch, "").pipe(Effect.result); - if (Result.isFailure(namesResult)) { - problems.push(`failed to walk matched directory: ${match}`); - continue; - } - const sqlRelative = [...namesResult.success].sort(); - for (const relative of sqlRelative) { - const relativeToWorkdir = `${match}/${relative}`; - if (!seen.has(relativeToWorkdir)) { - seen.add(relativeToWorkdir); - result.push(relativeToWorkdir); - } - } - } - } - - if (result.length === 0 && problems.length > 0) { - return yield* Effect.fail(new LegacyMigrationApplyError({ message: problems.join("\n") })); - } - return result; - }); - -/** - * Port of Go's `applySchemaFiles` (`internal/migration/apply/apply.go:50-61`): applies - * every file resolved by {@link legacyResolveSchemaPathFiles} directly, in order, WITHOUT - * inserting a migration-history row (Go sets `schema.Version = ""` before `ExecBatch`) and - * WITHOUT creating the history table or resetting connection state first (`applySchemaFiles` - * calls `ExecBatch` directly on each file, unlike `applyMigrationFiles`'s - * `migration.ApplyMigrations`) — `legacyExecSqlFile` already has exactly this shape. A failed - * `ExecBatch` sets `utils.CmdSuggestion = "See schema file: "` (`apply.go:57`, `fp` bolded) - * immediately, so the failing file is attached as the error's `suggestion` here too — the - * generic `normalizeCliError` fallback (`shared/output/normalize-error.ts`) surfaces any - * error's `suggestion` field verbatim, matching root.go's plain `CmdSuggestion` stderr line. - */ -const legacyApplySchemaFiles = ( - session: LegacyDbSession, - fs: FileSystem.FileSystem, - path: Path.Path, - workdir: string, - schemaPaths: ReadonlyArray, -) => - Effect.gen(function* () { - const declared = yield* legacyResolveSchemaPathFiles(fs, path, workdir, schemaPaths); - for (const relativePath of declared) { - const absPath = legacyResolveUnderWorkdir(path, workdir, relativePath); - yield* legacyExecSqlFile( - session, - fs, - path, - absPath, - (message) => - new LegacyMigrationApplyError({ - message, - suggestion: `See schema file: ${legacyBold(relativePath)}`, - }), - ); - } - }); - /** * Reapplies local migrations up to `version`, then runs seed files. Port of Go's * `apply.MigrateAndSeed` (`internal/migration/apply/apply.go:16-26`): when `experimental` is * set, `version` is empty, and `pgDeltaEnabled` is false, the declarative `schemaPaths` - * files are applied INSTEAD of migration files (bypassing `migrationsEnabled` entirely — Go's - * `applySchemaFiles` has no such gate, only `applyMigrationFiles` does); otherwise migration - * apply is gated on `db.migrations.enabled` as before. Seeding (`db.seed.enabled`, inside the - * seed helper) always runs, on either branch. + * files are applied INSTEAD of migration files via the shared {@link legacyApplySchemaFiles} + * (`legacy-migration-apply.ts` — also used by `db reset`'s own `--experimental` remote path, + * so both callers share one Go-quirk-preserving implementation instead of two), bypassing + * `migrationsEnabled` entirely — Go's `applySchemaFiles` has no such gate, only + * `applyMigrationFiles` does; otherwise migration apply is gated on `db.migrations.enabled` as + * before. Seeding (`db.seed.enabled`, inside the seed helper) always runs, on either branch. */ export const legacyMigrateAndSeed = ( session: LegacyDbSession, @@ -175,7 +52,14 @@ export const legacyMigrateAndSeed = ( Effect.gen(function* () { const output = yield* Output; if (config.experimental && version.length === 0 && !config.pgDeltaEnabled) { - yield* legacyApplySchemaFiles(session, fs, path, workdir, config.schemaPaths); + yield* legacyApplySchemaFiles( + session, + fs, + path, + workdir, + config.schemaPaths, + (message, suggestion) => new LegacyMigrationApplyError({ message, suggestion }), + ); } else if (config.migrationsEnabled) { const migrationsDir = path.join(workdir, "supabase", "migrations"); const pending = yield* legacyLoadPartialMigrations(fs, path, migrationsDir, version).pipe( diff --git a/apps/cli/src/legacy/shared/legacy-migration-apply.ts b/apps/cli/src/legacy/shared/legacy-migration-apply.ts index 96d545f10e..dc8aec9b24 100644 --- a/apps/cli/src/legacy/shared/legacy-migration-apply.ts +++ b/apps/cli/src/legacy/shared/legacy-migration-apply.ts @@ -1,14 +1,17 @@ import { Data, Effect, type FileSystem, type Path } from "effect"; import { Output } from "../../shared/output/output.service.ts"; +import { legacyBold } from "./legacy-colors.ts"; import type { LegacyDbExecError } from "./legacy-db-connection.errors.ts"; import type { LegacyDbSession } from "./legacy-db-connection.service.ts"; +import { legacyErrorMessage, legacyRelativizeErrorMessage } from "./legacy-error-message.ts"; import { INSERT_MIGRATION_VERSION, MIGRATE_FILE_PATTERN, legacyCreateMigrationTable, } from "./legacy-migration-history.ts"; -import { legacySplitAndTrim } from "./legacy-sql-split.ts"; +import { legacySqlFilesGlob } from "./legacy-sql-files-glob.ts"; +import { legacySplitAndTrim, legacySplitSqlTokens } from "./legacy-sql-split.ts"; /** * Applying a migration file failed (Go's `ApplyMigrations` / `ExecBatch` error). @@ -102,13 +105,299 @@ type LegacyBatchItem = | { readonly kind: "exec"; readonly sql: string } | { readonly kind: "version" }; -const errMessage = (e: unknown): string => - typeof e === "object" && e !== null && "message" in e && typeof e.message === "string" - ? e.message - : String(e); - const utf8ByteLength = (value: string): number => new TextEncoder().encode(value).length; +// Go's `startBufSize` (`pkg/parser/token.go:15`) — the fixed initial `bufio.Scanner` +// buffer `parser.Split` pre-allocates before applying the configured/default max +// (`scanner.Buffer(buf, maxbuf)` where `buf := make([]byte, startBufSize)`). The +// scanner's buffer therefore starts at exactly this size regardless of how small +// `SUPABASE_SCANNER_BUFFER_SIZE` is set, and `bufio.Scanner`'s too-long check +// (`len(s.buf) >= s.maxTokenSize`, `$GOROOT/src/bufio/scan.go:200`) only fires once +// the buffer is full — so a statement must reach at least this many raw bytes +// before Go can ever raise `bufio.ErrTooLong`, no matter how small the override. +// Verified empirically against `apps/cli-go/pkg/parser` (`parser.SplitAndTrim`): a +// single-statement probe of exactly 4096 raw bytes always succeeds — even with +// `SUPABASE_SCANNER_BUFFER_SIZE` set to 10 bytes — while 4097 bytes always fails; +// with the override set above this floor (e.g. 5000 bytes), the exact same +// pattern repeats at the override's own value (5000 succeeds, 5001 fails). +const GO_SCANNER_START_BUF_SIZE = 4096; + +// Go's `parser.MaxScannerCapacity` (`pkg/parser/token.go:19`) — the hardcoded default +// `parser.Split` falls back to when `viper.GetSizeInBytes("SCANNER_BUFFER_SIZE")` +// returns `0`. Reached whenever the env var is SET but resolves to a non-positive size +// — including a value `legacyParseScannerBufferSize` can't parse at all. Verified +// empirically against `apps/cli-go/pkg/parser` + vendored `viper@v1.21.0`: +// `SUPABASE_SCANNER_BUFFER_SIZE=5M` (a bare multiplier suffix with NO trailing `b`/`B`) +// behaves byte-for-byte identically to the var being completely unset from the default +// cap's own first-failure point on — because `parseSizeInBytes` only ever recognizes a +// `k`/`m`/`g` multiplier when it immediately precedes a trailing `b`/`B` +// (`util.go:156-174`); "5M" never strips a suffix, so it falls through to +// `cast.ToInt("5M")`, which fails whole (not a leading-digits prefix parse — unlike +// JS's lenient `Number.parseInt`) and returns `0`. This is NOT the same as truly unset, +// though: `viper.IsSet("SCANNER_BUFFER_SIZE")` is still `true` (the var IS present, just +// unparseable), so `parseFile`'s file-size auto-growth (see `checkScannerBufferSize`'s +// doc comment) never runs — the cap stays pinned at this hardcoded default regardless of +// the real file's size, unlike the genuinely-unset case where it grows to match. +const GO_DEFAULT_MAX_SCANNER_CAPACITY = 256 * 1024; + +/** + * Go's `viper.GetSizeInBytes("SCANNER_BUFFER_SIZE")` (env-prefixed + * `SUPABASE_SCANNER_BUFFER_SIZE`, `pkg/parser/token.go:87`): an integer byte count, + * optionally suffixed `k`/`K`/`m`/`M`/`g`/`G` (× 1024/1024²/1024³) plus a trailing + * `b`/`B` (e.g. `"5MB"`, `"256KB"`, or a bare byte count). Ported 1:1 from viper's + * own parser (`parseSizeInBytes`, `github.com/spf13/viper@v1.21.0/util.go:151-179`): + * an unparseable or non-positive result is treated as unset (`0`). + * + * The multiplier is recognized ONLY when the string's LAST character is literally + * `b`/`B` — a bare `"5M"` (no trailing `B`) is NOT 5 MiB in real Go: `sizeStr[lastChar]` + * isn't `b`/`B`, so the multiplier branch never runs and the whole (unstripped) string + * is handed to `cast.ToInt`, which fails on the trailing letter and yields `0`. Do NOT + * special-case a bare `k`/`m`/`g` suffix here; that would make this port accept a value + * real Go rejects. + * + * Go's inner `lastChar > 1` gate (`util.go:158`) means the trailing-`B`-strip ALSO never + * runs for a 2-character value like `"5B"`/`"5b"` — only 3+ characters (an actual + * multiplier letter, or at least one digit, before the `B`) reach the switch at all — so + * `"5B"` is unstripped, `cast.ToInt("5B")` fails, and the whole thing is `0` too, same as + * `"5M"`. `cast.ToInt` (`strconv.ParseInt`) also requires the ENTIRE remaining string to + * be a clean integer — trailing garbage fails the WHOLE parse, unlike JS's lenient + * `Number.parseInt`, which stops at the first non-digit and returns whatever numeric + * prefix it found (`Number.parseInt("5M", 10) === 5`, silently discarding the "M", where + * Go's parse rejects the string outright). `cast.ToInt` does tolerate one decimal point + * via its own `trimDecimal` (keeps only the integer part, e.g. `"5.5"` → `"5"`), so allow + * exactly that one exception. All of the above verified empirically against + * `apps/cli-go/pkg/parser` + vendored `viper@v1.21.0` + * (`"5"→5, "5B"→0, "50B"→50, "5KB"→5120, "5M"→0, "0"→0, "5.5"→5, "5.5MB"→5242880`). + * + * `cast.ToInt`'s underlying `strconv.ParseInt(s, 0, 0)` (review CLI-1958) uses base + * `0`, so the remaining (post multiplier-strip) string is ALSO accepted as a + * `0x`/`0X`-prefixed hex literal, an `0o`/`0O`-prefixed OR bare-leading-zero octal + * literal, or a `0b`/`0B`-prefixed binary literal — handled below by + * {@link parseGoBaseZeroInt}. Verified empirically against vendored `viper@v1.21.0` + * (via `viper.GetSizeInBytes`, since `cast.ToInt` is itself unexported call + * plumbing): `"0x100000"→1048576`, `"0o40000"→16384`, + * `"0b100000000000000000000"→2097152`, `"0755"` (legacy octal, no `"o"`)`→493`, + * `"0x"/"garbage"/"0x1g"→unparseable (0)`. A hex/octal/binary value ending in a + * literal `b`/`B` digit (e.g. `"0x1B"`) still hits the multiplier-strip switch + * ABOVE first, same as any other value — that consumes the trailing `B` before + * base-0 parsing ever sees it (`"0x1B"→1`, not `27`), which is a genuine Go quirk + * this port reproduces automatically by keeping the same two-step order, not a bug. + * Go's base-0 grammar also permits `_` digit separators (e.g. `"1_048_576"`) — see + * {@link parseGoBaseZeroInt}'s own doc comment for the exact placement grammar. + */ +/** + * Go's underscore digit-separator grammar (`go.dev/ref/spec#Integer_literals`, + * reproduced by `strconv.ParseInt`'s base-0 mode, review CLI-1958): a SINGLE `_` + * may sit immediately after a base prefix (explicit `0x`/`0o`/`0b`, or the bare + * leading `"0"` of legacy octal) or between two digits of the same base — never + * doubled, never leading a plain (no-prefix) decimal literal, and never trailing. + * Verified empirically against the real `strconv.ParseInt(s, 0, 64)`: + * `"1_048_576"→1048576`, `"0x_100000"/"0x10_0000"→1048576`, + * `"0o_40000"/"0o4_0000"→16384`, `"0b_100000000000000000000"→1048576`, + * `"0_755"/"07_55"→493` (legacy octal, underscore right after the leading `"0"` + * or between later octal digits); while `"_1048576"`, `"1048576_"`, + * `"1__048576"`, `"0x100000_"`, and `"0_x100000"` (underscore splitting the + * leading `"0"` from the `"x"` — not a real prefix, so it's parsed as legacy + * octal digits `"x100000"`) all fail, matching Go exactly. + */ +// `strconv.ParseInt(s, 0, 0)`'s bitSize-0 mode requires the result to fit in Go's `int` +// — 64 bits on every platform this CLI ships for (amd64/arm64). A magnitude outside +// this range is a range error (`strconv.ErrRange`), and `cast.ToInt` (`spf13/cast@v1.10.0 +// /number.go:407-414`'s `parseInt[T]`) discards ANY `parseFn` error — range or +// syntax — and returns exactly `0`, not the (possibly huge, saturated-to-max-magnitude) +// value `strconv.ParseInt` itself returns alongside that error. Verified empirically +// against the pinned `spf13/cast@v1.10.0`: `cast.ToInt("9223372036854775808")` (one over +// `math.MaxInt64`) → `0`. `Number.parseInt` has no such range check (it silently rounds +// via IEEE-754 double precision instead), so this port must reject the same magnitudes +// Go does, or it would treat an out-of-range override as an enormous-but-finite limit +// instead of falling back to the 256KiB default like Go (review CLI-1958 round 18). +const GO_MAX_INT64 = 9223372036854775807n; +const GO_MIN_INT64 = -9223372036854775808n; + +const parseGoBaseZeroInt = (value: string): number | undefined => { + const negative = value.startsWith("-"); + const unsigned = negative || value.startsWith("+") ? value.slice(1) : value; + if (unsigned.length === 0) return undefined; + + let base = 10; + let digits = unsigned; + const prefix = unsigned.slice(0, 2).toLowerCase(); + if (prefix === "0x") { + base = 16; + digits = unsigned.slice(2); + } else if (prefix === "0o") { + base = 8; + digits = unsigned.slice(2); + } else if (prefix === "0b") { + base = 2; + digits = unsigned.slice(2); + } else if (unsigned.length > 1 && unsigned[0] === "0") { + // Legacy (no "o") leading-zero octal, e.g. "0755". + base = 8; + digits = unsigned.slice(1); + } + if (digits.length === 0) return undefined; + + // Only a real base prefix (or the legacy-octal leading "0") may be followed + // immediately by an underscore; a plain decimal literal has no prefix to + // follow, so a leading underscore there is always invalid (matches Go). + const hadPrefix = base !== 10; + const digitClass = base === 16 ? "0-9a-fA-F" : base === 8 ? "0-7" : base === 2 ? "01" : "0-9"; + const validPattern = new RegExp( + `^${hadPrefix ? "_?" : ""}[${digitClass}](?:_?[${digitClass}])*$`, + ); + if (!validPattern.test(digits)) return undefined; + + const cleanDigits = digits.replace(/_/g, ""); + // Exact-magnitude range check via BigInt — `Number.parseInt` below loses precision + // past 2^53 and never errors, so the int64 bound must be checked independently of it. + const bigPrefix = base === 16 ? "0x" : base === 8 ? "0o" : base === 2 ? "0b" : ""; + const magnitude = BigInt(`${bigPrefix}${cleanDigits}`); + const signedMagnitude = negative ? -magnitude : magnitude; + if (signedMagnitude > GO_MAX_INT64 || signedMagnitude < GO_MIN_INT64) return undefined; + + const n = Number.parseInt(cleanDigits, base); + return negative ? -n : n; +}; + +// `cast.ToInt`'s `trimDecimal` (`spf13/cast@v1.10.0/number.go:507-525`) runs BEFORE +// `strconv.ParseInt`: when the whole string is a sign + plain decimal digits + an +// optional ".digits" tail (`stringNumberRe`, `^([-+]?\d*)(\.\d*)?$` — never matches +// a `0x`/`0o`/`0b` literal, which contains letters), it drops the fractional part +// outright rather than rounding (`"5.5"` → `"5"`). Anything else (including a +// non-decimal-looking string that merely contains a ".") passes through unchanged +// and is left for {@link parseGoBaseZeroInt} to accept or reject. +const trimGoDecimal = (value: string): string => { + if (!value.includes(".")) return value; + const match = /^([+-]?\d*)(?:\.\d*)?$/.exec(value); + if (!match) return value; + const intPart = match[1] ?? ""; + if (intPart === "+" || intPart === "-") return `${intPart}0`; + return intPart === "" ? "0" : intPart; +}; + +const legacyParseScannerBufferSize = (raw: string): number => { + let value = raw.trim(); + let multiplier = 1; + const lastIndex = value.length - 1; + if (lastIndex > 1 && (value[lastIndex] === "b" || value[lastIndex] === "B")) { + switch (value[lastIndex - 1]!.toLowerCase()) { + case "k": + multiplier = 1 << 10; + value = value.slice(0, lastIndex - 1).trim(); + break; + case "m": + multiplier = 1 << 20; + value = value.slice(0, lastIndex - 1).trim(); + break; + case "g": + multiplier = 1 << 30; + value = value.slice(0, lastIndex - 1).trim(); + break; + default: + value = value.slice(0, lastIndex).trim(); + break; + } + } + const size = parseGoBaseZeroInt(trimGoDecimal(value)); + return size !== undefined && Number.isFinite(size) && size > 0 ? size * multiplier : 0; +}; + +/** + * Go's `parser.Split`/`SplitAndTrim` (`pkg/parser/token.go:81-119`) enforces + * `SUPABASE_SCANNER_BUFFER_SIZE` as the `bufio.Scanner`'s max token size — but only + * when the env var is actually SET: `parseFile` (`pkg/migration/file.go:55-70`) + * otherwise grows the package-level `parser.MaxScannerCapacity` to the real file's + * byte length before the scan even starts (`viper.IsSet("SCANNER_BUFFER_SIZE")` + * gates the auto-growth), so the DEFAULT (unset) path can never hit + * `bufio.ErrTooLong` for a file read this way — no single statement can be bigger + * than the whole file. Every caller of `execMigrationBatch` mirrors exactly this + * Go call site (`ApplyMigrations`, `SeedGlobals`, `applySchemaFiles` all read their + * file via `NewMigrationFromFile`/`parseFile`), so this is the correct single home + * for the check (CLI-1958 review) rather than duplicating it per caller. + * + * Fails the same way `parser.Split` does on the FIRST raw (pre-trim) statement + * whose byte length exceeds the effective limit (`Math.max(configured, + * GO_SCANNER_START_BUF_SIZE)` — see that constant's comment). `"After statement + * : …"` reports the count and RAW text of the last statement successfully + * scanned BEFORE the oversized one: Go's loop body (`token = scanner.Text()`) + * never runs for the failing `Scan()` call, so `token` still holds whatever the + * previous iteration left it as (`""` if the very first statement is already + * oversized) — verified empirically against the same `apps/cli-go/pkg/parser` + * probe. This is a "read"-phase failure (`NewMigrationFromFile`/`parseFile` + * returns before `apply.go`'s `CmdSuggestion` is ever set), so it carries no + * suggestion, same as the file-open failure above. + * + * `projectEnv`, when given, is the caller's already-loaded `legacyLoadProjectEnv` map: + * Go's `loadNestedEnv` (`pkg/config/config.go:1220`) `os.Setenv`s every project-`.env` + * key that isn't already in the shell env BEFORE `ParseDatabaseConfig` returns — i.e. + * before ANY command body (including this scan) runs — so `viper.AutomaticEnv()` sees a + * `supabase/.env`-only `SUPABASE_SCANNER_BUFFER_SIZE` exactly like a real shell-exported + * one. Defaults to `{}` for callers that haven't threaded a project-env map through + * (shell-only, same as before this parameter existed). + */ +export const checkScannerBufferSize = ( + content: string, + mapError: (message: string, phase: "read" | "exec") => E, + projectEnv: Readonly> = {}, +): Effect.Effect => { + const raw = + process.env["SUPABASE_SCANNER_BUFFER_SIZE"] ?? projectEnv["SUPABASE_SCANNER_BUFFER_SIZE"]; + if (raw === undefined) return Effect.void; + const configuredLimit = legacyParseScannerBufferSize(raw); + // `configuredLimit <= 0` covers both an explicit non-positive size and an unparseable + // value (e.g. a bare "5M", see `GO_DEFAULT_MAX_SCANNER_CAPACITY`'s comment) — Go's + // `viper.GetSizeInBytes` collapses all of these to `0` too, and `parser.Split` then + // falls back to its OWN hardcoded default cap, not to "no limit". + const limit = + configuredLimit > 0 + ? Math.max(configuredLimit, GO_SCANNER_START_BUF_SIZE) + : GO_DEFAULT_MAX_SCANNER_CAPACITY; + // Go's suggestion reports `maxbuf>>10` — the EFFECTIVE cap actually passed to + // `scanner.Buffer` (`pkg/parser/token.go:110`), which is the raw configured value + // (even below the `GO_SCANNER_START_BUF_SIZE` floor — the floor only affects when + // `bufio.ErrTooLong` can fire, never the number Go prints) when positive, or the + // hardcoded default once Go has fallen back to it. + const reportedLimit = configuredLimit > 0 ? configuredLimit : GO_DEFAULT_MAX_SCANNER_CAPACITY; + let emitted = 0; + let lastRaw = ""; + for (const token of legacySplitSqlTokens(content)) { + // A delimiter-terminated token is found (and emitted) by `parser.Split`'s scan in + // the SAME `Scan()` call that fills the buffer to capacity — before Go's too-long + // check is ever reached — so a token exactly AT `limit` still succeeds; only + // strictly-over fails (`>`). The trailing, unterminated token (only ever the LAST + // one `legacySplitSqlTokens` returns, if any — see `LegacySplitSqlToken.terminated`) + // has no delimiter to find: once the buffer fills to `limit` bytes without one, the + // too-long check fires immediately, without Go ever attempting the extra `Read()` + // that would reveal real EOF — so a trailing token AT `limit` already fails (`>=`). + const tooLong = token.terminated + ? utf8ByteLength(token.raw) > limit + : utf8ByteLength(token.raw) >= limit; + if (tooLong) { + const suggestion = `Try setting SUPABASE_SCANNER_BUFFER_SIZE=5MB (current size is ${Math.floor(reportedLimit / 1024)}KB)`; + return Effect.fail( + mapError( + `bufio.Scanner: token too long\nAfter statement ${emitted}: ${lastRaw}\n${suggestion}`, + "read", + ), + ); + } + // Go's `token = scanner.Text()` (`pkg/parser/token.go:96`) runs on EVERY successful + // `Scan()` — unconditionally, before the `len(trim) > 0` gate that decides whether to + // `append` to `stats` — so `token` (and therefore the eventual `bufio.ErrTooLong` + // message) reflects the last RAW text scanned even when that statement trimmed to + // empty and was never appended (e.g. a lone `;` immediately before an oversized + // statement reports "After statement N: ;", not a blank token). `emitted` mirrors + // Go's `len(stats)` (append-gated); `lastRaw` must NOT share that gate — verified + // against the Go source directly (review CLI-1958 round 18). + lastRaw = token.raw; + if (token.trimmed.length > 0) { + emitted += 1; + } + } + return Effect.void; +}; + /** * Port of Go's `markError` (`pkg/migration/file.go:117-132`): renders a `^` caret * line under the error position of the failing statement. `pos` is the server's @@ -157,101 +446,163 @@ const TYPE_NAME_PATTERN = /type "([^"]+)" does not exist/; * apply.go:65-69), so role/globals files (`legacySeedGlobals`) stay reset-free like Go. * When `forceNoVersion` is set the history insert is skipped regardless of filename * (Go's `SeedGlobals` clears `Version`). + * + * `projectEnv` is forwarded to {@link checkScannerBufferSize} — see its own doc comment + * for why a project-`.env`-only `SUPABASE_SCANNER_BUFFER_SIZE` must be visible here too. */ const execMigrationBatch = ( session: LegacyDbSession, fs: FileSystem.FileSystem, path: Path.Path, migrationPath: string, - mapError: (message: string) => E, + mapError: (message: string, phase: "read" | "exec") => E, forceNoVersion: boolean, + displayPath: string = migrationPath, + projectEnv: Readonly> = {}, ): Effect.Effect => Effect.gen(function* () { - const content = yield* fs.readFileString(migrationPath); - const statements = legacySplitAndTrim(content); - const filename = path.basename(migrationPath); - const matches = MIGRATE_FILE_PATTERN.exec(filename); - const version = forceNoVersion ? "" : (matches?.[1] ?? ""); - const name = matches?.[2] ?? ""; - - // Mirror Go's `MigrationFile.ExecBatch` error context (`pkg/migration/file.go:88-113`): - // on a failed statement, render the `^` caret under the server-reported error - // position, the `Detail` line when present, the SQLSTATE-42704 extension hint, - // then `At statement: ` and the (caret-marked) statement text. The - // structured `detail`/`position` fields are only set by the driver for server - // ErrorResponses, mirroring Go's `errors.As(err, &pgErr)` gate. - const atStatement = (e: LegacyDbExecError, index: number, stat: string) => { - const marked = legacyMarkError(stat, e.position ?? 0); - const msg: Array = []; - if (e.detail !== undefined && e.detail.length > 0) { - msg.push(e.detail); - } - // Provide helpful hint for extension type errors (SQLSTATE 42704: undefined_object) - const typeName = TYPE_NAME_PATTERN.exec(e.message)?.[1]; - if (typeName !== undefined && e.code === "42704" && !typeName.includes(".")) { - msg.push(""); - msg.push("Hint: This type may be defined in a schema that's not in your search_path."); - msg.push(" Use schema-qualified type references to avoid this error:"); - msg.push(` CREATE TABLE example (col extensions.${typeName});`); - msg.push(" Learn more: supabase migration new --help"); - } - msg.push(`At statement: ${index}`, marked); - return new Error(`${errMessage(e)}\n${msg.join("\n")}`); - }; - - // `executed` is the global statement index of the next statement to run, so the - // error context stays accurate across flushed batches and standalone statements - // (Go threads the same counter through `ExecBatch`). - let pending: ReadonlyArray = []; - let executed = 0; - - const flushBatch = Effect.gen(function* () { - if (pending.length === 0) return; - const items = pending; - pending = []; - const base = executed; - const body = Effect.gen(function* () { - for (const [offset, item] of items.entries()) { - const index = base + offset; - if (item.kind === "version") { - // Go defaults to the version-insert statement when all listed statements succeed. - yield* session - .query(INSERT_MIGRATION_VERSION, [version, name, statements]) - .pipe( - Effect.mapError((cause) => atStatement(cause, index, INSERT_MIGRATION_VERSION)), - ); - } else { - yield* session - .exec(item.sql) - .pipe(Effect.mapError((cause) => atStatement(cause, index, item.sql))); - } + // Go's `MigrationFile.ExecBatch` receives an already-read/parsed file (the read + // happens earlier, in `NewMigrationFromFile`/`parseFile`, which wraps the open + // failure as `"failed to open migration file: %w"`, `pkg/migration/file.go:57-58`) + // — so a read failure here is a DIFFERENT error class than a statement-execution + // failure below, and needs the same Go prefix so stderr/JSON errors don't surface + // the bare platform error text. Tagged "read" so callers that attach a suggestion + // only around execution failures (`apply.go:61-63`) can tell the two apart. + // + // Go opens `fp` — the workdir-RELATIVE form `[db.migrations].schema_paths`/ + // `[db.seed].sql_paths` already resolved to at config-load time — because Go's + // process cwd is always the workdir (`ChangeWorkDir`, `cmd/root.go:104`). This + // module deliberately never `process.chdir`s (only `bootstrap` does, as its own + // documented one-off), so callers must pass an ABSOLUTE `migrationPath` for the + // real read to work — but that means the platform error's embedded path is + // absolute too. When it differs from `displayPath` (the caller's Go-equivalent + // relative path), substitute it in so the wrapped message still reports the + // relative form Go would, not a leaked local temp/absolute path. + // + // Known residual delta (CLI-1958 review): `readFileString` decodes via `TextDecoder` + // with `fatal: false` (the Effect `FileSystem` default), so an invalid-UTF-8 byte + // sequence in the file is lossily replaced with U+FFFD before it ever reaches + // `legacySplitAndTrim`/`session.exec`. Go's `parseFile` instead scans the raw byte + // stream and preserves those bytes verbatim into the statement strings it sends to + // PostgreSQL. Reading raw bytes here (`fs.readFile`) and mapping them 1:1 into a + // "binary string" would fix the split/parse stage, but the fix dies at the wire: the + // shared `pg`/`pg-protocol` layer this session is built on unconditionally UTF-8- + // encodes query text before writing it (`pg-protocol/dist/serializer.js` — + // `buff.write(string, offset, 'utf-8')`, no raw-byte send API), so ANY string + // representation still gets re-mangled at that boundary, just differently. Faithful + // byte parity would require patching that shared wire-serializer — infrastructure + // every legacy DB command's `session.exec` funnels through, not something scoped to + // this file's read path — so it's flagged here rather than "fixed" underneath it. + const content = yield* fs.readFileString(migrationPath).pipe( + Effect.mapError((error) => { + const message = legacyRelativizeErrorMessage( + legacyErrorMessage(error), + migrationPath, + displayPath, + ); + return mapError(`failed to open migration file: ${message}`, "read"); + }), + ); + + // Still `NewMigrationFromFile`/`parseFile`'s territory (`pkg/migration/file.go:55-70`) — + // `parser.SplitAndTrim` runs INSIDE `parseFile`, before `ExecBatch` ever sees the + // statements, so a `SUPABASE_SCANNER_BUFFER_SIZE` violation is a "read"-phase + // failure like the open failure above, not an "exec"-phase one. See + // `checkScannerBufferSize`'s comment for why this is a no-op unless the env var + // is explicitly set. + yield* checkScannerBufferSize(content, mapError, projectEnv); + + // Everything below mirrors Go's `(*MigrationFile).ExecBatch` (`pkg/migration/file.go`), + // which runs against an already-read file — so every failure from here on is an + // execution failure, tagged "exec" (as opposed to the "read" failure above, which + // mirrors `NewMigrationFromFile`). Only execution failures get `CmdSuggestion` + // (`apply.go:61-63`); callers rely on this tag to replicate that split. + yield* Effect.gen(function* () { + const statements = legacySplitAndTrim(content); + const filename = path.basename(migrationPath); + const matches = MIGRATE_FILE_PATTERN.exec(filename); + const version = forceNoVersion ? "" : (matches?.[1] ?? ""); + const name = matches?.[2] ?? ""; + + // Mirror Go's `MigrationFile.ExecBatch` error context (`pkg/migration/file.go:88-113`): + // on a failed statement, render the `^` caret under the server-reported error + // position, the `Detail` line when present, the SQLSTATE-42704 extension hint, + // then `At statement: ` and the (caret-marked) statement text. The + // structured `detail`/`position` fields are only set by the driver for server + // ErrorResponses, mirroring Go's `errors.As(err, &pgErr)` gate. + const atStatement = (e: LegacyDbExecError, index: number, stat: string) => { + const marked = legacyMarkError(stat, e.position ?? 0); + const msg: Array = []; + if (e.detail !== undefined && e.detail.length > 0) { + msg.push(e.detail); } - yield* session.exec("COMMIT"); + // Provide helpful hint for extension type errors (SQLSTATE 42704: undefined_object) + const typeName = TYPE_NAME_PATTERN.exec(e.message)?.[1]; + if (typeName !== undefined && e.code === "42704" && !typeName.includes(".")) { + msg.push(""); + msg.push("Hint: This type may be defined in a schema that's not in your search_path."); + msg.push(" Use schema-qualified type references to avoid this error:"); + msg.push(` CREATE TABLE example (col extensions.${typeName});`); + msg.push(" Learn more: supabase migration new --help"); + } + msg.push(`At statement: ${index}`, marked); + return new Error(`${legacyErrorMessage(e)}\n${msg.join("\n")}`); + }; + + // `executed` is the global statement index of the next statement to run, so the + // error context stays accurate across flushed batches and standalone statements + // (Go threads the same counter through `ExecBatch`). + let pending: ReadonlyArray = []; + let executed = 0; + + const flushBatch = Effect.gen(function* () { + if (pending.length === 0) return; + const items = pending; + pending = []; + const base = executed; + const body = Effect.gen(function* () { + for (const [offset, item] of items.entries()) { + const index = base + offset; + if (item.kind === "version") { + // Go defaults to the version-insert statement when all listed statements succeed. + yield* session + .query(INSERT_MIGRATION_VERSION, [version, name, statements]) + .pipe( + Effect.mapError((cause) => atStatement(cause, index, INSERT_MIGRATION_VERSION)), + ); + } else { + yield* session + .exec(item.sql) + .pipe(Effect.mapError((cause) => atStatement(cause, index, item.sql))); + } + } + yield* session.exec("COMMIT"); + }); + yield* session.exec("BEGIN"); + yield* body.pipe(Effect.tapError(() => session.exec("ROLLBACK").pipe(Effect.ignore))); + executed += items.length; }); - yield* session.exec("BEGIN"); - yield* body.pipe(Effect.tapError(() => session.exec("ROLLBACK").pipe(Effect.ignore))); - executed += items.length; - }); - - for (const statement of statements) { - if (legacyIsPipelineIncompatible(statement)) { - // Flush the open batch, then run the incompatible statement on its own (no - // surrounding transaction) so PostgreSQL accepts it. - yield* flushBatch; - const index = executed; - yield* session - .exec(statement) - .pipe(Effect.mapError((cause) => atStatement(cause, index, statement))); - executed += 1; - } else { - pending = [...pending, { kind: "exec", sql: statement }]; + + for (const statement of statements) { + if (legacyIsPipelineIncompatible(statement)) { + // Flush the open batch, then run the incompatible statement on its own (no + // surrounding transaction) so PostgreSQL accepts it. + yield* flushBatch; + const index = executed; + yield* session + .exec(statement) + .pipe(Effect.mapError((cause) => atStatement(cause, index, statement))); + executed += 1; + } else { + pending = [...pending, { kind: "exec", sql: statement }]; + } } - } - if (version.length > 0) { - pending = [...pending, { kind: "version" }]; - } - yield* flushBatch; - }).pipe(Effect.mapError((error) => mapError(errMessage(error)))); + if (version.length > 0) { + pending = [...pending, { kind: "version" }]; + } + yield* flushBatch; + }).pipe(Effect.mapError((error) => mapError(legacyErrorMessage(error), "exec"))); + }); /** * Go's per-migration connection reset (`apply.go:65-69`): `RESET ALL` clears any @@ -264,7 +615,7 @@ const resetConnectionState = ( session: LegacyDbSession, mapError: (message: string) => E, ): Effect.Effect => - session.exec("RESET ALL").pipe(Effect.mapError((e) => mapError(errMessage(e)))); + session.exec("RESET ALL").pipe(Effect.mapError((e) => mapError(legacyErrorMessage(e)))); /** * Applies a single migration file to the connected database and records it in @@ -286,7 +637,7 @@ export const legacyApplyMigrationFile = ( Effect.gen(function* () { yield* resetConnectionState(session, mapError); yield* legacyCreateMigrationTable(session).pipe( - Effect.mapError((e) => mapError(errMessage(e))), + Effect.mapError((e) => mapError(legacyErrorMessage(e))), ); yield* execMigrationBatch(session, fs, path, migrationPath, mapError, false); }); @@ -308,7 +659,7 @@ export const legacyApplyMigrations = ( const output = yield* Output; if (pending.length === 0) return; yield* legacyCreateMigrationTable(session).pipe( - Effect.mapError((e) => mapError(errMessage(e))), + Effect.mapError((e) => mapError(legacyErrorMessage(e))), ); for (const migrationPath of pending) { yield* output.raw(`Applying migration ${path.basename(migrationPath)}...\n`, "stderr"); @@ -352,11 +703,98 @@ export const legacySeedGlobals = ( * would print an extra line Go never prints. Callers write the in-memory SQL * constant to a temp file first (this module only reads files, like * `execMigrationBatch`'s other callers). + * + * `displayPath`, when given, is the path a read-failure's wrapped message should + * report instead of `filePath` — see `execMigrationBatch`'s comment on why the two + * can differ (an absolute path is required for the real read, but Go's equivalent + * error names the workdir-relative form). `projectEnv`, when given, is forwarded to + * {@link checkScannerBufferSize} via `execMigrationBatch` — see that helper's doc + * comment. */ export const legacyExecSqlFile = ( session: LegacyDbSession, fs: FileSystem.FileSystem, path: Path.Path, filePath: string, - mapError: (message: string) => E, -): Effect.Effect => execMigrationBatch(session, fs, path, filePath, mapError, true); + mapError: (message: string, phase: "read" | "exec") => E, + displayPath?: string, + projectEnv?: Readonly>, +): Effect.Effect => + execMigrationBatch(session, fs, path, filePath, mapError, true, displayPath, projectEnv); + +/** + * Applies Go's EXPERIMENTAL declarative schema-files branch of `apply.MigrateAndSeed` + * (`apps/cli-go/internal/migration/apply/apply.go:19,51-68`). Reads `[db.migrations] + * schema_paths` (already resolved to Go's config-load form — supabase-joined when + * relative, verbatim when absolute) via the shared `Glob.SQLFiles` port + * ({@link legacySqlFilesGlob}), then runs each matched file's statements with + * {@link legacyExecSqlFile} in glob order — no history table, no history row, and no + * `RESET ALL` between files, matching Go's `schema.Version = ""` discard (`apply.go:61`) + * and the fact that `ExecBatch` (unlike `ApplyMigrations`) never resets connection state. + * + * Callers gate the call on Go's three-conjunct condition (`--experimental` + no resolved + * version + pg-delta NOT enabled, `apply.go:19`) themselves — this function only performs + * the branch's body, mirroring `applySchemaFiles`'s own signature (it never re-checks the + * gate). It is the caller's responsibility to skip `legacyApplyMigrations` entirely when + * this is called (Go's `if`/`else if` is mutually exclusive, `apply.go:19-27`). + * + * Faithfully reproduces two undocumented, unfixed-upstream Go quirks that are load-bearing + * for the strict 1:1 contract (CLI-1958): + * - **Empty `schema_paths` (the `supabase init` default) silently applies nothing** and + * returns success — `Config.Db.Migrations.SchemaPaths.SQLFiles` returns a `nil` error + * when there are zero patterns to glob (`errors.Join()` with no arguments is `nil`), so + * `applySchemaFiles` returns `nil` too (`apply.go:53-54`). + * - **A PARTIAL glob failure is silently dropped**: per-pattern warnings are only + * surfaced (as the returned failure) when NO pattern matched anything at all + * (`declared` empty, `apply.go:53-55`); once at least one file is found, every other + * pattern's warning is discarded — unlike the seed path's `WARN:` line. + * + * On a per-file EXECUTION failure only, attaches Go's `CmdSuggestion = "See schema file: + * "` (`apply.go:63`) via the optional second argument of `mapError`. A file-READ + * failure (Go's `NewMigrationFromFile`, `apply.go:57-59`) returns before `CmdSuggestion` is + * ever set, so it must NOT carry the suggestion — {@link legacyExecSqlFile}'s `mapError` + * receives the `"read"`/`"exec"` phase precisely so this call site can tell them apart. + * + * `projectEnv` is the caller's already-loaded `legacyLoadProjectEnv` map, forwarded to + * {@link checkScannerBufferSize} (via `legacyExecSqlFile`/`execMigrationBatch`) so a + * `SUPABASE_SCANNER_BUFFER_SIZE` set only in `supabase/.env` is honored here exactly like + * a real Go run — see that helper's doc comment. + */ +export const legacyApplySchemaFiles = ( + session: LegacyDbSession, + fs: FileSystem.FileSystem, + path: Path.Path, + workdir: string, + schemaPaths: ReadonlyArray, + mapError: (message: string, suggestion?: string) => E, + projectEnv: Readonly> = {}, +): Effect.Effect => + Effect.gen(function* () { + const { files, warnings } = yield* legacySqlFilesGlob(fs, path, schemaPaths, workdir); + if (files.length === 0) { + // Go: `if len(declared) == 0 { return err }` — `err` is `nil` when there were no + // patterns to glob at all, and the joined per-pattern warnings otherwise. + if (warnings.length > 0) { + return yield* Effect.fail(mapError(warnings.join("\n"))); + } + return; + } + for (const file of files) { + const absolutePath = path.isAbsolute(file) ? file : path.join(workdir, file); + // `file` is already Go's `fp` form (workdir-relative when the declared pattern + // was relative, verbatim when absolute) — pass it through as the display path so + // a read failure reports it instead of the `absolutePath` the real read needs. + yield* legacyExecSqlFile( + session, + fs, + path, + absolutePath, + (message, phase) => + phase === "exec" + ? mapError(message, `See schema file: ${legacyBold(file)}`) + : mapError(message), + file, + projectEnv, + ); + } + }); diff --git a/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts b/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts index 749fb9b70b..3a5ed138b3 100644 --- a/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-migration-apply.unit.test.ts @@ -1,4 +1,4 @@ -import { mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { chmodSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { BunServices } from "@effect/platform-bun"; @@ -9,6 +9,7 @@ import { mockOutput } from "../../../tests/helpers/mocks.ts"; import type { LegacyDbSession } from "./legacy-db-connection.service.ts"; import { legacyApplyMigrationFile, + legacyApplySchemaFiles, legacyIsPipelineIncompatible, legacyMarkError, legacySeedGlobals, @@ -131,6 +132,33 @@ describe("legacyApplyMigrationFile", () => { ); }); + it.effect( + "wraps a read failure with Go's parse-file error text (Go NewMigrationFromFile parity)", + () => { + // Go's `NewMigrationFromFile`/`parseFile` wraps the open failure as + // `"failed to open migration file: %w"` (`pkg/migration/file.go:57-58`) before + // `ApplyMigrations`/`applySchemaFiles` ever get a chance to attach a + // `CmdSuggestion` — a read failure here must carry the same prefix, not the bare + // platform error text. + const dir = mkdtempSync(join(tmpdir(), "legacy-apply-read-fail-")); + const missingFile = join(dir, "20240101120000_missing.sql"); + const { session } = fakeSession(); + return run(session, missingFile).pipe( + Effect.exit, + Effect.tap((exit) => + Effect.sync(() => { + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const msg = JSON.stringify(exit.cause); + expect(msg).toContain("failed to open migration file: "); + } + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + it.effect("runs a pipeline-incompatible statement outside the surrounding transaction", () => { const dir = mkdtempSync(join(tmpdir(), "legacy-apply-")); const file = join(dir, "20240101120000_add_index.sql"); @@ -437,3 +465,574 @@ describe("legacySeedGlobals", () => { ); }); }); + +describe("legacyApplySchemaFiles", () => { + it.effect( + "reports a read failure with the workdir-relative path, not the absolute path used to read it (Go open supabase/... parity)", + () => { + // Go opens the workdir-relative `fp` from `schema_paths` directly (its process + // cwd is always the workdir, `ChangeWorkDir`), so a read failure reports + // `open supabase/unreadable.sql: ...`. This module never `process.chdir`s, so + // the real read needs an absolute path — but the wrapped read-failure message + // must still show the relative `supabase/...` form, not that absolute path. An + // unreadable file (a real permission failure, not a missing-path one) reproduces + // a genuine read failure while still passing the glob's own stat/type check — + // `stat` only needs directory execute permission, not read permission on the + // file itself, so this still resolves as a `"File"` match, unlike a directory + // (which the glob would instead expand via `legacyWalkSqlFiles`). + const dir = mkdtempSync(join(tmpdir(), "legacy-schema-files-read-fail-")); + const file = join(dir, "supabase", "unreadable.sql"); + mkdirSync(join(dir, "supabase"), { recursive: true }); + writeFileSync(file, "select 1;"); + chmodSync(file, 0o000); + const { session } = fakeSession(); + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const exit = yield* legacyApplySchemaFiles( + session, + fs, + path, + dir, + ["supabase/unreadable.sql"], + (message, suggestion) => + new TestError({ message: suggestion ? `${message} (${suggestion})` : message }), + ).pipe(Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const msg = JSON.stringify(exit.cause); + expect(msg).toContain("failed to open migration file: "); + expect(msg).toContain("supabase/unreadable.sql"); + expect(msg).not.toContain(dir); + } + chmodSync(file, 0o644); + rmSync(dir, { recursive: true, force: true }); + }).pipe(Effect.provide(BunServices.layer)); + }, + ); + + it.effect( + "rejects an oversized statement when SUPABASE_SCANNER_BUFFER_SIZE is configured (Go bufio.Scanner: token too long parity)", + () => { + // Go's `parser.Split` (`pkg/parser/token.go:81-119`) only enforces + // `SUPABASE_SCANNER_BUFFER_SIZE` when it's explicitly set — `parseFile` + // otherwise auto-grows the scanner to the real file's byte length, so the + // DEFAULT path can never hit `bufio.ErrTooLong`. With it set below a single + // statement's raw byte length, Go fails with `bufio.Scanner: token too long` + // instead of silently applying the oversized statement — verified empirically + // against `apps/cli-go/pkg/parser` (a `parser.SplitAndTrim` scratch probe). + const dir = mkdtempSync(join(tmpdir(), "legacy-schema-files-scanner-")); + mkdirSync(join(dir, "supabase"), { recursive: true }); + const file = join(dir, "supabase", "big.sql"); + // A single, un-splittable statement whose raw text exceeds the 4096-byte floor + // (Go's `bufio.Scanner` starts at that size regardless of the configured limit). + writeFileSync(file, `SELECT 1;\nSELECT '${"a".repeat(5000)}';\n`); + const { session } = fakeSession(); + const previous = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = "100b"; + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const exit = yield* legacyApplySchemaFiles( + session, + fs, + path, + dir, + ["supabase/big.sql"], + (message, suggestion) => + new TestError({ message: suggestion ? `${message} (${suggestion})` : message }), + ).pipe(Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const msg = JSON.stringify(exit.cause); + expect(msg).toContain("bufio.Scanner: token too long"); + expect(msg).toContain("After statement 1: SELECT 1;"); + expect(msg).toContain("Try setting SUPABASE_SCANNER_BUFFER_SIZE=5MB"); + } + }).pipe( + Effect.ensuring( + Effect.sync(() => { + if (previous === undefined) delete process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + else process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = previous; + rmSync(dir, { recursive: true, force: true }); + }), + ), + Effect.provide(BunServices.layer), + ); + }, + ); + + it.effect( + "reports the last scanned RAW token in the too-long error even when it trimmed to empty (Go scanner.Text() parity, review CLI-1958)", + () => { + // Go's `token = scanner.Text()` (`pkg/parser/token.go:96`) runs on EVERY + // successful `Scan()`, unconditionally — BEFORE the `len(trim) > 0` gate that + // decides whether to append to `stats`. So when a statement trims to empty + // (a lone ";") immediately before an oversized one, Go's `bufio.ErrTooLong` + // message still reports that lone ";" as the last-scanned text, not a blank + // token — `len(stats)` (this port's `emitted`) stays gated on non-empty trim, + // but the reported RAW text must not share that gate. + const dir = mkdtempSync(join(tmpdir(), "legacy-schema-files-scanner-empty-token-")); + mkdirSync(join(dir, "supabase"), { recursive: true }); + const file = join(dir, "supabase", "big.sql"); + writeFileSync(file, `;\nSELECT '${"a".repeat(5000)}';\n`); + const { session } = fakeSession(); + const previous = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = "100b"; + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const exit = yield* legacyApplySchemaFiles( + session, + fs, + path, + dir, + ["supabase/big.sql"], + (message, suggestion) => + new TestError({ message: suggestion ? `${message} (${suggestion})` : message }), + ).pipe(Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const msg = JSON.stringify(exit.cause); + expect(msg).toContain("bufio.Scanner: token too long"); + // 0 statements were EMITTED (the lone ";" trimmed to empty and was never + // appended), but the last scanned RAW token (";") must still show — not a + // blank token, which a trim-gated tracker would wrongly report instead. + expect(msg).toContain("After statement 0: ;"); + } + }).pipe( + Effect.ensuring( + Effect.sync(() => { + if (previous === undefined) delete process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + else process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = previous; + rmSync(dir, { recursive: true, force: true }); + }), + ), + Effect.provide(BunServices.layer), + ); + }, + ); + + it.effect( + "applies an oversized statement fine when SUPABASE_SCANNER_BUFFER_SIZE is unset (Go's default auto-grows to file size)", + () => { + const dir = mkdtempSync(join(tmpdir(), "legacy-schema-files-scanner-default-")); + mkdirSync(join(dir, "supabase"), { recursive: true }); + const file = join(dir, "supabase", "big.sql"); + writeFileSync(file, `SELECT '${"a".repeat(5000)}';\n`); + const { session, calls } = fakeSession(); + const previous = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + delete process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + yield* legacyApplySchemaFiles( + session, + fs, + path, + dir, + ["supabase/big.sql"], + (message, suggestion) => + new TestError({ message: suggestion ? `${message} (${suggestion})` : message }), + ); + expect(calls.some((c) => c.kind === "exec" && c.sql.startsWith("SELECT 'a"))).toBe(true); + }).pipe( + Effect.ensuring( + Effect.sync(() => { + if (previous !== undefined) process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = previous; + rmSync(dir, { recursive: true, force: true }); + }), + ), + Effect.provide(BunServices.layer), + ); + }, + ); + + it.effect( + "falls back to Go's hardcoded default cap when SUPABASE_SCANNER_BUFFER_SIZE is set but unparseable (viper parity, not '5M' == 5MiB)", + () => { + // Verified empirically against `apps/cli-go/pkg/parser` + vendored + // `viper@v1.21.0`: a bare multiplier suffix with NO trailing "b"/"B" (e.g. "5M") + // is NOT 5 MiB in real Go — `parseSizeInBytes` only recognizes a multiplier when + // the string's LAST character is literally "b"/"B", so "5M" never strips a + // suffix and `cast.ToInt("5M")` fails whole, yielding 0. `viper.IsSet` is still + // true (the var IS present), so `parseFile`'s file-size auto-growth never runs + // — `parser.Split` falls back to its OWN hardcoded default cap + // (`MaxScannerCapacity`, 256KiB), not to "no limit" and not to a tiny 5-byte + // limit either. A statement past that hardcoded default must still fail. + const dir = mkdtempSync(join(tmpdir(), "legacy-schema-files-scanner-garbage-")); + mkdirSync(join(dir, "supabase"), { recursive: true }); + const file = join(dir, "supabase", "big.sql"); + writeFileSync(file, `SELECT 1;\nSELECT '${"a".repeat(300_000)}';\n`); + const { session } = fakeSession(); + const previous = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = "5M"; + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const exit = yield* legacyApplySchemaFiles( + session, + fs, + path, + dir, + ["supabase/big.sql"], + (message, suggestion) => + new TestError({ message: suggestion ? `${message} (${suggestion})` : message }), + ).pipe(Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const msg = JSON.stringify(exit.cause); + expect(msg).toContain("bufio.Scanner: token too long"); + // 256KiB (Go's `parser.MaxScannerCapacity` default), not "5MB" and not ~0KB. + expect(msg).toContain( + "Try setting SUPABASE_SCANNER_BUFFER_SIZE=5MB (current size is 256KB)", + ); + } + }).pipe( + Effect.ensuring( + Effect.sync(() => { + if (previous === undefined) delete process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + else process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = previous; + rmSync(dir, { recursive: true, force: true }); + }), + ), + Effect.provide(BunServices.layer), + ); + }, + ); + + it.effect( + "accepts a hex-literal SUPABASE_SCANNER_BUFFER_SIZE (Go strconv.ParseInt base-0 parity, review CLI-1958)", + () => { + // `viper.GetSizeInBytes` → `cast.ToInt` → `strconv.ParseInt(s, 0, 0)` parses + // with base 0, so a `0x`-prefixed literal is a valid byte count in real Go: + // "0x1400" is 5120 (5KiB) — verified empirically against vendored + // `viper@v1.21.0` (`viper.GetSizeInBytes("SCANNER_BUFFER_SIZE")` with the env + // var set to "0x1400" returns 5120). A decimal-only parser would reject this + // string outright and silently fall back to the 256KiB default instead, so a + // statement between 5120 and 262144 bytes would apply in TS but Go would + // already have failed with "bufio.Scanner: token too long" at 5121 bytes. + const dir = mkdtempSync(join(tmpdir(), "legacy-schema-files-scanner-hex-")); + mkdirSync(join(dir, "supabase"), { recursive: true }); + const file = join(dir, "supabase", "big.sql"); + writeFileSync(file, `SELECT 1;\nSELECT '${"a".repeat(5116)}';\n`); + const { session } = fakeSession(); + const previous = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = "0x1400"; + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const exit = yield* legacyApplySchemaFiles( + session, + fs, + path, + dir, + ["supabase/big.sql"], + (message, suggestion) => + new TestError({ message: suggestion ? `${message} (${suggestion})` : message }), + ).pipe(Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const msg = JSON.stringify(exit.cause); + expect(msg).toContain("bufio.Scanner: token too long"); + // 5KiB (0x1400 bytes), not the 256KiB hardcoded fallback a decimal-only + // parser would have silently used instead. + expect(msg).toContain( + "Try setting SUPABASE_SCANNER_BUFFER_SIZE=5MB (current size is 5KB)", + ); + } + }).pipe( + Effect.ensuring( + Effect.sync(() => { + if (previous === undefined) delete process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + else process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = previous; + rmSync(dir, { recursive: true, force: true }); + }), + ), + Effect.provide(BunServices.layer), + ); + }, + ); + + it.effect( + "accepts underscore digit separators in a decimal SUPABASE_SCANNER_BUFFER_SIZE (Go strconv.ParseInt base-0 underscore-literal parity, review CLI-1958)", + () => { + // Go's base-0 integer grammar (`go.dev/ref/spec#Integer_literals`, reproduced + // by `strconv.ParseInt`) permits a single `_` between digits: "5_120" is the + // same 5120 (5KiB) byte count as the hex-literal test above's "0x1400" — + // verified empirically against the real `strconv.ParseInt("5_120", 0, 64)`. + // A parser that rejects underscores outright would silently fall back to the + // 256KiB default instead, so a statement between 5120 and 262144 bytes would + // apply in TS but Go would already have failed with "bufio.Scanner: token too + // long" at 5121 bytes. + const dir = mkdtempSync(join(tmpdir(), "legacy-schema-files-scanner-underscore-")); + mkdirSync(join(dir, "supabase"), { recursive: true }); + const file = join(dir, "supabase", "big.sql"); + writeFileSync(file, `SELECT 1;\nSELECT '${"a".repeat(5116)}';\n`); + const { session } = fakeSession(); + const previous = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = "5_120"; + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const exit = yield* legacyApplySchemaFiles( + session, + fs, + path, + dir, + ["supabase/big.sql"], + (message, suggestion) => + new TestError({ message: suggestion ? `${message} (${suggestion})` : message }), + ).pipe(Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const msg = JSON.stringify(exit.cause); + expect(msg).toContain("bufio.Scanner: token too long"); + // 5KiB (5_120 bytes), not the 256KiB hardcoded fallback an + // underscore-rejecting parser would have silently used instead. + expect(msg).toContain( + "Try setting SUPABASE_SCANNER_BUFFER_SIZE=5MB (current size is 5KB)", + ); + } + }).pipe( + Effect.ensuring( + Effect.sync(() => { + if (previous === undefined) delete process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + else process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = previous; + rmSync(dir, { recursive: true, force: true }); + }), + ), + Effect.provide(BunServices.layer), + ); + }, + ); + + it.effect( + "rejects an invalid underscore placement in SUPABASE_SCANNER_BUFFER_SIZE, unlike a valid digit separator (Go strconv.ParseInt underscore-grammar parity, review CLI-1958)", + () => { + // Go only permits a SINGLE underscore immediately after a base prefix or + // between two digits — never leading a plain (no-prefix) decimal literal, + // never doubled, never trailing. "_5120" (leading underscore, no prefix) is + // invalid in real Go (`strconv.ParseInt("_5120", 0, 64)` errors), so it falls + // back to the same 256KiB default as a genuinely unset/unparseable value — + // verified empirically against the real Go `strconv.ParseInt`. + const dir = mkdtempSync(join(tmpdir(), "legacy-schema-files-scanner-bad-underscore-")); + mkdirSync(join(dir, "supabase"), { recursive: true }); + const file = join(dir, "supabase", "big.sql"); + writeFileSync(file, `SELECT 1;\nSELECT '${"a".repeat(5116)}';\n`); + const { session } = fakeSession(); + const previous = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = "_5120"; + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const exit = yield* legacyApplySchemaFiles( + session, + fs, + path, + dir, + ["supabase/big.sql"], + (message, suggestion) => + new TestError({ message: suggestion ? `${message} (${suggestion})` : message }), + ).pipe(Effect.exit); + // The 5116-byte statement fits comfortably under the 256KiB default + // fallback, so an invalid underscore placement must NOT fail the apply. + expect(Exit.isSuccess(exit)).toBe(true); + }).pipe( + Effect.ensuring( + Effect.sync(() => { + if (previous === undefined) delete process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + else process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = previous; + rmSync(dir, { recursive: true, force: true }); + }), + ), + Effect.provide(BunServices.layer), + ); + }, + ); + + it.effect( + "falls back to Go's hardcoded default cap when SUPABASE_SCANNER_BUFFER_SIZE overflows Go's signed int range (strconv.ParseInt/cast.ToInt range-error parity, review CLI-1958)", + () => { + // "9223372036854775808" is one more than `math.MaxInt64`. Go's + // `strconv.ParseInt(s, 0, 0)` rejects it with a range error, and `cast.ToInt` + // (`spf13/cast@v1.10.0/number.go:407-414`) discards ANY `parseFn` error — + // range or syntax — returning exactly `0`, never the huge (if imprecise) + // magnitude `Number.parseInt` would otherwise accept. `viper.IsSet` is still + // true, so this falls back to the 256KiB hardcoded default, same as a + // genuinely unparseable value ("5M" above) — NOT to "no limit". Verified + // empirically against the pinned `spf13/cast@v1.10.0` + // (`cast.ToInt("9223372036854775808")` → `0`). + const dir = mkdtempSync(join(tmpdir(), "legacy-schema-files-scanner-int64-overflow-")); + mkdirSync(join(dir, "supabase"), { recursive: true }); + const file = join(dir, "supabase", "big.sql"); + writeFileSync(file, `SELECT 1;\nSELECT '${"a".repeat(300_000)}';\n`); + const { session } = fakeSession(); + const previous = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = "9223372036854775808"; + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const exit = yield* legacyApplySchemaFiles( + session, + fs, + path, + dir, + ["supabase/big.sql"], + (message, suggestion) => + new TestError({ message: suggestion ? `${message} (${suggestion})` : message }), + ).pipe(Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const msg = JSON.stringify(exit.cause); + expect(msg).toContain("bufio.Scanner: token too long"); + // 256KiB (Go's hardcoded default), not "no limit" — a treat-as-unbounded + // bug would let this 300_000-byte statement apply successfully instead. + expect(msg).toContain( + "Try setting SUPABASE_SCANNER_BUFFER_SIZE=5MB (current size is 256KB)", + ); + } + }).pipe( + Effect.ensuring( + Effect.sync(() => { + if (previous === undefined) delete process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + else process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = previous; + rmSync(dir, { recursive: true, force: true }); + }), + ), + Effect.provide(BunServices.layer), + ); + }, + ); + + it.effect( + "still accepts the exact int64 boundary magnitudes for SUPABASE_SCANNER_BUFFER_SIZE (Go strconv.ParseInt range-boundary parity, review CLI-1958)", + () => { + // `math.MaxInt64` itself ("9223372036854775807", one less than the overflow + // test above) is NOT a range error in Go — only magnitudes strictly beyond it + // are. A range check that's off-by-one in the strict direction would wrongly + // reject this legitimate (if enormous) configured size and fall back to the + // 256KiB default instead of the requested cap. + const dir = mkdtempSync(join(tmpdir(), "legacy-schema-files-scanner-int64-boundary-")); + mkdirSync(join(dir, "supabase"), { recursive: true }); + const file = join(dir, "supabase", "big.sql"); + writeFileSync(file, `SELECT 1;\nSELECT '${"a".repeat(5116)}';\n`); + const { session } = fakeSession(); + const previous = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = "9223372036854775807"; + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const exit = yield* legacyApplySchemaFiles( + session, + fs, + path, + dir, + ["supabase/big.sql"], + (message, suggestion) => + new TestError({ message: suggestion ? `${message} (${suggestion})` : message }), + ).pipe(Effect.exit); + // The 5116-byte statement fits comfortably under the (enormous) configured + // limit, so this must succeed, not fall back to the 256KiB default. + expect(Exit.isSuccess(exit)).toBe(true); + }).pipe( + Effect.ensuring( + Effect.sync(() => { + if (previous === undefined) delete process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + else process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = previous; + rmSync(dir, { recursive: true, force: true }); + }), + ), + Effect.provide(BunServices.layer), + ); + }, + ); + + it.effect( + "rejects an oversized statement when SUPABASE_SCANNER_BUFFER_SIZE is set only in the project env (Go loadNestedEnv parity)", + () => { + // Go's `loadNestedEnv` (`pkg/config/config.go:1220`) `os.Setenv`s every + // project-`.env` key that isn't already in the shell env BEFORE the command body + // runs, so `viper.AutomaticEnv()` sees a `supabase/.env`-only + // `SUPABASE_SCANNER_BUFFER_SIZE` exactly like a real shell-exported one. + // `legacyApplySchemaFiles`'s `projectEnv` parameter threads the caller's already + // -loaded `legacyLoadProjectEnv` map through to the same check. + const dir = mkdtempSync(join(tmpdir(), "legacy-schema-files-scanner-projectenv-")); + mkdirSync(join(dir, "supabase"), { recursive: true }); + const file = join(dir, "supabase", "big.sql"); + writeFileSync(file, `SELECT 1;\nSELECT '${"a".repeat(5000)}';\n`); + const { session } = fakeSession(); + const previous = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + delete process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const exit = yield* legacyApplySchemaFiles( + session, + fs, + path, + dir, + ["supabase/big.sql"], + (message, suggestion) => + new TestError({ message: suggestion ? `${message} (${suggestion})` : message }), + { SUPABASE_SCANNER_BUFFER_SIZE: "100b" }, + ).pipe(Effect.exit); + expect(Exit.isFailure(exit)).toBe(true); + if (Exit.isFailure(exit)) { + const msg = JSON.stringify(exit.cause); + expect(msg).toContain("bufio.Scanner: token too long"); + } + }).pipe( + Effect.ensuring( + Effect.sync(() => { + if (previous !== undefined) process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = previous; + rmSync(dir, { recursive: true, force: true }); + }), + ), + Effect.provide(BunServices.layer), + ); + }, + ); + + it.effect( + "shell env still wins over the project env for SUPABASE_SCANNER_BUFFER_SIZE (Go godotenv 'never overrides' parity)", + () => { + // `godotenv.Load`'s `overload=false` never sets a key already present in + // `os.Environ()` (`godotenv@v1.5.1/godotenv.go:184-200`) — the shell value must + // win even when a (different) project-env value is also threaded through. + const dir = mkdtempSync(join(tmpdir(), "legacy-schema-files-scanner-shellwins-")); + mkdirSync(join(dir, "supabase"), { recursive: true }); + const file = join(dir, "supabase", "big.sql"); + writeFileSync(file, `SELECT '${"a".repeat(5000)}';\n`); + const { session, calls } = fakeSession(); + const previous = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + // Shell explicitly unsets enforcement (0 → treated as unset, no check) while the + // project env sets a tiny limit — the shell value must win. + process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = "0"; + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + yield* legacyApplySchemaFiles( + session, + fs, + path, + dir, + ["supabase/big.sql"], + (message, suggestion) => + new TestError({ message: suggestion ? `${message} (${suggestion})` : message }), + { SUPABASE_SCANNER_BUFFER_SIZE: "100b" }, + ); + expect(calls.some((c) => c.kind === "exec" && c.sql.startsWith("SELECT 'a"))).toBe(true); + }).pipe( + Effect.ensuring( + Effect.sync(() => { + if (previous === undefined) delete process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + else process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = previous; + rmSync(dir, { recursive: true, force: true }); + }), + ), + Effect.provide(BunServices.layer), + ); + }, + ); +}); diff --git a/apps/cli/src/legacy/shared/legacy-path-match.ts b/apps/cli/src/legacy/shared/legacy-path-match.ts index b7dadac49b..f57f2acd75 100644 --- a/apps/cli/src/legacy/shared/legacy-path-match.ts +++ b/apps/cli/src/legacy/shared/legacy-path-match.ts @@ -1,19 +1,30 @@ /** - * Faithful port of Go's stdlib `path.Match` (`$GOROOT/src/path/match.go`), used - * by the seed-file globber to expand `[db.seed] sql_paths` exactly like the Go + * Faithful, BYTE-level port of Go's stdlib `path.Match` (`$GOROOT/src/path/match.go`), + * used by the seed-file globber to expand `[db.seed] sql_paths` exactly like the Go * CLI's `config.Glob.Files` → `io/fs.Glob` → `path.Match` chain. * - * Why a hand port instead of a JS `RegExp`: Go's glob grammar and JS regex - * character classes diverge — POSIX classes (`[[:alpha:]]`), `\d`/`\w`, and a - * leading `^` mean different things, and Go reports a malformed class as an - * error (`path.ErrBadPattern`) where JS would silently reinterpret it. Compiling - * each segment to a `RegExp` leaked those JS-only semantics; porting the - * algorithm keeps seed globbing byte-compatible with Go, including the - * malformed-pattern handling. + * Why byte-level, not code-point-level: Go strings are raw byte slices — every index, + * slice, and length in `path.Match` operates on UTF-8 BYTES, not decoded characters. + * This matters most in the `*`-retry loop (`legacyPathMatch`'s inner `for` below): + * Go retries the starred chunk at every BYTE offset of `name`, including offsets that + * land in the middle of a multibyte UTF-8 character. When that happens, Go's + * `unicode/utf8.DecodeRuneInString` decodes the LEADING (invalid, mid-character) + * continuation byte as a single-byte `U+FFFD` "rune" — it never throws and never + * consumes more than one byte for invalid input — so a `?` operator in the retried + * chunk can advance past exactly one such byte and let the retry succeed where a + * code-point-stepping port would not. Verified empirically against `apps/cli-go` + * (a `path.Match` scratch probe): `Match("*??.sql", "！.sql")` — a single fullwidth + * exclamation mark, U+FF01, 3 UTF-8 bytes — returns `true`: the second `?` in the + * retried chunk lands on the fullwidth character's 2nd and 3rd bytes (both mid-character + * continuation bytes, each decoded as one `U+FFFD` "rune"), not on a real code point. A + * prior code-point-based port of this file returned `false` for that same case. * - * Pure — no Effect / service dependencies. Operates on code points; Go mixes - * byte and rune indexing, which is equivalent for the BMP characters that occur - * in real seed paths. + * Why a hand port instead of a JS `RegExp`: Go's glob grammar and JS regex character + * classes diverge — POSIX classes (`[[:alpha:]]`), `\d`/`\w`, and a leading `^` mean + * different things, and Go reports a malformed class as an error (`path.ErrBadPattern`) + * where JS would silently reinterpret it. Compiling each segment to a `RegExp` leaked + * those JS-only semantics; porting the algorithm keeps seed globbing byte-compatible + * with Go, including the malformed-pattern handling and the byte-offset retry above. */ /** Mirrors Go's `path.Match` return `(matched bool, err error)`; `badPattern` ↔ `path.ErrBadPattern`. */ @@ -27,106 +38,198 @@ export const LEGACY_BAD_PATTERN_MESSAGE = "syntax error in pattern"; const BAD_PATTERN: LegacyPathMatchResult = { matched: false, badPattern: true }; -/** UTF-16 width (1 or 2 code units) of a code point. */ -const runeWidth = (cp: number): number => (cp > 0xffff ? 2 : 1); +const UTF8_ENCODER = new TextEncoder(); + +/** + * `TextEncoder.encode` returns `Uint8Array` (never a + * `SharedArrayBuffer`-backed view) — naming that explicitly so every `.subarray()` + * slice threaded through this module's helpers keeps that narrower type instead of + * widening to the generic `Uint8Array` default. + */ +type Bytes = Uint8Array; + +const RUNE_ERROR = 0xfffd; +const SLASH = 0x2f; +const STAR = 0x2a; +const QUESTION = 0x3f; +const LBRACKET = 0x5b; +const RBRACKET = 0x5d; +const CARET = 0x5e; +const HYPHEN = 0x2d; +const BACKSLASH = 0x5c; + +interface DecodedRune { + readonly r: number; + readonly size: number; +} + +/** + * Port of Go's `unicode/utf8.DecodeRuneInString`, decoding the rune starting at byte + * offset `i` of `b`. Any invalid or truncated sequence decodes as `(RuneError, 1)` — + * never throws, never consumes more than the single invalid lead byte — matching Go's + * documented behaviour exactly (`$GOROOT/src/unicode/utf8/utf8.go`'s `first` table and + * `acceptRanges`, transcribed here as explicit range checks per lead byte rather than + * the table itself, for readability; verified to agree with the table for every lead + * byte class, including the overlong/surrogate/out-of-range exclusions on `0xE0`, + * `0xED`, `0xF0`, and `0xF4`). + */ +const decodeRune = (b: Bytes, i: number): DecodedRune => { + const n = b.length - i; + if (n <= 0) return { r: RUNE_ERROR, size: 0 }; + const b0 = b[i]!; + if (b0 < 0x80) return { r: b0, size: 1 }; + let size: number; + let lo: number; + let hi: number; + if (b0 >= 0xc2 && b0 <= 0xdf) { + size = 2; + lo = 0x80; + hi = 0xbf; + } else if (b0 === 0xe0) { + size = 3; // Excludes the overlong 3-byte encoding. + lo = 0xa0; + hi = 0xbf; + } else if ((b0 >= 0xe1 && b0 <= 0xec) || b0 === 0xee || b0 === 0xef) { + size = 3; + lo = 0x80; + hi = 0xbf; + } else if (b0 === 0xed) { + size = 3; // Excludes the UTF-16 surrogate range U+D800-U+DFFF. + lo = 0x80; + hi = 0x9f; + } else if (b0 === 0xf0) { + size = 4; // Excludes the overlong 4-byte encoding. + lo = 0x90; + hi = 0xbf; + } else if (b0 >= 0xf1 && b0 <= 0xf3) { + size = 4; + lo = 0x80; + hi = 0xbf; + } else if (b0 === 0xf4) { + size = 4; // Caps the range at U+10FFFF. + lo = 0x80; + hi = 0x8f; + } else { + // 0x80-0xC1: a bare continuation byte or an overlong 2-byte lead. 0xF5-0xFF: past + // the max valid lead byte. Both are invalid lead bytes. + return { r: RUNE_ERROR, size: 1 }; + } + if (n < size) return { r: RUNE_ERROR, size: 1 }; + const b1 = b[i + 1]!; + if (b1 < lo || b1 > hi) return { r: RUNE_ERROR, size: 1 }; + if (size === 2) return { r: ((b0 & 0x1f) << 6) | (b1 & 0x3f), size: 2 }; + const b2 = b[i + 2]!; + if (b2 < 0x80 || b2 > 0xbf) return { r: RUNE_ERROR, size: 1 }; + if (size === 3) return { r: ((b0 & 0x0f) << 12) | ((b1 & 0x3f) << 6) | (b2 & 0x3f), size: 3 }; + const b3 = b[i + 3]!; + if (b3 < 0x80 || b3 > 0xbf) return { r: RUNE_ERROR, size: 1 }; + return { + r: ((b0 & 0x07) << 18) | ((b1 & 0x3f) << 12) | ((b2 & 0x3f) << 6) | (b3 & 0x3f), + size: 4, + }; +}; interface ScanChunk { readonly star: boolean; - readonly chunk: string; - readonly rest: string; + readonly chunk: Bytes; + readonly rest: Bytes; } /** Go's `scanChunk`: the next non-`*` segment, possibly preceded by a `*`. */ -const scanChunk = (pattern: string): ScanChunk => { +const scanChunk = (pattern: Bytes): ScanChunk => { let star = false; let p = pattern; - while (p.length > 0 && p[0] === "*") { - p = p.slice(1); + while (p.length > 0 && p[0] === STAR) { + p = p.subarray(1); star = true; } let inrange = false; for (let i = 0; i < p.length; i++) { - const c = p[i]; - if (c === "\\") { + const c = p[i]!; + if (c === BACKSLASH) { if (i + 1 < p.length) i++; - } else if (c === "[") { + } else if (c === LBRACKET) { inrange = true; - } else if (c === "]") { + } else if (c === RBRACKET) { inrange = false; - } else if (c === "*" && !inrange) { - return { star, chunk: p.slice(0, i), rest: p.slice(i) }; + } else if (c === STAR && !inrange) { + return { star, chunk: p.subarray(0, i), rest: p.subarray(i) }; } } - return { star, chunk: p, rest: "" }; + return { star, chunk: p, rest: p.subarray(p.length) }; }; interface GetEsc { readonly r: number; - readonly rest: string; + readonly rest: Bytes; readonly bad: boolean; } /** Go's `getEsc`: a possibly-escaped character from inside a class. */ -const getEsc = (chunk: string): GetEsc => { - if (chunk.length === 0 || chunk[0] === "-" || chunk[0] === "]") { +const getEsc = (chunk: Bytes): GetEsc => { + if (chunk.length === 0 || chunk[0] === HYPHEN || chunk[0] === RBRACKET) { return { r: 0, rest: chunk, bad: true }; } let c = chunk; - if (c[0] === "\\") { - c = c.slice(1); + if (c[0] === BACKSLASH) { + c = c.subarray(1); if (c.length === 0) return { r: 0, rest: c, bad: true }; } - const r = c.codePointAt(0)!; - const rest = c.slice(runeWidth(r)); - // Go errors when the class has no closing `]` after this character. + const { r, size } = decodeRune(c, 0); + // Go: `if r == utf8.RuneError && n == 1 { err = ErrBadPattern }` — a genuinely + // invalid byte, not a literal (valid, 3-byte-encoded) U+FFFD character. + if (r === RUNE_ERROR && size === 1) return { r, rest: c.subarray(1), bad: true }; + const rest = c.subarray(size); return { r, rest, bad: rest.length === 0 }; }; interface MatchChunk { - readonly rest: string; + readonly rest: Bytes; readonly ok: boolean; readonly bad: boolean; } -const BAD_CHUNK: MatchChunk = { rest: "", ok: false, bad: true }; +const EMPTY_BYTES = new Uint8Array(0); +const BAD_CHUNK: MatchChunk = { rest: EMPTY_BYTES, ok: false, bad: true }; /** * Go's `matchChunk`: match the all-single-char-operators `chunk` against the * start of `s`. Once the match fails the loop keeps walking `chunk` (no longer * reading `s`) so a malformed pattern is still reported. */ -const matchChunk = (chunkIn: string, sIn: string): MatchChunk => { +const matchChunk = (chunkIn: Bytes, sIn: Bytes): MatchChunk => { let chunk = chunkIn; let s = sIn; let failed = false; while (chunk.length > 0) { if (!failed && s.length === 0) failed = true; - const op = chunk[0]; - if (op === "[") { + const op = chunk[0]!; + if (op === LBRACKET) { let r = 0; if (!failed) { - r = s.codePointAt(0)!; - s = s.slice(runeWidth(r)); + const decoded = decodeRune(s, 0); + r = decoded.r; + s = s.subarray(decoded.size); } - chunk = chunk.slice(1); + chunk = chunk.subarray(1); let negated = false; - if (chunk.length > 0 && chunk[0] === "^") { + if (chunk.length > 0 && chunk[0] === CARET) { negated = true; - chunk = chunk.slice(1); + chunk = chunk.subarray(1); } let match = false; let nrange = 0; for (;;) { - if (chunk.length > 0 && chunk[0] === "]" && nrange > 0) { - chunk = chunk.slice(1); + if (chunk.length > 0 && chunk[0] === RBRACKET && nrange > 0) { + chunk = chunk.subarray(1); break; } const lo = getEsc(chunk); if (lo.bad) return BAD_CHUNK; chunk = lo.rest; let hi = lo.r; - if (chunk[0] === "-") { - const hiEsc = getEsc(chunk.slice(1)); + if (chunk.length > 0 && chunk[0] === HYPHEN) { + const hiEsc = getEsc(chunk.subarray(1)); if (hiEsc.bad) return BAD_CHUNK; chunk = hiEsc.rest; hi = hiEsc.r; @@ -135,30 +238,30 @@ const matchChunk = (chunkIn: string, sIn: string): MatchChunk => { nrange++; } if (match === negated) failed = true; - } else if (op === "?") { + } else if (op === QUESTION) { if (!failed) { - const cp = s.codePointAt(0)!; - if (cp === 0x2f) failed = true; // '/' - s = s.slice(runeWidth(cp)); + if (s[0] === SLASH) failed = true; + const { size } = decodeRune(s, 0); + s = s.subarray(size); } - chunk = chunk.slice(1); - } else if (op === "\\") { - chunk = chunk.slice(1); + chunk = chunk.subarray(1); + } else if (op === BACKSLASH) { + chunk = chunk.subarray(1); if (chunk.length === 0) return BAD_CHUNK; if (!failed) { if (chunk[0] !== s[0]) failed = true; - s = s.slice(1); + s = s.subarray(1); } - chunk = chunk.slice(1); + chunk = chunk.subarray(1); } else { if (!failed) { if (chunk[0] !== s[0]) failed = true; - s = s.slice(1); + s = s.subarray(1); } - chunk = chunk.slice(1); + chunk = chunk.subarray(1); } } - return failed ? { rest: "", ok: false, bad: false } : { rest: s, ok: true, bad: false }; + return failed ? { rest: EMPTY_BYTES, ok: false, bad: false } : { rest: s, ok: true, bad: false }; }; /** @@ -167,14 +270,16 @@ const matchChunk = (chunkIn: string, sIn: string): MatchChunk => { * pattern is malformed, mirroring Go's `path.ErrBadPattern`. */ export const legacyPathMatch = (pattern: string, name: string): LegacyPathMatchResult => { - let pat = pattern; - let nm = name; + let pat = UTF8_ENCODER.encode(pattern); + let nm = UTF8_ENCODER.encode(name); while (pat.length > 0) { const scan = scanChunk(pat); pat = scan.rest; - if (scan.star && scan.chunk === "") { - // Trailing `*` matches the rest of the name unless it contains a `/`. - return { matched: !nm.includes("/"), badPattern: false }; + if (scan.star && scan.chunk.length === 0) { + // Trailing `*` matches the rest of the name unless it contains a `/`. `/` is + // never a UTF-8 continuation byte, so a raw byte scan is safe here regardless + // of any multibyte characters elsewhere in `nm`. + return { matched: !nm.includes(SLASH), badPattern: false }; } const m = matchChunk(scan.chunk, nm); if (m.bad) return BAD_PATTERN; @@ -185,10 +290,11 @@ export const legacyPathMatch = (pattern: string, name: string): LegacyPathMatchR continue; } if (scan.star) { - // Look for a match skipping one code point at a time; `*` cannot cross `/`. + // Look for a match skipping one BYTE at a time (see this file's top comment + // for why byte-, not code-point-, stepping matters here); `*` cannot cross `/`. let advanced = false; - for (let i = 0; i < nm.length && nm[i] !== "/"; i++) { - const skip = matchChunk(scan.chunk, nm.slice(i + 1)); + for (let i = 0; i < nm.length && nm[i] !== SLASH; i++) { + const skip = matchChunk(scan.chunk, nm.subarray(i + 1)); if (skip.bad) return BAD_PATTERN; if (skip.ok) { if (pat.length === 0 && skip.rest.length > 0) continue; @@ -203,7 +309,7 @@ export const legacyPathMatch = (pattern: string, name: string): LegacyPathMatchR while (pat.length > 0) { const tail = scanChunk(pat); pat = tail.rest; - if (matchChunk(tail.chunk, "").bad) return BAD_PATTERN; + if (matchChunk(tail.chunk, EMPTY_BYTES).bad) return BAD_PATTERN; } return { matched: false, badPattern: false }; } diff --git a/apps/cli/src/legacy/shared/legacy-path-match.unit.test.ts b/apps/cli/src/legacy/shared/legacy-path-match.unit.test.ts index 66a849d48b..7285f5c3e4 100644 --- a/apps/cli/src/legacy/shared/legacy-path-match.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-path-match.unit.test.ts @@ -49,6 +49,23 @@ describe("legacyPathMatch", () => { }); }); + describe("byte-offset `*` retry against multibyte characters (Go path.Match parity)", () => { + it.each([ + // Go's byte-offset `*`-retry loop can land mid-multibyte-character and have a `?` + // consume the resulting invalid continuation byte as a single-byte `U+FFFD` "rune" + // — producing matches a code-point-stepping port would miss. Verified empirically + // against `apps/cli-go`'s `path.Match` (a fullwidth exclamation mark, U+FF01, is a + // single character but 3 UTF-8 bytes; an emoji, U+1F600, is 4 UTF-8 bytes): + ["*??.sql", "！.sql", true], + ["*??.sql", "😀.sql", true], + ["*?.sql", "！.sql", true], + ["*???.sql", "！.sql", false], + ["schemas/*??.sql", "schemas/！.sql", true], + ] as const)("%s ~ %s => %s", (pattern, name, expected) => { + expect(legacyPathMatch(pattern, name).matched).toBe(expected); + }); + }); + describe("escapes", () => { it.each([ ["\\*.sql", "*.sql", true], diff --git a/apps/cli/src/legacy/shared/legacy-pgdelta.cache.ts b/apps/cli/src/legacy/shared/legacy-pgdelta.cache.ts index d9f292dd25..d999a0ea7a 100644 --- a/apps/cli/src/legacy/shared/legacy-pgdelta.cache.ts +++ b/apps/cli/src/legacy/shared/legacy-pgdelta.cache.ts @@ -332,12 +332,30 @@ const parseCatalogTimestamp = (name: string): Option.Option => { return Number.isInteger(ts) ? Option.some(ts) : Option.none(); }; +/** + * Mirrors Go's `ensureTempDir` + `ReadDir` pairing (`pgcache/cache.go`, + * `declarative.go`): the temp dir's existence is already guaranteed by the + * `MkdirAll` that runs before every write into it, so Go's `ReadDir` only ever + * needs to tolerate a genuinely missing directory (a cache that was never + * written to) — every OTHER read failure (e.g. permission denied) propagates, + * same as {@link legacyListLocalMigrations} above. Swallowing every failure + * (as an earlier version of this did) let a real read error silently look like + * "no cached catalogs", which both bypasses catalog resolution's cache HIT and + * — for cleanup's caller — bypasses the retention limit indefinitely, since + * the caller's own warning path never fires without a propagated failure. + */ const listJsonEntries = Effect.fnUntraced(function* (fs: FileSystem.FileSystem, tempDir: string) { - const exists = yield* fs.exists(tempDir).pipe(Effect.orElseSucceed(() => false)); - if (!exists) return [] as ReadonlyArray; - return yield* fs - .readDirectory(tempDir) - .pipe(Effect.orElseSucceed(() => [] as ReadonlyArray)); + return yield* fs.readDirectory(tempDir).pipe( + Effect.catchTag("PlatformError", (error) => + error.reason._tag === "NotFound" + ? Effect.succeed([] as ReadonlyArray) + : Effect.fail( + new LegacyMigrationsReadError({ + message: `failed to read directory: ${error.message}`, + }), + ), + ), + ); }); /** @@ -511,6 +529,20 @@ export const legacyWriteMigrationCatalogSnapshot = Effect.fnUntraced(function* ( * `diff/pgdelta.go` `ExportCatalogPgDelta`) rather than porting a second copy, * so this can't reintroduce the `/workspace` mount bug `pgcache/cache.go` had * (supabase/cli#5921). + * + * The snapshot's timestamp is read from `Clock` HERE — after `legacyHashMigrations` + * and `legacyExportCatalogPgDelta` (the network round-trip) have both resolved, + * immediately before the write — never accepted as a caller-supplied parameter. + * This mirrors Go's own call order exactly: `TryCacheMigrationsCatalog` + * (`pgcache/cache.go:71-91`) resolves `hash` and `snapshot` FIRST, and only THEN + * calls `WriteMigrationCatalogSnapshot`, which itself reads `time.Now().UTC()` + * (`pgcache/cache.go:151-163`) — i.e. Go's clock read happens LAST, right before + * the file write, not before the export. A caller capturing the timestamp before + * calling this function (review CLI-1958) would race a concurrent cache write + * from another process: Go would order the two snapshots by real write-time, but + * the early-captured timestamp could sort the wrong one as "latest" during + * catalog resolution/retention (`legacyResolveMigrationCatalogPath`, + * `legacyCleanupOldMigrationCatalogs`). */ export const legacyTryCacheMigrationsCatalog = Effect.fnUntraced(function* ( fs: FileSystem.FileSystem, @@ -527,7 +559,6 @@ export const legacyTryCacheMigrationsCatalog = Effect.fnUntraced(function* ( }; readonly isLocal: boolean; readonly migrationsDir: string; - readonly nowMillis: number; }, ) { if (!params.enabled) return; @@ -537,6 +568,7 @@ export const legacyTryCacheMigrationsCatalog = Effect.fnUntraced(function* ( targetRef: params.targetUrl, role: "postgres", }); + const nowMillis = yield* Clock.currentTimeMillis; yield* legacyWriteMigrationCatalogSnapshot( fs, path, @@ -544,7 +576,7 @@ export const legacyTryCacheMigrationsCatalog = Effect.fnUntraced(function* ( prefix, hash, snapshot, - params.nowMillis, + nowMillis, ); }); diff --git a/apps/cli/src/legacy/shared/legacy-pgdelta.cache.unit.test.ts b/apps/cli/src/legacy/shared/legacy-pgdelta.cache.unit.test.ts index fc32436de8..55ccfa61c5 100644 --- a/apps/cli/src/legacy/shared/legacy-pgdelta.cache.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-pgdelta.cache.unit.test.ts @@ -1,13 +1,16 @@ import { createHash } from "node:crypto"; -import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { chmodSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; -import { Effect, FileSystem, Layer, Option, Path } from "effect"; +import { Effect, Exit, FileSystem, Layer, Option, Path } from "effect"; import { Output } from "../../shared/output/output.service.ts"; import { mockOutput } from "../../../tests/helpers/mocks.ts"; +import { LegacyEdgeRuntimeScript } from "./legacy-edge-runtime-script.service.ts"; +import { LegacyPgDeltaSslProbe } from "./legacy-pgdelta-ssl-probe.service.ts"; +import { type LegacyPgDeltaContext } from "./legacy-pgdelta.ts"; import { type LegacySetupInputs, legacyBaselineCatalogFileName, @@ -28,6 +31,7 @@ import { legacyResolveSetupInputs, legacySanitizedCatalogPrefix, legacySetupInputsToken, + legacyTryCacheMigrationsCatalog, legacyWriteMigrationCatalogSnapshot, } from "./legacy-pgdelta.cache.ts"; @@ -518,6 +522,71 @@ describe("legacyWriteMigrationCatalogSnapshot + cleanup", () => { }); }); +describe("legacyTryCacheMigrationsCatalog — timestamp ordering (review CLI-1958)", () => { + // `it.live` (not `it.effect`): the mocked export below uses a real `Effect.sleep` + // to create a measurable time gap, which needs the real wall clock, not + // `it.effect`'s virtual `TestClock` (which never auto-advances and would hang). + it.live( + "reads the clock AFTER the pg-delta export resolves, matching Go's WriteMigrationCatalogSnapshot ordering", + () => { + // Go's `TryCacheMigrationsCatalog` (`pgcache/cache.go:71-91`) resolves `hash` + // and `snapshot` FIRST and only THEN calls `WriteMigrationCatalogSnapshot`, + // which itself reads `time.Now().UTC()` (`pgcache/cache.go:151-163`) — i.e. + // Go's clock read happens LAST, right before the file write. The mocked + // edge-runtime export below sleeps for a real, measurable interval before + // resolving; the written snapshot's embedded timestamp must reflect a moment + // AFTER that sleep, proving the clock was read after the export — not + // captured up front by a caller before this function even started (the + // pre-fix bug). + const dir = withTemp(); + const migrationsDir = join(dir, "supabase", "migrations"); + mkdirSync(migrationsDir, { recursive: true }); + // Mirrors `legacyPgDeltaTempPath` (`/supabase/.temp/pgdelta`). + const tempDir = join(dir, "supabase", ".temp", "pgdelta"); + const beforeCallMillis = Date.now(); + const edge = Layer.succeed(LegacyEdgeRuntimeScript, { + run: () => + Effect.gen(function* () { + yield* Effect.sleep("30 millis"); + return { stdout: "{}", stderr: "" }; + }), + }); + const sslProbe = Layer.succeed(LegacyPgDeltaSslProbe, { + requireSsl: () => Effect.succeed(false), + requireSslForHost: () => Effect.succeed(false), + }); + const ctx: LegacyPgDeltaContext = { + projectId: "test", + cwd: dir, + npmVersion: undefined, + denoVersion: 1, + }; + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + yield* legacyTryCacheMigrationsCatalog(fs, path, ctx, { + enabled: true, + targetUrl: "postgresql://postgres:postgres@127.0.0.1:5432/postgres", + conn: { host: "127.0.0.1", port: 5432, user: "postgres", database: "postgres" }, + isLocal: true, + migrationsDir, + }); + const names = (yield* fs.readDirectory(tempDir)).filter((n) => + n.startsWith("catalog-local-migrations-"), + ); + expect(names.length).toBe(1); + const match = /-(\d+)\.json$/.exec(names[0]!); + expect(match).not.toBeNull(); + const embeddedMillis = Number(match![1]); + expect(embeddedMillis).toBeGreaterThanOrEqual(beforeCallMillis + 25); + }).pipe( + Effect.provide(Layer.mergeAll(BunServices.layer, mockOutput().layer, edge, sslProbe)), + Effect.tap(() => Effect.sync(() => rmSync(dir, { recursive: true, force: true }))), + ); + }, + ); +}); + describe("legacyCleanupOldMigrationCatalogs", () => { it.effect("only prunes files matching the given prefix's family", () => { const dir = withTemp(); @@ -539,4 +608,32 @@ describe("legacyCleanupOldMigrationCatalogs", () => { }), ).pipe(Effect.tap(() => Effect.sync(() => rmSync(dir, { recursive: true, force: true })))); }); + + it.effect( + "propagates a permission-denied directory read instead of treating it as empty (Go ReadDir parity)", + () => { + // Go's CleanupOldMigrationCatalogs only tolerates a genuinely MISSING temp dir + // (ensureTempDir already created it before ReadDir runs) — any other ReadDir + // failure propagates, so a permission-denied listing must fail here too rather + // than silently look like "no cached catalogs" (which would bypass retention + // indefinitely, since the caller's own best-effort warning never fires without + // a propagated failure). + const dir = withTemp(); + const tempDir = join(dir, "pgdelta"); + mkdirSync(tempDir, { recursive: true }); + writeFileSync(join(tempDir, "catalog-local-migrations-h-100.json"), "{}"); + chmodSync(tempDir, 0o000); + return withServices((fs, path) => + legacyCleanupOldMigrationCatalogs(fs, path, tempDir, "local").pipe(Effect.exit), + ).pipe( + Effect.tap((exit) => + Effect.sync(() => { + chmodSync(tempDir, 0o755); + expect(Exit.isFailure(exit)).toBe(true); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); }); diff --git a/apps/cli/src/legacy/shared/legacy-seed-ops.ts b/apps/cli/src/legacy/shared/legacy-seed-ops.ts index bbea4d4fcc..2690f5fcf9 100644 --- a/apps/cli/src/legacy/shared/legacy-seed-ops.ts +++ b/apps/cli/src/legacy/shared/legacy-seed-ops.ts @@ -1,11 +1,12 @@ import { createHash } from "node:crypto"; -import { Effect, type FileSystem, Option, type Path } from "effect"; +import { Effect, type FileSystem, type Path } from "effect"; import { Output } from "../../shared/output/output.service.ts"; import type { LegacyDbExecError } from "./legacy-db-connection.errors.ts"; import type { LegacyDbSession } from "./legacy-db-connection.service.ts"; +import { checkScannerBufferSize } from "./legacy-migration-apply.ts"; import { legacyCreateSeedTable } from "./legacy-migration-history.ts"; -import { LEGACY_BAD_PATTERN_MESSAGE, legacyPathMatch } from "./legacy-path-match.ts"; +import { legacySqlFilesGlob } from "./legacy-sql-files-glob.ts"; import { legacySplitAndTrim } from "./legacy-sql-split.ts"; /** @@ -26,171 +27,6 @@ export interface LegacySeedFile { readonly dirty: boolean; } -const META_CHARS = /[*?[\\]/u; - -/** Result of resolving `[db.seed].sql_paths` against the workspace. */ -interface LegacyGlobResult { - /** Workdir-relative, forward-slashed matches, deduplicated in pattern order. */ - readonly files: ReadonlyArray; - /** Per-pattern warnings (`no files matched pattern: …`), joined by Go's `errors.Join`. */ - readonly warning: Option.Option; -} - -/** - * Resolves seed glob patterns to existing files, porting Go's `config.Glob.Files` - * over `fs.Glob` (`pkg/config/config.go:102-124`). Each pattern is first joined - * under the `supabase/` directory (Go resolves `sql_paths` at config load, - * `config.go:884`). Matches per pattern are sorted; the overall result preserves - * first-seen order across patterns. A pattern that matches nothing, or is malformed - * (Go's `path.ErrBadPattern`, e.g. an unterminated `[` class), contributes a warning - * but is not fatal — mirroring `fs.Glob`'s up-front `Match(pattern, "")` validation - * (`io/fs/glob.go`) and the sibling seed pipeline's `legacy-seed.ts:resolveSeedFiles`. - */ -const legacyGlobSeedFiles = Effect.fnUntraced(function* ( - fs: FileSystem.FileSystem, - path: Path.Path, - patterns: ReadonlyArray, - workdir: string, -) { - const seen = new Set(); - const files: Array = []; - const errors: Array = []; - - for (const rawPattern of patterns) { - // Patterns arrive already resolved to Go's config-load form (relative entries - // supabase/-joined, absolute preserved) via `legacyResolveSeedSqlPath` — the reader - // for `[db.seed].sql_paths`, the caller for `--sql-paths`. Go's `config.Glob.Files` - // globs those resolved paths without re-prefixing (`config.go:102-124`), so only - // normalize separators here; re-joining `supabase/` would double-prefix. - const pattern = toSlash(rawPattern); - // Go's `fs.Glob` validates the whole pattern up front (`Match(pattern, "")`); a - // malformed glob is reported as `failed to glob files: ` and - // contributes no matches, rather than the misleading "no files matched" below. - if (legacyPathMatch(pattern, "").badPattern) { - errors.push(`failed to glob files: ${LEGACY_BAD_PATTERN_MESSAGE}`); - continue; - } - const matches = yield* globOne(fs, path, workdir, pattern); - if (matches.length === 0) { - errors.push(`no files matched pattern: ${pattern}`); - continue; - } - for (const match of [...matches].sort()) { - const fp = toSlash(match); - // Go's `GetPendingSeeds` globs via `Glob.SQLFiles`, which `Stat`s each match: a - // directory is expanded to its regular `.sql` files recursively (`walkMatchedDir`, - // sorted) while a file match is kept verbatim (`config.go:157-183`). Without this a - // directory `sql_paths` entry (e.g. `["seeds"]`) would flow into - // `readFileString()` and fail — Go's `db push --include-seed` / remote reset - // seed the directory's SQL children instead. - const matchType = yield* fs.stat(path.isAbsolute(fp) ? fp : path.join(workdir, fp)).pipe( - Effect.map((info) => info.type), - Effect.orElseSucceed(() => "File" as const), - ); - if (matchType === "Directory") { - for (const file of yield* legacyWalkSeedSqlFiles(fs, path, workdir, fp)) { - if (!seen.has(file)) { - seen.add(file); - files.push(file); - } - } - continue; - } - if (!seen.has(fp)) { - seen.add(fp); - files.push(fp); - } - } - } - - return { - files, - warning: errors.length > 0 ? Option.some(errors.join("\n")) : Option.none(), - } satisfies LegacyGlobResult; -}); - -const toSlash = (p: string): string => p.replaceAll("\\", "/"); - -/** Splits a forward-slashed path into its directory prefix and final element. */ -const splitPath = (p: string): { readonly dir: string; readonly file: string } => { - const slash = p.lastIndexOf("/"); - return slash === -1 ? { dir: "", file: p } : { dir: p.slice(0, slash), file: p.slice(slash + 1) }; -}; - -/** Faithful port of Go's `fs.Glob` for one pattern, rooted at `workdir`. */ -const globOne = ( - fs: FileSystem.FileSystem, - path: Path.Path, - workdir: string, - pattern: string, -): Effect.Effect, never> => - Effect.gen(function* () { - // Absolute patterns resolve against the filesystem root (Go preserves absolute - // seed paths); relative ones are rooted at the workdir. - const resolve = (p: string): string => (path.isAbsolute(p) ? p : path.join(workdir, p)); - // No metacharacters: a direct existence check (Go's `fs.Glob` fast path). - if (!META_CHARS.test(pattern)) { - const exists = yield* fs.exists(resolve(pattern)).pipe(Effect.orElseSucceed(() => false)); - return exists ? [pattern] : []; - } - const { dir, file } = splitPath(pattern); - // Resolve the directory level first (recursively if it, too, is a glob). - const dirs = - dir === "" || !META_CHARS.test(dir) ? [dir] : yield* globOne(fs, path, workdir, dir); - const result: Array = []; - for (const d of dirs) { - const absDir = d === "" ? workdir : resolve(d); - const names = yield* fs - .readDirectory(absDir) - .pipe(Effect.orElseSucceed((): ReadonlyArray => [])); - for (const name of names) { - if (legacyPathMatch(file, name).matched) { - result.push(d === "" ? name : `${d}/${name}`); - } - } - } - return result; - }); - -/** - * Recursively collects the regular `.sql` files under a matched seed directory, porting - * Go's `walkMatchedDir` with the `SQLFiles` include filter (`entry.Type().IsRegular() && - * filepath.Ext(path) == ".sql"`, `config.go:126-131,194-211`). Paths are workdir-relative - * (matching the glob output), forward-slashed, and sorted for deterministic application. - */ -const legacyWalkSeedSqlFiles = ( - fs: FileSystem.FileSystem, - path: Path.Path, - workdir: string, - dir: string, -): Effect.Effect, never> => - Effect.gen(function* () { - const collected: Array = []; - const walk = (rel: string): Effect.Effect => - Effect.gen(function* () { - const absDir = path.isAbsolute(rel) ? rel : path.join(workdir, rel); - const names = yield* fs - .readDirectory(absDir) - .pipe(Effect.orElseSucceed((): ReadonlyArray => [])); - for (const name of names) { - const childRel = `${rel}/${name}`; - const childType = yield* fs - .stat(path.isAbsolute(childRel) ? childRel : path.join(workdir, childRel)) - .pipe( - Effect.map((info) => info.type), - Effect.orElseSucceed(() => "Unknown" as const), - ); - if (childType === "Directory") { - yield* walk(childRel); - } else if (childType === "File" && childRel.endsWith(".sql")) { - collected.push(toSlash(childRel)); - } - } - }); - yield* walk(dir); - return collected.sort(); - }); - /** `SELECT path, hash FROM supabase_migrations.seed_files`, `42P01` → empty map. */ const readRemoteSeeds = (session: LegacyDbSession) => session.query(SELECT_SEED_TABLE).pipe( @@ -212,10 +48,16 @@ const isUndefinedTable = (error: LegacyDbExecError): boolean => /** * Resolves the pending seed files for `db push --include-seed`. Mirrors Go's - * `GetPendingSeeds` (`pkg/migration/seed.go:34-63`): glob the configured paths - * (warn, don't fail, on empty patterns), read the remote `seed_files` hashes, - * and emit each local file that is new (`dirty=false`) or hash-changed - * (`dirty=true`); files whose hash already matches are skipped. + * `GetPendingSeeds` (`pkg/migration/seed.go:34-63`): glob the configured paths via + * the shared {@link legacySqlFilesGlob} traversal (also used by `[db.migrations]. + * schema_paths`, `legacy-migration-apply.ts`, and by `legacy-seed.ts`'s own + * `resolveSeedFiles` for the `migration down`/`start` seed step), warn — don't fail — + * on empty patterns, read the remote `seed_files` hashes, and emit each local file + * that is new (`dirty=false`) or hash-changed (`dirty=true`); files whose hash + * already matches are skipped. Per-pattern warnings are joined with Go's `errors.Join` + * newline semantics and surfaced unconditionally (`seed.go:36-38`) — unlike the + * schema-files apply path (see `legacyApplySchemaFiles`), which only surfaces a + * warning when it is the ONLY outcome. */ export const legacyGetPendingSeeds = Effect.fnUntraced(function* ( session: LegacyDbSession, @@ -225,9 +67,9 @@ export const legacyGetPendingSeeds = Effect.fnUntraced(function* ( workdir: string, ) { const output = yield* Output; - const { files, warning } = yield* legacyGlobSeedFiles(fs, path, patterns, workdir); - if (Option.isSome(warning)) { - yield* output.raw(`WARN: ${warning.value}\n`, "stderr"); + const { files, warnings } = yield* legacySqlFilesGlob(fs, path, patterns, workdir); + if (warnings.length > 0) { + yield* output.raw(`WARN: ${warnings.join("\n")}\n`, "stderr"); } const pending: Array = []; if (files.length === 0) return pending; @@ -285,12 +127,16 @@ export const legacySeedData = ( // Go's `ExecBatchWithCache` parses the file (read + `SplitAndTrim`) // UNCONDITIONALLY before the dirty check (`file.go:198-211`), so a dirty seed // that is unreadable or contains malformed SQL still fails and leaves the - // previous hash — only the queueing of statements is gated on `Dirty`. - const lines = legacySplitAndTrim( - yield* fs.readFileString( - path.isAbsolute(seed.path) ? seed.path : path.join(workdir, seed.path), - ), + // previous hash — only the queueing of statements is gated on `Dirty`. Parsing + // includes the same `SUPABASE_SCANNER_BUFFER_SIZE` enforcement every other + // `parseFile` caller gets (`checkScannerBufferSize`'s own doc comment) — Go's + // `SeedFile.ExecBatchWithCache` runs through the identical `parseFile`, so an + // oversized seed statement must fail here too, not execute silently. + const content = yield* fs.readFileString( + path.isAbsolute(seed.path) ? seed.path : path.join(workdir, seed.path), ); + yield* checkScannerBufferSize(content, (message) => new Error(message)); + const lines = legacySplitAndTrim(content); const statements = seed.dirty ? [] : lines; yield* session.exec("BEGIN"); const body = Effect.gen(function* () { diff --git a/apps/cli/src/legacy/shared/legacy-seed-ops.unit.test.ts b/apps/cli/src/legacy/shared/legacy-seed-ops.unit.test.ts index 2282ea54ef..72fd1800f6 100644 --- a/apps/cli/src/legacy/shared/legacy-seed-ops.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-seed-ops.unit.test.ts @@ -122,6 +122,35 @@ describe("legacySeedData (dirty parse)", () => { ); }); + it.effect( + "rejects an oversized seed statement when SUPABASE_SCANNER_BUFFER_SIZE is configured (Go SeedFile.ExecBatchWithCache parity)", + () => { + // Go's SeedFile.ExecBatchWithCache parses through the same parseFile every + // other file type does, so an oversized statement must abort the seed run — + // same as legacy-migration-apply.unit.test.ts's equivalent case for migrations. + const dir = mkdtempSync(join(tmpdir(), "legacy-seed-scanner-")); + // Raw text must exceed the 4096-byte floor Go's bufio.Scanner starts at + // regardless of the configured limit (see legacy-migration-apply.unit.test.ts's + // equivalent case for the exact same 4096-byte floor). + writeFileSync(join(dir, "big.sql"), `select '${"x".repeat(5000)}';`); + const { session, calls } = fakeSeedSession(); + const previous = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = "100b"; + return runSeed(session, dir, [{ path: "big.sql", hash: "newhash", dirty: false }]).pipe( + Effect.exit, + Effect.tap((exit) => + Effect.sync(() => { + expect(Exit.isFailure(exit)).toBe(true); + expect(calls.some((c) => c.sql.includes("select"))).toBe(false); + rmSync(dir, { recursive: true, force: true }); + if (previous === undefined) delete process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + else process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = previous; + }), + ), + ); + }, + ); + it.effect("refreshes the hash for a dirty seed that parses, without running statements", () => { const dir = mkdtempSync(join(tmpdir(), "legacy-seed-")); writeFileSync(join(dir, "data.sql"), "insert into t values (1);"); diff --git a/apps/cli/src/legacy/shared/legacy-seed.ts b/apps/cli/src/legacy/shared/legacy-seed.ts index 53247f0dff..c023e7f159 100644 --- a/apps/cli/src/legacy/shared/legacy-seed.ts +++ b/apps/cli/src/legacy/shared/legacy-seed.ts @@ -1,15 +1,16 @@ import { createHash } from "node:crypto"; -import { Data, Effect, FileSystem, Path, Result } from "effect"; +import { Data, Effect, FileSystem, Path } from "effect"; import { Output } from "../../shared/output/output.service.ts"; import type { LegacyDbSession } from "./legacy-db-connection.service.ts"; -import { legacyGlobPattern, legacyResolveUnderWorkdir, legacyWalkSqlFiles } from "./legacy-glob.ts"; +import { legacyResolveUnderWorkdir } from "./legacy-glob.ts"; +import { checkScannerBufferSize } from "./legacy-migration-apply.ts"; import { legacyCreateSeedTable, legacyReadSeedTable, UPSERT_SEED_FILE, } from "./legacy-migration-history.ts"; -import { LEGACY_BAD_PATTERN_MESSAGE, legacyPathMatch } from "./legacy-path-match.ts"; +import { legacySqlFilesGlob } from "./legacy-sql-files-glob.ts"; import { legacySplitAndTrim } from "./legacy-sql-split.ts"; /** Applying a seed file failed (Go's `SeedData` / `ExecBatchWithCache` errors). */ @@ -35,20 +36,13 @@ interface LegacyPendingSeed { } /** - * Port of Go's `Glob.SQLFiles` (`pkg/config/config.go:122-128` → `files`/`walkMatchedDir`) as - * called by `GetPendingSeeds` (`locals.SQLFiles(fsys)`, `pkg/migration/seed.go:35`) — the SAME - * method `db.migrations.schema_paths` resolves through (`legacyResolveSchemaPathFiles` in - * `legacy-migrate-and-seed.ts`), not the plainer `Glob.Files`: each pattern is glob-matched via - * {@link legacyGlobPattern}, and a matched DIRECTORY is expanded to its sorted, regular `.sql` - * files, recursively (via the shared {@link legacyWalkSqlFiles}), rather than kept as-is — a - * plain glob match (e.g. `[db.seed] sql_paths = ["./seeds"]` with no metacharacters) previously - * resolved a directory entry to itself, which then failed reading it as a seed file. A matched - * plain file is kept as-is, even a non-`.sql` one, matching `expandDir`'s `IsDir()`-only gate. - * - * Unlike `legacyResolveSchemaPathFiles`, a bad pattern, an empty match, or a directory-walk - * failure is NEVER a hard failure here — `GetPendingSeeds` only ever warns - * (`fmt.Fprintln(os.Stderr, "WARN:", err)`) and proceeds with whatever it already collected, - * even if that ends up empty (`len(locals) == 0` just means no pending seeds, not an error). + * Resolves `[db.seed].sql_paths` to existing files, porting Go's `config.Glob.SQLFiles` + * (`pkg/migration/seed.go:35`, via the shared {@link legacySqlFilesGlob} traversal — + * also used by `legacyGetPendingSeeds` (`legacy-seed-ops.ts`) for the same Go field on + * the `db push`/`db reset` path, and by `legacyApplySchemaFiles` (`legacy-migration-apply.ts`) + * for `[db.migrations].schema_paths`). Go's `GetPendingSeeds` prints a single unconditional + * `WARN: ` line for any glob problem (`seed.go:36-38`) — unlike the schema-files + * apply path, which only warns when NO pattern matched anything at all. */ const resolveSeedFiles = ( fs: FileSystem.FileSystem, @@ -58,55 +52,9 @@ const resolveSeedFiles = ( ) => Effect.gen(function* () { const output = yield* Output; - const seen = new Set(); - const result: Array = []; - const unmatched: Array = []; - for (const pattern of patterns) { - // Go's `fs.Glob` validates the whole pattern up front (`Match(pattern, "")`); - // a malformed glob is reported as `failed to glob files: ` and - // contributes no matches, exactly like `Glob.Files`'s error branch. - if (legacyPathMatch(pattern, "").badPattern) { - unmatched.push(`failed to glob files: ${LEGACY_BAD_PATTERN_MESSAGE}`); - continue; - } - const matches = [...(yield* legacyGlobPattern(fs, path, workdir, pattern))].sort(); - if (matches.length === 0) unmatched.push(`no files matched pattern: ${pattern}`); - for (const match of matches) { - const absMatch = legacyResolveUnderWorkdir(path, workdir, match); - const statResult = yield* fs.stat(absMatch).pipe(Effect.result); - if (Result.isFailure(statResult)) { - unmatched.push(`failed to stat matched file: ${match}`); - continue; - } - if (statResult.success.type !== "Directory") { - if (!seen.has(match)) { - seen.add(match); - result.push(match); - } - continue; - } - // Go's `walkMatchedDir`: recursively list the matched directory, keep only regular - // `.sql` files, sorted (a global sort over the full relative-to-fsys-root path, not - // per-directory — matches `sort.Strings(files)` running once after the whole walk). - const namesResult = yield* legacyWalkSqlFiles(fs, absMatch, "").pipe(Effect.result); - if (Result.isFailure(namesResult)) { - unmatched.push(`failed to walk matched directory: ${match}`); - continue; - } - const sqlRelative = [...namesResult.success].sort(); - for (const relative of sqlRelative) { - const relativeToWorkdir = `${match}/${relative}`; - if (!seen.has(relativeToWorkdir)) { - seen.add(relativeToWorkdir); - result.push(relativeToWorkdir); - } - } - } - } - // Go collects all glob/walk errors into one `errors.Join` and prints a single - // `WARN: ` line (`Glob.SQLFiles` → `seed.go:35-36`), not one per pattern. - if (unmatched.length > 0) yield* output.raw(`WARN: ${unmatched.join("\n")}\n`, "stderr"); - return result; + const { files, warnings } = yield* legacySqlFilesGlob(fs, path, patterns, workdir); + if (warnings.length > 0) yield* output.raw(`WARN: ${warnings.join("\n")}\n`, "stderr"); + return files; }); /** @@ -182,20 +130,27 @@ export const legacyApplySeedFiles = ( // statements are in memory at a time, matching Go's `ExecBatchWithCache` → // `parseFile` inside the apply loop (`file.go:198-203`). A dirty seed only // updates its recorded hash, so Go never re-reads it — skip the read. - const statements = seed.dirty - ? [] - : legacySplitAndTrim( - new TextDecoder().decode( - yield* fs.readFile(legacyResolveUnderWorkdir(path, workdir, seed.path)).pipe( - Effect.mapError( - (cause) => - new LegacyMigrationSeedError({ - message: `failed to open seed file: ${cause.message}`, - }), - ), - ), + let statements: ReadonlyArray = []; + if (!seed.dirty) { + const content = new TextDecoder().decode( + yield* fs.readFile(legacyResolveUnderWorkdir(path, workdir, seed.path)).pipe( + Effect.mapError( + (cause) => + new LegacyMigrationSeedError({ + message: `failed to open seed file: ${cause.message}`, + }), ), - ); + ), + ); + // Go's `SeedFile.ExecBatchWithCache` parses through the same `parseFile` every + // other caller does, so it enforces `SUPABASE_SCANNER_BUFFER_SIZE` here too — + // see `checkScannerBufferSize`'s own doc comment. + yield* checkScannerBufferSize( + content, + (message) => new LegacyMigrationSeedError({ message }), + ); + statements = legacySplitAndTrim(content); + } const txn = Effect.gen(function* () { yield* session.exec("BEGIN"); if (!seed.dirty) { diff --git a/apps/cli/src/legacy/shared/legacy-seed.unit.test.ts b/apps/cli/src/legacy/shared/legacy-seed.unit.test.ts index e3f672ef51..177084c845 100644 --- a/apps/cli/src/legacy/shared/legacy-seed.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-seed.unit.test.ts @@ -3,7 +3,7 @@ import { tmpdir } from "node:os"; import { join } from "node:path"; import { BunServices } from "@effect/platform-bun"; import { describe, expect, it } from "@effect/vitest"; -import { Effect, FileSystem, Layer, Path } from "effect"; +import { Effect, Exit, FileSystem, Layer, Path } from "effect"; import { mockOutput } from "../../../tests/helpers/mocks.ts"; import type { LegacyDbSession } from "./legacy-db-connection.service.ts"; @@ -111,3 +111,35 @@ describe("legacyApplySeedFiles seed glob", () => { }, ); }); + +describe("legacyApplySeedFiles scanner buffer size", () => { + it.effect( + "rejects an oversized seed statement when SUPABASE_SCANNER_BUFFER_SIZE is configured (Go SeedFile.ExecBatchWithCache parity)", + () => { + // Ports the same `parseFile` every migration/globals/schema-file caller goes + // through (see `checkScannerBufferSize`'s doc comment), so an oversized + // statement must abort here too, not execute silently. + const dir = mkdtempSync(join(tmpdir(), "legacy-seed-scanner-")); + // Raw text must exceed the 4096-byte floor Go's bufio.Scanner starts at + // regardless of the configured limit (see legacy-migration-apply.unit.test.ts's + // equivalent case for the exact same 4096-byte floor). + writeFileSync(join(dir, "big.sql"), `insert into t values ('${"x".repeat(5000)}');`); + const { session, queries } = fakeSession(); + const out = mockOutput(); + const previous = process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = "100b"; + return run(session, dir, ["big.sql"], out).pipe( + Effect.exit, + Effect.tap((exit) => + Effect.sync(() => { + expect(Exit.isFailure(exit)).toBe(true); + expect(queries.some((q) => q.sql.includes("insert into t"))).toBe(false); + rmSync(dir, { recursive: true, force: true }); + if (previous === undefined) delete process.env["SUPABASE_SCANNER_BUFFER_SIZE"]; + else process.env["SUPABASE_SCANNER_BUFFER_SIZE"] = previous; + }), + ), + ); + }, + ); +}); diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts new file mode 100644 index 0000000000..dba95222a6 --- /dev/null +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.ts @@ -0,0 +1,506 @@ +import { Effect, type FileSystem, type Path, Result } from "effect"; + +import { legacyErrorMessage, legacyRelativizeErrorMessage } from "./legacy-error-message.ts"; +import { LEGACY_BAD_PATTERN_MESSAGE, legacyPathMatch } from "./legacy-path-match.ts"; + +const META_CHARS = /[*?[\\]/u; + +// Go's `config.hasGlobMeta` (`apps/cli-go/pkg/config/config.go:211-213`) — a DIFFERENT, +// narrower set than `META_CHARS` above (which mirrors `io/fs.hasMeta`'s `path.Match` +// escape handling and includes `\`). Only used to gate `WithSkipEmptyGlobs()` below. +const GLOB_META_CHARS = /[*?[]/u; + +// Go's `filepath.ToSlash` replaces `os.PathSeparator` with `/` — a no-op on the +// non-Windows platforms this shell mostly runs on, since their separator already IS +// `/`. Gating on `win32` (rather than converting unconditionally) matters: on +// non-Windows a `\` in a pattern is never a path separator, only a `path.Match` +// escape (`foo\.sql`, `seed\*.sql`), and unconditionally slashing it would corrupt +// that escape — see `legacyPathMatch`'s escape handling below. +const toSlash = (p: string): string => (process.platform === "win32" ? p.replaceAll("\\", "/") : p); + +// Go's `sort.Strings` (used by both `Glob.SQLFiles`, `config.go:155`, and +// `walkMatchedDir`, `config.go:207`) orders strings by their raw UTF-8 BYTES — +// Go strings are just byte slices, so `strings.Compare` never decodes runes. JS's +// default `Array.prototype.sort()` instead compares UTF-16 CODE UNITS, which +// disagrees with UTF-8 byte order for any character outside the Basic Multilingual +// Plane: a supplementary-plane code point (`U+10000`+, a UTF-16 surrogate PAIR +// starting `0xD800`-`0xDBFF`) always UTF-8-encodes to 4 bytes leading `0xF0`-`0xF4`, +// while every 3-byte-encoded BMP character (`U+0800`-`U+FFFF`, UTF-8 lead byte +// `0xE0`-`0xEF`) is numerically SMALLER as a lead byte but can have a LARGER lone +// UTF-16 code unit than the surrogate pair's lead unit — so the two orderings can +// disagree. Verified empirically: sorting a 4-byte emoji filename against a +// 3-byte fullwidth-exclamation filename, Go's `sort.Strings` places the fullwidth +// exclamation FIRST, while JS's default `.sort()` places the emoji first. +const UTF8_ENCODER = new TextEncoder(); +const utf8Compare = (a: string, b: string): number => { + const bytesA = UTF8_ENCODER.encode(a); + const bytesB = UTF8_ENCODER.encode(b); + const len = Math.min(bytesA.length, bytesB.length); + for (let i = 0; i < len; i++) { + const diff = bytesA[i]! - bytesB[i]!; + if (diff !== 0) return diff; + } + return bytesA.length - bytesB.length; +}; + +// Joins a matched directory (or glob-split directory prefix) with a child/entry name, +// delegating to the injected `Path.Path` service for Go's `path.Join`-equivalent +// cleaning. Two Go call sites build a path exactly this way, and both need the same +// cleaning: +// +// - Direct glob-match construction (`globOne`, below): the real runtime glob path — +// `config.Glob.SQLFiles`'s `fs.Glob(fsys, pattern)` call (`apps/cli-go/pkg/config/ +// config.go:145`) resolves to `afero.IOFS.Glob` (it implements `fs.GlobFS`), which +// delegates to `afero.Glob` (`github.com/spf13/afero@v1.15.0/iofs.go:56-65`). Its +// `glob()` helper appends each match as `filepath.Join(dir, n)` +// (`match.go:99`), so a glob whose directory portion has a cleanable segment +// (`/tmp/./schemas/*.sql`, `/tmp/x/../schemas/*.sql`, a doubled `/tmp/schemas//*.sql`) +// still records the CLEANED path, not a raw concatenation. Verified empirically: a +// scratch `afero.Glob` probe against all three shapes above returns +// `.../tmp/schemas/a.sql` in every case. +// - Walked-child construction (`legacyWalkSqlFiles`, below): Go's `fs.WalkDir` builds +// each child path via `path.Join(dirname, name)` (`io/fs/walk.go`). +// +// `path.Join`/`filepath.Join` both run Clean on the joined result — collapsing doubled +// slashes, dropping a bare `.` root, and lexically resolving `.`/`..` segments anywhere +// else in the path. Node's `path.join` (via this module's injected `Path.Path` service, +// backed by `node:path`) runs the same POSIX lexical-cleaning algorithm and was verified +// empirically to match byte-for-byte across every case either call site can hit — +// dot-root, trailing slash, embedded `.`/`..`, and doubled slashes: +// +// Go: path.Join(".", "foo.sql") = "foo.sql" +// filepath.Join("/tmp/schemas/", "a.sql") = "/tmp/schemas/a.sql" +// filepath.Join("/tmp/./schemas", "a.sql") = "/tmp/schemas/a.sql" +// filepath.Join("/tmp/x/../schemas", "a.sql") = "/tmp/schemas/a.sql" +// path.Join("..", "foo.sql") = "../foo.sql" +// Node: path.join(".", "foo.sql") = "foo.sql" +// path.join("/tmp/schemas/", "a.sql") = "/tmp/schemas/a.sql" +// path.join("/tmp/./schemas", "a.sql") = "/tmp/schemas/a.sql" +// path.join("/tmp/x/../schemas", "a.sql") = "/tmp/schemas/a.sql" +// path.join("..", "foo.sql") = "../foo.sql" +// +// For seeds, the resulting path becomes the `supabase_migrations.seed_files.path` hash +// key, so any of these cleaning differences would make a TS-resolved path fail to match +// an already-recorded Go-CLI key and re-run/re-record the seed; for schema files it +// changes what path is suggested on an apply failure. +const joinRelChild = (path: Path.Path, rel: string, name: string): string => path.join(rel, name); + +/** + * Splits a forward-slashed path into its directory prefix and final element. + * + * A bare root prefix is kept as `"/"`, never chopped to `""`. This mirrors + * the real runtime glob path — `config.Glob.SQLFiles`'s `fs.Glob` call + * resolves to `afero.IOFS.Glob` (it implements `fs.GlobFS`), which delegates + * to `afero.Glob`/`match.go`'s `filepath.Split` followed by a switch on + * `dir` that leaves a bare `filepath.Separator` alone — every OTHER trailing + * separator is chopped, but the root one is deliberately preserved. Verified + * empirically against `apps/cli-go`: with cwd elsewhere, a pattern rooted at + * `/`, with a metacharacter in the FIRST component after the root slash + * (e.g. `tmp` + wildcard + `probe-dir` + wildcard + `.sql`), still resolves + * that first component against the filesystem ROOT, not cwd. Collapsing this + * to `dir: ""` would make `globOne` below treat such an absolute root-level + * pattern as relative to the workdir instead of the filesystem root. + * + * On Windows, a bare drive-root prefix (`"C:/"`) needs the exact same + * preservation, for the same reason but a different mechanism: Go's + * `filepath.Split` treats `"C:"` as the volume name (`volumeNameLen`, + * `internal/filepathlite/path_windows.go`) and always keeps the following + * separator attached to `dir` — verified against that source directly, since + * there is no Windows machine available to run the compiled stdlib on: + * `Split("C:/*.sql")` returns `dir: "C:/"`, not `dir: "C:"`. Chopping the + * separator here would matter downstream: Node's `path.isAbsolute("C:")` is + * `false` (a bare drive letter is a *drive-relative* path in Windows + * semantics, not absolute), so `globOne`'s `resolve()` would wrongly `join` + * it under the workdir instead of resolving the real drive root, while + * `path.isAbsolute("C:/")` is `true`. + */ +const splitPath = (p: string): { readonly dir: string; readonly file: string } => { + const slash = p.lastIndexOf("/"); + if (slash === -1) return { dir: "", file: p }; + if (slash === 0) return { dir: "/", file: p.slice(1) }; + if (process.platform === "win32" && slash === 2 && p.charAt(1) === ":") { + return { dir: p.slice(0, 3), file: p.slice(3) }; + } + return { dir: p.slice(0, slash), file: p.slice(slash + 1) }; +}; + +/** Faithful port of Go's `fs.Glob` for one pattern, rooted at `workdir`. */ +const globOne = ( + fs: FileSystem.FileSystem, + path: Path.Path, + workdir: string, + pattern: string, +): Effect.Effect, never> => + Effect.gen(function* () { + // Go's `fs.Glob`/`afero.Glob` resolve a literal (no-metacharacter) pattern via + // `Lstat`, which errors on an empty path — so `""` always yields no matches. An + // unguarded `path.join(workdir, "")` resolves to `workdir` itself, which would + // wrongly report the workdir as a match for an empty `schema_paths`/`sql_paths` + // entry (e.g. `schema_paths = [""]`). + if (pattern.length === 0) { + return []; + } + // Absolute patterns resolve against the filesystem root; relative ones are + // rooted at the workdir. + const resolve = (p: string): string => (path.isAbsolute(p) ? p : path.join(workdir, p)); + // No metacharacters: Go's `fs.Glob`/`afero.Glob` fast path (`match.go:34-40`) probes + // via `Lstat` (`OsFs.LstatIfPossible` → `os.Lstat`, verified empirically against + // `afero@v1.15.0`), which does NOT follow a symlink — so a literal pattern naming a + // BROKEN symlink still Lstat-succeeds (the link itself exists) and is reported as a + // match; the follow-up `fs.Stat(fsys, fp)` in `legacySqlFilesGlob` below (which DOES + // follow it) is what fails, with `failed to stat matched file: ...`. `fs.exists` here + // is Effect's `access`-based check (Node's `fs.access`), which follows the symlink + // like a normal `Stat` and would wrongly report "no files matched pattern" instead — + // verified empirically: `fs.access` on a broken symlink resolves ENOENT while + // `fs.lstat` on the same path succeeds. Probe for the entry itself the same + // no-follow way `legacyWalkSqlFiles` below already does for a walked child: `readLink` + // succeeds only for a symlink (broken or not), so treat that as an Lstat success + // before falling back to the normal existence check for everything else. + if (!META_CHARS.test(pattern)) { + const resolved = resolve(pattern); + const isSymlink = yield* fs.readLink(resolved).pipe( + Effect.map(() => true), + Effect.orElseSucceed(() => false), + ); + const exists = + isSymlink || (yield* fs.exists(resolved).pipe(Effect.orElseSucceed(() => false))); + return exists ? [pattern] : []; + } + const { dir, file } = splitPath(pattern); + // Resolve the directory level first (recursively if it, too, is a glob). + const dirs = + dir === "" || !META_CHARS.test(dir) ? [dir] : yield* globOne(fs, path, workdir, dir); + const result: Array = []; + for (const d of dirs) { + const absDir = d === "" ? workdir : resolve(d); + const names = yield* fs + .readDirectory(absDir) + .pipe(Effect.orElseSucceed((): ReadonlyArray => [])); + for (const name of names) { + if (legacyPathMatch(file, name).matched) { + // `joinRelChild` (above) is the same Path-service clean-join the walked-child + // path below uses — see its comment for why a raw `${d}/${name}` concatenation + // isn't enough here (Go's `afero.Glob` cleans via `filepath.Join(dir, n)`, so + // `d === "/"`, `d === ""`, and a `dir` containing `.`/`..`/doubled-slash + // segments must all clean the same way as the walked-child case does). + // + // Deliberately NOT `toSlash`'d here, on Windows: Go's `config.Glob.SQLFiles` + // sorts the RAW backslash-joined matches from `fs.Glob`/`afero.Glob` (built via + // `filepath.Join`, `config.go:145-155`) and only converts each surviving match + // to forward slash AFTER that sort (`fp := filepath.ToSlash(item)`, + // `config.go:156`). This function's own caller (`legacySqlFilesGlob`) already + // sorts the array `globOne` returns and slashes each item only after — slashing + // here too would sort forward-slash-joined strings instead of the raw + // backslash-joined ones, which can disagree: comparing `a\x.sql` vs `a0\x.sql` + // byte-for-byte puts `a0\x.sql` first (`\` is `0x5C`, greater than `0`'s + // `0x30`), while `a/x.sql` vs `a0/x.sql` puts `a/x.sql` first (`/` is `0x2F`, + // less than `0x30`) — a different order for the same two matches. + result.push(joinRelChild(path, d, name)); + } + } + } + return result; + }); + +/** + * Recursively collects the regular `.sql` files under a matched directory, porting + * Go's `walkMatchedDir` with the `SQLFiles` include filter (`entry.Type().IsRegular() && + * filepath.Ext(path) == ".sql"`, `apps/cli-go/pkg/config/config.go:126-131,194-211`). + * Paths are workdir-relative (matching the glob output), forward-slashed, and sorted + * for deterministic application. + * + * A `ReadDir` failure anywhere in the tree (e.g. a permissions error on a nested + * directory) fails the WHOLE walk with `failed to walk matched directory: `, + * discarding every file collected so far — Go's `fs.WalkDir` callback returns that + * `err` unchanged, which stops the traversal immediately and makes `walkMatchedDir` + * return `(nil, err)` rather than the partial list (`config.go:196-208`; verified + * empirically: an unreadable matched directory makes `Glob.SQLFiles` return a + * `failed to walk matched directory: ...` error with zero files, not an empty match). + */ +const legacyWalkSqlFiles = ( + fs: FileSystem.FileSystem, + path: Path.Path, + workdir: string, + dir: string, +): Effect.Effect, string> => + Effect.gen(function* () { + const collected: Array = []; + const walk = (rel: string): Effect.Effect => + Effect.gen(function* () { + const absDir = path.isAbsolute(rel) ? rel : path.join(workdir, rel); + // Go's `fs.WalkDir` runs against `afero.OsFs` with the process cwd already the + // workdir (`ChangeWorkDir`, `cmd/root.go`), so a `ReadDir` failure — on the + // matched root `dir` itself, or on any nested directory the walk descends into — + // embeds the workdir-relative `rel` in its error text, never an absolute path. + // This module never `process.chdir`s, so the real read needs `absDir`, but the + // wrapped warning must still report `rel` (same substitution pattern as the + // matched-file stat failure below and `legacyApplySchemaFiles`'s read errors). + const names = yield* fs + .readDirectory(absDir) + .pipe( + Effect.mapError( + (error) => + `failed to walk matched directory: ${legacyRelativizeErrorMessage(legacyErrorMessage(error), absDir, rel)}`, + ), + ); + // Go's `fs.WalkDir` visits directory entries in lexical byte order — its own + // `ReadDir` (`os.ReadDir`/`afero.OsFs`) contract guarantees results "sorted by + // filename" before `walkDir` ever iterates them. This FileSystem service's + // `readDirectory` makes no such promise (raw OS enumeration order), so when a + // directory contains MULTIPLE problematic children (e.g. two unreadable + // subdirectories), which one's failure short-circuits this loop — and therefore + // which single error message this walk fails with, or which `WARN:`/fatal text a + // caller (the experimental schema branch, or seed-path globbing) surfaces — + // would otherwise depend on filesystem enumeration order instead of matching + // Go's deterministic choice (review CLI-1958). Sort with the same UTF-8 + // byte-order comparator `globOne`/the top-level match list already use above. + for (const name of [...names].sort(utf8Compare)) { + const childRel = joinRelChild(path, rel, name); + const childAbs = path.isAbsolute(childRel) ? childRel : path.join(workdir, childRel); + // Go's `fs.WalkDir` types each child from the parent's `ReadDir` entry + // (`os.ReadDir`'s Lstat-based `DirEntry`) and never re-`Stat`s through it — + // so a symlinked file or subdirectory found below the matched root is + // neither included nor recursed into, regardless of what it points to + // (`io/fs/walk.go:114-115`: only the matched root itself, resolved once by + // the caller before reaching this walk, may be a symlink). `readLink` + // succeeds only for symlinks, so use it as the no-follow probe in place of + // the `Lstat` this FileSystem service doesn't expose. + const isSymlink = yield* fs.readLink(childAbs).pipe( + Effect.map(() => true), + Effect.orElseSucceed(() => false), + ); + if (isSymlink) { + continue; + } + const statResult = yield* fs.stat(childAbs).pipe(Effect.result); + if (Result.isFailure(statResult)) { + // TOCTOU: this child existed a moment ago in `names` (this directory's + // `readDirectory` snapshot) but is gone by the time we stat it here — e.g. removed + // by a concurrent process. Go never hits this window for the child's TYPE: `fs.WalkDir` + // decides file-vs-directory from the SAME `DirEntry` its parent `ReadDir` already + // returned and never re-`Stat`s a child, so a `.sql` file that disappears here + // stays in Go's declared file list, and only the later, real file-open fails + // loudly. Verified empirically: a scratch `filepath.WalkDir` probe that deletes a + // sibling `.sql` file between `ReadDir` and that file's own visit still reports + // it `IsRegular` from the cached entry, keeps it in `declared`, and the + // subsequent `os.Open` on it fails with "no such file or directory" — never a + // silent drop. Losing the stat here must not silently drop the file and let the + // walk "succeed" having applied nothing — best-effort include it, matching Go's + // outcome, and let the real downstream read surface the failure. + if (childRel.endsWith(".sql")) { + collected.push(toSlash(childRel)); + continue; + } + // TOCTOU, directory variant: the vanished entry could equally have been a + // SUBDIRECTORY of the matched tree, not a harmless non-`.sql` file — and Go's + // outcome for those two cases is NOT the same. For a directory `DirEntry`, + // `fs.WalkDir` unconditionally attempts a second `ReadDir` to recurse into it + // (`io/fs/walk.go`'s `walkDir`); when the child has since been removed, that + // second `ReadDir` fails, and `walkMatchedDir`'s callback propagates the error + // unchanged (`if err != nil { return err }`, `config.go:198-199`) — `fs.WalkDir` + // returns it, and `walkMatchedDir` wraps it as `failed to walk matched + // directory: `, discarding every file already collected + // (`config.go:205-206`). Verified empirically: a scratch `fs.WalkDir` probe that + // removes a nested subdirectory between the parent's `ReadDir` and the + // subdirectory's own `ReadDir` reproduces exactly this — zero files, error + // `failed to walk matched directory: open .../nested: no such file or + // directory` — never a silent skip, unlike the vanished-`.sql`-file case above + // (round 6). This FileSystem service can't recover the lost entry's type after + // the fact — Go's `DirEntry` type came for free from the same `ReadDir` syscall + // as the listing, whereas this port's `stat` is a second, separate syscall, so a + // raced disappearance here always loses the type along with the entry. Treat any + // non-`.sql` disappearance as a potential directory and fail the whole walk the + // same way an unreadable still-present directory does (the `readDirectory` + // failure above) — matching Go's fail-loud design intent (never silently apply a + // partial schema/seed set) rather than risk silently dropping an entire nested + // subtree of schema files. This `stat` stands in for the second `ReadDir` Go + // itself would issue on the vanished directory, so its display path needs the + // same absolute-to-relative substitution as the `readDirectory` failure above — + // `childAbs` is what the real (stand-in) syscall needed, `childRel` is what Go's + // own `ReadDir` error would embed. + return yield* Effect.fail( + `failed to walk matched directory: ${legacyRelativizeErrorMessage(legacyErrorMessage(statResult.failure), childAbs, childRel)}`, + ); + } + const childType = statResult.success.type; + if (childType === "Directory") { + yield* walk(childRel); + } else if (childType === "File" && childRel.endsWith(".sql")) { + collected.push(toSlash(childRel)); + } + } + }); + yield* walk(dir); + return collected.sort(utf8Compare); + }); + +/** Result of resolving SQL-file glob patterns against the workspace. */ +interface LegacySqlFilesGlobResult { + /** Workdir-relative, forward-slashed matches, deduplicated in first-seen order across patterns. */ + readonly files: ReadonlyArray; + /** + * Per-pattern/per-match problems (`no files matched pattern: …` / `failed to glob + * files: …` / `failed to walk matched directory: …`), in pattern order. Never fatal + * by itself — callers decide when a warning matters + * (e.g. the seed path always surfaces it; the schema-files apply path only surfaces + * it when NO pattern matched anything at all, mirroring Go's `apply.go:53-55`). + */ + readonly warnings: ReadonlyArray; +} + +/** + * Mirrors Go's `GlobOption`s (`apps/cli-go/pkg/config/config.go:100-117`). Neither + * caller in this shared module needs them today (`legacyApplySchemaFiles`'s + * `[db.migrations].schema_paths`, and the two `[db.seed].sql_paths` callers — + * `legacyGetPendingSeeds` and `legacy-seed.ts`'s `resolveSeedFiles` — all pass none, + * matching Go's own zero-option call sites, `pkg/migration/seed.go:35` and + * `internal/migration/apply/apply.go:52`). Added for `db diff`'s declarative path, + * which calls `Glob.files` `WithSkipEmptyGlobs()` + `WithErrorOnAllSkippedGlobs()`. + */ +export interface LegacySqlFilesGlobOptions { + /** + * Go's `WithSkipEmptyGlobs()`: a pattern that contains a glob metacharacter + * (`config.hasGlobMeta` — `*`, `?`, `[`; NOT `\`, unlike `META_CHARS` above) and + * matches nothing is silently skipped — no "no files matched pattern" warning — + * unless `errorOnAllSkipped` retroactively un-skips it (below). A LITERAL pattern + * (no glob metacharacter) that doesn't exist always warns, regardless of this flag. + */ + readonly skipEmptyGlobs?: boolean; + /** + * Go's `WithErrorOnAllSkippedGlobs()`: only meaningful together with + * `skipEmptyGlobs`. If the overall result ends up empty AND at least one pattern + * was silently skipped, every skipped pattern's silence is retroactively turned + * back into a "no files matched pattern" warning — so a `skipEmptyGlobs` caller + * can still detect the "nothing matched anything" case. + */ + readonly errorOnAllSkipped?: boolean; +} + +/** + * Resolves SQL-file glob patterns to existing files, porting Go's `config.Glob.SQLFiles` + * over `fs.Glob` (`apps/cli-go/pkg/config/config.go:123-211`). Shared by + * `[db.seed].sql_paths` (via `legacyGetPendingSeeds`, `legacy-seed-ops.ts`, and + * `legacy-seed.ts`'s `resolveSeedFiles`) and `[db.migrations].schema_paths` (via + * `legacyApplySchemaFiles`, `legacy-migration-apply.ts`) — all three Go call sites + * resolve through the exact same `Glob` type and `SQLFiles` method, so the traversal + * logic lives here once. + * + * Each pattern is matched independently: matches are sorted per-pattern (Go's + * `sort.Strings`, `config.go:155`), but the overall result preserves cross-pattern + * DECLARATION order (no global re-sort), with first-seen dedup. A directory match is + * expanded to its regular `.sql` files, recursively, sorted by full path + * (`walkMatchedDir`); a non-directory match is kept verbatim regardless of extension. A + * pattern that matches nothing, or is malformed (Go's `path.ErrBadPattern`, e.g. an + * unterminated `[` class), contributes a warning but does not stop the loop — mirroring + * `fs.Glob`'s up-front `Match(pattern, "")` validation (`io/fs/glob.go`). + * + * Patterns are assumed already resolved to Go's config-load form (a relative entry + * `supabase/`-joined, an absolute entry verbatim) — callers resolve that once at + * config-read time (`legacyResolveSeedSqlPath`), matching Go's `config.resolve` step, + * which runs once at config load, before any glob. + */ +export const legacySqlFilesGlob = Effect.fnUntraced(function* ( + fs: FileSystem.FileSystem, + path: Path.Path, + patterns: ReadonlyArray, + workdir: string, + options?: LegacySqlFilesGlobOptions, +) { + const skipEmptyGlobs = options?.skipEmptyGlobs ?? false; + const errorOnAllSkipped = options?.errorOnAllSkipped ?? false; + const seen = new Set(); + const files: Array = []; + const warnings: Array = []; + const skipped: Array = []; + + for (const rawPattern of patterns) { + // Go's `filepath.ToSlash(pattern)` (`config.go:145`) is passed only as an ARGUMENT + // to `fs.Glob` — the loop's own `pattern` variable (Go's range variable) is never + // reassigned, so every later reference to it in THIS iteration (`hasGlobMeta`, the + // skipped-pattern list, and the "no files matched pattern" warning, all below) + // still reports the ORIGINAL, un-slashed pattern. On Windows, an absolute pattern + // with backslashes (`C:\schemas\*.sql`) must therefore warn with that raw backslash + // form, even though matching itself runs against the slashed form. + const pattern = toSlash(rawPattern); + // Go's `fs.Glob` validates the whole pattern up front (`Match(pattern, "")`); a + // malformed glob is reported as `failed to glob files: ` and + // contributes no matches, rather than the misleading "no files matched" below. + if (legacyPathMatch(pattern, "").badPattern) { + warnings.push(`failed to glob files: ${LEGACY_BAD_PATTERN_MESSAGE}`); + continue; + } + const matches = yield* globOne(fs, path, workdir, pattern); + if (matches.length === 0) { + if (skipEmptyGlobs && GLOB_META_CHARS.test(rawPattern)) { + skipped.push(rawPattern); + } else { + warnings.push(`no files matched pattern: ${rawPattern}`); + } + continue; + } + for (const match of [...matches].sort(utf8Compare)) { + const fp = toSlash(match); + // A directory match is expanded to its regular `.sql` files recursively + // (`walkMatchedDir`, sorted); a file match is kept verbatim (`config.go:157-183`). + // Go: `if err != nil { allErrors = append(allErrors, errors.Errorf("failed to + // stat matched file: %w", err)); continue }` (`config.go:157-161`) — a match + // that disappears (or is a broken symlink) between the glob and this stat + // becomes a warning and is skipped, same as a walk failure below. Falling back + // to treating it as a regular file (the previous behaviour here) would instead + // hand a nonexistent path to the caller's later read, turning a warned-but- + // otherwise-successful reset into a hard apply error. + // + // Go's `fs.Stat(fsys, fp)` (`config.go:157`) runs with its process cwd already + // the workdir (`ChangeWorkDir`, `cmd/root.go`), so `fp` IS the exact string the + // real syscall sees and the resulting error's path is that workdir-relative + // `fp`. This module never `process.chdir`s, so the real stat needs an absolute + // path here — but the wrapped message must still report the relative `fp`, not + // the absolute path used to make the syscall work. Same substitution pattern as + // `legacyApplySchemaFiles`'s read-error display path (`legacy-migration-apply.ts`). + const absoluteFp = path.isAbsolute(fp) ? fp : path.join(workdir, fp); + const statResult = yield* fs.stat(absoluteFp).pipe(Effect.result); + if (Result.isFailure(statResult)) { + const message = legacyRelativizeErrorMessage( + legacyErrorMessage(statResult.failure), + absoluteFp, + fp, + ); + warnings.push(`failed to stat matched file: ${message}`); + continue; + } + const matchType = statResult.success.type; + if (matchType === "Directory") { + // Go: `if err != nil { allErrors = append(allErrors, err); continue }` — a walk + // failure on this match becomes a warning (never a hard Effect failure, like + // every other per-match/per-pattern problem here) and the loop moves on to the + // next match, exactly like a malformed pattern or a "no files matched" miss. + const walked = yield* legacyWalkSqlFiles(fs, path, workdir, fp).pipe(Effect.result); + if (Result.isFailure(walked)) { + warnings.push(walked.failure); + continue; + } + for (const file of walked.success) { + if (!seen.has(file)) { + seen.add(file); + files.push(file); + } + } + continue; + } + if (!seen.has(fp)) { + seen.add(fp); + files.push(fp); + } + } + } + + // Go: `if opts.errorOnAllSkipped && len(result) == 0 && len(skipped) > 0` — only + // escalate silently-skipped patterns back into warnings when NOTHING matched at all. + if (errorOnAllSkipped && files.length === 0 && skipped.length > 0) { + for (const pattern of skipped) { + warnings.push(`no files matched pattern: ${pattern}`); + } + } + + return { files, warnings } satisfies LegacySqlFilesGlobResult; +}); diff --git a/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts new file mode 100644 index 0000000000..a40acc4cc5 --- /dev/null +++ b/apps/cli/src/legacy/shared/legacy-sql-files-glob.unit.test.ts @@ -0,0 +1,828 @@ +import { chmodSync, mkdirSync, mkdtempSync, rmSync, symlinkSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { BunFileSystem, BunPath, BunServices } from "@effect/platform-bun"; +import { describe, expect, it } from "@effect/vitest"; +import { Effect, FileSystem, Layer, Path } from "effect"; + +import { legacySqlFilesGlob } from "./legacy-sql-files-glob.ts"; + +const run = (patterns: ReadonlyArray, workdir: string) => + Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + return yield* legacySqlFilesGlob(fs, path, patterns, workdir); + }).pipe(Effect.provide(BunServices.layer)); + +describe("legacySqlFilesGlob", () => { + it.effect( + "treats an empty pattern as no match, not the workdir itself (Go fs.Glob parity)", + () => { + // Go's `fs.Glob`/`afero.Glob` resolve a no-metacharacter pattern via `Lstat`, which + // errors on an empty path — an empty `schema_paths`/`sql_paths` entry (e.g. + // `schema_paths = [""]`) always yields no matches, never the workdir itself. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-empty-")); + return run([""], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual([]); + expect(result.warnings).toEqual(["no files matched pattern: "]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "does not follow a symlinked .sql file below a matched directory (Go WalkDir parity)", + () => { + // Go's `Glob.SQLFiles` expands a matched directory with `fs.WalkDir`, which types + // each child from its parent's `ReadDir` entry (`os.ReadDir`'s Lstat-based + // `DirEntry`) and never re-`Stat`s through it — so a symlinked `.sql` file is + // never included, regardless of what it points to. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-symlink-file-")); + const schemasDir = join(dir, "schemas"); + mkdirSync(schemasDir); + writeFileSync(join(schemasDir, "real.sql"), "select 1;"); + const outsideDir = join(dir, "outside"); + mkdirSync(outsideDir); + writeFileSync(join(outsideDir, "evil.sql"), "select 2;"); + symlinkSync(join(outsideDir, "evil.sql"), join(schemasDir, "linked.sql")); + return run(["schemas"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["schemas/real.sql"]); + expect(result.warnings).toEqual([]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "does not recurse into a symlinked subdirectory below a matched directory (Go WalkDir parity)", + () => { + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-symlink-dir-")); + const schemasDir = join(dir, "schemas"); + mkdirSync(schemasDir); + writeFileSync(join(schemasDir, "real.sql"), "select 1;"); + const outsideSubdir = join(dir, "outside-subdir"); + mkdirSync(outsideSubdir); + writeFileSync(join(outsideSubdir, "nested.sql"), "select 3;"); + symlinkSync(outsideSubdir, join(schemasDir, "linked-dir")); + return run(["schemas"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["schemas/real.sql"]); + expect(result.warnings).toEqual([]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "surfaces a stat failure on a matched file as a warning instead of treating it as a regular file (Go parity)", + () => { + // Go: `if info, err := fs.Stat(fsys, fp); err != nil { allErrors = append(allErrors, + // errors.Errorf("failed to stat matched file: %w", err)); continue }` (config.go:157-161) + // — a match that disappears (or is a broken symlink) between the glob and this stat + // becomes a warning and is skipped entirely, never silently treated as a regular file. + // + // Go's `fsys` here is always `afero.NewOsFs()` with the process cwd already the + // workdir (`ChangeWorkDir`, `cmd/root.go`), so `fs.Stat(fsys, fp)`'s embedded path + // in the resulting error is the workdir-RELATIVE `fp` (verified directly against + // `os.Stat`/`afero.OsFs.Stat`, which pass the name through to `os.Stat` unchanged). + // This module never `process.chdir`s, so the real stat needs an absolute path — but + // the warning must still report the relative form, not that absolute (temp-dir) + // path, or it would leak a local filesystem path Go never would. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-stat-fail-")); + const schemasDir = join(dir, "schemas"); + mkdirSync(schemasDir); + writeFileSync(join(schemasDir, "good.sql"), "select 1;"); + symlinkSync(join(schemasDir, "does-not-exist.sql"), join(schemasDir, "broken.sql")); + return run(["schemas/*.sql"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["schemas/good.sql"]); + expect(result.warnings).toHaveLength(1); + expect(result.warnings[0]).toMatch(/^failed to stat matched file: /); + expect(result.warnings[0]).toContain("schemas/broken.sql"); + expect(result.warnings[0]).not.toContain(dir); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "surfaces a stat failure for a LITERAL (no-metacharacter) pattern naming a broken symlink, instead of reporting no match (Go afero.Glob Lstat parity)", + () => { + // Go's `fs.Glob`/`afero.Glob` no-metacharacter fast path (`match.go:34-40`) probes + // via `Lstat` (`OsFs.LstatIfPossible` → `os.Lstat`), which does NOT follow a + // symlink — so a LITERAL pattern naming a broken symlink still Lstat-succeeds (the + // link itself exists) and is reported as a match; the follow-up `fs.Stat` above is + // what then fails with `failed to stat matched file: ...`, exactly like the + // wildcard-pattern case the previous test covers. Verified empirically against + // `apps/cli-go` (`afero.Glob`/`fs.Stat` scratch probe): a literal broken-symlink + // pattern always Globs to a match and always fails the follow-up Stat — never + // "no files matched pattern". + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-literal-symlink-")); + const schemasDir = join(dir, "schemas"); + mkdirSync(schemasDir); + symlinkSync(join(schemasDir, "does-not-exist.sql"), join(schemasDir, "broken.sql")); + return run(["schemas/broken.sql"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual([]); + expect(result.warnings).toHaveLength(1); + expect(result.warnings[0]).toMatch(/^failed to stat matched file: /); + expect(result.warnings[0]).toContain("schemas/broken.sql"); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "keeps a bare root ('/') as the directory when a glob pattern's meta character is in the first path component (Go afero.Glob parity)", + () => { + // Go's real runtime glob path — `config.Glob.SQLFiles`'s `fs.Glob` call resolves to + // `afero.IOFS.Glob` (it implements `fs.GlobFS`), which delegates to `afero.Glob` + // (`match.go`): `filepath.Split` followed by a switch that leaves a bare + // `filepath.Separator` alone — every OTHER trailing separator is chopped, but the + // root one is deliberately preserved. Verified empirically against `apps/cli-go`: + // with cwd elsewhere, a pattern rooted at "/" with a metacharacter in the first + // component after the root slash still resolves against the filesystem ROOT, not + // cwd. A canary file placed in the WORKDIR (never the real "/") proves this native + // port does not fall back to treating the root component as workdir-relative. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-abs-root-")); + writeFileSync(join(dir, "__legacy_sql_glob_canary__.sql"), "select 1;"); + return run(["/*__legacy_sql_glob_canary__*.sql"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual([]); + expect(result.warnings).toEqual([ + "no files matched pattern: /*__legacy_sql_glob_canary__*.sql", + ]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "recurses through a root-anchored directory component without falling back to the workdir (Go afero.Glob parity)", + () => { + // Same bug as above, but for a two-level pattern (`/foo*/*.sql`) — the recursive + // call that resolves the "foo*" directory component must also treat "/" as the + // real filesystem root, not "" (which `globOne` maps to the workdir). The workdir + // here contains a subdirectory that WOULD match "foo*" if (and only if) the + // recursive call incorrectly fell back to reading the workdir instead of "/". + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-abs-root-nested-")); + const canaryDir = join(dir, "__legacy_sql_glob_root_canary_dir__"); + mkdirSync(canaryDir); + writeFileSync(join(canaryDir, "a.sql"), "select 1;"); + return run(["/__legacy_sql_glob_root_canary_dir__*/*.sql"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual([]); + expect(result.warnings).toEqual([ + "no files matched pattern: /__legacy_sql_glob_root_canary_dir__*/*.sql", + ]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "preserves a Windows drive root ('C:/') as the directory when splitting a glob pattern (Go filepath.Split parity)", + () => { + // Go's `filepath.Split` treats `"C:"` as the volume name on Windows + // (`volumeNameLen`, `internal/filepathlite/path_windows.go`) and always + // keeps the following separator attached to `dir` — so `Split("C:/*.sql")` + // returns `dir: "C:/"`, not `dir: "C:"` (verified directly against that + // stdlib source; there is no Windows machine available to run the + // compiled binary on). Losing the trailing slash matters: Node's + // `path.isAbsolute("C:")` is `false` (a bare drive letter is + // *drive-relative*, not absolute, in Windows semantics), so `globOne`'s + // `resolve()` would wrongly `join` it under the workdir instead of + // resolving the real drive root. Force win32 path semantics + // (`BunPath.layerWin32`) and this module's own `process.platform` gate + // so the test exercises the same branch a real Windows install takes. + const originalPlatform = process.platform; + Object.defineProperty(process, "platform", { value: "win32" }); + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-drive-root-")); + writeFileSync(join(dir, "a.sql"), "select 1;"); + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + // A "C:/" drive root doesn't exist on this (non-Windows) test host, so + // fake just the two calls that must resolve against it, reusing a real + // file's stat info to avoid hand-rolling a `File.Info`. + const realFileInfo = yield* fs.stat(join(dir, "a.sql")); + const driveRootFs: FileSystem.FileSystem = { + ...fs, + readDirectory: (p: string) => + p === "C:/" ? Effect.succeed(["a.sql"]) : fs.readDirectory(p), + stat: (p: string) => (p === "C:/a.sql" ? Effect.succeed(realFileInfo) : fs.stat(p)), + }; + return yield* legacySqlFilesGlob(driveRootFs, path, ["C:/*.sql"], dir); + }).pipe( + Effect.provide(Layer.mergeAll(BunFileSystem.layer, BunPath.layerWin32)), + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["C:/a.sql"]); + expect(result.warnings).toEqual([]); + }), + ), + Effect.ensuring( + Effect.sync(() => { + Object.defineProperty(process, "platform", { value: originalPlatform }); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "reports the raw backslash pattern in a 'no files matched' warning on Windows, not the slashed form used for matching (Go filepath.ToSlash parity)", + () => { + // Go passes `filepath.ToSlash(pattern)` only as an ARGUMENT to `fs.Glob` + // (`config.go:145`) — the loop's own `pattern` variable (Go's range variable) is + // never reassigned, so the "no files matched pattern: %s" warning (`config.go:155`) + // still reports the ORIGINAL backslash form. An absolute Windows pattern with + // backslashes that matches nothing must therefore warn with that backslash form, + // not the slashed one used internally to glob. Force win32 path semantics + // (`BunPath.layerWin32`) and this module's own `process.platform` gate, same as + // the drive-root test above. + const originalPlatform = process.platform; + Object.defineProperty(process, "platform", { value: "win32" }); + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-win-warn-")); + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + return yield* legacySqlFilesGlob(fs, path, ["C:\\schemas\\*.sql"], dir); + }).pipe( + Effect.provide(Layer.mergeAll(BunFileSystem.layer, BunPath.layerWin32)), + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual([]); + expect(result.warnings).toEqual(["no files matched pattern: C:\\schemas\\*.sql"]); + }), + ), + Effect.ensuring( + Effect.sync(() => { + Object.defineProperty(process, "platform", { value: originalPlatform }); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "sorts raw backslash-joined Windows matches BEFORE slashing, not after (Go afero.Glob/filepath.ToSlash ordering parity)", + () => { + // Go's `config.Glob.SQLFiles` (`config.go:145-156`) calls `fs.Glob`, which resolves + // to `afero.Glob`'s `glob()` helper — it builds each match with `filepath.Join(dir, + // n)` (OS-separator-joined, backslash on Windows) and NEVER slashes it. Only back in + // `SQLFiles`, AFTER `sort.Strings(matches)` sorts those raw backslash matches, does + // each surviving item get `filepath.ToSlash`'d. For a pattern like `a*/x.sql` + // matching both `a\x.sql` and `a0\x.sql`, sorting the RAW backslash strings byte-for + // -byte puts `a0\x.sql` first (`\` is `0x5C`, greater than `0`'s `0x30`) — but + // sorting the SLASHED strings instead would put `a/x.sql` first (`/` is `0x2F`, less + // than `0x30`), a different order for the same two matches. `globOne` must therefore + // push the raw joined match (slashing only happens in the caller's post-sort loop), + // matching Go's real order exactly. Fully faked filesystem (no real Windows host + // available): `readDirectory`/`stat` return canned results keyed by the exact + // backslash-joined paths `BunPath.layerWin32`'s `path.join` computes. + const originalPlatform = process.platform; + Object.defineProperty(process, "platform", { value: "win32" }); + const scratchDir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-win-sort-")); + const canaryFile = join(scratchDir, "canary.sql"); + writeFileSync(canaryFile, "select 1;"); + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const fileInfo = yield* fs.stat(canaryFile); + const workdir = "/workdir"; + const winFs: FileSystem.FileSystem = { + ...fs, + readDirectory: (p: string) => { + if (p === workdir) return Effect.succeed(["a", "a0"]); + if (p === "\\workdir\\a" || p === "\\workdir\\a0") return Effect.succeed(["x.sql"]); + return fs.readDirectory(p); + }, + stat: (p: string) => + p === "\\workdir\\a\\x.sql" || p === "\\workdir\\a0\\x.sql" + ? Effect.succeed(fileInfo) + : fs.stat(p), + }; + return yield* legacySqlFilesGlob(winFs, path, ["a*/x.sql"], workdir); + }).pipe( + Effect.provide(Layer.mergeAll(BunFileSystem.layer, BunPath.layerWin32)), + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["a0/x.sql", "a/x.sql"]); + expect(result.warnings).toEqual([]); + }), + ), + Effect.ensuring( + Effect.sync(() => { + Object.defineProperty(process, "platform", { value: originalPlatform }); + rmSync(scratchDir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "normalizes a doubled slash when the matched directory itself has a trailing slash (Go path.Join parity)", + () => { + // Go's `fs.WalkDir` builds each child path via `path.Join(dirname, name)` + // (`io/fs/walk.go`), and `path.Join` runs `path.Clean` on the result, collapsing a + // doubled `/`. A literal (no-metacharacter) `schema_paths`/`sql_paths` entry like + // `"schemas/"` resolves via `fs.Glob`'s fast path to the pattern VERBATIM, trailing + // slash and all — so the walk over its children must not produce `schemas//a.sql`. + // Verified empirically against `apps/cli-go`: a scratch probe calling + // `config.Glob{"/"}.SQLFiles(...)` on a real trailing-slash directory returns + // the single-slash path, not a doubled one. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-trailing-slash-")); + const schemasDir = join(dir, "schemas"); + mkdirSync(schemasDir); + writeFileSync(join(schemasDir, "a.sql"), "select 1;"); + return run(["schemas/"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["schemas/a.sql"]); + expect(result.warnings).toEqual([]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "drops the './' prefix when the matched directory cleans to '.' (Go path.Join parity)", + () => { + // Go's `fs.WalkDir` builds each child path via `path.Join(dirname, name)`, and + // `path.Join` runs `path.Clean`, which drops a bare `.` root entirely rather than + // joining it as a prefix. A matched directory can clean to exactly `.` — e.g. + // `[db.migrations].schema_paths = [".."]`/`[db.seed].sql_paths = [".."]`, which + // `baseConfig.resolve`'s own `path.Join(builder.SupabaseDirPath, pattern)` collapses + // to `.` (`apps/cli-go/pkg/config/config.go:969-980`) — so the walk over its children + // must record `a.sql`, not `./a.sql`. This matters beyond cosmetics: for seeds, the + // walked path becomes the `supabase_migrations.seed_files.path` hash key, so a + // `./`-prefixed path would never match an already-recorded Go-CLI key. Verified + // empirically: `path.Join(".", "foo.sql")` and a real `fs.WalkDir` rooted at `.` both + // drop the `./` prefix entirely. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-dot-root-")); + writeFileSync(join(dir, "a.sql"), "select 1;"); + const nestedDir = join(dir, "nested"); + mkdirSync(nestedDir); + writeFileSync(join(nestedDir, "b.sql"), "select 2;"); + return run(["."], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["a.sql", "nested/b.sql"]); + expect(result.warnings).toEqual([]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "cleans a '..'-segment matched directory when walking its children (Go path.Join parity)", + () => { + // Go's `fs.WalkDir` builds each child path via `path.Join(dirname, name)`, and + // `path.Join` runs `path.Clean`, which lexically resolves an embedded `..` segment — + // not just a bare `.` root or a trailing slash. A matched directory can contain a + // `..` anywhere, e.g. `[db.migrations].schema_paths = ["nested/../schemas"]`, and the + // walk over its children must record `schemas/a.sql`, not `nested/../schemas/a.sql`. + // Verified empirically: `path.Join("/tmp/x/../schemas", "a.sql")` and Node's + // `path.join("/tmp/x/../schemas", "a.sql")` both clean to `/tmp/schemas/a.sql`. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-dotdot-segment-")); + const schemasDir = join(dir, "schemas"); + mkdirSync(schemasDir); + writeFileSync(join(schemasDir, "a.sql"), "select 1;"); + return run(["nested/../schemas"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["schemas/a.sql"]); + expect(result.warnings).toEqual([]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "cleans a direct glob match whose directory portion has a '.' segment (Go afero.Glob parity)", + () => { + // Distinct from the walked-child cleaning above: this pattern's glob metacharacter + // (`*`) is in the FINAL component, so `globOne` matches `a.sql` directly against + // the directory entries of `schemas/.` — it never goes through + // `legacyWalkSqlFiles`. Go's real runtime glob path resolves through + // `afero.IOFS.Glob` -> `afero.Glob`'s `glob()` helper, which appends each match as + // `filepath.Join(dir, n)` (`match.go:99`) — so the recorded match is the CLEANED + // `schemas/a.sql`, not a raw `schemas/./a.sql` concatenation. Verified empirically: + // a scratch `afero.Glob(fs, ".../tmp/./schemas/*.sql")` probe against a real + // filesystem returns the cleaned path. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-direct-dot-segment-")); + const schemasDir = join(dir, "schemas"); + mkdirSync(schemasDir); + writeFileSync(join(schemasDir, "a.sql"), "select 1;"); + return run(["schemas/./*.sql"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["schemas/a.sql"]); + expect(result.warnings).toEqual([]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "cleans a direct glob match whose directory portion has a '..' segment (Go afero.Glob parity)", + () => { + // Same distinction as above (a direct match via `globOne`, not a walked directory + // expansion), but for an embedded `..` rather than a `.` segment — e.g. an absolute + // `schema_paths`/`sql_paths` entry like `/tmp/x/../schemas/*.sql`. `filepath.Join` + // lexically resolves `..` the same way it drops a bare `.` root, so Go still records + // the cleaned `schemas/a.sql`, not `nested/../schemas/a.sql`. For seed files, that + // recorded path is the `supabase_migrations.seed_files.path` hash key, so leaving it + // uncleaned would make a TS-resolved match fail to line up with an already-recorded + // Go-CLI key and re-run/re-record the seed. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-direct-dotdot-segment-")); + const schemasDir = join(dir, "schemas"); + mkdirSync(schemasDir); + writeFileSync(join(schemasDir, "a.sql"), "select 1;"); + return run(["nested/../schemas/*.sql"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["schemas/a.sql"]); + expect(result.warnings).toEqual([]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "normalizes a doubled slash for a direct glob match under a trailing-slash directory component (Go afero.Glob parity)", + () => { + // Same distinction again: `splitPath` on `"schemas//*.sql"` yields a `dir` of + // `"schemas/"` (a single trailing slash survives the split), so the old raw + // `` `${d}/${name}` `` concatenation inserted a SECOND slash on top of it + // (`"schemas//a.sql"`). `filepath.Join`/`path.join` collapse doubled slashes + // regardless of where they came from, so the recorded match must be the + // single-slash `schemas/a.sql`. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-direct-doubled-slash-")); + const schemasDir = join(dir, "schemas"); + mkdirSync(schemasDir); + writeFileSync(join(schemasDir, "a.sql"), "select 1;"); + return run(["schemas//*.sql"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["schemas/a.sql"]); + expect(result.warnings).toEqual([]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "still includes a '.sql' child whose stat fails after it's already listed (Go WalkDir parity)", + () => { + // Go's `fs.WalkDir` types each child from the parent's `ReadDir`-returned `DirEntry` + // and never re-`Stat`s through it, so a `.sql` file that disappears between `ReadDir` + // and its own visit stays in Go's declared file list — only the later, real file-open + // fails. Simulate the stat failure directly (mocking a real race is flaky) by pointing + // the matched directory at one that lists a child but whose child path is unreadable: + // a broken symlink target used as a bare filename via a `readLink` failure isn't + // enough here (that's the earlier symlink test), so exercise the `fs.stat` failure + // path itself by removing the file the instant after `readDirectory` returns it, via + // a `FileSystem` layer that deletes on first `stat` call for that path. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-stat-race-")); + const schemasDir = join(dir, "schemas"); + mkdirSync(schemasDir); + const racyFile = join(schemasDir, "racy.sql"); + writeFileSync(racyFile, "select 1;"); + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const racyFs: FileSystem.FileSystem = { + ...fs, + stat: (p: string) => + p === racyFile + ? Effect.sync(() => rmSync(racyFile)).pipe(Effect.andThen(fs.stat(p))) + : fs.stat(p), + }; + return yield* legacySqlFilesGlob(racyFs, path, ["schemas"], dir); + }).pipe( + Effect.provide(BunServices.layer), + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["schemas/racy.sql"]); + expect(result.warnings).toEqual([]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "fails the whole walk when a non-'.sql' child whose stat fails could have been a subdirectory (Go WalkDir parity)", + () => { + // Round 6 (test above) established that a `.sql` FILE child racing away between + // `readDirectory` and this port's own `stat` call is best-effort included, matching + // Go's cached-`DirEntry` behaviour for regular files. But Go's `fs.WalkDir` does NOT + // treat every vanished child the same way: for a DIRECTORY `DirEntry`, it unconditionally + // attempts a second `ReadDir` to recurse into it; when that child is gone, the second + // `ReadDir` fails, and `walkMatchedDir`'s callback propagates the error unchanged + // (`if err != nil { return err }`, `config.go:198-199`) — `fs.WalkDir` returns it, and + // `walkMatchedDir` wraps it as `failed to walk matched directory: `, discarding + // every file already collected (`config.go:205-206`). Verified empirically with a + // scratch `fs.WalkDir` probe against `apps/cli-go`'s real `walkMatchedDir`: removing a + // nested subdirectory between the parent's `ReadDir` and the subdirectory's own `ReadDir` + // reproduces exactly this — zero files, `failed to walk matched directory: open + // .../nested: no such file or directory` — never a silent skip. This port's `stat` call + // is a second, separate syscall from the parent's `readDirectory` (unlike Go, which gets + // the child's type for free from the SAME syscall as the listing), so a raced + // disappearance here always loses the type along with the entry — a non-`.sql` name + // could equally have been the now-missing subdirectory, and must fail the same way an + // unreadable still-present directory does, not silently vanish along with the files it + // may have held. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-dir-race-")); + const schemasDir = join(dir, "schemas"); + mkdirSync(schemasDir); + writeFileSync(join(schemasDir, "a.sql"), "select 1;"); + const nestedDir = join(schemasDir, "nested"); + mkdirSync(nestedDir); + writeFileSync(join(nestedDir, "b.sql"), "select 2;"); + return Effect.gen(function* () { + const fs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const racyFs: FileSystem.FileSystem = { + ...fs, + stat: (p: string) => + p === nestedDir + ? Effect.sync(() => rmSync(nestedDir, { recursive: true })).pipe( + Effect.andThen(fs.stat(p)), + ) + : fs.stat(p), + }; + return yield* legacySqlFilesGlob(racyFs, path, ["schemas"], dir); + }).pipe( + Effect.provide(BunServices.layer), + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual([]); + expect(result.warnings).toHaveLength(1); + expect(result.warnings[0]).toMatch(/^failed to walk matched directory: /); + // This `stat` failure stands in for the second `ReadDir` Go's own `fs.WalkDir` + // would issue on the vanished directory — whose error, like every other Go + // filesystem error here, embeds the workdir-relative path, not this port's + // absolute stand-in syscall path. + expect(result.warnings[0]).toContain("schemas/nested"); + expect(result.warnings[0]).not.toContain(dir); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect("still expands a real (non-symlinked) nested directory recursively", () => { + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-nested-")); + const schemasDir = join(dir, "schemas"); + const nestedDir = join(schemasDir, "nested"); + mkdirSync(nestedDir, { recursive: true }); + writeFileSync(join(schemasDir, "a.sql"), "select 1;"); + writeFileSync(join(nestedDir, "b.sql"), "select 2;"); + return run(["schemas"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["schemas/a.sql", "schemas/nested/b.sql"]); + expect(result.warnings).toEqual([]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }); + + const isRoot = typeof process.getuid === "function" && process.getuid() === 0; + + it.effect.skipIf(isRoot)( + "surfaces a directory-read failure during walk as a warning instead of an empty match (Go WalkDir parity)", + () => { + // Go's `fs.WalkDir` returns the `ReadDir` error from its walkFn unchanged, which + // stops the walk immediately; `walkMatchedDir` then wraps it as `failed to walk + // matched directory: ...` and discards every file already found — never an empty + // (successful) match. Verified empirically against `apps/cli-go`: an unreadable + // matched directory makes `Glob.SQLFiles` return that error with zero files. + // + // Go's `fsys` here is always `afero.OsFs` with the process cwd already the workdir + // (`ChangeWorkDir`, `cmd/root.go`), so the `ReadDir` error's embedded path is the + // workdir-relative matched directory (`schemas`), never an absolute one. This + // module never `process.chdir`s, so the real read needs an absolute path, but the + // warning must still report the relative form. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-walk-fail-")); + const schemasDir = join(dir, "schemas"); + mkdirSync(schemasDir); + writeFileSync(join(schemasDir, "a.sql"), "select 1;"); + chmodSync(schemasDir, 0o000); + return run(["schemas"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual([]); + expect(result.warnings).toHaveLength(1); + expect(result.warnings[0]).toMatch(/^failed to walk matched directory: /); + expect(result.warnings[0]).toContain("schemas"); + expect(result.warnings[0]).not.toContain(dir); + }), + ), + Effect.ensuring( + Effect.sync(() => { + chmodSync(schemasDir, 0o755); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect.skipIf(isRoot)( + "surfaces a NESTED directory-read failure during walk with a workdir-relative path, not the matched root's (Go WalkDir parity)", + () => { + // Same leak as the matched-root-directory case above, but for a failure during the + // RECURSIVE walk of an already-descended subdirectory — a distinct code path inside + // Go's `fs.WalkDir` callback (it recurses via the SAME `ReadDir` call the matched + // root used, `io/fs/walk.go`), and this port's `walk()` closure recurses the same + // way. The matched root ("schemas") itself is readable; only "schemas/nested" is + // not, so the warning must report "schemas/nested", never the workdir's absolute + // temp-dir path. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-walk-fail-nested-")); + const schemasDir = join(dir, "schemas"); + const nestedDir = join(schemasDir, "nested"); + mkdirSync(nestedDir, { recursive: true }); + writeFileSync(join(schemasDir, "a.sql"), "select 1;"); + writeFileSync(join(nestedDir, "b.sql"), "select 2;"); + chmodSync(nestedDir, 0o000); + return run(["schemas"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual([]); + expect(result.warnings).toHaveLength(1); + expect(result.warnings[0]).toMatch(/^failed to walk matched directory: /); + expect(result.warnings[0]).toContain("schemas/nested"); + expect(result.warnings[0]).not.toContain(dir); + }), + ), + Effect.ensuring( + Effect.sync(() => { + chmodSync(nestedDir, 0o755); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect.skipIf(isRoot)( + "keeps files from a sibling pattern when only one matched directory fails to walk", + () => { + // Go: `if err != nil { allErrors = append(allErrors, err); continue }` — a walk + // failure on one match doesn't stop the loop over the REST of the matches/patterns; + // whether it's ultimately fatal is the caller's decision (`legacyApplySchemaFiles`'s + // `len(declared) == 0` gate), not this function's. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-walk-fail-partial-")); + const goodDir = join(dir, "good"); + const badDir = join(dir, "bad"); + mkdirSync(goodDir); + mkdirSync(badDir); + writeFileSync(join(goodDir, "a.sql"), "select 1;"); + writeFileSync(join(badDir, "b.sql"), "select 2;"); + chmodSync(badDir, 0o000); + return run(["good", "bad"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["good/a.sql"]); + expect(result.warnings).toHaveLength(1); + expect(result.warnings[0]).toMatch(/^failed to walk matched directory: /); + }), + ), + Effect.ensuring( + Effect.sync(() => { + chmodSync(badDir, 0o755); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect.skipIf(isRoot)( + "picks the lexically-first failing subdirectory as the fatal error, matching Go's fs.WalkDir sorted-visit order (review CLI-1958)", + () => { + // Go's `fs.WalkDir` visits directory entries in lexical byte order — its + // `ReadDir` (`os.ReadDir`/`afero.OsFs`) contract guarantees results "sorted by + // filename" before `walkDir` ever iterates them, so when a matched directory + // has MULTIPLE unreadable subdirectories, Go deterministically fails on the + // FIRST one lexically ("aaa" before "bbb") and never even attempts the second. + // This module's own `readDirectory` makes no such ordering promise, so this + // test provides a fake `FileSystem` whose `readDirectory` deliberately returns + // "schemas"'s children in REVERSE order ("bbb" before "aaa") — the opposite of + // Go's guaranteed order — to prove the walk sorts them back (`utf8Compare`) + // before iterating, rather than trusting raw (here: adversarial) enumeration + // order. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-walk-order-")); + const schemasDir = join(dir, "schemas"); + const aaaDir = join(schemasDir, "aaa"); + const bbbDir = join(schemasDir, "bbb"); + mkdirSync(aaaDir, { recursive: true }); + mkdirSync(bbbDir, { recursive: true }); + chmodSync(aaaDir, 0o000); + chmodSync(bbbDir, 0o000); + return Effect.gen(function* () { + const realFs = yield* FileSystem.FileSystem; + const path = yield* Path.Path; + const reorderedFs: FileSystem.FileSystem = { + ...realFs, + readDirectory: (p, opts) => + realFs + .readDirectory(p, opts) + .pipe( + Effect.map((names) => (p === schemasDir ? [...names].sort().reverse() : names)), + ), + }; + const result = yield* legacySqlFilesGlob(reorderedFs, path, ["schemas"], dir); + expect(result.files).toEqual([]); + expect(result.warnings).toHaveLength(1); + // Go descends into "aaa" first (lexical order), fails reading it, and stops — + // "bbb" is never even attempted. + expect(result.warnings[0]).toContain("schemas/aaa"); + expect(result.warnings[0]).not.toContain("schemas/bbb"); + }).pipe( + Effect.provide(BunServices.layer), + Effect.ensuring( + Effect.sync(() => { + chmodSync(aaaDir, 0o755); + chmodSync(bbbDir, 0o755); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); + + it.effect( + "sorts direct wildcard matches by UTF-8 byte order, not UTF-16 code units (Go sort.Strings parity)", + () => { + // Go's `sort.Strings` (`Glob.SQLFiles`, `config.go:155`) orders the raw UTF-8 bytes + // of each match. A supplementary-plane character (here, an emoji — 4-byte UTF-8, + // lead byte 0xF0) always sorts AFTER a 3-byte-encoded BMP character (here, a + // fullwidth exclamation mark — lead byte 0xEF) in Go, because 0xF0 > 0xEF. JS's + // default `Array.prototype.sort()` instead compares UTF-16 code units, under which + // the emoji's surrogate-pair lead unit (0xD83D) sorts BEFORE the fullwidth + // exclamation mark's code unit (0xFF01) — the opposite order. Verified empirically + // against a real Go `sort.Strings` call: it places the fullwidth-exclamation file + // first. + const dir = mkdtempSync(join(tmpdir(), "legacy-sql-glob-utf8-sort-")); + const schemasDir = join(dir, "schemas"); + mkdirSync(schemasDir); + writeFileSync(join(schemasDir, "\u{1F600}.sql"), "select 1;"); // 😀 + writeFileSync(join(schemasDir, "！.sql"), "select 2;"); // ！ + return run(["schemas/*.sql"], dir).pipe( + Effect.tap((result) => + Effect.sync(() => { + expect(result.files).toEqual(["schemas/！.sql", "schemas/\u{1F600}.sql"]); + expect(result.warnings).toEqual([]); + rmSync(dir, { recursive: true, force: true }); + }), + ), + ); + }, + ); +}); diff --git a/apps/cli/src/legacy/shared/legacy-sql-split.ts b/apps/cli/src/legacy/shared/legacy-sql-split.ts index 4eec9072a3..a2205847f0 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-split.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-split.ts @@ -19,7 +19,11 @@ interface State { const BEGIN_ATOMIC = "ATOMIC"; const END_ATOMIC = "END"; -const isIdentifierRune = (rune: string): boolean => /[\p{L}\p{N}_$]/u.test(rune); +// `\p{Nd}` (decimal digits only), not `\p{N}` (all Unicode numbers): Go's +// `unicode.IsDigit` — what `isIdentifierRune`/`TagState.next` port — is an alias for +// category `Nd` alone, so it rejects `No`/`Nl` runes like superscript-2 (`²`) that +// `\p{N}` would wrongly accept as a valid identifier/dollar-tag character. +const isIdentifierRune = (rune: string): boolean => /[\p{L}\p{Nd}_$]/u.test(rune); function isBeginAtomic(data: string): boolean { let offset = data.length - BEGIN_ATOMIC.length; @@ -114,8 +118,9 @@ class TagState implements State { constructor(private readonly offset: number) {} next(rune: string, data: string): State | null { if (rune === "$") return new DollarState(data.slice(this.offset)); - // Valid dollar-tag characters. - if (/[\p{L}\p{N}_]/u.test(rune)) return this; + // Valid dollar-tag characters — see `isIdentifierRune`'s comment on why `\p{Nd}`, + // not `\p{N}`. + if (/[\p{L}\p{Nd}_]/u.test(rune)) return this; return new ReadyState().next(rune, data); } } @@ -144,23 +149,28 @@ class AtomicState implements State { } /** - * Splits `sql` into raw statements (comments/whitespace preserved), then applies - * the optional transforms to each. Mirrors Go's `parser.Split`. + * One raw token from {@link splitRaw}. `terminated` is `false` only for a + * trailing statement emitted at EOF with no closing delimiter (the + * `acc.length > 0` fallback below) — every other token was emitted because the + * FSM itself found a boundary (a bare `;` in `ReadyState`, or `AtomicState` + * closing). Only ever `false` on the LAST element `splitRaw` returns, since + * that fallback fires at most once, after the main loop. */ -export function legacySplitSql( - sql: string, - ...transform: ReadonlyArray<(s: string) => string> -): string[] { +interface RawToken { + readonly text: string; + readonly terminated: boolean; +} + +/** The FSM traversal shared by every `legacySplitSql*` entry point below. */ +function splitRaw(sql: string): RawToken[] { let state: State = new ReadyState(); - const statements: string[] = []; + const tokens: RawToken[] = []; let acc = ""; for (const rune of Array.from(sql)) { acc += rune; const next = state.next(rune, acc); if (next === null) { - let token = acc; - for (const apply of transform) token = apply(token); - if (token.length > 0) statements.push(token); + tokens.push({ text: acc, terminated: true }); acc = ""; state = new ReadyState(); } else { @@ -168,21 +178,85 @@ export function legacySplitSql( } } // Trailing non-terminated statement at EOF. - if (acc.length > 0) { - let token = acc; + if (acc.length > 0) tokens.push({ text: acc, terminated: false }); + return tokens; +} + +/** + * Splits `sql` into raw statements (comments/whitespace preserved), then applies + * the optional transforms to each. Mirrors Go's `parser.Split`. + */ +export function legacySplitSql( + sql: string, + ...transform: ReadonlyArray<(s: string) => string> +): string[] { + const statements: string[] = []; + for (const { text: raw } of splitRaw(sql)) { + let token = raw; for (const apply of transform) token = apply(token); if (token.length > 0) statements.push(token); } return statements; } +/** Go's `parser.SplitAndTrim`'s per-token transform: trim trailing `;` then surrounding whitespace. */ +const legacyTrimStatement = (token: string): string => token.replace(/;+$/u, "").trim(); + /** Mirrors Go's `parser.SplitAndTrim`: trim trailing `;` then surrounding whitespace. */ export function legacySplitAndTrim(sql: string): string[] { - return legacySplitSql( - sql, - (token) => token.replace(/;+$/u, ""), - (token) => token.trim(), - ); + return legacySplitSql(sql, legacyTrimStatement); +} + +/** One statement, paired with both its RAW and trimmed forms. */ +export interface LegacySplitSqlToken { + /** The exact text `legacySplitSql(sql)` (no transforms) would emit for this statement. */ + readonly raw: string; + /** `legacyTrimStatement(raw)` — what `legacySplitAndTrim` emits, including when empty. */ + readonly trimmed: string; + /** + * `false` only for a trailing statement with no closing delimiter, emitted at + * real EOF (`splitRaw`'s `acc.length > 0` fallback) — see {@link RawToken}. + * `checkScannerBufferSize` (`legacy-migration-apply.ts`) needs this to decide + * `>` vs `>=` against the effective buffer limit: Go's `bufio.Scanner` can + * only apply its too-long check (`len(s.buf) >= s.maxTokenSize`) once it has + * given up looking for a delimiter and still needs more data — for a + * delimiter-terminated token the delimiter is found (and the token emitted) + * in the SAME `Scan()` call that fills the buffer to capacity, before that + * check is ever reached, so a token exactly AT the limit still succeeds. An + * unterminated trailing token has no delimiter to find: once the buffer + * fills to the effective limit without one, the too-long check fires + * immediately — Go never gets to attempt the extra `Read()` that would + * reveal real EOF and let the split function emit the trailing token + * instead. Verified empirically against `apps/cli-go/pkg/parser.Split`: a + * single terminated statement of exactly `maxbuf` bytes always succeeds, + * while an unterminated one of exactly `maxbuf` bytes always fails with + * `bufio.ErrTooLong` (one byte under still succeeds; one byte over always + * fails either way). + */ + readonly terminated: boolean; +} + +/** + * Same FSM traversal as {@link legacySplitAndTrim}, but pairs each statement's RAW + * (pre-trim) text with its trimmed form instead of discarding the raw text once + * emitted. Go's `bufio.Scanner`-based `parser.Split` (`pkg/parser/token.go:81-119`) + * enforces `SUPABASE_SCANNER_BUFFER_SIZE` against the untransformed + * `scanner.Text()` — the RAW form — and its `bufio.ErrTooLong` message reports + * that same raw text for the LAST successfully scanned statement, so a caller + * replicating that check (`legacy-migration-apply.ts`'s `execMigrationBatch`) + * needs both forms, not just the trimmed one `legacySplitAndTrim` returns. + * + * Unlike `legacySplitSql`/`legacySplitAndTrim`, this does NOT drop a statement + * whose trimmed form is empty — callers that replicate Go's `len(stats)` counter + * (which only increments for a non-empty trimmed statement) need to see every raw + * token, including the ones `legacySplitAndTrim` itself would filter out. + */ +export function legacySplitSqlTokens(sql: string): ReadonlyArray { + return splitRaw(sql).map(({ text: raw, terminated }) => ({ + raw, + trimmed: legacyTrimStatement(raw), + terminated, + })); } // `(?i)drop\s+` — Go's `dropStatementPattern` (`internal/db/diff/diff.go:100`, diff --git a/apps/cli/src/legacy/shared/legacy-sql-split.unit.test.ts b/apps/cli/src/legacy/shared/legacy-sql-split.unit.test.ts index b04a4793a0..6acf7cc5b1 100644 --- a/apps/cli/src/legacy/shared/legacy-sql-split.unit.test.ts +++ b/apps/cli/src/legacy/shared/legacy-sql-split.unit.test.ts @@ -39,6 +39,17 @@ describe("legacySplitAndTrim", () => { ]); }); + it("treats a non-decimal Unicode digit as an invalid dollar-tag character, like Go's unicode.IsDigit", () => { + // Go's TagState.Next gates on unicode.IsDigit (category Nd only), which is false + // for superscript-2 (U+00B2, category No) — the tag "a²" is therefore invalid, + // Go falls back out of the tag and the embedded `;` becomes a real boundary. + const sql = "CREATE FUNCTION f() AS $a²$foo; bar$a²$ LANGUAGE sql;"; + expect(legacySplitAndTrim(sql)).toEqual([ + "CREATE FUNCTION f() AS $a²$foo", + "bar$a²$ LANGUAGE sql", + ]); + }); + it("respects named dollar tags", () => { const sql = "CREATE FUNCTION f() AS $body$ SELECT ';'; $body$ LANGUAGE sql; SELECT 2;"; expect(legacySplitAndTrim(sql)).toEqual([