Skip to content

Commit e339a38

Browse files
committed
fix(db): reconcile folders stranded by old pods during the cutover drain
0272 and 0274 both ran while the previous app version was still serving. During the blue/green window the old pods write the legacy folder tables exclusively while the new pods write `folder` exclusively, so any folder an old pod creates between the migration and its own termination exists only in the legacy table and is invisible to the new code. 0274 asked to be re-run once the deploy drained; a journaled migration never replays, so the re-run needs its own file. Covers both trees, where 0274 covered only `file`. The workflow side is the higher-volume one and is where the stranded row actually landed in production. Insert-only for ids missing from `folder`, never a mirror upsert: 0272's dedup renamed 47 workflow folders, and an upsert would revert all 47 to their colliding names and violate the active-unique index doing it. Names are deduplicated against the first free suffix rather than a row number, since the target name may already be held by a row this block is not inserting. Only active rows dedupe — the partial index ignores soft-deleted rows, so an archived folder keeps its name and reappears under it if restored. Rows insert in depth order so a parent always precedes its children, which the self-referencing FK and the parent-resource-type trigger both require. Wrapped in a DO block for the same reason 0274 is: 0272 ends in an embedded COMMIT, so files after it are not guaranteed to run inside drizzle's batch transaction. Verified by executing the file against Postgres 17 with adversarial fixtures: a 0272-renamed row is left untouched, a stranded active row whose name and whose "(2)" are both taken lands on "(3)", a stranded archived row keeps its colliding name, a fully-stranded three-level chain inserts parents first, sibling dedup scopes to the parent, and cross-workspace and cross-resource-type names do not interfere. Idempotent across three consecutive runs with byte-identical state. Production at 06:52Z has 1 row to insert (archived, empty, no parent) on the workflow side and 0 on the file side.
1 parent a3413cf commit e339a38

2 files changed

Lines changed: 143 additions & 0 deletions

File tree

Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
-- Post-drain reconcile for the generic-folder cutover, covering BOTH legacy trees.
2+
--
3+
-- 0272 mirrored `workflow_folder` and `workspace_file_folders` into `folder`, and 0274
4+
-- reconciled the file tree. Both ran while the previous app version was still serving. During
5+
-- the blue/green rolling window the old pods keep writing the LEGACY tables exclusively while
6+
-- the new pods write `folder` exclusively, so every folder an old pod created between the
7+
-- migration and its own termination exists only in the legacy table and is invisible to the
8+
-- new code. 0274 anticipated this and asked to be run again once the deploy drained; a
9+
-- journaled migration never replays, so that re-run has to be its own file. This is it.
10+
--
11+
-- Production measurement at 2026-07-29T06:40Z, mid-drain: exactly 1 stranded row
12+
-- (`workflow_folder`, archived, no parent, no workflows inside) and 0 on the file side, with 0
13+
-- name collisions and 0 absent parents on both trees. The count at execution time may be
14+
-- higher, since old pods were still serving when that was taken.
15+
--
16+
-- INSERT-ONLY, NOT A MIRROR. 0274 could upsert every column because file folders can never
17+
-- diverge in name — `workspace_file_folders` enforces the same active-unique key `folder`
18+
-- does. That does NOT hold here: 0272's dedup renamed 47 workflow folders across 35 collision
19+
-- groups, so `folder` is deliberately different from `workflow_folder` for those rows. An
20+
-- upsert would revert all 47 to their colliding names and violate
21+
-- `folder_workspace_resource_parent_name_active_unique` in the process. Only ids absent from
22+
-- `folder` are touched; every row already mirrored is left exactly as it is.
23+
--
24+
-- Names are deduplicated against the FIRST FREE suffix rather than a row number, because the
25+
-- target name may already be taken by a row this block is not inserting. Only ACTIVE rows are
26+
-- deduplicated — the partial unique index ignores soft-deleted rows, so an archived folder
27+
-- keeps its original name and reappears under it if restored.
28+
--
29+
-- Rows are inserted in depth order so a parent always lands before its children: the
30+
-- self-referencing FK `folder_parent_id_folder_id_fk` is checked immediately, and the
31+
-- `folder_parent_resource_type_match` trigger reads the parent row. A child inserted before
32+
-- its parent would fail both.
33+
--
34+
-- RUN THIS BEFORE THE SOFT-DELETE CLEANUP JOB PURGES EXPIRED `folder` ROWS. Cleanup
35+
-- hard-deletes from `folder` but never from the legacy tables, so a run that lands after a
36+
-- purge reinstates every purged folder as a soft-deleted phantom in Recently Deleted whose
37+
-- files and workflows are already gone. The retention window is far longer than any drain, so
38+
-- running promptly after the deploy is sufficient — but this is the one ordering constraint
39+
-- that makes the file dangerous rather than merely redundant if deferred indefinitely.
40+
--
41+
-- Idempotent and a clean no-op once reconciled: the driving queries select only ids missing
42+
-- from `folder`, so a second execution finds nothing and the `ON CONFLICT (id) DO NOTHING`
43+
-- guards against a concurrent writer inserting the same id between the SELECT and the INSERT.
44+
45+
-- Wrapped in a DO block for the same reason 0274 is: 0272 ends with an embedded COMMIT for its
46+
-- trailing CONCURRENTLY index builds, so files after it are not guaranteed to run inside
47+
-- drizzle's batch transaction. Both trees must land together or not at all.
48+
DO $$
49+
DECLARE
50+
rec RECORD;
51+
candidate TEXT;
52+
suffix INT;
53+
BEGIN
54+
-- Workflow tree. `archived_at` is this table's soft-delete column; `folder` calls it
55+
-- `deleted_at`. Depth is computed over the SOURCE table so a stranded child of a stranded
56+
-- parent still orders correctly.
57+
FOR rec IN
58+
WITH RECURSIVE tree AS (
59+
SELECT s.id, 0 AS depth
60+
FROM "workflow_folder" s
61+
WHERE s."parent_id" IS NULL
62+
UNION ALL
63+
SELECT s.id, t.depth + 1
64+
FROM "workflow_folder" s
65+
JOIN tree t ON t.id = s."parent_id"
66+
)
67+
SELECT s."id", s."name", s."user_id", s."workspace_id", s."parent_id",
68+
s."sort_order", s."created_at", s."updated_at", s."archived_at" AS deleted_at
69+
FROM "workflow_folder" s
70+
JOIN tree t ON t.id = s."id"
71+
WHERE NOT EXISTS (SELECT 1 FROM "folder" f WHERE f."id" = s."id")
72+
ORDER BY t.depth, s."created_at"
73+
LOOP
74+
candidate := rec."name";
75+
IF rec.deleted_at IS NULL THEN
76+
suffix := 1;
77+
WHILE EXISTS (
78+
SELECT 1 FROM "folder" f
79+
WHERE f."workspace_id" = rec."workspace_id"
80+
AND f."resource_type" = 'workflow'
81+
AND coalesce(f."parent_id", '') = coalesce(rec."parent_id", '')
82+
AND f."name" = candidate
83+
AND f."deleted_at" IS NULL
84+
) LOOP
85+
suffix := suffix + 1;
86+
candidate := rec."name" || ' (' || suffix || ')';
87+
END LOOP;
88+
END IF;
89+
90+
INSERT INTO "folder" (id, resource_type, name, user_id, workspace_id, parent_id, locked, sort_order, created_at, updated_at, deleted_at)
91+
VALUES (rec."id", 'workflow', candidate, rec."user_id", rec."workspace_id", rec."parent_id",
92+
false, rec."sort_order", rec."created_at", rec."updated_at", rec.deleted_at)
93+
ON CONFLICT (id) DO NOTHING;
94+
END LOOP;
95+
96+
-- File tree. Same shape; this table already uses `deleted_at`. 0274 reconciled it fully, so
97+
-- this pass only picks up what old pods wrote after 0274 ran.
98+
FOR rec IN
99+
WITH RECURSIVE tree AS (
100+
SELECT s.id, 0 AS depth
101+
FROM "workspace_file_folders" s
102+
WHERE s."parent_id" IS NULL
103+
UNION ALL
104+
SELECT s.id, t.depth + 1
105+
FROM "workspace_file_folders" s
106+
JOIN tree t ON t.id = s."parent_id"
107+
)
108+
SELECT s."id", s."name", s."user_id", s."workspace_id", s."parent_id",
109+
s."sort_order", s."created_at", s."updated_at", s."deleted_at"
110+
FROM "workspace_file_folders" s
111+
JOIN tree t ON t.id = s."id"
112+
WHERE NOT EXISTS (SELECT 1 FROM "folder" f WHERE f."id" = s."id")
113+
ORDER BY t.depth, s."created_at"
114+
LOOP
115+
candidate := rec."name";
116+
IF rec."deleted_at" IS NULL THEN
117+
suffix := 1;
118+
WHILE EXISTS (
119+
SELECT 1 FROM "folder" f
120+
WHERE f."workspace_id" = rec."workspace_id"
121+
AND f."resource_type" = 'file'
122+
AND coalesce(f."parent_id", '') = coalesce(rec."parent_id", '')
123+
AND f."name" = candidate
124+
AND f."deleted_at" IS NULL
125+
) LOOP
126+
suffix := suffix + 1;
127+
candidate := rec."name" || ' (' || suffix || ')';
128+
END LOOP;
129+
END IF;
130+
131+
INSERT INTO "folder" (id, resource_type, name, user_id, workspace_id, parent_id, locked, sort_order, created_at, updated_at, deleted_at)
132+
VALUES (rec."id", 'file', candidate, rec."user_id", rec."workspace_id", rec."parent_id",
133+
false, rec."sort_order", rec."created_at", rec."updated_at", rec."deleted_at")
134+
ON CONFLICT (id) DO NOTHING;
135+
END LOOP;
136+
END $$;

packages/db/migrations/meta/_journal.json

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1919,6 +1919,13 @@
19191919
"when": 1785281897202,
19201920
"tag": "0274_file_folder_cutover_reconcile",
19211921
"breakpoints": true
1922+
},
1923+
{
1924+
"idx": 275,
1925+
"version": "7",
1926+
"when": 1785281897203,
1927+
"tag": "0275_folder_cutover_post_drain_reconcile",
1928+
"breakpoints": true
19221929
}
19231930
]
19241931
}

0 commit comments

Comments
 (0)