-
Notifications
You must be signed in to change notification settings - Fork 337
CS-656 Create script to Migrate all users from old domain to new domain - remove duplicate people records #3304
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
github-actions
wants to merge
29
commits into
main
Choose a base branch
from
chas/email-migration-script
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
29 commits
Select commit
Hold shift + click to select a range
81582b1
fix(api): add merge-duplicate-user task to consolidate users after em…
chasprowebdev df9a837
fix(api): create trigger job to find duplicate members and migrate
chasprowebdev 4bcfe57
fix(api): normalize emails and domains to lowercase before matching i…
chasprowebdev 1f91792
fix(api): update the logger details in merge-duplicate-user trigger job
chasprowebdev 4ae255c
fix(api): add guard against normalized old/new domains in merge-org-e…
chasprowebdev 6a647f3
fix(api): re-point user-level relations before deleting old user in m…
chasprowebdev e06d1a1
fix(api): migrate SOA and ISMS document approver before deleting old …
chasprowebdev bba164e
fix(api): above user deletion inn merge-duplicate-user
chasprowebdev 7f30f83
fix(api): dedupe offboarding and background check records before memb…
chasprowebdev fc47152
fix(api): remove user after merging in merge-duplicate-user
chasprowebdev 39d0d89
fix(api): avoid deleting user record in merge-duplicate-user
chasprowebdev 1298f76
Merge branch 'main' into chas/email-migration-script
tofikwest eec2b5f
Merge branch 'main' into chas/email-migration-script
tofikwest b1faa18
fix(api): skip user-level merge for old users in multiple orgs
chasprowebdev f8e0cc8
fix(api): re-point remaining user-level relations in merge-duplicate-…
chasprowebdev 85c134d
fix(api): correct surviving user/member ids in merge-duplicate-user log
chasprowebdev 5f2825e
fix(api): clean up stale comments in merge-duplicate-user
chasprowebdev 9985da7
Merge branch 'chas/email-migration-script' of https://github.com/tryc…
chasprowebdev ade97fa
Merge branch 'main' of https://github.com/trycompai/comp into chas/em…
chasprowebdev c56dd32
fix(api): remove scripts from .gitignore and make seeded DB test for …
chasprowebdev 95c5779
fix(api): put local scripts to gitignore
chasprowebdev 2a45f2f
fix(api): write jest tests for email domain migration scripts
chasprowebdev 21cad2d
fix(api): keep old user record instead of deleting in merge-duplicate…
chasprowebdev f1dad21
fix(api): scope OffboardingAccessRevocation.revokedById to the multi-…
chasprowebdev 3976d84
fix(api): re-point IsmsObjective.ownerMemberId in merge-duplicate-user
chasprowebdev e071d0d
Merge branch 'main' of https://github.com/trycompai/comp into chas/em…
chasprowebdev 0297b43
fix(api): add inequality guard to merge-duplicate-user
chasprowebdev f134a35
fix(api): recompute org-membership check inside merge-duplicate-user …
chasprowebdev 842eb0a
Merge branch 'main' into chas/email-migration-script
tofikwest File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
162 changes: 162 additions & 0 deletions
162
apps/api/src/scripts/seed-email-domain-migration-test-data.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,162 @@ | ||
| #!/usr/bin/env bun | ||
| /** | ||
| * Seed a small test organization for exercising migrate-org-email-domain | ||
| * (and the merge-duplicate-user task it triggers for each matched pair). | ||
| * | ||
| * Usage: | ||
| * bun run apps/api/src/scripts/seed-email-domain-migration-test-data.ts | ||
| * | ||
| * Idempotent: re-running upserts the same org/users by slug/email instead of | ||
| * creating duplicates, so you can reseed after a merge run to reset state. | ||
| * | ||
| * Creates one org with member/email pairs covering: | ||
| * - alice: a clean pair with no other data attached | ||
| * - bob: a pair where the old member has a task + comment to re-point | ||
| * - carol: old-domain only, no new-domain counterpart (should be skipped) | ||
| * - dave: new-domain only, no old-domain counterpart (unaffected) | ||
| * - frank: old-domain email in a different case (tests normalization) | ||
| * - erin: old user is also a member of a second org (tests the | ||
| * merge-duplicate-user guard that keeps the User row when it | ||
| * belongs to more than one org) | ||
| */ | ||
|
|
||
| import { db } from '@db'; | ||
|
|
||
| const ORG_SLUG = 'email-migration-test-org'; | ||
| const OTHER_ORG_SLUG = 'email-migration-test-other-org'; | ||
| const OLD_DOMAIN = 'oldcorp.test'; | ||
| const NEW_DOMAIN = 'newcorp.test'; | ||
|
|
||
| async function upsertOrg(slug: string, name: string) { | ||
| return db.organization.upsert({ | ||
| where: { slug }, | ||
| create: { slug, name, onboardingCompleted: true, hasAccess: true }, | ||
| update: { name }, | ||
| }); | ||
| } | ||
|
|
||
| async function upsertUser(email: string, name: string) { | ||
| return db.user.upsert({ | ||
| where: { email }, | ||
| create: { email, name, emailVerified: true }, | ||
| update: { name }, | ||
| }); | ||
| } | ||
|
|
||
| async function upsertMember({ | ||
| organizationId, | ||
| userId, | ||
| role = 'employee', | ||
| }: { | ||
| organizationId: string; | ||
| userId: string; | ||
| role?: string; | ||
| }) { | ||
| const existing = await db.member.findFirst({ | ||
| where: { organizationId, userId }, | ||
| }); | ||
| if (existing) return existing; | ||
| return db.member.create({ data: { organizationId, userId, role } }); | ||
| } | ||
|
|
||
| async function main() { | ||
| console.log('Seeding email-domain-migration test data...\n'); | ||
|
|
||
| const org = await upsertOrg(ORG_SLUG, 'Email Migration Test Org'); | ||
| const otherOrg = await upsertOrg( | ||
| OTHER_ORG_SLUG, | ||
| 'Email Migration Test Other Org', | ||
| ); | ||
|
|
||
| // 1. Clean pair — merges with no other data attached. | ||
| const alice = { | ||
| old: await upsertUser(`alice@${OLD_DOMAIN}`, 'Alice Old'), | ||
| new: await upsertUser(`alice@${NEW_DOMAIN}`, 'Alice New'), | ||
| }; | ||
| await upsertMember({ organizationId: org.id, userId: alice.old.id }); | ||
| await upsertMember({ organizationId: org.id, userId: alice.new.id }); | ||
|
|
||
| // 2. Pair where the old member owns real data that should get re-pointed. | ||
| const bob = { | ||
| old: await upsertUser(`bob@${OLD_DOMAIN}`, 'Bob Old'), | ||
| new: await upsertUser(`bob@${NEW_DOMAIN}`, 'Bob New'), | ||
| }; | ||
| const bobOldMember = await upsertMember({ | ||
| organizationId: org.id, | ||
| userId: bob.old.id, | ||
| }); | ||
| await upsertMember({ organizationId: org.id, userId: bob.new.id }); | ||
|
|
||
| const bobTask = await db.task.create({ | ||
|
chasprowebdev marked this conversation as resolved.
|
||
| data: { | ||
| organizationId: org.id, | ||
| title: 'Review vendor security questionnaire', | ||
| description: | ||
| 'Seeded task assigned to the old member — should re-point to the new member after merge.', | ||
| assigneeId: bobOldMember.id, | ||
| }, | ||
| }); | ||
| await db.comment.create({ | ||
| data: { | ||
| organizationId: org.id, | ||
| authorId: bobOldMember.id, | ||
| entityId: bobTask.id, | ||
| entityType: 'task', | ||
| content: 'Seeded comment authored by the old member.', | ||
| }, | ||
| }); | ||
|
|
||
| // 3. Old-domain only — no new-domain counterpart, so the migration must skip it. | ||
| const carolOld = await upsertUser(`carol@${OLD_DOMAIN}`, 'Carol NoMatch'); | ||
| await upsertMember({ organizationId: org.id, userId: carolOld.id }); | ||
|
|
||
| // 4. New-domain only — already migrated, unaffected either way. | ||
| const daveNew = await upsertUser(`dave@${NEW_DOMAIN}`, 'Dave New'); | ||
| await upsertMember({ organizationId: org.id, userId: daveNew.id }); | ||
|
|
||
| // 5. Case-insensitive match: uppercase old-domain email vs. lowercase new-domain email. | ||
| const frank = { | ||
| old: await upsertUser(`FRANK@${OLD_DOMAIN.toUpperCase()}`, 'Frank Old'), | ||
| new: await upsertUser(`frank@${NEW_DOMAIN}`, 'Frank New'), | ||
| }; | ||
| await upsertMember({ organizationId: org.id, userId: frank.old.id }); | ||
| await upsertMember({ organizationId: org.id, userId: frank.new.id }); | ||
|
|
||
| // 6. Old user also belongs to a second org — the User row must survive the merge. | ||
| const erin = { | ||
| old: await upsertUser(`erin@${OLD_DOMAIN}`, 'Erin Old'), | ||
| new: await upsertUser(`erin@${NEW_DOMAIN}`, 'Erin New'), | ||
| }; | ||
| await upsertMember({ organizationId: org.id, userId: erin.old.id }); | ||
| await upsertMember({ organizationId: org.id, userId: erin.new.id }); | ||
| await upsertMember({ organizationId: otherOrg.id, userId: erin.old.id }); | ||
|
|
||
| console.log('Seed complete.\n'); | ||
| console.log(`organizationId: ${org.id}`); | ||
| console.log(`otherOrganizationId: ${otherOrg.id} (holds erin's other membership)`); | ||
| console.log(`oldDomain: ${OLD_DOMAIN}`); | ||
| console.log(`newDomain: ${NEW_DOMAIN}`); | ||
|
|
||
| console.log('\nExpected to merge:'); | ||
| console.log(` alice@${OLD_DOMAIN} -> alice@${NEW_DOMAIN} (clean merge)`); | ||
| console.log(` bob@${OLD_DOMAIN} -> bob@${NEW_DOMAIN} (task + comment re-pointed)`); | ||
| console.log(` frank@${OLD_DOMAIN.toLowerCase()} -> frank@${NEW_DOMAIN} (case-insensitive match)`); | ||
| console.log(` erin@${OLD_DOMAIN} -> erin@${NEW_DOMAIN} (old user's account survives — still in ${OTHER_ORG_SLUG})`); | ||
|
|
||
| console.log('\nExpected to skip:'); | ||
| console.log(` carol@${OLD_DOMAIN} (no new-domain counterpart)`); | ||
| console.log(` dave@${NEW_DOMAIN} (no old-domain counterpart)`); | ||
|
|
||
| console.log('\nTrigger the migration with:'); | ||
| console.log( | ||
| ` migrateOrgEmailDomain.trigger({ organizationId: '${org.id}', oldDomain: '${OLD_DOMAIN}', newDomain: '${NEW_DOMAIN}' })`, | ||
| ); | ||
|
|
||
| await db.$disconnect(); | ||
| } | ||
|
|
||
| main().catch(async (error) => { | ||
| console.error('Seeding failed:', error); | ||
| await db.$disconnect(); | ||
| process.exit(1); | ||
| }); | ||
56 changes: 56 additions & 0 deletions
56
apps/api/src/trigger/tasks/people/merge-duplicate-user-db-mock.util.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| export type ModelMock = Record<string, jest.Mock>; | ||
|
|
||
| const MODEL_METHOD_DEFAULTS: Record<string, unknown> = { | ||
| findMany: [], | ||
| findUnique: null, | ||
| findFirst: null, | ||
| updateMany: { count: 0 }, | ||
| deleteMany: { count: 0 }, | ||
| count: 0, | ||
| }; | ||
|
|
||
| function createModelMock(): ModelMock { | ||
| const methods: ModelMock = {}; | ||
| return new Proxy(methods, { | ||
| get: (target, prop) => { | ||
| if (typeof prop !== 'string') return undefined; | ||
| if (!target[prop]) { | ||
| const hasDefault = Object.prototype.hasOwnProperty.call( | ||
| MODEL_METHOD_DEFAULTS, | ||
| prop, | ||
| ); | ||
| target[prop] = jest | ||
| .fn() | ||
| .mockResolvedValue(hasDefault ? MODEL_METHOD_DEFAULTS[prop] : {}); | ||
| } | ||
| return target[prop]; | ||
| }, | ||
| }); | ||
| } | ||
|
|
||
| /** | ||
| * Builds a `db`-shaped Proxy for tests: any model/method is auto-mocked on | ||
| * first access with a harmless default (empty lists / zero counts) unless a | ||
| * test overrides it. `$transaction` invokes its callback with the same | ||
| * proxy as `tx`, so assertions against the returned object see calls made | ||
| * inside a transaction too. Used from a `jest.mock('@db', ...)` factory via | ||
| * `require`, since factories can't close over module-scope variables. | ||
| */ | ||
| export function createDbProxyMock(): Record<string, unknown> { | ||
| const models: Record<string, ModelMock> = {}; | ||
| let dbProxy: Record<string, unknown>; | ||
| const dbMock = { | ||
| $transaction: jest.fn( | ||
| async (fn: (tx: unknown) => Promise<unknown>) => fn(dbProxy), | ||
| ), | ||
| }; | ||
| dbProxy = new Proxy(dbMock, { | ||
| get: (target, prop) => { | ||
| if (typeof prop !== 'string') return undefined; | ||
| if (prop in target) return target[prop as keyof typeof target]; | ||
| if (!models[prop]) models[prop] = createModelMock(); | ||
| return models[prop]; | ||
| }, | ||
| }); | ||
| return dbProxy; | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.