Add overwrite and skip options to Appwrite destination#168
Add overwrite and skip options to Appwrite destination#168premtsd-code wants to merge 6 commits intomainfrom
Conversation
- Add setOverwrite() / setSkip() methods to Appwrite destination - When overwrite is true, use upsertDocuments to update existing rows - When skip is true, pass ignore: true to createDocuments to silently skip duplicates - Point utopia-php/database to csv-import-upsert branch for ignore support
Greptile SummaryThis PR adds Confidence Score: 4/5Safe to merge once the upstream All P1 concerns from the prior review thread are resolved. The one remaining new finding is P2 (unnecessary
Important Files Changed
Reviews (5): Last reviewed commit: "Use skipDuplicates() scope guard instead..." | Re-trigger Greptile |
|
Tip: Greploops — Automatically fix all review issues by running Use the Greptile plugin for Claude Code to query reviews, search comments, and manage custom context directly from your terminal. |
| if ($this->overwrite) { | ||
| $dbForDatabases->skipRelationshipsExistCheck(fn () => $dbForDatabases->upsertDocuments( | ||
| $collectionName, | ||
| $this->rowBuffer | ||
| )); | ||
| } else { | ||
| $dbForDatabases->skipRelationshipsExistCheck(fn () => $dbForDatabases->createDocuments( | ||
| $collectionName, | ||
| $this->rowBuffer, | ||
| ignore: $this->skip | ||
| )); |
There was a problem hiding this comment.
Successful upserts not reflected in resource status
When overwrite is true, upsertDocuments is called but the return value is silently discarded and the method unconditionally returns true (line 1106). If the upsert fails for any reason (other than throwing an exception), the caller has no way to know — and even in the success case, there is no cache update after a successful upsert. A successfully upserted row won't be added to the cache, so a re-run would attempt to upsert it again instead of treating it as already migrated.
The same gap exists for the skip path: rows silently skipped at the database level (because ignore: true suppresses the duplicate error) are returned as true — indistinguishable from a genuine insert — meaning the status report will show them as successfully migrated when they were actually dropped.
There was a problem hiding this comment.
Cache is already updated at the import() level (line 368: $this->cache->update($responseResource)) for every resource regardless of the code path in createRecord(), so re-runs will see the row as cached.
For the skip path: returning true is the intended behavior — skip means "do not fail on duplicates, keep going." The migration completed successfully from the caller's perspective.
The bool $ignore parameter on Database::createDocuments() was replaced with a skipDuplicates(callable, bool) scope guard in utopia-php/database csv-import-upsert-v2. The previous named-argument call would throw "Unknown named argument" at runtime against the current database branch. Wrap createDocuments in skipDuplicates() so the orchestrator picks up the flag through its instance state instead of via parameter.
Summary
setOverwrite()andsetSkip()methods to the Appwrite destinationoverwriteis true, usesupsertDocuments()to update existing rows with matching IDsskipis true, passesignore: truetocreateDocuments()to silently skip duplicatesutopia-php/databasetocsv-import-upsertbranch which adds theignoreparameter tocreateDocumentsDepends on utopia-php/database#850
Test plan
skip: true— duplicate rows silently skippedoverwrite: true— existing rows updated