feat(webv2): export and import projects as .invk archives - #55
Open
joshistoast wants to merge 26 commits into
Open
feat(webv2): export and import projects as .invk archives#55joshistoast wants to merge 26 commits into
joshistoast wants to merge 26 commits into
Conversation
showPromptSyntaxHighlighting and preferNumericAttentionStyle describe how a person edits, not what a project is, so they no longer travel inside the project document. Drops the Settings row duplicating the Preview header's progress-image toggle.
A project document references its pixels by server image name, so the old single-JSON export opened perfectly on the machine that wrote it and showed nothing but missing layers anywhere else. Projects now export to the ZIP container the previous frontend already writes for canvas projects, at version 2: the same manifest skeleton and images/<image_name> convention, carrying project.json and a cover instead of the four canvas state files. Import restores only the images this server is missing, remaps every reference the server renamed, and always mints a fresh project id. A v1 canvas archive is recognized and refused by name rather than by schema dump. Covers now populate the library grid from a per-user index.
Four project-file modules and the shared concurrency helper enter both routes' initial graphs; all are pinned to route-shared so request counts are unchanged.
The page had two competing axes: heading, description and toolbar flush left, and a centered empty state under them sharing an edge with nothing. Empty states now align to start, on the same column as everything else. The toolbar spans the measure — search on the left edge, sort and layout on the right, under the header's buttons — so the top of the page reads as two vertical lines rather than one line and a drifting cluster. The layout choice becomes a SegmentGroup: two adjacent icon buttons read as two actions, one of which is somehow lit. The empty state's dashed tile is gone. It was a grid cell with no grid around it, at an arbitrary width, shown even in list view, repeating the call to action the header already carries. A button replaces it, outline rather than solid so the page keeps one accent.
Drops NewProjectCard from every route's source-owner set. No byte or request budget moved.
joshistoast
requested review from
JPPhoto,
Pfannkuchensack,
blessedcoolant and
lstein
as code owners
August 5, 2026 02:13
A document can reference a video today — `VideoField { video_name }` reaches
projectGraph through an imported workflow whose node value was authored
elsewhere — and the exporter dropped it silently. The collector now returns a
set per kind, the archive files bytes under images/ or videos/, and import
restores each through its own endpoints with a rename mapping per kind, since
the two namespaces are separate and a flat map would let one rewrite the other.
Filing by folder rather than pooling under one is what lets import know an
entry's kind without guessing it from a file extension. No manifest version
bump: the reader already ignores entries it does not recognize.
Also makes the gallery-selection exclusion deliberate. Those keys are pointers
into per-install gallery content, and were skipped only because GalleryItem
spells its name field 'name' — an accident that a rename would have quietly
undone.
compareImage is a GeneratedImageContract, so unlike the rest of the gallery selection it carries imageName and was genuinely being bundled. A comparison someone happened to leave open is a pointer into per-install gallery content, not part of the project.
The cover index is one KV blob and every write replaces it, so recording a cover from a store that had not loaded deleted every entry but that one. Autosave reaches recordProjectCover from the editor, which loads the index only when the project switcher or Open dialog opens, so anyone who reloaded straight into a project and generated hit exactly that ordering. Records made before the load are now held and merged onto the server's answer when it arrives. A failed read is no longer treated as an empty index: the store stays unloaded and the pending records wait, because merging onto a blank answer is the same destructive write by another route. Eviction at the 500-project cap now re-inserts the project being written, so it can never discard the very record it was asked to make. Drops the unused ensureProjectCoversLoaded export and puts the previously write-only isLoaded field to work as the guard.
unzip is fully buffered: by the time it returns a record of entries, every one has already been inflated into memory. The size and entry-count totals were accumulated over that result, which describes an out-of-memory crash rather than preventing one. The budget now runs inside fflate's entry filter, consulted per entry with the uncompressed size declared in the central directory, and refuses everything from the crossing point on. The refusal is raised after unzip settles rather than thrown from inside its walk, so too-large stays distinguishable from not-a-project. Lowers the ceiling from 4 GiB to 2 GiB. 4 GiB is exactly where ZIP32's 32-bit sizes and offsets stop being able to describe the file, and fflate reads zip64 records but never writes them, so an archive at that boundary would be produced structurally invalid rather than refused. Also bounds the write direction by entry count, which only the read direction had.
An unservable asset is skipped, which is right — half a project's pixels beats none. But an aborted signal makes every asset unservable at once, so a cancelled export skipped all of them, packed an archive of nothing and handed it to the browser as a finished download, then threw. isRequestCancellation tells the two apart where the request fails, and the signal is checked again before packing. Excluding the gallery selection from collection was only half of leaving it behind: a reference that is not bundled still travels, it just travels broken, and because import collects with the same skip list the restore pass could not even report it as dangling. stripGallerySelection now removes those keys in the planner, which is where what belongs in a project file is decided. Also gives mimeForEntryName a per-kind fallback, so a video entry with an unrecognised extension is no longer announced to the video endpoint as image/png.
…port The archive's cover entry is a thumbnail of an image the document also references, and the cover URL asks for a thumbnail anyway — so once that image is restored, pointing the index at it is the same picture without a second copy. Uploading the entry unconditionally left one orphan per import: covers go up under the canvas's private 'other' category, which appears in no gallery view and no board count, so nobody could find or delete them. The bundled bytes remain the fallback for a cover whose source image is dangling here, which is the case that made them worth carrying. Restoring the assets now happens before the cover is resolved, which also takes the cover upload off the front of the queue where it blocked the fan-out. Passes the entry's kind to mimeForEntryName so a video is announced as one.
A project file is the one operation here whose duration is set by how much someone has drawn: a few hundred full-resolution layers is a few hundred round trips and hundreds of megabytes. Every entry point started one and then said nothing until it finished or threw, and a button that appears to do nothing for two minutes gets pressed again. Progress is now reported throughout, as one live toast that updates in place rather than a line per phase. Both directions could also half-succeed: export skips assets the server will not serve, import leaves references dangling. Both were computed, sorted and then discarded, so a project that shed forty layers was indistinguishable from a clean round trip. They are returned now, and a lossy run finishes as a warning naming the count while only a clean one finishes as a success. Five surfaces had duplicated the same twenty lines of scope capture, picker, try/catch and toast, and had already drifted — the import sites checked the account scope before showing an error and the export sites did not, so cancelling an export by signing out produced a toast written for a developer. useProjectFileActions owns that sequence once.
The tooltip sits on the icon rather than the segment item for a good reason: Tooltip.Trigger is asChild and merges its own data-state onto whatever it clones, which on the item overwrites data-state="checked" — the selected segment then styles as unselected and the indicator measures 0x0. But an icon is an svg and cannot take focus, so the label had become hover-only; it used to sit on an IconButton, which was focusable. The hidden radio is the focusable control here, so it now drives the tooltip open alongside pointer hover. Screen readers were never affected — the radio carries the accessible name either way. Adds a browser test pinning both halves at once, since fixing either one the obvious way breaks the other.
Adds projectFileToasts and useProjectFileActions to both routes' source-owner sets — both routes offer Import and Export, so both need the reporter and the hooks that drive it. Launchpad owned JavaScript grows 62 bytes; editor owned JavaScript falls 161, since the export and import handlers moved out of the page components into the shared modules. Request counts are unchanged.
Records why the read budget runs inside fflate's entry filter rather than over its result, why the archive ceiling sits well under ZIP32's 4 GiB, that cancellation is not a skip, that excluding the gallery selection takes both a collector skip and a strip, how the cover index survives a write that precedes its first read, and why an imported cover reuses a restored image. Corrects restoreArchiveImages to restoreArchiveAssets.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A project document references its pixels by server image name, never by URL or inline data. The old export — a single
<name>.invokeproject.jsonwrapping the document — therefore opened perfectly on the machine that wrote it and showed missing layers elsewhere.Projects now export to
.invk, the ZIP container the previous frontend already writes for canvas projects, at version 2. Existing.invokeproject.jsonfiles remain importable for backward compatibility.The format
.invkv1 (canvas projects,invokeai/frontend/web) is kept verbatim as a container: same extension, same rootmanifest.jsonshape, sameimages/<image_name>convention. Version 2 changes only the payload.Where a v1 archive carried
canvas_state.json/params.json/ref_images.json/loras.json, a v2 archive carries oneproject.json— the webv2 equivalents all live inside it.Assets are filed by kind rather than pooled under one folder. Images and videos are separate backend namespaces with separate fetch and upload routes and no shared name space, so import has to know which an entry is before it can restore it.
Manifests parse through a zod discriminated union on
version, so a v1 archive is recognized and refused as a legacy canvas project rather than as a schema dump. The legacy canvas reader remains unchanged and continues to reject v2.What gets bundled
collectLiveAssetRefs()walks the document collecting every string underimageName/image_nameas an image and undervideo_nameas a video. Collecting by key rather than by path allows new control-adapter kinds and node fields to be picked up without updating a path list.Videos are included because imported workflows can already carry
VideoField { video_name }values even though webv2 cannot author that field yet.Gallery selections are excluded, of either kind.
selectedImage,selectedImageName,selectedImageNames, andcompareImagepoint into per-install gallery content rather than at anything the document renders.History is excluded. The walker skips the
queue,graphHistory, andeventsroots and thesnapshot/snapshots/recentImageskeys at any depth. Queue entries and recent-result lists can embed many canvas snapshots, so bundling them can inflate otherwise normal projects dramatically.Remapping still walks the whole canonical document. If a bundled asset returns under a new server name, every reference follows, including history references that were not bundled.
Import
.invkimport is: read → assert the current account → mint a fresh identity and name → validate and canonicalize → restore missing assets → remap references → create the project → update the local summary and cover index.Validation happens before any asset lookup, upload, or project mutation. A damaged document therefore causes zero backend mutations. Legacy
.invokeproject.jsonimports take the same fresh-identity and validation/canonicalization path but have no bundled assets to restore.Images use the bulk
images_by_nameslookup. Videos have no bulk equivalent and fan outGET /videos/i/{name}behind a concurrency limit of five. Video403and404responses mean the asset is absent and should be restored; any other non-success response aborts import. Rename mappings stay separate by media kind.Import never overwrites a project. Re-importing an archive always mints a new project id, while existing matching assets are reused without upload.
Restored assets use category
otherandis_intermediate: false. If project creation fails after uploads, the importer performs best-effort rollback only when the same account still owns the operation and a follow-up lookup confirms the project was not created; ambiguous creation and account rotation deliberately avoid destructive deletion.A referenced asset that is neither on the server nor in the archive is reported as dangling and its reference is left unchanged, so the project opens with a broken reference rather than failing entirely.
Covers
ProjectSummary.coverUrlis populated from aprojectId → coverImageNameindex in the existing client-state KV. The library can render thumbnails without fetching every complete project document.Cover-index whole-blob writes are serialized per account. Failed writes remain dirty and can be retried by an identical later mutation; clears made before the first server read are retained. A successful load is final for the current account epoch, preventing a later stale read from overwriting an optimistic mutation.
The index is not a source of truth: older projects and projects without output show the folder glyph.
Safety and cancellation
INVK_MAX_ARCHIVE_BYTESis a fixed 2 GiB ceiling andINVK_MAX_ENTRIESremains 20,000.File.sizeis rejected beforearrayBuffer()or legacytext()allocation.Blobconstruction.Also in this PR
Prompt-editing toggles move to user preferences.
showPromptSyntaxHighlightingandpreferNumericAttentionStyledescribe how a person edits rather than project content. The duplicate Settings row for the Preview progress-image toggle is removed while the preference remains available to the Preview header.The projects page is aligned to one grid. Empty states align to the page measure, the toolbar spans it, and the layout choice is represented as a segmented group. Keyboard access to the layout toggle is preserved.
Active-project export flushes registered debounced Generate drafts before reading the project, so an export made immediately after typing includes the latest prompt.
Notes
ROUTE_SHARED_MODULESinvite.config.mtsincludes the cross-route project-file modules needed to keep the launchpad/editor ownership split stable.fflateis dynamically imported insideinvk/archive.tsand emitted as its own chunk, so it does not enter an initial route graph.Verification
pnpm run check:releasepasses, including formatting, Oxc, TypeScript, architecture policy tests and inventories, 408 node test files / 5,369 tests, the browser and fixture suites, architecture performance checks, the project-file browser journey, and all 12 accessibility journeys.Focused tests cover real fflate round trips and malformed archives, allocation boundaries, validation-before-mutation, legacy JSON compatibility, asset lookup statuses, streaming export limits, post-pack cancellation, upload rollback, cover write serialization/retry/account ownership, and debounced-draft flushing.
The mock backend already had video upload and retrieval support; only image upload was missing for this round trip. The new Playwright journey uses the real Projects UI to export Fixture Project 002, resets the backend to the empty profile, imports through the real file chooser in a fresh browser context, and then verifies the HTTP contracts and persisted state: one fresh project, four uploaded image DTOs, one uploaded video DTO, returned upload names remapped through the stored document,
other/non-intermediate media, retrievable DTO/full/thumbnail routes, and a restored cover-index entry. The verified journey completed in 1.633 seconds under its 60-second deadline during the final release gate.This browser journey verifies HTTP contracts, persistence, and reference remapping. It does not claim byte-for-byte media fidelity.