Skip to content

feat(recall): support direct model reference images in recall API#9045

Open
lstein wants to merge 10 commits intomainfrom
lstein/recall-reference-images
Open

feat(recall): support direct model reference images in recall API#9045
lstein wants to merge 10 commits intomainfrom
lstein/recall-reference-images

Conversation

@lstein
Copy link
Copy Markdown
Collaborator

@lstein lstein commented Apr 12, 2026 •
edited
Loading

Summary

The current recall_parameters API can recall reference images that are attached to the denoiser via ip_adapters, but fails when the reference image is attached directly to the model, as happens with Flux.2 and Qwen Image Edit. This PR corrects the problem and allows both types of reference image to be recalled.

In addition, there was no test coverage of the recall_parameters API at all. This PR adds tests for recall of basic generation parameters, ip_adapters, direct_reference images, LoRAs, and controlNets.

Related Issues / Discussions

QA Instructions

It is easiest to test using the Swagger doc execution feature.

  1. Select an image from the current running instance's outputs/images directory.
  2. "Try out" the endpoint /api/v1/recall/{queue_id}
  3. Enter "default" for the queue ID
  4. Use this request body for the POST:
{
 "positive_prompt": "test prompt",
 "model": "FLUX.2 Klein 9B",
 "reference_images": [
       {
           "image_name" : <name of the selected image>
       }
   ]
}

When submitted, the positive prompt and the model should change as specified (these are the positive controls), and the selected reference image should appear.

Merge Plan

Simple merge.

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • ❗Changes to a redux slice have a corresponding migration
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

lstein and others added 2 commits April 11, 2026 22:08
@lstein @claude
feat(recall): support model-free reference images in recall API
4da9fab 
The recall parameters API previously exposed only `loras`, `control_layers`,
and `ip_adapters`. This meant reference images used by architectures that
feed images directly into the main model — FLUX.2 Klein, FLUX Kontext, and
Qwen Image Edit — could not be sent through the recall endpoint at all:
they have no adapter model to resolve, so they could not ride in the
`ip_adapters` list.

This change adds a new `reference_images` field on RecallParameter that
carries only an `image_name`. The backend validates the file exists in
outputs/images and forwards the resolved metadata (width/height) in the
broadcast event. The frontend's recall handler picks the right config type
(`flux2_reference_image` / `flux_kontext_reference_image` / `ip_adapter`
fallback) via getDefaultRefImageConfig() based on the currently-selected
main model, matching the behavior of a manual drag-and-drop, and dispatches
`refImagesRecalled` with replace:false so these append rather than clobber
any adapters already applied in the same event.

Also consolidates the two existing docs under docs/contributing/RECALL_PARAMETERS/
(RECALL_PARAMETERS_API.md and RECALL_API_LORAS_CONTROLNETS_IMAGES.md) into
a single RECALL_PARAMETERS_API.md that documents the full request schema
including the new field.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@lstein @claude
test(recall): cover loras, control layers, and ip_adapters paths
4938d0a 
The original recall_parameters router (PR #8758) shipped without any
unit tests for its three collection fields. This commit backfills that
coverage alongside the reference_images tests added in the previous
commit.

The resolver helpers (resolve_model_name_to_key, load_image_file,
process_controlnet_image) are monkey-patched via module-level attribute
replacement so each test can pin down a specific resolution outcome
without spinning up the model manager or an image-files service. Two
small factory helpers (make_name_to_key_stub / make_load_image_file_stub)
make that ergonomic.

New coverage:

* LoRAs — multi-entry resolution + weight/is_enabled pass-through,
  silent drop on unresolvable names, is_enabled default of True.
* Control layers — ControlNet resolution precedence, fall-through to
  T2I Adapter and Control LoRA in order, missing image gracefully
  warned-and-continued, processed_image attached when the processor
  returns data, unresolvable entries dropped.
* IP Adapters — IPAdapter-before-FluxRedux lookup order, method /
  image_influence pass-through, missing image gracefully warned-and-
  continued, unresolvable entries dropped.
* Combined happy path — full request with prompts + model + all four
  collection fields, verifying every resolved value reaches the
  broadcast payload.
* Main-model drop — an unresolvable main model is scrubbed from the
  broadcast so the frontend never receives a stale model name.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions github-actions bot added api python PRs that change python files frontend PRs that change frontend files python-tests PRs that change python tests docs PRs that change docs labels Apr 12, 2026
@lstein lstein moved this to 6.13.x Theme: MODELS in Invoke - Community Roadmap Apr 12, 2026
@lstein lstein added the v6.13.x label Apr 12, 2026
@lstein @claude
merge: resolve conflicts from main, extend image access check to refe…
bc9dfb1 
…rence images

Merge main into lstein/recall-reference-images, resolving conflicts in
recall_parameters.py and regenerating openapi.json + schema.ts. Extended
_assert_recall_image_access to also validate reference_images, since they
carry image_name fields that need the same authorization guard.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@lstein
Copy link
Copy Markdown
Collaborator Author

lstein commented Apr 13, 2026

Merged main to resolve conflicts. Changes in the merge commit:

  • recall_parameters.py: Kept both resolve_reference_images (this branch) and _assert_recall_image_access (from main). Extended _assert_recall_image_access to also validate reference_images, since they carry image_name fields that need the same authorization guard as control layers and IP adapters.
  • openapi.json and schema.ts: Regenerated from Python sources to include both the new recall endpoints and the workflow visibility endpoints from main.

@lstein lstein marked this pull request as ready for review April 13, 2026 23:00
@lstein lstein marked this pull request as draft April 13, 2026 23:08
lstein and others added 2 commits April 13, 2026 19:36
@lstein @claude
fix(test): patch ApiDependencies in auth_dependencies to fix recall t…
6108ca5 
…ests

The patched_dependencies fixture only monkeypatched ApiDependencies in
the recall_parameters module, but the endpoint also resolves
CurrentUserOrDefault via auth_dependencies, which accesses
ApiDependencies.invoker independently. Patch both import sites.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@lstein
chore(frontend): typegen
c43df36
@lstein lstein marked this pull request as ready for review April 13, 2026 23:44
lstein and others added 2 commits April 13, 2026 21:14
@lstein @claude
feat(recall): add strict mode to clear unset parameters on recall
b9a19af 
Add a `strict` query parameter (default false) to POST recall endpoint.
When true, parameters not in the request body are reset: list fields
(loras, control_layers, ip_adapters, reference_images) become [] and
scalar fields become null, so the frontend clears stale state.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@lstein
Merge branch 'main' into lstein/recall-reference-images
5b969af
@lstein
Copy link
Copy Markdown
Collaborator Author

lstein commented Apr 14, 2026

Commit b9a19af adds a new strict parameter to the POST recall_parameters route. When True, all parameters that are not explicitly mentioned in the POST body are set to their default values. This is helpful to prevent unused reference images from hanging around.

@lstein lstein changed the title feat(recall): support model-free reference images in recall API feat(recall): support direct model reference images in recall API Apr 14, 2026
@lstein @claude
fix(frontend): eliminate ref image doubling from race between Promise…
3404cd7 
….all chains

IP adapters and model-free reference images were dispatched via two
independent Promise.all chains — one with replace:true, the other with
replace:false.  When a previous recall's promises were still in-flight
they could resolve after the clear and re-append stale entries, doubling
the list.  Combine both into a single Promise.all with one replace:true
dispatch so the race is impossible.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@blessedcoolant blessedcoolant Awaiting requested review from blessedcoolant blessedcoolant is a code owner

@dunkeroni dunkeroni Awaiting requested review from dunkeroni dunkeroni is a code owner

@Pfannkuchensack Pfannkuchensack Awaiting requested review from Pfannkuchensack Pfannkuchensack is a code owner

@JPPhoto JPPhoto Awaiting requested review from JPPhoto JPPhoto is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

api docs PRs that change docs frontend PRs that change frontend files python PRs that change python files python-tests PRs that change python tests v6.13.x

Projects

Invoke - Community Roadmap
Status: 6.13.x Theme: MODELS

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@lstein