diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 00000000..0cf18a89 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,80 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Project + +PhotoMapAI is a local-first image browser for large photo collections. It uses CLIP embeddings to power semantic text/image search and builds a UMAP "semantic map" that clusters images by content. The backend is FastAPI; the frontend is vanilla ES6 modules (no framework) using Swiper.js and Plotly.js. All processing is local — nothing is sent to external services. + +## Common Commands + +```bash +# Install for development (Python 3.10–3.13) +pip install -e .[testing,development] +npm install + +# Run the server (entry point defined in pyproject.toml) +start_photomap # http://localhost:8050 + +# Tests +make test # runs npm test + pytest +pytest tests # backend only +pytest tests/backend/test_search.py::test_text_search # single test +npm test # frontend Jest only +NODE_OPTIONS='--experimental-vm-modules' jest tests/frontend/search.test.js # single JS test + +# Linting / formatting (CI enforces both) +make lint # runs backend-lint + frontend-lint +ruff check photomap tests --fix +npm run lint:fix +npm run format # prettier write +npm run format:check # CI check + +# Docs +make docs # mkdocs serve on :8000 +``` + +Ruff is configured for line-length 120, target py310, rules E/W/F/I/UP/B (see pyproject.toml). Jest runs in jsdom with experimental ESM (the project is `"type": "module"`). + +## Architecture + +### Backend layout (`photomap/backend/`) + +- `photomap_server.py` — FastAPI app entry point. Wires up routers, mounts `/static` and Jinja2 templates, and defines the top-level `/` route. `start_photomap` from `pyproject.toml` runs `main()` here. +- `routers/` — one router per API surface: `album`, `search`, `umap`, `index`, `curation`, `filetree`, `upgrade`. Routers are included in `photomap_server.py`; `curation_router` is mounted with an explicit `/api/curation` prefix while the others set their own prefixes. +- `config.py` — YAML-backed album config. Access via the `get_config_manager()` singleton (lru_cached). `Album` is a Pydantic model that expands `~` in image paths. Config lives in a platformdirs user config directory. +- `embeddings.py` — CLIP embedding generation and persistence (`.npz`). +- `imagetool.py` — shared CLI entry point for `index_images`, `update_images`, `search_images`, `search_text`, `find_duplicate_images` (all registered as scripts in `pyproject.toml`). +- `metadata_extraction.py` / `metadata_formatting.py` — pulls EXIF + generator metadata (InvokeAI) out of images and formats for the UI. + +### Metadata subsystem (`photomap/backend/metadata_modules/`) + +This is the area under active refactor (current branch: `lstein/feature/refactor-invoke-metadata`). InvokeAI writes several incompatible metadata schemas into PNG tEXt chunks; the parser must auto-detect and upgrade. + +- `invokemetadata.py` defines `GenerationMetadata` as a Pydantic `Annotated[Union[…], Field(discriminator="metadata_version")]` over `GenerationMetadata2`, `GenerationMetadata3`, and `GenerationMetadata5`. `GenerationMetadataAdapter.parse()` inspects fields like `canvas_v2_metadata`, `app_version`, and `model_weights` to inject the correct `metadata_version` when the source JSON predates the discriminator. +- `invoke/` holds the per-version schemas: `invoke2metadata.py`, `invoke3metadata.py`, `invoke5metadata.py`, plus `canvas2metadata.py` and `common_metadata_elements.py` for shared types. `invoke_metadata_view.py` is the version-agnostic facade consumed by `invoke_formatter.py`. When adding support for a new InvokeAI version, add a new `invokeNmetadata.py`, extend the Union in `invokemetadata.py`, teach `parse()` how to recognize legacy payloads that lack a `metadata_version` field, and extend `InvokeMetadataView`'s `isinstance` dispatch. +- `invoke_formatter.py` / `exif_formatter.py` render parsed metadata for the drawer UI; `slide_summary.py` produces the compact slideshow caption. +- `invoke-DELETE/` is a holdover from the refactor — leave it alone unless cleaning up. + +### Frontend layout (`photomap/frontend/`) + +- `static/javascript/` — one ES6 module per feature. No build step; modules are served directly and imported from `main.js` / `index.js`. +- `state.js` is the centralized application state. Prefer extending it over adding new globals. +- `events.js` owns global keyboard shortcuts; register new ones there rather than scattering listeners. +- `localStorage` is used for persisted user preferences, `sessionStorage` for per-navigation state. +- `templates/` — Jinja2 templates rendered by FastAPI. + +### Tests + +- `tests/backend/` — pytest. `conftest.py` + `fixtures.py` set up shared fixtures (test images in `tests/backend/test_images/`). Use the FastAPI `TestClient` for router tests; see `test_search.py`, `test_albums.py`, `test_curation.py` as templates. +- `tests/frontend/` — Jest with jsdom. `setup.js` provides DOM fixtures. See `tests/frontend/README.md` for setup notes. + +## Conventions to follow + +From `.github/copilot-instructions.md` — the parts that actually affect how you write code here: + +- **Python:** type hints on public functions, `pathlib.Path` (not `os.path`) for file operations, f-strings, imports ordered stdlib → third-party → local (`photomap` is first-party to isort). Code must pass `ruff check photomap tests`. +- **Pinned quirk:** `setuptools<67` is intentional — avoids a deprecation warning from the CLIP dependency. Don't "fix" it. +- **New API endpoints:** add/extend a router under `photomap/backend/routers/`, use Pydantic models for request/response, include the router in `photomap_server.py`, add a `tests/backend/test_.py`. +- **New frontend features:** create a module in `static/javascript/`, wire shared state through `state.js`, register shortcuts in `events.js`, add a Jest test. +- **JavaScript:** ES6 modules only, `const`/`let`, must pass `npm run lint` and `npm run format:check`. diff --git a/photomap/backend/metadata_modules/__init__.py b/photomap/backend/metadata_modules/__init__.py index 646cadd5..033f6e89 100644 --- a/photomap/backend/metadata_modules/__init__.py +++ b/photomap/backend/metadata_modules/__init__.py @@ -1,4 +1,3 @@ - from .exif_formatter import format_exif_metadata from .invoke_formatter import format_invoke_metadata from .slide_summary import SlideSummary diff --git a/photomap/backend/metadata_modules/invoke-DELETE/__init__.py b/photomap/backend/metadata_modules/invoke-DELETE/__init__.py new file mode 100644 index 00000000..63cfea4b --- /dev/null +++ b/photomap/backend/metadata_modules/invoke-DELETE/__init__.py @@ -0,0 +1,10 @@ +# from .legacy import InvokeLegacyMetadata +# from .v3 import Invoke3Metadata +# from .v5 import Invoke5Metadata + +# # reexport the main classes +# __all__ = [ +# "InvokeLegacyMetadata", +# "Invoke3Metadata", +# "Invoke5Metadata", +# ] diff --git a/photomap/backend/metadata_modules/invoke/invoke_metadata_abc.py b/photomap/backend/metadata_modules/invoke-DELETE/invoke_metadata_abc.py similarity index 100% rename from photomap/backend/metadata_modules/invoke/invoke_metadata_abc.py rename to photomap/backend/metadata_modules/invoke-DELETE/invoke_metadata_abc.py diff --git a/photomap/backend/metadata_modules/invoke/legacy.py b/photomap/backend/metadata_modules/invoke-DELETE/legacy.py similarity index 100% rename from photomap/backend/metadata_modules/invoke/legacy.py rename to photomap/backend/metadata_modules/invoke-DELETE/legacy.py diff --git a/photomap/backend/metadata_modules/invoke/v3.py b/photomap/backend/metadata_modules/invoke-DELETE/v3.py similarity index 100% rename from photomap/backend/metadata_modules/invoke/v3.py rename to photomap/backend/metadata_modules/invoke-DELETE/v3.py diff --git a/photomap/backend/metadata_modules/invoke/v5.py b/photomap/backend/metadata_modules/invoke-DELETE/v5.py similarity index 100% rename from photomap/backend/metadata_modules/invoke/v5.py rename to photomap/backend/metadata_modules/invoke-DELETE/v5.py diff --git a/photomap/backend/metadata_modules/invoke/__init__.py b/photomap/backend/metadata_modules/invoke/__init__.py deleted file mode 100644 index a962bdea..00000000 --- a/photomap/backend/metadata_modules/invoke/__init__.py +++ /dev/null @@ -1,11 +0,0 @@ - -from .legacy import InvokeLegacyMetadata -from .v3 import Invoke3Metadata -from .v5 import Invoke5Metadata - -# reexport the main classes -__all__ = [ - "InvokeLegacyMetadata", - "Invoke3Metadata", - "Invoke5Metadata", -] diff --git a/photomap/backend/metadata_modules/invoke/canvas2metadata.py b/photomap/backend/metadata_modules/invoke/canvas2metadata.py new file mode 100644 index 00000000..a2b92a19 --- /dev/null +++ b/photomap/backend/metadata_modules/invoke/canvas2metadata.py @@ -0,0 +1,125 @@ +from typing import Any + +from pydantic import BaseModel, ConfigDict, Field, model_serializer, model_validator + +from photomap.backend.metadata_modules.invoke.common_metadata_elements import ( + ControlAdapter, + Fill, + Object, + Position, + ReferenceImage, + RegionalGuidance, + tag_reference_images, +) + + +class Clip(BaseModel): + height: float + width: float + x: float + y: float + + +class Inpaintmask(BaseModel): + model_config = ConfigDict(populate_by_name=True) + fill: Fill + id: str + is_enabled: bool = Field(alias="isEnabled") + is_locked: bool = Field(alias="isLocked") + name: Any | None + objects: list[Object] + opacity: float + position: Position + type: str + + +class Rasterlayer(BaseModel): + model_config = ConfigDict(populate_by_name=True) + id: str + is_enabled: bool = Field(alias="isEnabled") + is_locked: bool = Field(alias="isLocked") + name: Any | None + objects: list[Object] + opacity: float + position: Position + type: str + + +class ControlLayer(BaseModel): + model_config = ConfigDict(populate_by_name=True) + control_adapter: ControlAdapter = Field(alias="controlAdapter") + id: str + is_enabled: bool = Field(alias="isEnabled") + is_locked: bool = Field(alias="isLocked") + name: Any | None + objects: list[Object] + opacity: float + position: Position + type: str + with_transparency_effect: bool = Field(alias="withTransparencyEffect") + + +class CanvasV2Metadata(BaseModel): + model_config = ConfigDict(populate_by_name=True) + raster_layers: list[Rasterlayer] | None = Field(None, alias="rasterLayers") + control_layers: list[ControlLayer] | None = Field(None, alias="controlLayers") + inpaint_masks: list[Inpaintmask] | None = Field(None, alias="inpaintMasks") + reference_images: list[ReferenceImage] | None = Field( + None, alias="referenceImages" + ) + regional_guidance: list[RegionalGuidance] | None = Field( + None, alias="regionalGuidance" + ) + + @model_validator(mode="before") + @classmethod + def _preprocess_canvas_metadata( + cls, canvas_metadata: dict[str, Any] + ) -> dict[str, Any]: + """Preprocess canvas metadata to add type discriminators to image objects.""" + + def process_image_in_dict(obj: dict[str, Any], key: str = "image") -> None: + """Add type discriminator to an image object if it exists.""" + if key in obj and obj[key]: + tag_reference_images(obj[key]) + + def process_objects(objects: list[dict[str, Any]]) -> None: + """Process a list of objects that may contain images.""" + for obj in objects: + process_image_in_dict(obj) + + def process_reference_images(ref_images: list[dict[str, Any]]) -> None: + """Process reference images with ipAdapter.""" + for ref_image in ref_images: + if "ipAdapter" in ref_image and ref_image["ipAdapter"]: + process_image_in_dict(ref_image["ipAdapter"]) + + # Process layers with objects (rasterLayers, inpaintMasks, controlLayers) + for layer_key in ["rasterLayers", "inpaintMasks", "controlLayers"]: + if layer_key in canvas_metadata and canvas_metadata[layer_key]: + for layer in canvas_metadata[layer_key]: + if "objects" in layer and layer["objects"]: + process_objects(layer["objects"]) + + # Process top-level referenceImages + if "referenceImages" in canvas_metadata and canvas_metadata["referenceImages"]: + process_reference_images(canvas_metadata["referenceImages"]) + + # Process regionalGuidance with objects and referenceImages + if ( + "regionalGuidance" in canvas_metadata + and canvas_metadata["regionalGuidance"] + ): + for region in canvas_metadata["regionalGuidance"]: + if "objects" in region and region["objects"]: + process_objects(region["objects"]) + if "referenceImages" in region and region["referenceImages"]: + process_reference_images(region["referenceImages"]) + + return canvas_metadata + + @model_serializer(mode="wrap") + def serialize_model(self, serializer, info): + """Exclude None values when serializing.""" + data = serializer(self) + return {k: v for k, v in data.items() if v is not None} diff --git a/photomap/backend/metadata_modules/invoke/common_metadata_elements.py b/photomap/backend/metadata_modules/invoke/common_metadata_elements.py new file mode 100644 index 00000000..b4f74a4c --- /dev/null +++ b/photomap/backend/metadata_modules/invoke/common_metadata_elements.py @@ -0,0 +1,189 @@ +from typing import Annotated, Any, Literal + +from pydantic import BaseModel, ConfigDict, Field, model_serializer, model_validator + + +class Color(BaseModel): + b: int + g: int + r: int + + +class Model(BaseModel): + model_config = ConfigDict(extra="forbid", populate_by_name=True) + name: str = Field(alias="model_name") + base: str | None = Field(default=None, alias="base_model") + hash: str | None = None + key: str | None = None + type: str = Field(default="main", alias="model_type") + + @model_serializer(mode="wrap") + def serialize_model(self, serializer, info): + """Exclude None values when serializing.""" + data = serializer(self) + return {k: v for k, v in data.items() if v is not None} + + +class T5Encoder(Model): + model_config = ConfigDict(extra="allow", populate_by_name=True) + + +class ClipEmbedModel(Model): + model_config = ConfigDict(extra="allow", populate_by_name=True) + + +class ImageData(BaseModel): + model_config = ConfigDict(populate_by_name=True) + type: Literal["dataURL"] = Field(default="dataURL", alias="image_type") + data_url: str = Field(alias="dataURL") + height: int | None = None + width: int | None = None + + +class ImageFile(BaseModel): + model_config = ConfigDict(populate_by_name=True) + type: Literal["file"] = Field(default="file", alias="image_type") + name: str = Field(alias="image_name") + height: int | None = None + width: int | None = None + + +Image = Annotated[ + ImageFile | ImageData, + Field(discriminator="type"), +] + + +class ControlAdapter(BaseModel): + model_config = ConfigDict(populate_by_name=True) + image: Image + begin_end_step_pct: list[int | float] = Field(alias="beginEndStepPct") + control_mode: str | None = Field(None, alias="controlMode") + model: Model = Field(alias="control_model") + type: str | None = Field(default=None) + weight: float = Field(alias="control_weight") + + @model_validator(mode="before") + @classmethod + def fixup_step_percentages(cls, json_data: dict[str, Any]) -> dict[str, Any]: + """Convert begin_step_percent and end_step_percent to beginEndStepPct if they exist, and ensure the values are in a list.""" + return fixup_step_percentages(json_data) + + @model_validator(mode="before") + @classmethod + def tag_reference_images(cls, data: dict[str, Any]) -> dict[str, Any]: + """Tag reference images with a discriminator for proper parsing.""" + if "image" in data and isinstance(data["image"], dict): + tag_reference_images(data["image"]) + return data + + +class Lora(BaseModel): + model_config = ConfigDict(extra="forbid", populate_by_name=True) + model: Model = Field(alias="lora") + weight: float + + +class IPAdapter(BaseModel): + model_config = ConfigDict(populate_by_name=True) + begin_end_step_pct: list[int | float] | None = Field( + None, alias="beginEndStepPct" + ) + model: Model + image: Image + type: str | None = None + method: str | None = None + weight: float | None = None + image_influence: str | None = Field(None, alias="imageInfluence") + + @model_validator(mode="before") + @classmethod + def fixup_step_percentages(cls, json_data: dict[str, Any]) -> dict[str, Any]: + """Convert begin_step_percent and end_step_percent to beginEndStepPct if they exist, and ensure the values are in a list.""" + return fixup_step_percentages(json_data) + + @model_validator(mode="before") + @classmethod + def tag_reference_images(cls, data: dict[str, Any]) -> dict[str, Any]: + """Tag reference images with a discriminator for proper parsing.""" + if "image" in data and isinstance(data["image"], dict): + tag_reference_images(data["image"]) + return data + + @model_validator(mode="before") + @classmethod + def consolidate_model_aliases(cls, data): + """Consolidate model aliases to ensure the model field is populated correctly.""" + for key in ["clip_vision_model", "ip_adapter_model", "t2i_adapter_model"]: + if "model" not in data and key in data: + data["model"] = data[key] + break + return data + + +class ReferenceImage(BaseModel): + model_config = ConfigDict(populate_by_name=True) + id: str + ip_adapter: IPAdapter = Field(alias="ipAdapter") + is_enabled: bool | None = Field(None, alias="isEnabled") + is_locked: bool | None = Field(None, alias="isLocked") + name: Any | None = None + type: str | None = None + + +class Fill(BaseModel): + color: Color + style: str + + +class Position(BaseModel): + x: int | float + y: int | float + + +class Object(BaseModel): + id: str + image: Image | None = None + type: str + + +class RegionalGuidance(BaseModel): + model_config = ConfigDict(populate_by_name=True) + auto_negative: bool = Field(alias="autoNegative") + fill: Fill + id: str + is_enabled: bool = Field(alias="isEnabled") + is_locked: bool = Field(alias="isLocked") + name: Any | None + positive_prompt: str | None = Field(None, alias="positivePrompt") + negative_prompt: str | None = Field(None, alias="negativePrompt") + objects: list[Object] + opacity: float + position: Position + reference_images: list[ReferenceImage] = Field(alias="referenceImages") + type: str + + +def tag_reference_images(image: dict[str, Any]) -> None: + """Mutates the input image dict to add an "image_type" field based on whether it has a "dataURL" or "image_name" field.""" + if "dataURL" in image: + image["type"] = "dataURL" + elif "image_name" in image: + image["type"] = "file" + elif "name" in image: + image["type"] = "file" + + +def fixup_step_percentages(json_data: dict[str, Any]) -> dict[str, Any]: + """ + Helper function to convert begin_step_percent and end_step_percent to beginEndStepPct if they exist, + and ensure the values are in a list. + """ + for key in ["begin_step_percent", "end_step_percent"]: + if key in json_data: + value = json_data.pop(key) + if isinstance(value, int | float): + json_data.setdefault("beginEndStepPct", []).append(value) + elif isinstance(value, list): + json_data.setdefault("beginEndStepPct", []).extend(value) + return json_data diff --git a/photomap/backend/metadata_modules/invoke/invoke2metadata.py b/photomap/backend/metadata_modules/invoke/invoke2metadata.py new file mode 100644 index 00000000..bce2961f --- /dev/null +++ b/photomap/backend/metadata_modules/invoke/invoke2metadata.py @@ -0,0 +1,78 @@ +from typing import Any, Literal + +from pydantic import BaseModel, ConfigDict, model_serializer, model_validator + +from photomap.backend.metadata_modules.invoke.common_metadata_elements import Model + + +class Prompt(BaseModel): + prompt: str + weight: float + + +class ImageVariation(BaseModel): + seed: int + weight: float + + +class Image(BaseModel): + cfg_scale: float + height: int + hires_fix: bool | None = None + perlin: int | float | None = None + postprocessing: Any | None + prompt: str | list[Prompt] + sampler: str + seamless: bool | None = None + seed: int + steps: int + threshold: int | float | None = None + type: str + variations: list[ImageVariation] | None = None + width: int + + +class ModelListElement(BaseModel): + model: Model + status: str + description: str + + +class GenerationMetadata2(BaseModel): + model_config = ConfigDict(extra="forbid", populate_by_name=True) + metadata_version: Literal[2] + app_id: str + model_id: str | None = None + app_version: str + image: Image | None = None + images: list[Image] | None = None + model: str + model_hash: str + model_weights: str | None = None + # This appears in a few old images, but is not well structured. + # We structure it a bit in the model validator. + model_list: list[ModelListElement] | None = None + + @model_validator(mode="before") + @classmethod + def validate_model_id(cls, data: dict[str, Any]) -> dict[str, Any]: + """Munge the model_list into a more compatible structure""" + if "model_list" in data and isinstance(data["model_list"], dict): + model_list = [] + for model_entry_key, model_entry_value in data["model_list"].items(): + new_model_entry = {} + new_model_entry["model"] = Model(model_name=model_entry_key) + if isinstance(model_entry_value, dict): + new_model_entry["status"] = model_entry_value.get("status", "") + new_model_entry["description"] = model_entry_value.get( + "description", "" + ) + model_list.append(new_model_entry) + data["model_list"] = model_list + return data + + @model_serializer(mode="wrap") + def serialize_model(self, serializer, info): + """Exclude None values when serializing.""" + data = serializer(self) + return {k: v for k, v in data.items() if v is not None} diff --git a/photomap/backend/metadata_modules/invoke/invoke3metadata.py b/photomap/backend/metadata_modules/invoke/invoke3metadata.py new file mode 100644 index 00000000..7e33499b --- /dev/null +++ b/photomap/backend/metadata_modules/invoke/invoke3metadata.py @@ -0,0 +1,144 @@ +from typing import Literal + +from pydantic import BaseModel, ConfigDict, Field, model_serializer, model_validator + +from photomap.backend.metadata_modules.invoke.canvas2metadata import Clip +from photomap.backend.metadata_modules.invoke.common_metadata_elements import ( + ControlAdapter, + IPAdapter, + Lora, + Model, + tag_reference_images, +) + + +class T2IAdapter(IPAdapter): + pass + + +class CanvasObject(BaseModel): + kind: str + layer: str + tool: str | None = None + stroke_width: int | None = Field(default=None, alias="strokeWidth") + x: int | None = None + y: int | None = None + width: int | None = None + height: int | None = None + image_name: str | None = Field(default=None, alias="imageName") + points: list[float] | None = None + clip: Clip | None = None + + +class PostProcessing(BaseModel): + type: str + orig_path: list[str] | None = None + orig_hash: str | None = None + scale: float | None = None + strength: float | None = None + + +# Most fields are optional because of various glitches and exceptions in v3 metadata +class GenerationMetadata3(BaseModel): + model_config = ConfigDict(extra="forbid", populate_by_name=True) + metadata_version: Literal[3] + app_version: str | None = Field(default="3.X.X", alias="imported_app_version") + generation_mode: str | None = None + positive_prompt: str | None = None + positive_style_prompt: str | None = None + negative_prompt: str | None = None + negative_style_prompt: str | None = None + height: int | None = None + width: int | None = None + rand_device: str | None = None + scheduler: str | None = None + seed: int | None = None + steps: int | None = None + strength: float | None = None + init_image: str | None = None + post_processing: list[PostProcessing] | None = None + model: Model | None = None + vae: Model | None = None + ip_adapters: list[IPAdapter] | None = Field(default=None, alias="ipAdapters") + t2iAdapters: list[T2IAdapter] | None = None + loras: list[Lora] | None = None + controlnets: list[ControlAdapter] | None = None + cfg_rescale_multiplier: float | None = None + cfg_scale: float | None = None + esrgan_model: str | None = None + clip_skip: int | None = None + seamless_x: bool | None = None + seamless_y: bool | None = None + # These fields appear in some app_version 3 images + refiner_model: Model | None = None + refiner_cfg_scale: float | None = None + refiner_steps: int | None = None + refiner_scheduler: str | None = None + refiner_positive_aesthetic_score: float | None = Field( + default=None, alias="refiner_positive_aesthetic_store" + ) + refiner_negative_aesthetic_score: float | None = Field( + default=None, alias="refiner_negative_aesthetic_store" + ) + refiner_start: float | None = None + # A few examples of these + hrf_enabled: bool | None = None + hrf_method: str | None = None + hrf_strength: float | None = None + hrf_width: int | None = None + hrf_height: int | None = None + # One example of this found! + canvas_objects: list[CanvasObject] | None = Field( + default=None, alias="_canvas_objects" + ) + + @model_validator(mode="before") + @classmethod + def fixup_orphan_images(cls, data: dict) -> dict: + """ + Fix up any orphaned reference images by sticking them into a postprocessing model. + """ + if ( + "image" in data + and isinstance(data["image"], dict) + and "postprocessing" in data["image"] + ): + post_processing_list = [] + for post_processing_entry in data["image"]["postprocessing"]: + post_processing_list.append( + PostProcessing( + orig_path=post_processing_entry.get("orig_path"), + orig_hash=post_processing_entry.get("orig_hash"), + type=post_processing_entry.get("type"), + strength=post_processing_entry.get("strength"), + scale=post_processing_entry.get("scale"), + ) + ) + data.pop("image") + data["post_processing"] = post_processing_list + return data + + @model_validator(mode="before") + @classmethod + def tag_reference_images(cls, data): + """Tag reference images with a discriminator for proper parsing.""" + if "ipAdapters" in data and isinstance(data["ipAdapters"], list): + for ref_image in data["ipAdapters"]: + if image := ref_image.get("image"): + tag_reference_images(image) + return data + + @model_validator(mode="before") + @classmethod + def fixup_aesthetic_score(cls, json_data: dict) -> dict: + """Replace the refiner_aesthetic_store and refiner_aesthetic_score fields with refiner_positive_aesthetic_score.""" + for key in ["refiner_positive_aesthetic_store", "refiner_aesthetic_store"]: + if key in json_data: + json_data["refiner_positive_aesthetic_score"] = json_data.pop(key) + return json_data + + @model_serializer(mode="wrap") + def serialize_model(self, serializer, info): + """Exclude None values when serializing.""" + data = serializer(self) + return {k: v for k, v in data.items() if v is not None} diff --git a/photomap/backend/metadata_modules/invoke/invoke5metadata.py b/photomap/backend/metadata_modules/invoke/invoke5metadata.py new file mode 100644 index 00000000..add790db --- /dev/null +++ b/photomap/backend/metadata_modules/invoke/invoke5metadata.py @@ -0,0 +1,245 @@ +from typing import Any, Literal + +from pydantic import BaseModel, ConfigDict, Field, model_serializer, model_validator + +from photomap.backend.metadata_modules.invoke.canvas2metadata import CanvasV2Metadata +from photomap.backend.metadata_modules.invoke.common_metadata_elements import ( + ClipEmbedModel, + Image, + IPAdapter, + Lora, + Model, + RegionalGuidance, + T5Encoder, + fixup_step_percentages, + tag_reference_images, +) + + +class ControlLayer(BaseModel): + id: str + type: str + is_enabled: bool = Field(alias="isEnabled") + is_selected: bool = Field(alias="isSelected") + control_adapter: IPAdapter | None = Field(default=None, alias="ipAdapter") + + +class ControlLayers(BaseModel): + version: int | float + layers: list[ControlLayer] + + +class ControlNet(BaseModel): + image: Image + model: Model = Field(alias="control_model") + weight: float | None = Field(alias="control_weight") + begin_end_step_pct: list[int | float] | None = Field( + None, alias="beginEndStepPct" + ) + control_mode: str + resize_mode: str + + @model_validator(mode="before") + @classmethod + def fixup_step_percentages(cls, json_data: dict[str, Any]) -> dict[str, Any]: + """Convert begin_step_percent and end_step_percent to beginEndStepPct if they exist, and ensure the values are in a list.""" + return fixup_step_percentages(json_data) + + +class RefImageConfig(BaseModel): + type: str + image: Image + model: Model | None = None + beginEndStepPct: list[float] | None = None + method: str | None = None + clipVisionModel: str | None = None + weight: float | None = None + image_influence: str | None = Field(default=None, alias="imageInfluence") + + +class RefImage(BaseModel): + id: str + isEnabled: bool + config: RefImageConfig + + @model_validator(mode="before") + @classmethod + def tag_reference_images(cls, data): + """Tag reference images with a discriminator for proper parsing.""" + if "config" in data and isinstance(data["config"], dict): + config = data["config"] + if "image" in config and isinstance(config["image"], dict): + tag_reference_images(config["image"]) + return data + + +# Empirically, pretty much all fields are optional in v5! +class GenerationMetadata5(BaseModel): + model_config = ConfigDict(extra="forbid", populate_by_name=True) + metadata_version: Literal[5] + app_version: str | None = Field(default="5.X.X") + model: Model | str | None = Field(default=None, alias="Model") + generation_mode: str | None = None + height: int | None = None + width: int | None = None + positive_prompt: str | None = Field(default=None, alias="Positive Prompt") + positive_style_prompt: str | None = None + negative_prompt: str | None = None + negative_style_prompt: str | None = None + scheduler: str | None = None + seed: int | None = None + steps: int | None = Field(default=None, alias="Steps") + guidance: int | float | None = None + ref_images: list[RefImage] | None = None + control_layers: ControlLayers | None = Field(default=None) + loras: list[Lora] | None = None + regions: list[RegionalGuidance] | None = None + t5_encoder: T5Encoder | None = None + qwen3_encoder: Model | None = None + qwen3_source: Model | None = None + vae: Model | None = None + clip_embed_model: ClipEmbedModel | None = None + dype_preset: str | None = None + rand_device: str | None = None + cfg_scale: float | None = None + cfg_rescale_multiplier: float | None = None + seamless_x: bool | None = None + seamless_y: bool | None = None + upscale_model: Model | None = None + upscale_initial_image: Image | None = None + upscale_scale: float | None = None + creativity: float | None = None + structure: float | None = None + tile_size: int | None = None + tile_overlap: int | None = None + clip_skip: int | None = None + canvas_v2_metadata: CanvasV2Metadata | None = None + # These fields appear in some ZiT images + seed_variance_strength: float | None = None + seed_variance_enabled: bool | None = Field( + default=None, alias="z_image_seed_variance_enabled" + ) + seed_variance_randomize_percentage: int | None = Field( + default=None, alias="z_image_seed_variance_randomize_percentage" + ) + # These fields appear in some Flux.1 images + dype_scale: float | None = None + dype_exponent: float | None = None + # These fields appear in some sdxl images + strength: float | None = None + init_image: str | None = None + hrf_enabled: bool | None = None + hrf_method: str | None = None + hrf_strength: float | None = None + refiner_cfg_scale: float | None = None + refiner_steps: int | None = None + refiner_scheduler: str | None = None + refiner_positive_aesthetic_score: float | None = None + refiner_negative_aesthetic_score: float | None = None + refiner_start: float | None = None + + @model_validator(mode="before") + @classmethod + def normalize_ref_images(cls, data): + """Flatten legacy ``ref_images`` structures before validation. + + Two historical quirks are handled here: + + * In some versions, ``ref_images`` was serialized as a list of lists. + Flatten the outer wrapper to a plain list. + * Earlier metadata nested the image under + ``config.image.original.image``; collapse that down to + ``config.image``. + """ + if not isinstance(data, dict): + return data + ref_images = data.get("ref_images") + if not isinstance(ref_images, list) or not ref_images: + return data + + # Flatten list-of-lists wrapper + if isinstance(ref_images[0], list): + ref_images = ref_images[0] + + # Unwrap nested ``config.image.original.image`` structures + for ref_image in ref_images: + if not isinstance(ref_image, dict): + continue + config = ref_image.get("config") + if not isinstance(config, dict): + continue + image_obj = config.get("image") + if not isinstance(image_obj, dict): + continue + original = image_obj.get("original") + if isinstance(original, dict) and "image" in original: + config["image"] = original["image"] + + data["ref_images"] = ref_images + return data + + @model_validator(mode="before") + @classmethod + def normalize_field_names(cls, data): + """Normalize alternative field name variations before validation.""" + if isinstance(data, dict): + # Map alternative names to canonical field names + aliases = { + "Seed variance strength": "seed_variance_strength", + "z_image_seed_variance_strength": "seed_variance_strength", + "z_image_seed_variance_randomize_percentage": "seed_variance_randomize_percentage", + "z_image_seed_variance_randomize_percent": "seed_variance_randomize_percentage", + "z_image_seed_variance_enabled": "seed_variance_enabled", + } + for alt_name, canonical_name in aliases.items(): + if alt_name in data and canonical_name not in data: + data[canonical_name] = data.pop(alt_name) + + return data + + @model_validator(mode="before") + def tag_reference_images(cls, data): + # NOTE: MOVE THIS TO THE PROPER IMAGE VALIDATOR + """Tag reference images with a discriminator for proper parsing.""" + if "upscale_initial_image" in data and isinstance( + data["upscale_initial_image"], dict + ): + tag_reference_images(data["upscale_initial_image"]) + if "controlnets" in data and isinstance(data["controlnets"], list): + for controlnet in data["controlnets"]: + if isinstance(controlnet, dict) and "image" in controlnet: + tag_reference_images(controlnet["image"]) + return data + + @model_validator(mode="before") + def fixup_controlnets(cls, data: dict[str, Any]) -> dict[str, Any]: + """ " + Massage the legacy controlnet format into the new control_layers format + """ + if "controlnets" in data and isinstance(data["controlnets"], list): + layers = [] + for cn in data["controlnets"]: + layer = { + "id": cn.get("id", ""), + "type": "controlnet", + "isEnabled": cn.get("isEnabled", True), + "isSelected": cn.get("isSelected", False), + "ipAdapter": { + "image": cn.get("image"), + "model": cn.get("control_model"), + "weight": cn.get("control_weight"), + "beginEndStepPct": cn.get("beginEndStepPct"), + "control_mode": cn.get("control_mode"), + "resize_mode": cn.get("resize_mode"), + }, + } + layers.append(layer) + data["control_layers"] = {"version": 1, "layers": layers} + del data["controlnets"] + return data + + @model_serializer(mode="wrap") + def serialize_model(self, serializer, info): + """Exclude None values when serializing.""" + data = serializer(self) + return {k: v for k, v in data.items() if v is not None} diff --git a/photomap/backend/metadata_modules/invoke/invoke_metadata_view.py b/photomap/backend/metadata_modules/invoke/invoke_metadata_view.py new file mode 100644 index 00000000..8bb2d8d2 --- /dev/null +++ b/photomap/backend/metadata_modules/invoke/invoke_metadata_view.py @@ -0,0 +1,274 @@ +""" +Version-agnostic view over a parsed GenerationMetadata object. + +The new invoke metadata system uses a Pydantic discriminated union over +``GenerationMetadata2`` / ``GenerationMetadata3`` / ``GenerationMetadata5``. +Each version stores the same logical pieces of information (prompt, model, +seed, LoRAs, reference images, control layers, raster images) in structurally +different places. Rather than sprinkling ``isinstance`` checks throughout the +formatter, this module centralizes the version-specific extraction rules in a +single ``InvokeMetadataView`` class that exposes a flat, stable interface for +the formatter to consume. +""" + +from collections import namedtuple + +from .canvas2metadata import CanvasV2Metadata +from .common_metadata_elements import ( + ControlAdapter, + ImageFile, + IPAdapter, +) +from .common_metadata_elements import Image as CommonImage +from .invoke2metadata import GenerationMetadata2 +from .invoke2metadata import Prompt as V2Prompt +from .invoke3metadata import GenerationMetadata3 +from .invoke5metadata import ControlLayer as V5ControlLayer +from .invoke5metadata import GenerationMetadata5, RefImage + +LoraTuple = namedtuple("LoraTuple", ["model_name", "weight"]) +ReferenceImageTuple = namedtuple( + "ReferenceImageTuple", ["model_name", "image_name", "weight"] +) +ControlLayerTuple = namedtuple( + "ControlLayerTuple", ["model_name", "image_name", "weight"] +) + +GenerationMetadataT = GenerationMetadata2 | GenerationMetadata3 | GenerationMetadata5 + + +def _image_name(image: CommonImage | None) -> str: + """Return the stored image name for an Image, or '' if unavailable. + + The discriminated ``Image`` union can be either an ``ImageFile`` (which + carries a ``name``) or an ``ImageData`` (inline data URL, no name). + """ + if isinstance(image, ImageFile): + return image.name or "" + return "" + + +def _effective_weight( + weight: float | None, image_influence: str | None +) -> float | str | None: + """Return the value to display in the weight column of a tuple table. + + Most InvokeAI IP-adapter-style reference images carry a numeric + ``weight``. Flux Redux adapters, however, don't use a numeric weight — + they use a categorical ``image_influence`` field with values like + ``"Low"``, ``"Medium"``, ``"High"``. When ``weight`` is absent we fall + back to ``image_influence`` so the column remains meaningful. + """ + if weight is not None: + return weight + return image_influence + + +class InvokeMetadataView: + """Read-only facade over a parsed InvokeAI ``GenerationMetadata``. + + Attributes expose exactly the fields required by ``invoke_formatter`` so + that the formatter stays ignorant of the underlying version layout. + """ + + def __init__(self, metadata: GenerationMetadataT) -> None: + self.metadata = metadata + + # ---- prompts --------------------------------------------------------- + + @property + def positive_prompt(self) -> str: + m = self.metadata + if isinstance(m, GenerationMetadata2): + if m.image is None: + return "" + prompt = m.image.prompt + if isinstance(prompt, list): + if not prompt: + return "" + first = prompt[0] + if isinstance(first, V2Prompt): + return first.prompt or "" + return "" + return prompt or "" + return m.positive_prompt or "" + + @property + def negative_prompt(self) -> str: + m = self.metadata + if isinstance(m, GenerationMetadata2): + return "" + return m.negative_prompt or "" + + # ---- model / seed ---------------------------------------------------- + + @property + def model_name(self) -> str: + m = self.metadata + if isinstance(m, GenerationMetadata2): + return m.model_weights or m.model or "" + if m.model is None: + return "" + if isinstance(m.model, str): + return m.model + return m.model.name or "" + + @property + def seed(self) -> int | None: + m = self.metadata + if isinstance(m, GenerationMetadata2): + return m.image.seed if m.image is not None else None + return m.seed + + # ---- loras ----------------------------------------------------------- + + @property + def loras(self) -> list[LoraTuple]: + m = self.metadata + if isinstance(m, GenerationMetadata2): + return [] + if not m.loras: + return [] + return [ + LoraTuple(model_name=lora.model.name, weight=lora.weight) for lora in m.loras + ] + + # ---- reference images (IPAdapter) ------------------------------------ + + @property + def reference_images(self) -> list[ReferenceImageTuple]: + m = self.metadata + if isinstance(m, GenerationMetadata2): + return [] + if isinstance(m, GenerationMetadata3): + return [ + self._ipadapter_to_tuple(ipa) + for ipa in (m.ip_adapters or []) + ] + # v5 + if m.ref_images: + return [self._refimage_to_tuple(r) for r in m.ref_images if r.isEnabled] + if m.canvas_v2_metadata is not None: + return self._canvas_reference_images(m.canvas_v2_metadata) + return [] + + @staticmethod + def _ipadapter_to_tuple(ipa: IPAdapter) -> ReferenceImageTuple: + return ReferenceImageTuple( + model_name=ipa.model.name if ipa.model else "", + image_name=_image_name(ipa.image), + weight=_effective_weight(ipa.weight, ipa.image_influence), + ) + + @staticmethod + def _refimage_to_tuple(ref: RefImage) -> ReferenceImageTuple: + cfg = ref.config + model_name = cfg.model.name if cfg.model else "" + return ReferenceImageTuple( + model_name=model_name, + image_name=_image_name(cfg.image), + weight=_effective_weight(cfg.weight, cfg.image_influence), + ) + + @staticmethod + def _canvas_reference_images( + canvas: CanvasV2Metadata, + ) -> list[ReferenceImageTuple]: + result: list[ReferenceImageTuple] = [] + for ref in canvas.reference_images or []: + if ref.is_enabled is False: + continue + ipa = ref.ip_adapter + result.append( + ReferenceImageTuple( + model_name=ipa.model.name if ipa.model else "", + image_name=_image_name(ipa.image), + weight=_effective_weight(ipa.weight, ipa.image_influence), + ) + ) + return result + + # ---- control layers -------------------------------------------------- + + @property + def control_layers(self) -> list[ControlLayerTuple]: + m = self.metadata + if isinstance(m, GenerationMetadata2): + return [] + if isinstance(m, GenerationMetadata3): + return [ + self._control_adapter_to_tuple(ca) + for ca in (m.controlnets or []) + ] + # v5 — prefer top-level control_layers, fall back to canvas_v2_metadata + if m.control_layers is not None and m.control_layers.layers: + return [ + self._v5_control_layer_to_tuple(layer) + for layer in m.control_layers.layers + if layer.is_enabled and layer.control_adapter is not None + ] + if m.canvas_v2_metadata is not None: + return self._canvas_control_layers(m.canvas_v2_metadata) + return [] + + @staticmethod + def _control_adapter_to_tuple(ca: ControlAdapter) -> ControlLayerTuple: + return ControlLayerTuple( + model_name=ca.model.name if ca.model else "", + image_name=_image_name(ca.image), + weight=ca.weight, + ) + + @staticmethod + def _v5_control_layer_to_tuple(layer: V5ControlLayer) -> ControlLayerTuple: + ca = layer.control_adapter + assert ca is not None # caller filters this + return ControlLayerTuple( + model_name=ca.model.name if ca.model else "", + image_name=_image_name(ca.image), + weight=_effective_weight(ca.weight, ca.image_influence), + ) + + @staticmethod + def _canvas_control_layers( + canvas: CanvasV2Metadata, + ) -> list[ControlLayerTuple]: + result: list[ControlLayerTuple] = [] + for layer in canvas.control_layers or []: + if not layer.is_enabled: + continue + ca = layer.control_adapter + image_names = [ + _image_name(obj.image) + for obj in layer.objects + if obj.image is not None + ] + image_names = [n for n in image_names if n] + result.append( + ControlLayerTuple( + model_name=ca.model.name if ca.model else "", + image_name=", ".join(image_names), + weight=ca.weight, + ) + ) + return result + + # ---- raster images --------------------------------------------------- + + @property + def raster_images(self) -> list[str]: + m = self.metadata + if not isinstance(m, GenerationMetadata5): + return [] + canvas = m.canvas_v2_metadata + if canvas is None or not canvas.raster_layers: + return [] + result: list[str] = [] + for layer in canvas.raster_layers: + if not layer.is_enabled: + continue + for obj in layer.objects: + name = _image_name(obj.image) if obj.image is not None else "" + if name: + result.append(name) + return result diff --git a/photomap/backend/metadata_modules/invoke_formatter.py b/photomap/backend/metadata_modules/invoke_formatter.py index bdc54473..cc313d83 100644 --- a/photomap/backend/metadata_modules/invoke_formatter.py +++ b/photomap/backend/metadata_modules/invoke_formatter.py @@ -1,134 +1,179 @@ """ backend.metadata_modules.invoke_formatter -Format metadata from invoke module, including human-readable tags. -Returns an HTML representation of the metadata. +Format InvokeAI generation metadata as HTML for the metadata drawer. + +This module is a thin HTML renderer over :class:`InvokeMetadataView`, which +provides a version-agnostic view over the Pydantic discriminated union of +v2 / v3 / v5 InvokeAI metadata. The formatter itself does not know anything +about the underlying metadata version layout. """ import logging +from collections.abc import Iterable from datetime import datetime from pathlib import Path -from typing import Any -from .invoke import Invoke3Metadata, Invoke5Metadata, InvokeLegacyMetadata +from pydantic import ValidationError + +from .invoke.invoke_metadata_view import ( + ControlLayerTuple, + InvokeMetadataView, + LoraTuple, + ReferenceImageTuple, +) +from .invokemetadata import GenerationMetadataAdapter from .slide_summary import SlideSummary logger = logging.getLogger(__name__) -def format_invoke_metadata(slide_data: SlideSummary, metadata: dict) -> SlideSummary: - """ - Format invoke metadata dictionary into an HTML string. +_COPY_SVG = ( + '' + '' + '" + "" +) - Args: - slide_data: SlideSummary containing the file name and path. - metadata (dict): Metadata dictionary containing invoke attributes. - Returns: - SlideSummary: structured metadata appropriate for an image with invoke data. +def format_invoke_metadata(slide_data: SlideSummary, metadata: dict) -> SlideSummary: + """Render InvokeAI metadata into an HTML table on ``slide_data.description``. + + Also populates ``slide_data.reference_images`` with the image names of any + reference images, control layer images, and raster images referenced by + the metadata — these are used by the drawer to render thumbnails. """ if not metadata: slide_data.description = "No invoke metadata available." return slide_data - # If all the values are simple scalars, then we're just going to return a table of key value pairs - # This is the case when an image was generated by Invoke using a workflow that manually - # collects values. + # Images produced by custom Invoke workflows sometimes carry a flat dict + # of hand-picked scalars. Render those as a plain key/value table. if all( isinstance(value, str | int | float | bool | type(None)) for value in metadata.values() ): - html = "" - for key, value in metadata.items(): - html += f"" - html += "
{key}{value}
" - slide_data.description = html + slide_data.description = _scalar_table(metadata) return slide_data - # pick the appropriate metadata class based on tags in the raw data - extractor_class = ( - Invoke5Metadata - if "canvas_v2_metadata" in metadata or "ref_images" in metadata - else ( - Invoke3Metadata - if "generation_mode" in metadata - else InvokeLegacyMetadata if "app_version" in metadata else None - ) - ) - - # get modification time for the file - modification_time = None - if mtime := ( - Path(slide_data.filepath).stat().st_mtime if slide_data.filepath else None - ): - modification_time = datetime.fromtimestamp(mtime).strftime("%Y-%m-%d %H:%M:%S") - - if not extractor_class: + try: + parsed = GenerationMetadataAdapter().parse(metadata) + except ValidationError as exc: + logger.warning("Failed to parse invoke metadata: %s", exc) slide_data.description = "Unknown invoke metadata format." return slide_data - extractor = extractor_class(raw_metadata=metadata) - positive_prompt = extractor.get_prompts().positive_prompt - negative_prompt = extractor.get_prompts().negative_prompt - model = extractor.get_model() - seed = extractor.get_seed() - loras = _format_list(extractor.get_loras()) - reference_images = extractor.get_reference_images() - reference_image_table = _format_list(reference_images) if reference_images else None - raster_images = extractor.get_raster_images() - control_layers = extractor.get_control_layers() - control_layer_table = _format_list(control_layers) if control_layers else None - - copy_svg = ( - '' - '' - '' - "" - ) + view = InvokeMetadataView(parsed) + + modification_time = _format_mtime(slide_data.filepath) + positive_prompt = view.positive_prompt + negative_prompt = view.negative_prompt + model = view.model_name + seed = view.seed + loras = view.loras + reference_images = view.reference_images + control_layers = view.control_layers + raster_images = view.raster_images - html = "" + rows: list[str] = [] if modification_time: - html += f"" - if positive_prompt: - html += f'' + rows.append(f"") + rows.append( + f'" + ) if negative_prompt: - html += f'' + rows.append( + f'" + ) if model: - html += f"" + rows.append(f"") if seed is not None: - html += f'' - if loras: - html += f"" + rows.append( + f'' + ) + if loras and (lora_html := _tuple_table(loras)): + rows.append(f"") if raster_images: - html += f"" - if reference_image_table: - html += f"" - if control_layer_table: - html += f"" - html += "
Date{modification_time}
Positive Prompt{positive_prompt}{copy_svg}
Date{modification_time}
Positive Prompt' + f"{positive_prompt}{_COPY_SVG if positive_prompt else ''}
Negative Prompt{negative_prompt}{copy_svg}
Negative Prompt' + f"{negative_prompt}{_COPY_SVG}
Model{model}
Model{model}
Seed{seed}{copy_svg}
Loras{loras}
Seed{seed}{_COPY_SVG}
Loras{lora_html}
Raster Images{', '.join(raster_images)}
Reference Images{reference_image_table}
Control Layers{control_layer_table}
" - - slide_data.description = html + rows.append( + f"Raster Images{', '.join(raster_images)}" + ) + if reference_images and (ref_html := _tuple_table(reference_images)): + rows.append(f"Reference Images{ref_html}") + if control_layers and (ctrl_html := _tuple_table(control_layers)): + rows.append(f"Control Layers{ctrl_html}") + + slide_data.description = "" + "".join(rows) + "
" slide_data.reference_images = [ - x.image_name for x in reference_images + control_layers - ] - slide_data.reference_images.extend(raster_images) + ri.image_name for ri in reference_images if ri.image_name + ] + [ + cl.image_name for cl in control_layers if cl.image_name + ] + list(raster_images) return slide_data -def _format_list(tuples: list[Any]) -> str | None: - """ - Format a list of tuples into an HTML table. - Args: - tuples (list): List of tuples, such as the Lora tuple defined in invoke_metadata_abc. - Returns: - str: HTML representation of the loras. +def _scalar_table(metadata: dict) -> str: + rows = "".join( + f"{key}{value}" for key, value in metadata.items() + ) + return f"{rows}
" + + +def _format_mtime(filepath: str | None) -> str | None: + if not filepath: + return None + try: + mtime = Path(filepath).stat().st_mtime + except (OSError, ValueError): + return None + return datetime.fromtimestamp(mtime).strftime("%Y-%m-%d %H:%M:%S") + + +def _tuple_table( + tuples: Iterable[LoraTuple | ReferenceImageTuple | ControlLayerTuple], +) -> str: + """Render a list of named tuples as a compact HTML table. + + Suppression rules: + + * **Row** — a row whose every field is empty (``None`` or ``""``) is dropped. + * **Column** — a column that is empty across *all* surviving rows is + dropped entirely, so a table of reference images with no IP adapter + model names, for example, renders without a model column at all. + * **Weight fallback** — when the ``weight`` column survives because at + least one row carries a weight, rows that are missing a weight are + rendered as ``1.0`` (the effective default InvokeAI uses when the + field is absent) rather than as a ragged empty cell. """ - if not tuples: - return - - html = "" - for tuple in tuples: - row = "".join([f"" for item in tuple]) - html += f"{row}" - html += "
{item}
" - return html + rows_data = [ + tup + for tup in tuples + if not all(v is None or v == "" for v in tup) + ] + if not rows_data: + return "" + + fields = rows_data[0]._fields + keep_column = [ + any(tup[idx] is not None and tup[idx] != "" for tup in rows_data) + for idx in range(len(fields)) + ] + + html_rows: list[str] = [] + for tup in rows_data: + cells: list[str] = [] + for idx, field_name in enumerate(fields): + if not keep_column[idx]: + continue + value = tup[idx] + if field_name == "weight" and (value is None or value == ""): + value = 1.0 + elif value is None: + value = "" + cells.append(f"{value}") + html_rows.append(f"{''.join(cells)}") + + return "" + "".join(html_rows) + "
" diff --git a/photomap/backend/metadata_modules/invokemetadata.py b/photomap/backend/metadata_modules/invokemetadata.py new file mode 100644 index 00000000..4ed57dd0 --- /dev/null +++ b/photomap/backend/metadata_modules/invokemetadata.py @@ -0,0 +1,55 @@ +""" +Wrapper for GenerationMetadata +""" + +from typing import Annotated, Any + +from pydantic import Field, TypeAdapter + +from .invoke.invoke2metadata import GenerationMetadata2 +from .invoke.invoke3metadata import GenerationMetadata3 +from .invoke.invoke5metadata import GenerationMetadata5 + +GenerationMetadata = Annotated[ + GenerationMetadata2 | GenerationMetadata3 | GenerationMetadata5, + Field(discriminator="metadata_version"), +] + + +class GenerationMetadataAdapter: + def __init__(self): + self.adapter = TypeAdapter(GenerationMetadata) + self.metadata = None + + def parse(self, json_data: dict[str, Any]) -> GenerationMetadata: + """ + Parse JSON data into a GenerationMetadata object. + + :param json_data: Dictionary containing metadata + :type json_data: dict[str, Any] + :return: Parsed generation metadata + :rtype: GenerationMetadata + """ + if "metadata_version" not in json_data: + if "canvas_v2_metadata" in json_data: + json_data = {"metadata_version": 5, **json_data} + elif "app_version" in json_data: + if any( + json_data["app_version"].startswith(x) for x in ["v1.", "2.", "v2."] + ): + json_data = {"metadata_version": 2, **json_data} + elif json_data["app_version"].startswith("3."): + if "model" in json_data and isinstance(json_data["model"], str): + json_data = {"metadata_version": 2, **json_data} + else: + json_data = {"metadata_version": 3, **json_data} + else: + json_data = {"metadata_version": 5, **json_data} + elif "model_weights" in json_data: + # v2 metadata has model_weights field + json_data = {"metadata_version": 2, **json_data} + else: + json_data = {"metadata_version": 3, **json_data} + + self.metadata = self.adapter.validate_python(json_data) + return self.metadata diff --git a/scripts/metadata_tests/dump_invoke_metadata.py b/scripts/metadata_tests/dump_invoke_metadata.py new file mode 100755 index 00000000..9e35da7c --- /dev/null +++ b/scripts/metadata_tests/dump_invoke_metadata.py @@ -0,0 +1,54 @@ +#!/usr/bin/env python3 +# +import json +import os +import sys +from pathlib import Path +from typing import Annotated, Any, Union + +from PIL import Image + + +def traverse_directory(directory: str) -> None: + """ + Traverse the specified directory to find PNG files and parse their + generation metadata from embedded JSON. + + Args: + directory: Path to the directory to traverse + """ + dir_path = Path(directory) + + for file_path in dir_path.rglob("*.png"): + parse_file(file_path) + + +def parse_file(file_path: Path) -> None: + """ + Retrieve the metadata from the PNG file and parse its + generation metadata from embedded JSON. + + Args: + file_path: Path to the file to parse + q""" + metadata_dict = {} + metadata_tags = ["invokeai_metadata", "Sd-metadata", "sd-metadata"] + with Image.open(file_path) as img: + for tag in metadata_tags: + if tag in img.info: + metadata_json = img.info[tag] + metadata_dict = json.loads(metadata_json) + print(f"## {file_path}") + print(json.dumps(metadata_dict, indent=4)) + + +if __name__ == "__main__": + path = sys.argv[1] if len(sys.argv) > 1 else "." + + if os.path.isfile(path): + parse_file(Path(path)) + elif os.path.isdir(path): + traverse_directory(path) + else: + print(f"Error: '{path}' is not a valid directory", file=sys.stderr) + sys.exit(1) diff --git a/scripts/metadata_tests/json2pydantic.py b/scripts/metadata_tests/json2pydantic.py new file mode 100755 index 00000000..2a0a22cb --- /dev/null +++ b/scripts/metadata_tests/json2pydantic.py @@ -0,0 +1,115 @@ +#!/usr/bin/env python3 +import json +import sys +from collections import defaultdict +from pathlib import Path +from typing import Any, Dict, Set + + +def json_to_pydantic_models( + json_data: Dict[str, Any], model_name: str = "Model" +) -> str: + """ + Convert a JSON object to Pydantic model class definitions. + + Args: + json_data: Dictionary containing the JSON structure + model_name: Name for the root model class + + Returns: + String containing Pydantic model class definitions + """ + models: Dict[str, Dict[str, str]] = {} + imports: Set[str] = {"from pydantic import BaseModel"} + + def infer_type(value: Any) -> str: + """Infer Python type from a JSON value.""" + if value is None: + return "Optional[Any]" + elif isinstance(value, bool): + return "bool" + elif isinstance(value, int): + return "int" + elif isinstance(value, float): + return "float" + elif isinstance(value, str): + return "str" + elif isinstance(value, list): + if value: + # Analyze first element to determine list type + item_type = infer_type(value[0]) + return f"List[{item_type}]" + return "List[Any]" + elif isinstance(value, dict): + return "Dict[str, Any]" + return "Any" + + def process_object(obj: Dict[str, Any], class_name: str) -> None: + """Recursively process JSON objects and create model classes.""" + fields: Dict[str, str] = {} + + for key, value in obj.items(): + if isinstance(value, dict): + # Create nested model + nested_class_name = "".join( + word.capitalize() for word in key.split("_") + ) + process_object(value, nested_class_name) + fields[key] = nested_class_name + elif isinstance(value, list) and value and isinstance(value[0], dict): + # List of objects - create model for first item + item_class_name = "".join( + word.capitalize() for word in key.rstrip("s").split("_") + ) + process_object(value[0], item_class_name) + fields[key] = f"List[{item_class_name}]" + else: + fields[key] = infer_type(value) + + models[class_name] = fields + + # Process the root object + process_object(json_data, model_name) + + # Add Optional import if needed + if any( + "Optional" in field_type + for fields in models.values() + for field_type in fields.values() + ): + imports.add("from typing import Optional") + + # Add List, Dict, Any imports if needed + all_types = "".join(str(fields) for fields in models.values()) + if "List[" in all_types: + imports.add("from typing import List") + if "Dict[" in all_types: + imports.add("from typing import Dict") + if "Any" in all_types: + imports.add("from typing import Any") + + # Generate output + output = "\n".join(sorted(imports)) + "\n\n" + + # Output models in dependency order (nested first) + for class_name in sorted(models.keys()): + fields = models[class_name] + output += f"class {class_name}(BaseModel):\n" + for field_name, field_type in sorted(fields.items()): + output += f" {field_name}: {field_type}\n" + output += "\n" + + return output + + +# Example usage +if __name__ == "__main__": + # read JSON from a file passed on the command line + if len(sys.argv) != 2: + print("Usage: python json2pydantic.py ") + sys.exit(1) + json_file_path = Path(sys.argv[1]) + with json_file_path.open("r", encoding="utf-8") as f: + example_json = json.load(f) + pydantic_code = json_to_pydantic_models(example_json, "GenerationMetadata") + print(pydantic_code) diff --git a/scripts/metadata_tests/parse_invoke_metadata.py b/scripts/metadata_tests/parse_invoke_metadata.py new file mode 100755 index 00000000..4d09f124 --- /dev/null +++ b/scripts/metadata_tests/parse_invoke_metadata.py @@ -0,0 +1,66 @@ +#!/usr/bin/env python3 +import json +import os +import sys +from pathlib import Path +from typing import Annotated, Any, Union + +from PIL import Image +from pydantic import Field, TypeAdapter + +from photomap.backend.metadata_modules.invokemetadata import ( + GenerationMetadata, + GenerationMetadataAdapter, +) + + +def traverse_directory(directory: str) -> None: + """ + Traverse the specified directory to find PNG files and parse their + generation metadata from embedded JSON. + + Args: + directory: Path to the directory to traverse + """ + dir_path = Path(directory) + + for file_path in dir_path.rglob("*.png"): + parse_file(file_path) + + +def parse_file(file_path: Path) -> None: + """ + Retrieve the metadata from the PNG file and parse its + generation metadata from embedded JSON. + + Args: + file_path: Path to the file to parse +q """ + metadata_dict = {} + try: + metadata_tags = ["invokeai_metadata", "Sd-metadata", "sd-metadata"] + metadata_adapter = GenerationMetadataAdapter() + with Image.open(file_path) as img: + for tag in metadata_tags: + if tag in img.info: + metadata_json = img.info[tag] + metadata_dict = json.loads(metadata_json) + generation_metadata = metadata_adapter.parse(metadata_dict) + print(f"## File: {file_path}") + print(generation_metadata.model_dump_json(indent=4)) + except Exception as e: + print(f"Error processing file {file_path}: {e}", file=sys.stderr) + if metadata_dict is not None: + print(f"Raw data = {json.dumps(metadata_dict, indent=4)}", file=sys.stderr) + + +if __name__ == "__main__": + path = sys.argv[1] if len(sys.argv) > 1 else "." + + if os.path.isfile(path): + parse_file(Path(path)) + elif os.path.isdir(path): + traverse_directory(path) + else: + print(f"Error: '{path}' is not a valid directory", file=sys.stderr) + sys.exit(1) diff --git a/scripts/metadata_tests/parse_invoke_metadata_from_file.py b/scripts/metadata_tests/parse_invoke_metadata_from_file.py new file mode 100755 index 00000000..414ec531 --- /dev/null +++ b/scripts/metadata_tests/parse_invoke_metadata_from_file.py @@ -0,0 +1,84 @@ +#!/usr/bin/env python3 +import argparse +import json +import os +import sys +from itertools import groupby +from pathlib import Path +from typing import Annotated, Any, Generator, Union + +from PIL import Image +from pydantic import Field, TypeAdapter + +from photomap.backend.metadata_modules.invokemetadata import GenerationMetadataAdapter + + +def read_records(file_path: Path) -> Generator[tuple[str, dict[str, Any]], None, None]: + """ + Read records from a file where each record starts with ## filename. + + Args: + file_path: Path to the file containing records + + Yields: + Tuple of (filename, json_data) for each record + """ + with open(file_path) as f: + filename = "" + for is_header, group in groupby(f, key=lambda line: line.startswith("##")): + if is_header: + filename = next(group).strip()[2:].strip() + else: + try: + json_data = json.loads("".join(group)) + yield filename, json_data + except json.JSONDecodeError as e: + print(f"Error parsing JSON for {filename}: {e}", file=sys.stderr) + + +def parse_file(file_path: Path, print_parse: bool = False) -> None: + """ + Retrieve the metadata from the PNG file and parse its + generation metadata from embedded JSON. + + Args: + file_path: Path to the file to parse + print_parse: If True, pretty-print serialized metadata on successful parse + """ + metadata_dict = {} + metadata_adapter = GenerationMetadataAdapter() + for filename, metadata_dict in read_records(file_path): + try: + generation_metadata = metadata_adapter.parse(metadata_dict) + print(f"## {filename}: successfully parsed") + if print_parse: + print(generation_metadata.model_dump_json(indent=4)) + except BrokenPipeError: + raise + except Exception as e: + print(f"## {filename}: parse failed: {e}", file=sys.stderr) + if metadata_dict is not None: + print( + f"Raw data = {json.dumps(metadata_dict, indent=4)}", file=sys.stderr + ) + + +if __name__ == "__main__": + parser = argparse.ArgumentParser( + description="Parse Invoke generation metadata from a file" + ) + parser.add_argument("file", help="Path to the file to parse") + parser.add_argument( + "--print_parse", + action="store_true", + help="Pretty-print serialized metadata on successful parse", + ) + + args = parser.parse_args() + path = args.file + + if os.path.isfile(path): + parse_file(Path(path), print_parse=args.print_parse) + else: + print(f"Error: '{path}' is not a valid file", file=sys.stderr) + sys.exit(1) diff --git a/tests/backend/test_invoke_metadata.py b/tests/backend/test_invoke_metadata.py new file mode 100644 index 00000000..78aff1ba --- /dev/null +++ b/tests/backend/test_invoke_metadata.py @@ -0,0 +1,819 @@ +"""Regression tests for the refactored InvokeAI metadata pipeline. + +Two layers are covered: + +1. ``InvokeMetadataView`` — the version-agnostic wrapper that mediates + between the Pydantic discriminated union (v2 / v3 / v5) and the + ``invoke_formatter``. Each logical field (prompts, model, seed, loras, + reference images, control layers, raster images) is asserted against + every metadata version that meaningfully carries it. +2. ``format_invoke_metadata`` — the HTML renderer consumed by + ``metadata-drawer``. Tests assert the rendered HTML contains the + expected table rows and that ``slide_data.reference_images`` is + populated for downstream consumers. +""" + +from __future__ import annotations + +import pytest + +from photomap.backend.metadata_modules.invoke.invoke_metadata_view import ( + ControlLayerTuple, + InvokeMetadataView, + LoraTuple, + ReferenceImageTuple, +) +from photomap.backend.metadata_modules.invoke_formatter import format_invoke_metadata +from photomap.backend.metadata_modules.invokemetadata import GenerationMetadataAdapter +from photomap.backend.metadata_modules.slide_summary import SlideSummary + +# --------------------------------------------------------------------------- +# Fixtures +# --------------------------------------------------------------------------- + + +def _slide() -> SlideSummary: + return SlideSummary(filename="sample.png", filepath="") + + +def _view(data: dict) -> InvokeMetadataView: + return InvokeMetadataView(GenerationMetadataAdapter().parse(data)) + + +@pytest.fixture +def v2_scalar_prompt_metadata() -> dict: + """Legacy InvokeAI v2 with a scalar prompt and model_weights.""" + return { + "app_id": "invoke-ai/InvokeAI-Stable-Diffusion", + "app_version": "2.3.5", + "model": "stable_diffusion_v1_5", + "model_weights": "stable-diffusion-1.5", + "model_hash": "abc123", + "image": { + "prompt": "a mountain landscape", + "seed": 1234, + "cfg_scale": 7.5, + "height": 512, + "width": 768, + "sampler": "k_euler_a", + "steps": 20, + "type": "txt2img", + "postprocessing": None, + }, + } + + +@pytest.fixture +def v2_list_prompt_metadata() -> dict: + """Very old InvokeAI v2 with the list-of-weighted-prompts variant.""" + return { + "app_id": "invoke-ai/InvokeAI-Stable-Diffusion", + "app_version": "2.2.0", + "model": "legacy_model", + "model_hash": "xyz", + "image": { + "prompt": [{"prompt": "old style prompt", "weight": 1.0}], + "seed": 999, + "cfg_scale": 7.0, + "height": 512, + "width": 512, + "sampler": "k_lms", + "steps": 10, + "type": "txt2img", + "postprocessing": None, + }, + } + + +@pytest.fixture +def v3_metadata() -> dict: + """InvokeAI v3 with LoRAs, IP adapters, and controlnets.""" + return { + "metadata_version": 3, + "app_version": "3.5.0", + "generation_mode": "txt2img", + "positive_prompt": "a cat riding a skateboard", + "negative_prompt": "blurry, low quality", + "seed": 42, + "model": {"model_name": "dreamshaper", "base_model": "sd-1"}, + "loras": [ + {"lora": {"model_name": "detail_lora"}, "weight": 0.8}, + {"lora": {"model_name": "style_lora"}, "weight": 0.5}, + ], + "ipAdapters": [ + { + "ip_adapter_model": {"model_name": "ip_adapter_sd15"}, + "image": {"image_name": "ref.png"}, + "weight": 0.5, + } + ], + "controlnets": [ + { + "control_model": {"model_name": "canny_sd15"}, + "image": {"image_name": "control.png"}, + "control_weight": 0.7, + "beginEndStepPct": [0.0, 1.0], + "control_mode": "balanced", + } + ], + } + + +@pytest.fixture +def v5_ref_images_metadata() -> dict: + """Modern InvokeAI v5 using the ``ref_images`` field.""" + return { + "metadata_version": 5, + "app_version": "5.6.0", + "positive_prompt": "a dog in a hat", + "negative_prompt": "ugly, deformed", + "seed": 100, + "model": {"name": "flux-schnell", "base": "flux"}, + "loras": [{"lora": {"name": "dog_lora"}, "weight": 0.6}], + "ref_images": [ + { + "id": "r1", + "isEnabled": True, + "config": { + "type": "ipAdapter", + "model": {"name": "ipa-flux"}, + "image": {"image_name": "ref1.png"}, + "weight": 0.7, + }, + }, + { + "id": "r2", + "isEnabled": False, # should be filtered out + "config": { + "type": "ipAdapter", + "model": {"name": "ipa-flux"}, + "image": {"image_name": "ref2.png"}, + "weight": 0.3, + }, + }, + ], + "control_layers": { + "version": 1, + "layers": [ + { + "id": "cl1", + "type": "control_layer", + "isEnabled": True, + "isSelected": False, + "ipAdapter": { + "model": {"name": "canny_flux"}, + "image": {"image_name": "edges.png"}, + "weight": 0.9, + }, + }, + { + "id": "cl2", + "type": "control_layer", + "isEnabled": False, # should be filtered out + "isSelected": False, + "ipAdapter": { + "model": {"name": "depth_flux"}, + "image": {"image_name": "depth.png"}, + "weight": 0.5, + }, + }, + ], + }, + } + + +@pytest.fixture +def v5_canvas_metadata() -> dict: + """InvokeAI v5 using the older ``canvas_v2_metadata`` layout.""" + return { + "app_version": "5.0.0", + "positive_prompt": "a bird over the ocean", + "negative_prompt": "blurry", + "seed": 7, + "model": {"name": "sdxl", "base": "sdxl"}, + "canvas_v2_metadata": { + "rasterLayers": [ + { + "id": "rl1", + "isEnabled": True, + "isLocked": False, + "name": None, + "objects": [ + {"id": "o1", "type": "image", + "image": {"image_name": "raster1.png"}}, + {"id": "o2", "type": "image", + "image": {"image_name": "raster2.png"}}, + ], + "opacity": 1.0, + "position": {"x": 0, "y": 0}, + "type": "raster_layer", + }, + { + "id": "rl2", + "isEnabled": False, # should be filtered out + "isLocked": False, + "name": None, + "objects": [ + {"id": "o3", "type": "image", + "image": {"image_name": "raster_disabled.png"}}, + ], + "opacity": 1.0, + "position": {"x": 0, "y": 0}, + "type": "raster_layer", + }, + ], + "referenceImages": [ + { + "id": "ri1", + "isEnabled": True, + "isLocked": False, + "name": None, + "ipAdapter": { + "model": {"name": "ipa-sdxl"}, + "image": {"image_name": "ip_ref.png"}, + "weight": 0.5, + }, + }, + ], + "controlLayers": [ + { + "id": "cl1", + "isEnabled": True, + "isLocked": False, + "name": None, + "objects": [ + {"id": "co1", "type": "image", + "image": {"image_name": "cl_img.png"}}, + ], + "opacity": 1.0, + "position": {"x": 0, "y": 0}, + "type": "control_layer", + "withTransparencyEffect": False, + "controlAdapter": { + "control_model": {"name": "canny_sdxl"}, + "image": {"image_name": "cl_img.png"}, + "control_weight": 0.6, + "beginEndStepPct": [0.0, 1.0], + "controlMode": "balanced", + }, + }, + ], + }, + } + + +# --------------------------------------------------------------------------- +# InvokeMetadataView — v2 +# --------------------------------------------------------------------------- + + +class TestInvokeMetadataViewV2: + def test_scalar_prompt(self, v2_scalar_prompt_metadata): + view = _view(v2_scalar_prompt_metadata) + assert view.positive_prompt == "a mountain landscape" + assert view.negative_prompt == "" + # model_weights takes precedence over plain model + assert view.model_name == "stable-diffusion-1.5" + assert view.seed == 1234 + + def test_list_prompt_takes_first_entry(self, v2_list_prompt_metadata): + view = _view(v2_list_prompt_metadata) + assert view.positive_prompt == "old style prompt" + assert view.negative_prompt == "" + # no model_weights — falls back to model + assert view.model_name == "legacy_model" + assert view.seed == 999 + + def test_empty_collections(self, v2_scalar_prompt_metadata): + view = _view(v2_scalar_prompt_metadata) + assert view.loras == [] + assert view.reference_images == [] + assert view.control_layers == [] + assert view.raster_images == [] + + +# --------------------------------------------------------------------------- +# InvokeMetadataView — v3 +# --------------------------------------------------------------------------- + + +class TestInvokeMetadataViewV3: + def test_prompts_model_seed(self, v3_metadata): + view = _view(v3_metadata) + assert view.positive_prompt == "a cat riding a skateboard" + assert view.negative_prompt == "blurry, low quality" + assert view.model_name == "dreamshaper" + assert view.seed == 42 + + def test_loras(self, v3_metadata): + view = _view(v3_metadata) + assert view.loras == [ + LoraTuple(model_name="detail_lora", weight=0.8), + LoraTuple(model_name="style_lora", weight=0.5), + ] + + def test_reference_images_from_ip_adapters(self, v3_metadata): + view = _view(v3_metadata) + assert view.reference_images == [ + ReferenceImageTuple( + model_name="ip_adapter_sd15", + image_name="ref.png", + weight=0.5, + ) + ] + + def test_control_layers_from_controlnets(self, v3_metadata): + view = _view(v3_metadata) + assert view.control_layers == [ + ControlLayerTuple( + model_name="canny_sd15", + image_name="control.png", + weight=0.7, + ) + ] + + def test_no_raster_images(self, v3_metadata): + view = _view(v3_metadata) + assert view.raster_images == [] + + +# --------------------------------------------------------------------------- +# InvokeMetadataView — v5 (ref_images path) +# --------------------------------------------------------------------------- + + +class TestInvokeMetadataViewV5RefImages: + def test_prompts_model_seed(self, v5_ref_images_metadata): + view = _view(v5_ref_images_metadata) + assert view.positive_prompt == "a dog in a hat" + assert view.negative_prompt == "ugly, deformed" + assert view.model_name == "flux-schnell" + assert view.seed == 100 + + def test_loras(self, v5_ref_images_metadata): + view = _view(v5_ref_images_metadata) + assert view.loras == [LoraTuple(model_name="dog_lora", weight=0.6)] + + def test_ref_images_skips_disabled(self, v5_ref_images_metadata): + view = _view(v5_ref_images_metadata) + assert view.reference_images == [ + ReferenceImageTuple( + model_name="ipa-flux", image_name="ref1.png", weight=0.7 + ) + ] + + def test_control_layers_skips_disabled(self, v5_ref_images_metadata): + view = _view(v5_ref_images_metadata) + assert view.control_layers == [ + ControlLayerTuple( + model_name="canny_flux", image_name="edges.png", weight=0.9 + ) + ] + + def test_no_raster_images_without_canvas(self, v5_ref_images_metadata): + view = _view(v5_ref_images_metadata) + assert view.raster_images == [] + + +# --------------------------------------------------------------------------- +# InvokeMetadataView — v5 (canvas_v2_metadata path) +# --------------------------------------------------------------------------- + + +class TestInvokeMetadataViewV5Canvas: + def test_prompts_model_seed(self, v5_canvas_metadata): + view = _view(v5_canvas_metadata) + assert view.positive_prompt == "a bird over the ocean" + assert view.negative_prompt == "blurry" + assert view.model_name == "sdxl" + assert view.seed == 7 + + def test_reference_images_from_canvas(self, v5_canvas_metadata): + view = _view(v5_canvas_metadata) + assert view.reference_images == [ + ReferenceImageTuple( + model_name="ipa-sdxl", image_name="ip_ref.png", weight=0.5 + ) + ] + + def test_control_layers_join_object_images(self, v5_canvas_metadata): + view = _view(v5_canvas_metadata) + assert view.control_layers == [ + ControlLayerTuple( + model_name="canny_sdxl", image_name="cl_img.png", weight=0.6 + ) + ] + + def test_raster_images_skip_disabled_layers(self, v5_canvas_metadata): + view = _view(v5_canvas_metadata) + assert view.raster_images == ["raster1.png", "raster2.png"] + + +# --------------------------------------------------------------------------- +# format_invoke_metadata — end-to-end HTML rendering +# --------------------------------------------------------------------------- + + +class TestFormatInvokeMetadata: + def test_empty_metadata_returns_placeholder(self): + slide = _slide() + result = format_invoke_metadata(slide, {}) + assert "No invoke metadata available" in result.description + + def test_scalar_only_metadata_uses_key_value_table(self): + slide = _slide() + metadata = { + "Custom Field": "value", + "Seed": 1234, + "Steps": 20, + } + result = format_invoke_metadata(slide, metadata) + assert "" in result.description + assert "" in result.description + assert "" in result.description + + def test_v2_renders_prompt_model_seed(self, v2_scalar_prompt_metadata): + slide = _slide() + result = format_invoke_metadata(slide, v2_scalar_prompt_metadata) + html = result.description + assert "
Custom FieldvalueSeed1234
" in html + assert "Positive Prompt" in html + assert "a mountain landscape" in html + assert "stable-diffusion-1.5" in html + assert "1234" in html + # No loras/controls/references in v2 + assert "Loras" not in html + assert "Reference Images" not in html + assert "Control Layers" not in html + assert result.reference_images == [] + + def test_v3_renders_all_sections(self, v3_metadata): + slide = _slide() + result = format_invoke_metadata(slide, v3_metadata) + html = result.description + for snippet in [ + "Positive Prompt", + "a cat riding a skateboard", + "Negative Prompt", + "blurry, low quality", + "", + "", + "42", + "Loras", + "detail_lora", + "style_lora", + "Reference Images", + "ip_adapter_sd15", + "ref.png", + "Control Layers", + "canny_sd15", + "control.png", + ]: + assert snippet in html, f"missing {snippet!r} in rendered HTML" + # Reference images collected for downstream thumbnail rendering + assert "ref.png" in result.reference_images + assert "control.png" in result.reference_images + + def test_v5_ref_images_path(self, v5_ref_images_metadata): + slide = _slide() + result = format_invoke_metadata(slide, v5_ref_images_metadata) + html = result.description + assert "a dog in a hat" in html + assert "flux-schnell" in html + assert "ipa-flux" in html + assert "ref1.png" in html + # disabled ref image must not appear + assert "ref2.png" not in html + # control layer (enabled) rendered; disabled one skipped + assert "canny_flux" in html + assert "depth_flux" not in html + assert "ref1.png" in result.reference_images + assert "edges.png" in result.reference_images + + def test_v5_canvas_path_raster_images(self, v5_canvas_metadata): + slide = _slide() + result = format_invoke_metadata(slide, v5_canvas_metadata) + html = result.description + assert "Raster Images" in html + assert "raster1.png" in html + assert "raster2.png" in html + # Disabled raster layer filtered out + assert "raster_disabled.png" not in html + # Canvas reference image + control layer rows present + assert "ipa-sdxl" in html + assert "ip_ref.png" in html + assert "canny_sdxl" in html + # reference_images is ref + control layer image names, plus raster + assert "ip_ref.png" in result.reference_images + assert "cl_img.png" in result.reference_images + assert "raster1.png" in result.reference_images + assert "raster2.png" in result.reference_images + + +# --------------------------------------------------------------------------- +# Empty-field handling in the rendered HTML +# --------------------------------------------------------------------------- + + +class TestFormatInvokeMetadataEmptyFields: + """Tweaks to how the formatter handles empty / missing field values.""" + + def test_empty_negative_prompt_suppresses_row(self): + """A blank negative_prompt should not produce a Negative Prompt row.""" + metadata = { + "metadata_version": 3, + "app_version": "3.5.0", + "positive_prompt": "anything", + "negative_prompt": "", + "seed": 1, + "model": {"model_name": "m"}, + } + html = format_invoke_metadata(_slide(), metadata).description + assert "Negative Prompt" not in html + + def test_empty_positive_prompt_suppresses_copy_icon_only(self): + """Blank positive_prompt keeps the row but drops the copy icon.""" + metadata = { + "metadata_version": 3, + "app_version": "3.5.0", + "positive_prompt": "", + "negative_prompt": "dont want this", + "seed": 1, + "model": {"model_name": "m"}, + } + html = format_invoke_metadata(_slide(), metadata).description + # Row is still present so the drawer always shows a positive prompt slot + assert "" in html + # But the copy icon (identifiable by its class) must NOT be on that row + # Extract just the Positive Prompt row to check + start = html.index("") + end = html.index("", start) + positive_row = html[start:end] + assert "copy-icon" not in positive_row + # The Negative Prompt row does still carry a copy icon (sanity check + # that we haven't accidentally dropped icons everywhere) + assert "Negative Prompt" in html + neg_start = html.index("") + neg_end = html.index("", neg_start) + assert "copy-icon" in html[neg_start:neg_end] + + def test_reference_image_row_omits_empty_model_and_weight(self): + """Tuple cells for missing model / weight should be dropped.""" + metadata = { + "metadata_version": 5, + "app_version": "5.6.0", + "positive_prompt": "x", + "seed": 1, + "model": {"name": "m"}, + "ref_images": [ + { + "id": "r1", + "isEnabled": True, + "config": { + "type": "ipAdapter", + # no model, no weight + "image": {"image_name": "only_image.png"}, + }, + } + ], + } + html = format_invoke_metadata(_slide(), metadata).description + assert "Reference Images" in html + assert "only_image.png" in html + # Extract the inner tuple-table row and assert it has exactly one " in tuple_table + + def test_flux_redux_image_influence_shown_in_weight_column(self): + """Flux Redux reference images don't carry a numeric weight — they + use an ``imageInfluence`` categorical like "Medium". That value + should surface in the weight column when no numeric weight exists. + """ + metadata = { + "metadata_version": 5, + "app_version": "5.6.0", + "positive_prompt": "x", + "seed": 1, + "model": {"name": "m"}, + "ref_images": [ + { + "id": "r1", + "isEnabled": True, + "config": { + "type": "ipAdapter", + "model": {"name": "flux_redux"}, + "image": {"image_name": "redux.png"}, + "imageInfluence": "Medium", + # no numeric weight + }, + } + ], + } + html = format_invoke_metadata(_slide(), metadata).description + start = html.index("
ModeldreamshaperSeedPositive PromptPositive Prompt
Negative Prompt
+ start = html.index("") + end = html.index("
", start) + tuple_table = html[start:end] + assert tuple_table.count("		") == 1 + assert "only_image.png
") + end = html.index("
", start) + tuple_table = html[start:end] + assert "Medium" in tuple_table + # And the formatter must NOT have replaced "Medium" with 1.0 + assert "1.0" not in tuple_table + + def test_flux_redux_image_influence_v3_ip_adapters(self): + """Same fallback applies to v3-style ipAdapters entries.""" + metadata = { + "metadata_version": 3, + "app_version": "3.5.0", + "positive_prompt": "x", + "seed": 1, + "model": {"model_name": "m"}, + "ipAdapters": [ + { + "ip_adapter_model": {"model_name": "flux_redux"}, + "image": {"image_name": "redux.png"}, + "imageInfluence": "High", + # no numeric weight + } + ], + } + html = format_invoke_metadata(_slide(), metadata).description + assert "High" in html + assert "flux_redux" in html + + def test_numeric_weight_wins_over_image_influence(self): + """When both weight and imageInfluence are present, the numeric + weight takes precedence — it's the more specific signal. + """ + metadata = { + "metadata_version": 5, + "app_version": "5.6.0", + "positive_prompt": "x", + "seed": 1, + "model": {"name": "m"}, + "ref_images": [ + { + "id": "r1", + "isEnabled": True, + "config": { + "type": "ipAdapter", + "model": {"name": "ipa"}, + "image": {"image_name": "ref.png"}, + "weight": 0.6, + "imageInfluence": "Medium", + }, + } + ], + } + html = format_invoke_metadata(_slide(), metadata).description + assert "0.6" in html + assert "Medium" not in html + + def test_mixed_weight_column_defaults_missing_weights_to_1_0(self): + """Regression for the ragged-row bug: when some ref_images have an + explicit weight and others don't, every row renders with a weight + cell and missing weights default to 1.0 instead of leaving a blank + column on the first row. + """ + metadata = { + "metadata_version": 5, + "app_version": "5.6.0", + "positive_prompt": "x", + "seed": 1, + "model": {"name": "m"}, + "ref_images": [ + { + "id": "r1", + "isEnabled": True, + "config": { + "type": "ipAdapter", + "image": {"image_name": "first.png"}, + # no weight — should render as 1.0 in a surviving column + }, + }, + { + "id": "r2", + "isEnabled": True, + "config": { + "type": "ipAdapter", + "image": {"image_name": "second.png"}, + "weight": 0.7, + }, + }, + ], + } + html = format_invoke_metadata(_slide(), metadata).description + start = html.index("") + end = html.index("
", start) + tuple_table = html[start:end] + # model_name column is empty on every row → dropped. + # Two surviving columns (image, weight) × two rows → 4 s total. + assert tuple_table.count("") == 2 + assert tuple_table.count("") == 4 + assert "first.png" in tuple_table + assert "second.png" in tuple_table + assert "1.0" in tuple_table + assert "0.7" in tuple_table + + def test_ref_images_list_of_lists_is_flattened(self): + """Some legacy metadata wraps ref_images in an outer list. The v5 + model validator should flatten it before parsing. + """ + metadata = { + "metadata_version": 5, + "app_version": "5.6.0", + "positive_prompt": "x", + "seed": 1, + "model": {"name": "m"}, + "ref_images": [ + [ + { + "id": "r1", + "isEnabled": True, + "config": { + "type": "ipAdapter", + "model": {"name": "ipa"}, + "image": {"image_name": "nested.png"}, + "weight": 0.5, + }, + }, + ] + ], + } + html = format_invoke_metadata(_slide(), metadata).description + assert "nested.png" in html + assert "ipa" in html + + def test_ref_images_unwrap_nested_original_image(self): + """Old metadata may nest the image under ``config.image.original.image``. + The v5 validator should collapse that to ``config.image`` before parsing. + """ + metadata = { + "metadata_version": 5, + "app_version": "5.6.0", + "positive_prompt": "x", + "seed": 1, + "model": {"name": "m"}, + "ref_images": [ + { + "id": "r1", + "isEnabled": True, + "config": { + "type": "ipAdapter", + "model": {"name": "ipa"}, + "image": { + "original": {"image": {"image_name": "deep.png"}} + }, + "weight": 0.5, + }, + } + ], + } + html = format_invoke_metadata(_slide(), metadata).description + assert "deep.png" in html + + def test_control_layer_row_skipped_when_fully_empty(self): + """A row with every cell empty should be dropped from the tuple table.""" + metadata = { + "metadata_version": 5, + "app_version": "5.6.0", + "positive_prompt": "x", + "seed": 1, + "model": {"name": "m"}, + "control_layers": { + "version": 1, + "layers": [ + { + "id": "cl1", + "type": "control_layer", + "isEnabled": True, + "isSelected": False, + "ipAdapter": { + # model name empty, image name empty, no weight + "model": {"name": ""}, + "image": {"image_name": ""}, + }, + }, + { + "id": "cl2", + "type": "control_layer", + "isEnabled": True, + "isSelected": False, + "ipAdapter": { + "model": {"name": "canny"}, + "image": {"image_name": "e.png"}, + "weight": 0.5, + }, + }, + ], + }, + } + html = format_invoke_metadata(_slide(), metadata).description + # The inner tuple table should contain exactly one — the + # "all empty" layer is dropped and only the "canny" layer survives. + start = html.index("") + end = html.index("
", start) + tuple_table = html[start:end] + assert tuple_table.count("") == 1 + assert "canny" in tuple_table + assert "e.png" in tuple_table