Skip to content

docs(architectures): document the registry and add a scaffolder for new architectures - #33

Draft
Pfannkuchensack wants to merge 1 commit into
refactor/arch-loader-flagsfrom
refactor/arch-docs-scaffolder
Draft

docs(architectures): document the registry and add a scaffolder for new architectures#33
Pfannkuchensack wants to merge 1 commit into
refactor/arch-loader-flagsfrom
refactor/arch-docs-scaffolder

Conversation

@Pfannkuchensack

Copy link
Copy Markdown
Collaborator

Stacked on #32#31#30#29#28#27. Base is refactor/arch-loader-flags.

Last of the series from .ideas/Backend Modularization Plan.md. No behaviour changes — docs and
a script.

The guide was a memory test

Its checklist never mentioned step_callback.py, dependencies.py's safe_globals list, or the
AnyVariant widening. That is precisely why forgetting them was easy — and none of those failed at
boot. They failed on the first generation that happened to use the new architecture.

All three are now either derived from the registry or checked in CI, so the guide can stop asking
people to remember them. The new section says what moved, and when each mistake used to surface:

Fact Was Now
Preview factors, spatial scale an elif chain in step_callback.py LatentSpaceFacet
safe_globals for conditioning a hand-kept list in dependencies.py derived from ConditioningFacet
Variant enum four hand-widened unions VariantFacet, all four CI-checked
fp8 storage exceptions config.base == inside the generic loader LoaderFlagsFacet

Two stale things fixed

§10 pointed at a file that no longer exists (starter_models.py). It now describes the package,
says shared components belong in common.py, and carries the constraint that shaped that split:
STARTER_MODELS order is what users see and is not sorted, so entries go where they belong, not
at the end.

The file tree was missing files a new architecture actually touchesfields.py,
primitives.py, conditioning_data.py, invocation_api. That list was derived from what the last
real architecture addition (ERNIE-Image, invoke-ai#9115) edited — 22 backend files — filtered against what
PRs 0–6 absorbed, rather than written from memory.

Worth noting from that derivation: ERNIE's load_default.py edit was not a base special case, it
added two SubModelType exclusions, which stay generic. And its hotfixes.py change was an
unrelated drive-by. Neither belongs in the guide.

scripts/new_architecture.py

Generates the two files that can be generated, inserts the import line that makes the definition load,
and prints the residual edits with the reason each cannot be generated. Dry run by default:

$ python scripts/new_architecture.py --enum-name NewModel --enum-value new-model
  would write  invokeai/backend/architectures/defs/new_model.py
  would write  invokeai/backend/model_manager/starter_models/new_model.py
  would patch  invokeai/backend/architectures/__init__.py  ->  new_model,  # noqa: F401

Still to do by hand -- these cannot be generated:
  ...

The templates are tested against reality, not themselves

A generator that drifts from what the registry expects is worse than none: it produces files that
look right and fail at boot. So the tests bind it to the fifteen architectures that already exist —
the import line it would emit must be the line already in architectures/__init__.py, and the module
path it computes must match registry.defs_module_path().

One test asserts the template scaffolds every facet that declares itself REQUIRED, so adding a
required facet without updating the scaffolder fails here rather than producing definition modules
that cannot boot.

Starter models turned out to be optional — SD2 ships none — so that test skips where there is no
module, with a second test pinning SD2 as the only such case so the skip cannot quietly spread.

Verification

  • pytest tests/test_new_architecture_scaffold.py tests/backend/architectures tests/test_docs_json_export.py — 240 passed, 1 skipped
  • ruff@0.11.2 check . + format --check — clean
  • openapi.json — identical.
  • Scaffolder dry run leaves the working tree untouched (asserted in a test, and checked by hand).

Not run: the docs site build. docs/node_modules is absent and installing it needs network. The
MDX was checked by hand instead — it uses only the two components the file already imports
(Steps, FileTree), and every aside type used (note, tip, caution) is already used elsewhere
in these docs. Worth a pnpm build in docs/ before merge.

🤖 Generated with Claude Code

…ew architectures

The integration guide described the world before the registry, and described it
incompletely. Its checklist never mentioned step_callback.py, dependencies.py's
safe_globals list, or the AnyVariant widening -- which is exactly why forgetting
them was easy. Those three are now either derived from the registry or checked in
CI, so the guide can stop being a memory test.

What the guide gains is a section on the one file that now holds most of it,
architectures/defs/<base>.py, with a table of what moved and, more importantly,
of when each mistake used to surface. That is the point of the change: none of
those omissions failed at boot. They failed on the first generation that happened
to use the new architecture.

The starter-models section was stale in a way that would have sent someone to a
file that no longer exists. It now describes the package, says where shared
components belong, and warns that the order of STARTER_MODELS is what users see
and is not sorted -- the constraint that shaped that split.

The file tree gained the files a new architecture actually touches. It was
missing fields.py, primitives.py, conditioning_data.py and invocation_api, all of
which the last real architecture addition (ERNIE-Image, invoke-ai#9115) edited. That list
was derived from that commit rather than from memory, then filtered against what
PRs 0-6 absorbed.

scripts/new_architecture.py generates the two files that can be generated -- the
definition module and the starter-models module -- inserts the import line that
makes the first one load, and prints the residual edits with the reason each one
cannot be generated. It defaults to a dry run.

The templates are tested against the fifteen architectures that already exist,
not against themselves. A generator that drifts from what the registry expects is
worse than none, because it produces files that look right and fail at boot: so
the import line it would emit must be the line already present in
architectures/__init__.py, and the module path it computes must match
registry.defs_module_path(). One test asserts the template scaffolds every facet
that declares itself REQUIRED, so adding a required facet without updating the
scaffolder fails rather than producing definition modules that cannot boot.

Starter models turned out to be optional -- SD2 ships none -- so that test skips
where there is no module, with a second test pinning SD2 as the only such case so
the skip cannot quietly spread.

The docs build was not run: docs/node_modules is absent and installing it needs
network. The MDX was checked by hand instead -- the components used are the two
already imported by the file, and every aside type used is already used elsewhere
in these docs.

openapi.json is unchanged.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant