docs(ui): add design document for the Forge UI system

[stormmuller]

Summary

Adds /design/ui-system.md, a full architecture and delivery plan for a
retained-mode, ECS-native UI system built on an anchored rect tree — a
hierarchy of rectangle-shaped elements, each anchored and pivoted against its
parent's rectangle. Deliberately not immediate-mode, and not a
markup-and-stylesheet model.

The name is descriptive rather than borrowed. The same approach appears in
Unity's uGUI, Godot's Control nodes, and Flash's display list; those are cited
as prior art, but Forge names the model for what it does.

Documentation only. No engine code changes.

The document covers:

• Model rationale — why a retained rect tree maps almost 1:1 onto ECS
  entities, where immediate-mode and markup-and-stylesheet models don't.
• Gap analysis of the current codebase, grounded in specific files, with one
  priority scale and a column naming what owns each gap.
• Architecture with seven diagrams: coordinate spaces, the component
  composition of a button, the per-frame system pipeline and its ordering
  constraints, RectTransform anchor/pivot resolution, the text pipeline, the
  pointer interaction state machine, and the two paths into activation.
• API sketch in the codebase's existing idiom (plain-data components,
  createComponentId keys, add-prefixed component factories, batched
  create-prefixed system factories), opening with the minimal path — two calls
  to a working button.
• A fourteen-entry decision log (DL-01…DL-14), each with options considered,
  the decision, rationale, and consequences. Two decisions were reversed during
  review and record that explicitly.
• A phased feature backlog with sizes and critical-path markers.
• Risks and tradeoffs, and a unit/e2e/demo testing strategy.

Central finding

Most of what a UI module needs is not UI. Three capabilities Forge lacks
outright — text rendering, rect clipping, and a canvas-space pointer — are each
generally useful with no UI involved. Two of them are now owned separately
(#584, #583), which shrank Phase 0 from eleven items to five and moved the
plan's highest-risk item (the sub-quad renderer refactor) off the UI critical
path onto the text work it exists to serve.

The dependency worth stating plainly: UI layout, anchoring, hit testing, and
interaction can all be built and unit-tested before #584 lands, but the module
is not useful without text — buttons need labels, HUDs need numbers. Text
should gate the UI module's first release rather than its development.

Issues filed from the review of this document

Reviewing the design surfaced three genuine engine bugs and two scope splits:

Issue
#581	Parented position offsets ignore the parent's rotation and scale — a child of a rotating parent spins in place instead of orbiting
#582	AGENTS.md overstates input capabilities: claims touch (which doesn't exist), omits gamepad (which does)
#583	Rect clipping/masking — out of scope for UI, belongs in /src/rendering
#584	Text rendering — out of scope for UI, belongs in /src/text
#585	SpriteEcsComponent.pivot is Y-down while the rest of the engine is Y-up

Two further pre-existing problems are recorded in the document rather than as
issues, since the UI work is what fixes them:

• MouseInputSource caches getBoundingClientRect() in its constructor
  (mouse-input-source.ts:61), so pointer coordinates are wrong after any
  resize, scroll, or layout shift.
• DepthEcsComponent is fully implemented and tested but referenced by nothing
  in /src. DL-06 concludes it is not the right vehicle for draw order either,
  so the plan deletes it in favour of a sortDepth field on the sprite.

Also documents the new /design directory in AGENTS.md, including that its
contents are proposals and must never be treated as a description of current
engine behavior.

Verification checklist

Documentation-only change; the TypeScript checks are unaffected (no files under
/src, /demo, or /e2e changed).

• npm run check-types passes with 0 errors — n/a, no source changes
• npm test passes — n/a, no source changes
• npm run lint passes with 0 errors — n/a, ESLint does not cover Markdown
• npm run cspell passes with 0 errors — verified locally on the changed
  files; new technical terms added to .cspell/project-words.txt
• npm run check-exports passes — n/a, no export changes
• Any new/changed public API is exported from the module's index.ts — n/a
• Documentation under /documentation-site/docs/docs is updated if this
  change affects documented behavior — n/a, this describes a proposed
  subsystem, not current behavior
• If this change touches a module with a demo — n/a, no module changed

Prettier formatting verified locally on both changed Markdown files, plus a
manual consistency sweep over the document (decision-log numbering, section
numbering, and module layout against referenced components).

⚠️ CI has not run on this branch since 031d13b (14:58). Neither the
CI / Build and Test nor the Changelog workflow has been instantiated for any
commit since, and the CodeQL jobs that are created never leave queued, so
mergeable_state is blocked on missing checks rather than failing ones. All
nine workflows are confirmed state: active, so this appears to be an
Actions capacity or account-limit problem rather than anything in this branch.
Everything above was therefore verified by hand.

Changelog

• Not required — this PR's Conventional Commits type is docs.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[stormmuller]

uGUI is a Unity term. Let's pick something else more generic or specific to Forge.

[stormmuller]

Agreed — fixed in 031d13b.

The model is now called an anchored rect tree throughout, which describes what it does rather than borrowing a product name. The comparison table names the alternatives generically too (immediate mode, markup + stylesheet) with a separate "Prior art" column citing Dear ImGui / the web DOM / Unity / Godot / Flash, so the lineage is still legible without any of those becoming Forge vocabulary.

uGUI now appears in exactly four places, all deliberate attribution: the summary's note on why the term was chosen, two prior-art cells, and the paragraph about which parts of Unity's implementation not to copy.

---

Generated by Claude Code

[stormmuller]

Checked, and it is a bug — filed as #581. My wording here was also wrong in a way worth correcting, fixed in 854effe.

Rotation and scale are inherited, and correctly:

world.rotation = parent.world.rotation + local.rotation      // correct
world.scale    = parent.world.scale    * local.scale         // correct
world.position = parent.world.position + local.position      // missing two terms

The narrower defect is that the child's local offset is never transformed by the parent before being added. It should be:

world.position = parent.world.position
               + rotate(local.position * parent.world.scale, parent.world.rotation)

Parent at the origin rotated 90°, child at local (10, 0): world position should be (0, 10), actual is (10, 0). The child's own rotation.world is correctly π/2, so the sprite spins but doesn't orbit — a turret on a rotating tank rotates fine and sits in the wrong place.

Two things I found while confirming it:

• transform-system.test.ts has seven tests, none of which attach a rotation or scale component to a parent, so nothing locks the current behaviour in either direction.
• parent-position-system.ts, parent-rotation-system.ts, and parent-scale-system.ts carry the same additive composition, but aren't exported from src/common/systems/index.ts and are referenced only by their own tests — dead code superseded by createTransformEcsSystem. Flagged in fix(common): parented position offsets ignore the parent's rotation and scale #581 as worth deleting alongside the fix rather than fixing in three places.

The document now treats this as a bug to be fixed rather than a constraint to design around, and DL-03's rationale no longer leans on it (the camera-based scaler is still the better option on its own merits — it keeps the scaler in one place instead of two that have to agree).

---

Generated by Claude Code

[stormmuller]

Not necessarily true, draw order of sprites are currently also influenced by camera layerMasks, camera ordering, y-positions etc.

[stormmuller]

You're right, that bullet overclaimed — fixed in 031d13b.

It now says the model admits a single ordering rule, explicitly flagged as a property of the model and not of Forge today, followed by a new paragraph spelling out the actual chain: Renderable.category vs the camera's cullingMask gates whether a sprite draws at all; cameras composite by CameraEcsComponent.layer across distinct render targets and in query order within one destination; and within a camera, commands sort by SpriteEcsComponent.layer then by depth, which render-system.ts derives from position.world.y.

So hierarchy-order-is-draw-order is something DL-06 has to build (layout writes a pre-order index into DepthEcsComponent), and even then it only holds within a single canvas and sprite layer — the camera and culling-mask rules still sit above it, and are what keep the UI pass separate from the world pass in the first place.

---

Generated by Claude Code

[stormmuller]

Entity hierarchy shouldn't ignore rotation and scale. If it does, it's a bug, create a github issue if that is the case.

[stormmuller]

Confirmed a bug — filed as #581. Fixed the document's wording in 854effe. (My reply to this went onto the uGUI thread by mistake a moment ago; reposting it on the right one.)

My phrasing here was also wrong. Rotation and scale are inherited, correctly:

world.rotation = parent.world.rotation + local.rotation      // correct
world.scale    = parent.world.scale    * local.scale         // correct
world.position = parent.world.position + local.position      // missing two terms

The narrower defect is that the child's local offset is never transformed by the parent before being added. It should be:

world.position = parent.world.position
               + rotate(local.position * parent.world.scale, parent.world.rotation)

Parent at the origin rotated 90°, child at local (10, 0): world position should be (0, 10), actual is (10, 0). The child's own rotation.world is correctly π/2 — so the sprite spins but doesn't orbit. A turret on a rotating tank rotates fine and sits in the wrong place.

Two things found while confirming:

• transform-system.test.ts has seven tests, none of which attach a rotation or scale component to a parent — nothing locks the current behaviour in either direction.
• parent-position-system.ts, parent-rotation-system.ts, and parent-scale-system.ts carry the same additive composition, but aren't exported from src/common/systems/index.ts and are referenced only by their own tests. Dead code superseded by createTransformEcsSystem; fix(common): parented position offsets ignore the parent's rotation and scale #581 flags them as worth deleting alongside the fix rather than fixing in three places.

The design no longer treats this as a constraint to work around — every diagram assumes it fixed, and DL-03's rationale was restated so it no longer leans on the bug.

---

Generated by Claude Code

[stormmuller]

Create an issue to update the AGENTS.md to accurately reflect the real capabilities of the input sources. Lets assume touch inputs are out of scope for the UI change.

[stormmuller]

Done — issue is #582, and touch is out of scope as of 51dbc63.

#582 is scoped to documentation accuracy only, not to building a touch source. It also suggests auditing the rest of AGENTS.md's capability list against /src in the same pass, since this bullet was wrong twice over — claiming touch, which doesn't exist, and omitting gamepad, which does.

Changes here:

• Backlog item 0.2 (TouchInputSource) removed. I left the numbering gap rather than renumbering, so the decision stays visible in the doc and the 0.6 cross-reference in the risks table stays valid.
• §4.2's gap row is re-labelled Out of scope and points at docs(agents): AGENTS.md overstates input source capabilities (claims touch support that doesn't exist) #582 instead of implying UI should close it.
• DL-07's consequences now say only the stale getBoundingClientRect fix is a genuine prerequisite.

One design consequence worth flagging: PointerStateEcsComponent is specified source-agnostically — position, buttons, delta, scroll, with no notion of where the input came from — and MouseInputSource is simply its only writer today. So if a touch source is ever added it becomes a second writer of the same component and every UI system above it keeps working unchanged. Nothing in this design needs revisiting to add touch later; until then the UI is mouse-and-gamepad only, which its docs should state plainly rather than leave implied.

---

Generated by Claude Code

[stormmuller]

Only if this is the idiomatic way to fix the problem. Is a seperate component the right move? if so, keep this in the plan. If not, explain what a better alternative is. (e.g. adding a sortDepth to the sprite component may be more ergonomic)

[stormmuller]

You're right — switched to sortDepth in 66036dc. DL-06 now records the reversal.

I checked the perf angle before deciding, and it reinforces your suggestion rather than trading against it. buildCameraCommands runs per camera and already does three world.getComponent calls per sprite (rotation, scale, flip). A separate component makes that four — 4 × sprites × cameras — and this design adds a second camera, doubling the multiplier. sortDepth is a property read on an object already sitting in the batch array.

Two more reasons it's the better shape:

• The two halves of one sort key belong together. SpriteEcsComponent.layer is already the primary key, and its own JSDoc says ties break "by depth (world Y position)" — sortDepth overrides exactly that clause. Splitting them across two components means someone tuning draw order has to know about two places and can attach one without the other.
• A separate component buys no composition here. Rotation and scale are meaningful on an entity with no sprite; a draw-order key is not. There's no entity that wants sort order and nothing to draw, so the split was arbitrary rather than expressive.

It stays undefined rather than taking a default of 0, so existing sprites are untouched — a 0 default would silently override world-Y sorting everywhere.

That leaves DepthEcsComponent dead with no prospective use, so the plan now deletes it (item 0.4) instead of resurrecting it.

---

Generated by Claude Code

[stormmuller]

Does this affect DL-07?

[stormmuller]

What does expected mean? is it a blocker or high priority. Rect needs to be built. Saying "expected" is not helpful.

[stormmuller]

Fair — "Expected" answered nothing. Rebuilt the table in 66036dc.

The whole column was incoherent, not just that row: it mixed urgency (Blocking/High/Medium/Minor), category (Bug, Opportunity), and scope (Out of scope, Expected). So it's now one ordered scale — Blocker > High > Medium > Low — stated above the table, plus a "Tracked as" column naming the backlog item or issue that owns each gap, so every row answers "how urgent" and "where does it get done".

On the specific row: you're right that rect needs building, and it's a Blocker, not a shrug. It now reads:

No rectangle type or rect concept in the transform — Blocker — 0.2, 1.1 — Every element in this design is a rectangle resolved against its parent's rectangle, so Rect2 and RectTransformEcsComponent are foundational and must be built first. src/math/Rect.ts exists but is a class predating the Vector2 class→plain-object migration, so it isn't the type to build on (DL-11).

Draw order moved from High to Blocker in the same pass — it isn't a tuning problem, it's visibly wrong output (a label draws behind its own panel).

---

Generated by Claude Code

[stormmuller]

Clipping and masking, similar to text, is useful outside of the UI module. Let's create a github issue for this and mark it out of scope for the UI module.

[stormmuller]

Done — #583, out of scope as of 66036dc.

Agreed on the reasoning: minimap viewports, wipe transitions, fog-of-war reveals, and fill-by-reveal health bars all want rect clipping with no UI involved, so it belongs in /src/rendering next to the sprite instance-data segment.

DL-09 stays in the document as the decision record — the UI design is what surfaced the requirement and worked out that per-instance clip rects beat gl.scissor (which forces a draw-call break per mask, destroying batching for exactly the scroll-list case with the most elements) — but #583 owns the work.

One dependency worth tracking: ScrollRectEcsComponent (backlog 3.4) can't ship before #583 lands; scroll views spill past their viewport without clipping. Nothing else in the UI backlog is affected, so this doesn't gate Phases 0–2. Noted on both the issue and item 3.4.

---

Generated by Claude Code

[stormmuller]

Make a github issue for text. And mark it as out-of-scope for the UI module.

[stormmuller]

Done — #584, out of scope as of 0fe84ae. This was also my own recommendation in the open questions, so it's good to have it settled rather than left as a question.

#584 covers MSDF atlas loading, the shader, shaping, and a default shipped atlas so createLabel works with no toolchain. The sub-quad expansion refactor of render-system.ts moved with it — it only ever existed to serve text, and it was the highest-risk item in this plan (hot path, existing tests, eight-plus demos depending on nine-slice). The UI critical path no longer carries it. DL-04 and DL-05 stay here as decision records with scope banners.

That triggered a real restructure rather than a one-line edit. Phase 0 has now lost touch (#582), text (#584), and clipping (#583), so it was renumbered and reduced to five items: the canvas-space pointer, Rect2, sprite.sortDepth, deleting DepthEcsComponent, and verifying UI-camera compositing. There's a table recording what moved out and where.

The dependency I want to be explicit about rather than bury: the UI module isn't useful without text. Buttons need labels; HUDs need numbers. Layout, anchoring, hit testing, and interaction can all be built and unit-tested before #584 lands, but the module can't ship a convincing demo — or arguably a convincing release — until it does. The document now states that in the summary, in the risks table, and in the open questions, with the recommendation to build in parallel and gate the UI module's release on text rather than shipping something text-less that reads as broken.

---

Generated by Claude Code

[stormmuller]

SpriteEcsComponent.pivot should not be Y-down.

If this is the case, it's a bug. Create a github issue for this.

[stormmuller]

Traced it, and you're right — it is Y-down, and it is a bug. Filed as #585; document updated in b61a0c0.

The derivation, since it's non-obvious:

1. createProjectionMatrix scales Y by -2 / height — the projection flips Y.
2. bindSpriteInstanceData pre-negates the entity position (buffer[POSITION_Y_OFFSET] = -position.world.y). Those two cancel, so a larger position.world.y draws higher. World space is Y-up. ✅
3. sprite.vert.glsl computes the quad's local offset in the shader, so it never gets that compensating negation — local +y passes through the projection flip unopposed and moves down.

With pivot = (0,0): normalizedPivot = (-1,-1), pivoted = a_position + 1 spans [0,2], scaled spans [0,size]. All offsets ≥ 0, and local +y is down, so the sprite hangs below the entity — the entity sits at the sprite's top edge. Top-left, matching the JSDoc.

So pivot is the one place a rendering-internal detail leaks into the public API. Everything else Y-facing is Y-up and explicitly compensated — position.world.y, RotationEcsComponent (negated, with a comment saying it matches position), screenToWorldSpace. Pivot is the only one that isn't, and it's invisible at the default centered pivot, which is presumably how it survived.

Two things #585 flags for the same pass:

• Nine-slice region offsets. render-system.ts adds region.offset straight onto entityPosition.world (Y-up) while the shader pivot is Y-down. Those spaces disagree; nine-slice always rendering with a centered pivot may be masking it.
• Test coverage. Given the 0.24.1 fix for pivot applying only half its offset, non-centered pivots look under-tested — worth pinning all four corners.

The design's §5.2 no longer presents sprite.pivot.y = 1 - rectTransform.pivot.y as a permanent gotcha; it's marked as pending #585, and the note says to delete that line once fixed rather than keep a compensating error.

---

Generated by Claude Code

[stormmuller]

Would it be better to make a broader component, rather than a button specific component? Other UI elements are clickable and should have onClick events, not just buttons.

[stormmuller]

Agreed — moved in b61a0c0, recorded as the new DL-13.

onClick and the pointer enter/exit pair now live on UiInteractableEcsComponent, which is the component that already means "this rect participates in pointer input". Toggles, sliders, scrollbar thumbs, list rows, inventory slots, cards, and close icons all get them without claiming to be buttons.

Following that through, ButtonEcsComponent had nothing left to hold — visual feedback belongs to a transition component, focus to UiFocusEcsComponent — so it's gone entirely. A button is fully described by interactable + transitions + a child label, which makes it createButton, an aggregate factory in the mould of createCamera. That's a better fit for this codebase than a component that exists only to name a preset.

---

Generated by Claude Code

[stormmuller]

All components are, by definition, state. Including State in the component name, leads me to suspect there is a code smell. They may be no code smell and this is completely valid. You decide.

[stormmuller]

You were right to be suspicious — it was a smell, and the fix was deleting the component rather than renaming it. Done in b61a0c0, folded into DL-13.

The State suffix was the tell. Splitting author-set config from system-written state into two components looked tidy, but nothing ever wants the interaction state without the interactable, or vice versa — so it was carved along the wrong seam and the name needed a filler word to justify itself.

UiInteractableEcsComponent now holds blocksRaycasts, interactable, the events, and isHovered / isPressed / isDragging / wasClickedThisFrame. That matches what the codebase already does where the two genuinely belong together: PositionEcsComponent holds author-set local beside system-written world. Interaction state is world to the interactable's local, so the system-written fields are documented as read-only to callers using the same convention.

Net effect: one component instead of three (with ButtonEcsComponent gone too), one query in the raycaster instead of two, one place to look.

I applied the same reasoning to /src/input — the pointer component is now PointerEcsComponent, not PointerStateEcsComponent.

---

Generated by Claude Code

[stormmuller]

Make sure this detail is included in the text issue you create

[stormmuller]

Added to #584 — new "Frame ordering constraints for createTextShapingEcsSystem" section.

It captures why that slot in the diagram is load-bearing rather than incidental:

• Before the render system — the render pass consumes the glyph quads, so they have to exist first.
• After anything that can change the wrap width. For text inside a UI container, the wrap width comes from the element's resolved rect, which only exists after the layout pass and createTransformEcsSystem. Shaping earlier wraps against last frame's width — invisible until something resizes, then one frame of visibly wrong line breaks.
• No layout dependency for standalone text. Damage numbers, floating names, and debug overlays have nothing constraining them, so their shaping depends only on the string, font, and size.

So the constraint is conditional on whether the text sits in a laid-out container, and the simplest correct default is to register shaping late — after layout and transform, before render — which is right for both cases. The issue points back at §5.4 for the full pipeline.

I also folded in the dirty-tracking note, since it belongs with the ordering: shaping is the one place this design deliberately doesn't recompute every frame (contrast DL-12, where everything else does), because a re-shaped paragraph per frame is real cost rather than theoretical. Re-shape only when text, font, fontSize, or the resolved wrap width actually changed.

---

Generated by Claude Code

[stormmuller]

Is there a scenario where the hovered state is skipped, the user manages to press on the very frame the cursor enters the frame?

[stormmuller]

Yes, and the diagram was wrong to imply otherwise — fixed in ddc5445.

Input is sampled per tick, not streamed. MouseInputSource accumulates button downs and ups into per-frame sets and reset()s them each tick, so the system sees the pointer's current position plus a set of edges — never their ordering within the frame. At 60 Hz a frame is ~16.7 ms and a flick-and-click fits inside one comfortably.

More decisively: touch has no hover phase at all. Its first event is simultaneously "entered" and "down". Touch is out of scope now (#582), but DL-07 specifies the pointer source-agnostically precisely so it can be added later without revisiting this design — and that promise is only real if the state machine already tolerates a missing hover. So this isn't a rare race, it's the only path on a whole input class.

Chasing it turned up a sibling case: press and release within one tick, from a synthetic click() or a fast enough real one. Both edges survive the frame in MouseInputSource's separate down/up sets, but a system reading only "is the button currently held" sees neither and silently drops the click.

The fix is that createUiInteractionEcsSystem must not apply one transition per tick. State is derived from the tick's facts; events are derived from the delta against the previous tick:

isOver           = this element was the raycast hit this tick
pressStartedHere = a pointer-down edge landed here (latched until release)

state = Disabled if !interactable | Pressed if isOver && pressStartedHere
      | Hovered  if isOver        | Normal

onPointerEnter  when !wasOver && isOver
onPointerDown   when a down edge occurred && isOver
onClick         when an up edge occurred && isOver && pressStartedHere

Your scenario then raises onPointerEnter and onPointerDown in the same tick, the state goes Normal → Pressed, and Hovered is never observed — nothing swallowed. I added that edge to the diagram so the path is legal rather than accidental.

The consequence for callers is now stated explicitly: treat these as sampled states, not a guaranteed sequence. A transition that only starts a press animation from a hover animation is wrong, and would fail first on touch and intermittently on fast mice. Both same-tick cases are added to the unit test plan — cheap to assert with a synthetic pointer, nearly impossible to reproduce by hand.

---

Generated by Claude Code

[stormmuller]

How will this work with a controller or other input source? There is no "click" on a controller. This is why the Input actions are decoupled from sources. I expect it to work that way for the UI module too.

The author needs to know that the "start" button was actioned. Not how it was actioned.

[stormmuller]

This has the added benefit of allowing me to drive UI actions programmatically for tests and tutorials.

[stormmuller]

You're right, and I should have followed the precedent that's already in the codebase. Restructured in 6a77d41 as DL-14.

CameraEcsComponent already does exactly what you're describing — zoomInput?: Axis1dAction, panInput?: Axis2dAction — consuming the action abstraction rather than reading keys. I built a pointer-centric model alongside it instead of following it.

onClick → onActivate, raised from either path, with the source never exposed:

• pointer — up edge inside, press started here (createUiInteractionEcsSystem)
• focus — submitInput triggered while focused (createUiNavigationEcsSystem)

The canvas takes submitInput / cancelInput / navigateInput as InputActions. The UI module never touches an input source. The author learns the Start button was activated, not how.

Two states stay deliberately separate, because collapsing them loses information:

• isHovered — pointer-only, meaningless on a gamepad.
• isFocused — source-agnostic, means "the currently selected element".

Transitions key off a derived visual state merging both, so an element highlights identically whether it's moused-over or stick-focused, and no transition component needs to know which input the player is holding.

Drag events stay explicitly pointer-shaped — a drag has no controller analogue, and pretending otherwise would be worse than admitting it.

The consequence I'd flag: UiFocusEcsComponent and createUiNavigationEcsSystem move from Phase 5 polish into Phase 2 core (item 2.4). Treating focus as a late accessibility add-on was the root mistake; it's half the interaction model.

---

Generated by Claude Code

[stormmuller]

Captured as a first-class consequence in DL-14, not just a footnote — it's a strong enough argument that it partly justifies the decision on its own:

UI becomes programmatically drivable for free. Because activation arrives through an InputAction, a test or a scripted tutorial calls action.trigger() and the UI responds exactly as it would to a real player — no synthesized DOM events, no fake pointer coordinates, no separate test-only code path.

Two things fall out that are worth naming:

• Unit tests get real interaction flows in jsdom. Previously the testing strategy could only assert the interaction state machine against a synthetic pointer; now a test can drive a whole menu — focus, submit, cancel — through the same actions a player uses. The /e2e suite stays for what genuinely needs a real browser (actual rendered pixels, real DOM events through the real input pipeline), rather than being the only place interaction can be exercised end to end.
• A tutorial can genuinely press its own buttons. Not a simulated highlight over a button plus a manual call to the handler — the actual activation path, so a scripted tutorial can't drift out of sync with what a real press does.

It also means the "does the button work with a controller" test doesn't need a controller.

---

Generated by Claude Code

[stormmuller]

Just make sure the DX is solid. I don't want the author of a game to be bogged down by setting up 2 cameras, canvases, etc. just to get a button on screen.

[stormmuller]

Agreed, and my previous example made it look worse than intended — fixed in 6a77d41.

Added a "minimal path" as the first thing in §6, before any of the option-heavy examples:

const canvas = createUiCanvas(world, renderContext);
const play = createButton(world, canvas, { label: 'Play' });

play.onActivate.registerListener(startGame);

That's a working, clickable, gamepad-navigable button. createUiCanvas creates the canvas entity, its rect transform, the dedicated static UI camera with a transparent clear color and culling mask, and its render target — and registers the UI systems in the right order. Every option is defaulted: 1920x1080 reference resolution, scale-with-screen-size, and submit/cancel/navigate bound on the supplied InputManager. You pass an option only to change it.

The second camera in DL-01 is an implementation detail of that one call, and the doc now says so explicitly: if an author ever has to think about culling masks or render targets to put a button on screen, the design has failed. I also reworked the full menu example so the option-heavy version is clearly labelled "what it looks like when you do reach for them" rather than reading as required setup.

Added a matching goal in §2 — "two calls to a working button" — so this is a stated design constraint rather than something I assert once in a reply and drift away from later.

---

Generated by Claude Code

[stormmuller]

And if the author wanted to apply an effect to the entire screen (world + UI) how would they achieve this or would this be impossible given this set up?

Post processing and other effects are very valid for UI too.

[stormmuller]

Not impossible, and it needs no new plumbing — but you're right that DL-01 read as though it were. Fixed in c2c9775.

The key fact I'd stated as a benefit without following through on: post-processing attaches to a render target, not to a camera. bloom-system.ts:205 dedupes with processedTargetsThisFrame.has(renderTarget), so an effect is applied once per target regardless of how many cameras drew into it. That dedupe only exists because multiple cameras per target was anticipated.

So all three scopes fall out of which target the UI camera renders into:

Want	Configuration
World only (default) — HUD stays crisp while the world blooms	Separate renderTargets; effects attach to the world's
Everything, world + UI	Point the UI camera's renderTarget at the same target as the world camera. Both draw in, the render system clears it once (clearedDestinationsThisFrame), effect applied once to the composite
UI only — blur behind a modal, vignette a pause menu	Attach effects to the UI camera, which owns its own target

Ordering already works for the shared case: post-processing systems are documented to register after the render system and before the present system, and both cameras have drawn by the time the render system's update returns.

The one real constraint, now written down: a shared target gives you one set of effects over both, so "bloom the world but not the UI, and tone-map everything" isn't expressible in a single pass — that needs world-on-its-own-target, effects there, then both composited into a shared target with the second effect. I've noted that as the argument for eventually letting a post-processing stack attach to the present step rather than only to a camera, and marked it out of scope for this design rather than pretending the limitation doesn't exist.

---

Generated by Claude Code

[stormmuller]

Does this decision change or remain as is once the parent transform bug is fixed? i.e. scale and rotation are also additively applied like position?

[stormmuller]

Scale might be multiplicatively applied?

[stormmuller]

The decision stands — but you've caught that I argued it badly. Answered explicitly in 2a69822 with a new "Does this decision survive #581?" subsection.

And yes to your follow-up: scale is multiplicative. The three channels compose differently, and only one of them is wrong:

Channel	Composition today	Correct?
Rotation	parent.world + local — additive	✅ nesting rotations adds angles
Scale	parent.world * local — multiplicative	✅ a 2x parent with a 3x child is 6x
Position	parent.world + local — additive	❌ the local offset is added raw, never scaled/rotated by the parent first

So #581 isn't "make scale and rotation additive like position" — those two are already right. The fix makes position account for the parent's already-correct rotation and scale:

world.position = parent.world.position
               + rotate(local.position * parent.world.scale, parent.world.rotation)

What that means for DL-03: once #581 lands, option (b) stops being broken. A scaled canvas root would correctly scale its children's offsets, and sprite sizes already scale (bindSpriteInstanceData multiplies by scale.world). But (b) still loses on the arguments that were always the real ones:

• Cost — (a) changes one number on one camera; (b) dirties position.world and scale.world for every UI entity on every resize.
• One coordinate space — with (a), rect stays in reference pixels regardless of screen size, so layout, hit testing, and every rect consumer speak one language. With (b) rects live in a screen-dependent space, or you maintain both and everyone has to know which they're holding.
• It breaks isStatic (DL-12) — a subtree frozen at one scale is wrong after a resize, so the freeze would need invalidating on every resize, which is exactly the dirty-tracking complexity DL-12 avoids.
• It breaks per-element scale — an author scaling a button 1.1x for a press animation would multiply against a screen-dependent base, so the same "pop" reads differently at 720p and 4K.

The honest correction, which the doc now states: leading with "(b) is broken" was lazy, because it's the one objection with an expiry date. The cost and coordinate-space arguments are load-bearing and hold either way.

---

Generated by Claude Code

[stormmuller]

Ensure this is captured in the text issue you create

[stormmuller]

Checked DL-04 against #584 line by line. Most of it was already there — the options table, the MSDF rationale, the atlas loader, the msdf-atlas-gen recipe, the kerning-pairs requirement, and the shipped default atlas.

One thing genuinely wasn't: the Canvas2D escape hatch. DL-04's decision is "(c) MSDF, with (a) available as a documented prototyping escape hatch", and #584 only dismissed Canvas2D as unsuitable without carrying the escape-hatch part. Now added as its own section.

The reasoning I wrote into the issue: MSDF is the shipping path, but atlas generation is a barrier at exactly the wrong moment — someone trying their own font for the first time, or throwing a debug readout on screen mid-session. So keep a fillText-to-texture path, with its limits stated plainly so nobody ships on it by accident:

• One draw call per string, no batching
• Re-uploads the texture on every change — unusable for anything per-frame (score, timer, FPS)
• Blurs when scaled
• No outline/glow/shadow

Plus a constraint worth designing to up front: both paths should share the same TextEcsComponent surface where practical, so moving a prototype onto a real atlas is a font swap rather than a rewrite. That only stays true if it's a requirement from the start.

I also strengthened the kerning note while there — missing kerning pairs produce text that's subtly wrong in a way that's hard to attribute later, which is worth flagging louder than a bullet in a list.

---

Generated by Claude Code

[stormmuller]

Does this affect DL-07? (re: the camera / culling-mask separation in §3's ordering note — answering here because the review thread isn't resolvable through the API yet)

Yes, in a way I hadn't drawn out. Addressed in a6c083f.

Because UI renders through its own camera (DL-01), a single canvas-space pointer maps to two different world positions — one through the world camera, one through the UI camera — and they diverge the moment the world camera pans or zooms.

So PointerEcsComponent must be camera-agnostic as well as source-agnostic: it publishes the pointer in canvas pixels only, and must not publish a "world position", because there is no single correct one. Conversion is the caller's job, and the existing helper is already camera-parameterized:

screenToWorldSpace(screenPosition, cameraPosition, cameraZoom, width, height, pixelsPerUnit)

The UI raycaster converts through the UI camera; game-world picking converts through the world camera; neither is privileged. That also keeps DL-07's original argument intact — a world-space consumer like drag-to-select or tower placement wants the world camera's conversion, which is only possible if the component stays raw.

And the consistency requirement in the other direction, which is the part that would have bitten: the raycaster should only hit-test what the UI camera can actually see. An element culled from the UI camera by cullingMask must not remain clickable, or the UI grows invisible hit regions. The same Renderable.category check that §3 describes for drawing has to gate hit testing too — otherwise the ordering note describes one rule for pixels and the raycaster quietly implements another.

---

Generated by Claude Code

[stormmuller]

Text is not part of the UI feature, but text inputs can be? Thoughts?

[stormmuller]

Maybe text inputs should be a separate issue?

[stormmuller]

Answered in full here and here — this thread wasn't reachable through the API at the time, so those landed on the PR conversation instead. Linking back so the thread isn't left hanging.

Short version: a "text input" is three concerns — text entry (IME, soft keyboards, clipboard → input), text display (→ #584), and the focusable rect (→ UI). So it's not inconsistent, but it was the most hard-blocked item in the backlog and its hard part isn't UI at all.

Both now resolved: split out as #586, with the DOM entry bridge living in /src/input. Item 3.5 marked out of scope. Pushed in 2ffa81f and 317ef93.

---

Generated by Claude Code

[stormmuller]

Text is not part of the UI feature, but text inputs can be? Thoughts? (re: DL-10 — replying here as the review threads still aren't resolvable through the API)

They look inconsistent but aren't — and chasing it found a real gap in my bookkeeping. Addressed in 2ffa81f.

A "text input" is three separable concerns wearing one name:

Concern	Belongs to	Why
Text entry — keystrokes, caret, selection, IME, clipboard, soft keyboard	Input	Almost always implies a focused interactive widget. Unlike damage numbers or world signage, there's no "entry" without something to type into.
Text display — drawing the string, caret, selection highlight	#584	Identical to drawing any other text.
The focusable rect it hangs off	UI	An interactive rectangle is exactly what this module is.

So the field belongs here. But your question exposed that I'd been sloppy: item 3.4 (ScrollRect) is marked blocked on #583, while 3.5 (text input) wasn't marked blocked on #584 — despite being far more blocked. A scroll view merely looks wrong without clipping; a text field is unusable without text rendering, because you can't see what you type. Now marked as the most hard-blocked item in the backlog.

And it prompted a change I think is a genuine improvement. The hidden-<input> bridge — "give me a DOM input synced to this screen rect, and tell me its value, caret, and selection" — is where all the IME and mobile-keyboard complexity lives, and it's reusable by anything needing text entry rather than specific to one widget. So it moves to /src/input as a small primitive, for exactly the reason DL-07 moved the pointer there, with TextInputEcsComponent consuming it.

The payoff: §2's "no DOM-backed widgets" non-goal becomes a real invariant instead of one with an exception carved out of it. The UI module is now DOM-free, full stop, and the one unavoidable piece of browser interop sits with the other input-source complexity where it belongs.

---

Generated by Claude Code

[stormmuller]

Yes, this is the plan now. We will have a separate Text issue.

[stormmuller]

Maybe text inputs should be a separate issue?

Agreed — #586, done in 317ef93. It's the strongest of the three splits, for two reasons that only became clear once we'd worked through the previous question:

1. It was the most hard-blocked item in the backlog — unusable, not merely imperfect, without feat(text): text rendering — MSDF font atlases, shaping, and a TextEcsComponent #584.
2. Its genuinely hard part isn't UI at all. IME composition, mobile soft keyboards, clipboard, selection semantics, and screen-reader accessibility are all input concerns. What's left in UI is a focusable rect, which is nothing special.

#586 covers the text-entry primitive in /src/input plus the thin component consuming it, with the multiline / caret-rendering / masking questions written down as things to settle early rather than discover mid-build. DL-10 stays as the decision record with a scope banner; item 3.5 is marked out of scope.

One thing I'd flag for #584 specifically, since it's cheap now and disruptive later: rendering a caret and a selection highlight needs per-glyph x-positions, so TextMeshEcsComponent should expose glyph advances/positions rather than only quads. If #584's API gets fixed without that, #586 has to retrofit it. Noted on both issues.

---

Yes, this is the plan now. We will have a separate Text issue.

Confirmed — that's #584, already filed and out of scope as of 0fe84ae. The design defers to it, and Phase 0 was rebuilt around its removal.

Where that leaves the scope splits: #583 clipping, #584 text rendering, #586 text input, plus #582 for the AGENTS.md accuracy fix. The UI module itself is now just the rect tree, layout, hit testing, interaction, and controls — which is what it should have been from the start.

---

Generated by Claude Code

[stormmuller]

Yes, and there is no need to document it as a breaking change.

Please don't call it Rect2 just Rect is fine. Replace the existing implementation.

[stormmuller]

Done in cabf569 — full reply here (this thread wasn't reachable through the API at the time).

Rect replaced in place, no parallel type, no breaking-change note. Checking the blast radius supports replacing over deprecating: Rect is constructed nowhere in /src outside its own tests, and its only consumer is CameraEcsComponent.scissorRect, whose type name doesn't change.

One thing still needs your call, in §11: Vector2/Vec2 puts the long name on the type and the short one on the namespace, which doesn't translate — Rect has no natural abbreviation. I've proposed Rects for the static namespace; the more literal mirror would be Rectangle/Rect, but that moves Rect off the type, which may be the opposite of what you meant.

---

Generated by Claude Code

[stormmuller]

Please don't call it Rect2 — just Rect is fine. Replace the existing implementation. / no need to document it as a breaking change

Done in cabf569. DL-11 now replaces Rect outright instead of introducing a parallel type, and the breaking-change note is gone.

Checking the blast radius supports replacing rather than deprecating — it's smaller than I'd assumed when I wrote the coexistence plan: Rect is constructed nowhere in /src outside its own tests, and its only consumer is CameraEcsComponent.scissorRect, whose type name doesn't change. What actually changes is construction (new Rect(origin, size) → { min, max }) and the two methods becoming static calls. So the "two types coexist temporarily, migrate later" hedging I'd written was solving a problem that doesn't exist.

Shape mirrors vector2.ts exactly — plain interface for the data, static-only class for the operations, mutating helpers taking target first:

export interface Rect {
  min: Vector2;
  max: Vector2;
}

Kept { min, max } over the current { origin, size }: anchor math reads min/max far more often, and size is one subtraction away.

Two things I folded in, shout if either is unwanted:

• Rename the file to rect.ts. src/math/Rect.ts breaks the kebab-case convention in AGENTS.md, and a full rewrite is the cheapest moment to fix it.
• One naming question left open, because I don't want to guess at it. Vector2/Vec2 puts the long name on the type and the short one on the namespace — that doesn't translate, since Rect has no natural abbreviation. I've proposed Rects for the static namespace (Rects.contains(rect, point) reads fine). The more literal mirror would be Rectangle for the type and Rect for the namespace — but that moves Rect off the type, which may be the opposite of what you meant. Your call; it's in §11.

---

Generated by Claude Code