Skip to content

Move wizard composition to specify artifact CLI (blocked by spec-kit#4305) - #17

Closed
nicolehaugen wants to merge 13 commits into
github:mainfrom
nicolehaugen:nicolehaugen-effective-pancake
Closed

Move wizard composition to specify artifact CLI (blocked by spec-kit#4305)#17
nicolehaugen wants to merge 13 commits into
github:mainfrom
nicolehaugen:nicolehaugen-effective-pancake

Conversation

@nicolehaugen

Copy link
Copy Markdown
Contributor

Warning

Blocked by github/spec-kit#4305.
Do not merge until #4305 lands and a specify-cli release containing
specify artifact list --json is published to PyPI. Once that ships, bump
the version floor note in skills/speckit-cli-setup/SKILL.md and un-draft.

Summary

Move the wizard's Composition tab off direct filesystem inspection and onto a
single specify artifact list --json call. The CLI returns one row per
artifact carrying the full composition stack: [...], which is everything the
wizard needs to render commands, templates, scripts, and hooks.

Depends on

What changes

  • composition/artifact-cli.mjs (new sole source of composition): calls
    specify artifact list --json once at boot, maps rows into the wizard's
    artifact / preset / extension shapes, and produces the composition summary.
  • project-scanner.mjs: no longer reads .specify/{presets,extensions}.json
    for composition. Starts empty and lets overlayCachedComposition apply the
    CLI-derived data after the scan.
  • composition/pipeline-fast-path.mjs: decides between the deterministic
    pipeline (canonical spine + replace-only overrides) and the LLM pipeline
    (prompts/composition.mjs::inferPipeline) needed when an extension adds a
    non-canonical command or uses wrap / prepend / append.
  • Boot UX: "Loading catalogs" split into distinct catalog + composition
    tracker steps so the user sees which phase is running.
  • Removed: the live-CLI integration test (test/artifact-cli.integration.test.mjs)
    and its fixture — that layer belongs in the spec-kit repo alongside the CLI
    it exercises. Unit tests here inject a fake runner so CI is green regardless
    of which specify-cli version is installed.

Verification

  • 207/207 unit tests pass (unit tests inject a fake specify runner).

  • Playwright DOM-diff matrix — captured the wizard side panel from a main-branch
    plugin variant vs. this branch across five scenarios:

    1. baseline (0 presets / 0 extensions)
    2. copilot-sub-agents preset
    3. pirate-full-preset
    4. agent-context extension only
    5. all three stacked

    Each scenario snapshots five surfaces (Composition → Commands / Templates /
    Scripts / Hooks, plus top-level Phases). 25/25 pairs are byte-identical
    after normalizing internal [ref=...] handles. No user-visible regressions.

Runtime behavior when the CLI is too old

Right now the wizard will surface an error and empty composition if the
installed specify-cli lacks artifact list --json. That's acceptable while
this PR is draft; when we un-draft, the floor version bump in
speckit-cli-setup guarantees users get a compatible CLI.

nicolehaugen and others added 12 commits August 25, 2026 13:07
Complements artifact-cli.test.mjs (fixture round-trip) with three real-shape
guards:

* Live-CLI test invokes real `specify artifact list/info` on a scaffolded
  workspace and asserts wizard-contract fields (id/kind/stack, layer
  vocabulary, exactly-one-active). Skips when `specify` isn't on PATH.
* Fixture-drift tests replay a committed snapshot of real CLI output and
  assert the field set the shape mapper reads is present. Guards against
  silent CLI shape changes without needing the binary in CI.
* fixtures/README.md documents regeneration.

Full suite: 211/211 pass.
On a warm cache the server's `bootAsync` reaches `phase: ready` before
the browser's first paint. The old flow relied on JS to (a) populate the
overlay content and (b) hide `main.app-body`, then almost immediately
flipped the overlay to `is-hidden` in the same microtask cycle. The
browser composited populate + hide into one frame and the user saw a
blank body flip straight to the loaded app with no boot indicator.

Three-part fix so the overlay is guaranteed to paint:

* Static markup in `index.html` — pre-render the overlay panel with
  title + subtitle so it is visible from the very first paint, before
  any module fetch/parse.
* CSS-level `main.app-body { visibility: hidden }` — no longer
  depends on JS running to keep the app body hidden underneath.
* JS-side minimum visible time (`MIN_OVERLAY_MS = 450`) — even when
  the state fetch resolves in a single frame, the hide is deferred via
  `setTimeout` so the overlay stays up long enough to register.
Boot's `hydrateCatalogs` used to walk preset → extension → bundle
serially, and each hydrator walked its 2–3 source URLs serially inside
`hydrateFromCatalogSources`. That's ~8 GitHub GETs strictly serial on a
cold cache, plus 3 sequential `specify <kind> list` shell-outs, for
what is entirely disjoint state.

Two changes:

* Run `hydratePresetsForSources` / `hydrateExtensionsForSources` /
  `hydrateBundlesForSources` via `Promise.all` — they touch
  independent cache slices.
* Inside `hydrateFromCatalogSources`, `Promise.all` the per-source
  `fetchCatalogJson` calls before folding into the items array.
  Order of items is preserved because we still iterate the resolved
  array in source order.

All 211 tests pass.
The catalog boot step called specify artifact info once per artifact
via �xecFileSync in a serial loop. With a real workspace stack (~70
artifacts across 4 presets + 1 extension), that's ~70 shell-outs, each
one blocking the Node event loop for its full duration.

Impact: /api/state and SSE could not be answered during boot, so the UI
sat on 'Loading catalogs' for the full 108s wall time of the loop, even
though the HTTP server was up. From the user's perspective the wizard
'hung'.

Fix:
- Swap the default runner to a promisified execFile so each shell-out
  yields the event loop instead of hard-blocking it.
- Fan the info-per-id calls out with Promise.all — safe now that spawn
  is non-blocking.
- Await the runner return so injected sync test runners (which return a
  Buffer/string) still work unchanged.

Measured on the current workspace (68 artifacts):
  before: 108s serial sync, HTTP frozen throughout
  after:  13s parallel async, HTTP responsive throughout (~8x faster).

All 211 tests still pass.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2d44fd06-6323-4654-8a80-b756561ef669
The catalog boot step was fragile in three separate ways beyond the
sync exec loop already fixed in the artifact-info fan-out:

1. fetchCatalogJson had NO timeout. A stalled socket (slow DNS, TCP
   loss, CDN outage) would block the fetch forever and freeze boot on
   'Loading catalogs' with no recovery path. Added AbortSignal.timeout
   (15s per fetch).

2. specifyRun had NO timeout either. A wedged CLI (uv resolver stuck,
   PATH resolution hang) had the same failure mode. Added a 20s kill
   timer; on expiry we resolve with the partial stdout (callers already
   tolerate empty output).

3. hydrateCatalogs awaited the three groups (presets, extensions,
   bundles) serially, and hydrateFromCatalogSources awaited each source
   inside a group serially. Both are independent I/O — swapped for
   Promise.all at each level so total time is bounded by the slowest
   single call, not the sum.

Measured on the current workspace (7 catalogs, 68 artifacts):
  before: 108s serial sync, HTTP frozen throughout
  after:  ~14s parallel async, HTTP responsive throughout (~8x faster,
          and now failure-bounded instead of unbounded)

All 211 tests still pass.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2d44fd06-6323-4654-8a80-b756561ef669
Boot was making 68 sequential `specify artifact info` shell-outs per artifact to reconstruct composition stacks — the flow that regressed catalog-loading to 27-60s on Windows. spec-kit PR #4305 makes `artifact list --json` return the full per-row stack, so the wizard needs exactly ONE shell-out to build composition.

Wizard-side changes:

- `buildCompositionFromCli` now consumes a single `list --json` call and feeds rows directly through `shapeArtifact`. Presets/extensions summaries are folded from artifact stacks (accepted edge case: a preset that contributes zero currently-active artifacts won't appear — living with it until upstream ships `preset list --json`).

- Removed `specifyArtifactInfo` — dead post-refactor. Don't reintroduce a per-artifact fan-out on the boot critical path.

- Deleted `project-scanner.mjs::scanComposition` and its `.specify/{presets,extensions}.json` reads. Neither file is written by any CLI version; the code was dead. Removed the now-unused `readBoundedJson` import.

- Governing principle: CLI is the source of truth for composition. No direct fs reads of `.registry`/`.yml` from the wizard, ever.

Test-side changes:

- Unit fixtures flattened: list rows now embed `stack` (no separate `info` map). `fakeRunner` simplified to only handle `list`.

- Fixture-drift test rewritten to require `stack` on list rows. Skips gracefully when the on-disk snapshot is pre-#4305 (regen once upstream ships).

- Live-CLI test skips when the installed CLI's `list --json` doesn't yet emit `stack` — same rationale.

- Deleted obsolete `live-cli-info.json` fixture; updated README.

- Deleted `scanWorkspace drops malformed composition entries` test — it exercised the deleted `scanComposition` path.

208 tests: 207 pass, 1 skip (drift, until fixture regen).
Real `specify artifact list --json` output now carries per-row `stack` (spec-kit#4305 has landed). Fixture-drift and live-CLI tests are now actively guarding (208/208 pass, 0 skips) instead of skipping under the pre-#4305 detector.
Boot overlay previously bundled two unrelated phases into one 'catalog' step: remote catalog JSON fetches (~150ms parallel) AND the composition CLI build (specify artifact list --json, ~2s cold). When boot felt slow, you couldn't tell which side was blocked.

Split them so each shows independently in the overlay:

- `catalog` (Loading catalogs) \u2014 hydratePresets/Extensions/Bundles, remote HTTPS + `specify <group> list` per group, all parallel.

- `composition` (Building composition) \u2014 single `specify artifact list --json` call + shape mapping.

Now a hang on either side is visible at a glance without log scraping.
Comments now describe current behavior only.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2d44fd06-6323-4654-8a80-b756561ef669
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2d44fd06-6323-4654-8a80-b756561ef669
- project-scanner.mjs: rewrite composition-block comment to describe current

  behavior only; drop 'used to look at' language and the fabricated AGENTS.md

  citation.

- artifact-cli.mjs: drop the phantom 'AGENTS.md: CLI is the source of truth

  for composition.' line from the doc header.

- pipeline-fast-path.mjs: rename 'LLM Stage 2' → 'LLM path' (there was no

  Stage 1) and 'Fast path' → 'Deterministic path' to match the actual code.

- ui/index.html, ui/boot.js: drop '- Dev' suffix from the wizard title in

  all four spots to prepare for check-in.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 2d44fd06-6323-4654-8a80-b756561ef669
Copilot AI balanced review requested due to automatic review settings August 26, 2026 16:17
@nicolehaugen
nicolehaugen marked this pull request as ready for review August 26, 2026 16:18
@nicolehaugen
nicolehaugen requested a review from mnriem as a code owner August 26, 2026 16:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Moves wizard composition assembly to the forthcoming specify artifact list --json API, pending spec-kit#4305.

Changes:

  • Adds CLI-backed artifact mapping, hook enrichment, and pipeline fast-path selection.
  • Separates composition boot progress and parallelizes timeout-bounded catalog loading.
  • Removes filesystem composition assembly and updates tests/dependencies.
Show a summary per file
File Description
ui/styles/boot.css Hides app content during boot.
ui/index.html Adds initial boot markup.
ui/boot.js Adds composition status and minimum display time.
test/state-and-scanner.test.mjs Removes obsolete scanner test.
test/composition.test.mjs Removes legacy composition tests.
test/boot-progress.test.mjs Covers the new boot step.
test/artifact-cli.test.mjs Tests CLI composition mapping.
project-scanner.mjs Removes filesystem composition scanning.
package.json Updates js-yaml.
package-lock.json Locks the dependency update.
extension.mjs Splits catalog and composition boot phases.
composition/pipeline-fast-path.mjs Adds deterministic pipeline selection.
composition/hooks.mjs Extracts hook metadata.
composition/collect.mjs Removes legacy filesystem collection.
composition/assembler.mjs Removes legacy composition assembly.
composition/artifact-cli.mjs Adds the CLI-backed composition adapter.
catalog/sources.mjs Adds fetch timeouts.
catalog/shared.mjs Parallelizes hydration and bounds CLI calls.
canvas-runtime/composition-apply.mjs Uses the new composition pipeline.
canvas-runtime/boot-progress.mjs Registers composition boot progress.

Review details

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Files not reviewed (1)
  • plugins/spec-kit-copilot-wizard/extensions/speckit-wizard-canvas/package-lock.json: Generated file
  • Files reviewed: 19/20 changed files
  • Comments generated: 4
  • Review effort level: Balanced

return {
// CLI `null` layer = built-in; wizard code expects "core".
layer: layer.layer == null ? "core" : layer.layer,
presetId: layer.presetId ?? null,

await applyComposition(inst, payload);
return { ok: true, reason, stage2Needed: stage2.needed };
return { ok: true, reason, pipelineFastPath: fastPath.canSynthesize };

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@copilot The decision was to remove references to "stage2" thorughout code and instead is the 'fastpath' - make the fix accordingly

Comment on lines +210 to +212
await runFastComposition(inst, { reason: "boot" });
await snapshot(inst);
tracker.ok("composition");
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 26, 2026 18:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review details

Files not reviewed (1)
  • plugins/spec-kit-copilot-wizard/extensions/speckit-wizard-canvas/package-lock.json: Generated file
Suppressed comments (7)

Previously missed (4) — in code that hasn't changed since the last review.

plugins/spec-kit-copilot-wizard/extensions/speckit-wizard-canvas/catalog/shared.mjs:61

  • On timeout, returning partial stdout is unsafe for these line-based list parsers: a truncated prefix can still parse successfully and mark only the first installed presets/extensions active. Treat a timed-out command as failed so callers use their empty/fallback path rather than publishing a plausible but incomplete installed set.
        const timer = setTimeout(() => {
            try { child.kill(); } catch { /* best-effort */ }
            done(stdout || null);
        }, timeoutMs);

plugins/spec-kit-copilot-wizard/extensions/speckit-wizard-canvas/test/artifact-cli.test.mjs:6

  • The replacement suite covers the new CLI mapper and fast path, but deleting composition.test.mjs also removes all tests for still-live canonical.mjs, effective-phases.mjs, and resolvePipelineEntry behavior. No equivalent tests remain elsewhere, including regression coverage for bare extension IDs and hook filtering. Preserve those unaffected tests in focused test files.
import {
    buildCompositionFromCli,
} from "../composition/artifact-cli.mjs";
import { computePipelineFastPath } from "../composition/pipeline-fast-path.mjs";

plugins/spec-kit-copilot-wizard/extensions/speckit-wizard-canvas/project-scanner.mjs:132

  • Starting the scan with empty composition breaks the existing phase gate in canvas-runtime/snapshot-builder.mjs:113-116, which checks scan.composition.extensions before snapshot() overlays inst.cachedComposition. Consequently, taskstoissues remains gated even when its provider extension is installed. Apply the cached CLI composition before building the snapshot, or move this gate to a post-overlay source.
    // Composition data comes from `runFastComposition` (CLI-driven, see
    // composition/artifact-cli.mjs) and is applied via `overlayCachedComposition`
    // after this scan runs. Start empty so the overlay step has a clean base.
    const composition = { presets: [], extensions: [] };

plugins/spec-kit-copilot-wizard/extensions/speckit-wizard-canvas/composition/artifact-cli.mjs:350

  • This filter removes the CLI-derived command row, and the appended synthetic hook row has a newly constructed stack. If a project or preset overrides an extension's hook command, the replacement reports the extension as active and loses the resolver's actual winning/hidden layers. Reclassify the original artifact as a hook while preserving its stack and description instead of replacing it.
    // Strip extension-provided command artifacts whose name matches a
    // declared hook command from the same extension. The hook artifact
    // above replaces them.
    const filtered = artifacts.filter((artifact) => {

plugins/spec-kit-copilot-wizard/extensions/speckit-wizard-canvas/composition/artifact-cli.mjs:108

  • The upstream artifact contract sets presetId/presetName to null for extension layers and identifies their owner through sourceId. Passing those fields through means extension rows never satisfy the presetId checks later in this module, and UI lookups such as phase-runtime.js:793-796 cannot associate extension commands with their extension. Translate extension sourceId into the wizard's ownership shape (or migrate all consumers to sourceId) before summarizing/rendering.
        layer: layer.layer == null ? "core" : layer.layer,
        presetId: layer.presetId ?? null,
        presetName: layer.presetName ?? null,
        sourceId: layer.sourceId ?? null,

plugins/spec-kit-copilot-wizard/extensions/speckit-wizard-canvas/canvas-runtime/composition-apply.mjs:302

  • This return shape removed stage2Needed, but the unchanged refresh dispatcher still tests !fast.stage2Needed (canvas-runtime/dispatch.mjs:100). For a novel command or stack directive, the missing field is undefined, so the dispatcher incorrectly treats the run as complete and never invokes LLM pipeline inference. Preserve the existing result contract or update the dispatcher in the same change.
        return { ok: true, reason, pipelineFastPath: fastPath.canSynthesize };

plugins/spec-kit-copilot-wizard/extensions/speckit-wizard-canvas/extension.mjs:212

  • runFastComposition catches CLI failures and returns { ok: false }, so this await never enters the catch block. The code then marks composition successful and immediately marks boot ready, hiding the overlay; therefore an old CLI lacking artifact list is not surfaced as described. Inspect the returned result and expose its failure through a persistent boot or composition error state.
    try {
        await runFastComposition(inst, { reason: "boot" });
        await snapshot(inst);
        tracker.ok("composition");
  • Files reviewed: 19/20 changed files
  • Comments generated: 0 new
  • Review effort level: Balanced

@nicolehaugen

Copy link
Copy Markdown
Contributor Author

Superseded by #18, which contains the same commits on a branch created directly in this repo off main (instead of coming from a fork).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants