Move wizard composition to specify artifact CLI (blocked by spec-kit#4305) - #17
Move wizard composition to specify artifact CLI (blocked by spec-kit#4305)#17nicolehaugen wants to merge 13 commits into
Conversation
Complements artifact-cli.test.mjs (fixture round-trip) with three real-shape guards: * Live-CLI test invokes real `specify artifact list/info` on a scaffolded workspace and asserts wizard-contract fields (id/kind/stack, layer vocabulary, exactly-one-active). Skips when `specify` isn't on PATH. * Fixture-drift tests replay a committed snapshot of real CLI output and assert the field set the shape mapper reads is present. Guards against silent CLI shape changes without needing the binary in CI. * fixtures/README.md documents regeneration. Full suite: 211/211 pass.
On a warm cache the server's `bootAsync` reaches `phase: ready` before
the browser's first paint. The old flow relied on JS to (a) populate the
overlay content and (b) hide `main.app-body`, then almost immediately
flipped the overlay to `is-hidden` in the same microtask cycle. The
browser composited populate + hide into one frame and the user saw a
blank body flip straight to the loaded app with no boot indicator.
Three-part fix so the overlay is guaranteed to paint:
* Static markup in `index.html` — pre-render the overlay panel with
title + subtitle so it is visible from the very first paint, before
any module fetch/parse.
* CSS-level `main.app-body { visibility: hidden }` — no longer
depends on JS running to keep the app body hidden underneath.
* JS-side minimum visible time (`MIN_OVERLAY_MS = 450`) — even when
the state fetch resolves in a single frame, the hide is deferred via
`setTimeout` so the overlay stays up long enough to register.
Boot's `hydrateCatalogs` used to walk preset → extension → bundle serially, and each hydrator walked its 2–3 source URLs serially inside `hydrateFromCatalogSources`. That's ~8 GitHub GETs strictly serial on a cold cache, plus 3 sequential `specify <kind> list` shell-outs, for what is entirely disjoint state. Two changes: * Run `hydratePresetsForSources` / `hydrateExtensionsForSources` / `hydrateBundlesForSources` via `Promise.all` — they touch independent cache slices. * Inside `hydrateFromCatalogSources`, `Promise.all` the per-source `fetchCatalogJson` calls before folding into the items array. Order of items is preserved because we still iterate the resolved array in source order. All 211 tests pass.
This reverts commit bb5edf2.
The catalog boot step called specify artifact info once per artifact via �xecFileSync in a serial loop. With a real workspace stack (~70 artifacts across 4 presets + 1 extension), that's ~70 shell-outs, each one blocking the Node event loop for its full duration. Impact: /api/state and SSE could not be answered during boot, so the UI sat on 'Loading catalogs' for the full 108s wall time of the loop, even though the HTTP server was up. From the user's perspective the wizard 'hung'. Fix: - Swap the default runner to a promisified execFile so each shell-out yields the event loop instead of hard-blocking it. - Fan the info-per-id calls out with Promise.all — safe now that spawn is non-blocking. - Await the runner return so injected sync test runners (which return a Buffer/string) still work unchanged. Measured on the current workspace (68 artifacts): before: 108s serial sync, HTTP frozen throughout after: 13s parallel async, HTTP responsive throughout (~8x faster). All 211 tests still pass. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2d44fd06-6323-4654-8a80-b756561ef669
The catalog boot step was fragile in three separate ways beyond the
sync exec loop already fixed in the artifact-info fan-out:
1. fetchCatalogJson had NO timeout. A stalled socket (slow DNS, TCP
loss, CDN outage) would block the fetch forever and freeze boot on
'Loading catalogs' with no recovery path. Added AbortSignal.timeout
(15s per fetch).
2. specifyRun had NO timeout either. A wedged CLI (uv resolver stuck,
PATH resolution hang) had the same failure mode. Added a 20s kill
timer; on expiry we resolve with the partial stdout (callers already
tolerate empty output).
3. hydrateCatalogs awaited the three groups (presets, extensions,
bundles) serially, and hydrateFromCatalogSources awaited each source
inside a group serially. Both are independent I/O — swapped for
Promise.all at each level so total time is bounded by the slowest
single call, not the sum.
Measured on the current workspace (7 catalogs, 68 artifacts):
before: 108s serial sync, HTTP frozen throughout
after: ~14s parallel async, HTTP responsive throughout (~8x faster,
and now failure-bounded instead of unbounded)
All 211 tests still pass.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 2d44fd06-6323-4654-8a80-b756561ef669
Boot was making 68 sequential `specify artifact info` shell-outs per artifact to reconstruct composition stacks — the flow that regressed catalog-loading to 27-60s on Windows. spec-kit PR #4305 makes `artifact list --json` return the full per-row stack, so the wizard needs exactly ONE shell-out to build composition.
Wizard-side changes:
- `buildCompositionFromCli` now consumes a single `list --json` call and feeds rows directly through `shapeArtifact`. Presets/extensions summaries are folded from artifact stacks (accepted edge case: a preset that contributes zero currently-active artifacts won't appear — living with it until upstream ships `preset list --json`).
- Removed `specifyArtifactInfo` — dead post-refactor. Don't reintroduce a per-artifact fan-out on the boot critical path.
- Deleted `project-scanner.mjs::scanComposition` and its `.specify/{presets,extensions}.json` reads. Neither file is written by any CLI version; the code was dead. Removed the now-unused `readBoundedJson` import.
- Governing principle: CLI is the source of truth for composition. No direct fs reads of `.registry`/`.yml` from the wizard, ever.
Test-side changes:
- Unit fixtures flattened: list rows now embed `stack` (no separate `info` map). `fakeRunner` simplified to only handle `list`.
- Fixture-drift test rewritten to require `stack` on list rows. Skips gracefully when the on-disk snapshot is pre-#4305 (regen once upstream ships).
- Live-CLI test skips when the installed CLI's `list --json` doesn't yet emit `stack` — same rationale.
- Deleted obsolete `live-cli-info.json` fixture; updated README.
- Deleted `scanWorkspace drops malformed composition entries` test — it exercised the deleted `scanComposition` path.
208 tests: 207 pass, 1 skip (drift, until fixture regen).
Real `specify artifact list --json` output now carries per-row `stack` (spec-kit#4305 has landed). Fixture-drift and live-CLI tests are now actively guarding (208/208 pass, 0 skips) instead of skipping under the pre-#4305 detector.
Boot overlay previously bundled two unrelated phases into one 'catalog' step: remote catalog JSON fetches (~150ms parallel) AND the composition CLI build (specify artifact list --json, ~2s cold). When boot felt slow, you couldn't tell which side was blocked. Split them so each shows independently in the overlay: - `catalog` (Loading catalogs) \u2014 hydratePresets/Extensions/Bundles, remote HTTPS + `specify <group> list` per group, all parallel. - `composition` (Building composition) \u2014 single `specify artifact list --json` call + shape mapping. Now a hang on either side is visible at a glance without log scraping.
Comments now describe current behavior only. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2d44fd06-6323-4654-8a80-b756561ef669
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2d44fd06-6323-4654-8a80-b756561ef669
- project-scanner.mjs: rewrite composition-block comment to describe current behavior only; drop 'used to look at' language and the fabricated AGENTS.md citation. - artifact-cli.mjs: drop the phantom 'AGENTS.md: CLI is the source of truth for composition.' line from the doc header. - pipeline-fast-path.mjs: rename 'LLM Stage 2' → 'LLM path' (there was no Stage 1) and 'Fast path' → 'Deterministic path' to match the actual code. - ui/index.html, ui/boot.js: drop '- Dev' suffix from the wizard title in all four spots to prepare for check-in. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 2d44fd06-6323-4654-8a80-b756561ef669
There was a problem hiding this comment.
Pull request overview
Moves wizard composition assembly to the forthcoming specify artifact list --json API, pending spec-kit#4305.
Changes:
- Adds CLI-backed artifact mapping, hook enrichment, and pipeline fast-path selection.
- Separates composition boot progress and parallelizes timeout-bounded catalog loading.
- Removes filesystem composition assembly and updates tests/dependencies.
Show a summary per file
| File | Description |
|---|---|
ui/styles/boot.css |
Hides app content during boot. |
ui/index.html |
Adds initial boot markup. |
ui/boot.js |
Adds composition status and minimum display time. |
test/state-and-scanner.test.mjs |
Removes obsolete scanner test. |
test/composition.test.mjs |
Removes legacy composition tests. |
test/boot-progress.test.mjs |
Covers the new boot step. |
test/artifact-cli.test.mjs |
Tests CLI composition mapping. |
project-scanner.mjs |
Removes filesystem composition scanning. |
package.json |
Updates js-yaml. |
package-lock.json |
Locks the dependency update. |
extension.mjs |
Splits catalog and composition boot phases. |
composition/pipeline-fast-path.mjs |
Adds deterministic pipeline selection. |
composition/hooks.mjs |
Extracts hook metadata. |
composition/collect.mjs |
Removes legacy filesystem collection. |
composition/assembler.mjs |
Removes legacy composition assembly. |
composition/artifact-cli.mjs |
Adds the CLI-backed composition adapter. |
catalog/sources.mjs |
Adds fetch timeouts. |
catalog/shared.mjs |
Parallelizes hydration and bounds CLI calls. |
canvas-runtime/composition-apply.mjs |
Uses the new composition pipeline. |
canvas-runtime/boot-progress.mjs |
Registers composition boot progress. |
Review details
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Files not reviewed (1)
- plugins/spec-kit-copilot-wizard/extensions/speckit-wizard-canvas/package-lock.json: Generated file
- Files reviewed: 19/20 changed files
- Comments generated: 4
- Review effort level: Balanced
| return { | ||
| // CLI `null` layer = built-in; wizard code expects "core". | ||
| layer: layer.layer == null ? "core" : layer.layer, | ||
| presetId: layer.presetId ?? null, |
|
|
||
| await applyComposition(inst, payload); | ||
| return { ok: true, reason, stage2Needed: stage2.needed }; | ||
| return { ok: true, reason, pipelineFastPath: fastPath.canSynthesize }; |
There was a problem hiding this comment.
@copilot The decision was to remove references to "stage2" thorughout code and instead is the 'fastpath' - make the fix accordingly
| await runFastComposition(inst, { reason: "boot" }); | ||
| await snapshot(inst); | ||
| tracker.ok("composition"); |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Review details
Files not reviewed (1)
- plugins/spec-kit-copilot-wizard/extensions/speckit-wizard-canvas/package-lock.json: Generated file
Suppressed comments (7)
Previously missed (4) — in code that hasn't changed since the last review.
plugins/spec-kit-copilot-wizard/extensions/speckit-wizard-canvas/catalog/shared.mjs:61
- On timeout, returning partial stdout is unsafe for these line-based list parsers: a truncated prefix can still parse successfully and mark only the first installed presets/extensions active. Treat a timed-out command as failed so callers use their empty/fallback path rather than publishing a plausible but incomplete installed set.
const timer = setTimeout(() => {
try { child.kill(); } catch { /* best-effort */ }
done(stdout || null);
}, timeoutMs);
plugins/spec-kit-copilot-wizard/extensions/speckit-wizard-canvas/test/artifact-cli.test.mjs:6
- The replacement suite covers the new CLI mapper and fast path, but deleting
composition.test.mjsalso removes all tests for still-livecanonical.mjs,effective-phases.mjs, andresolvePipelineEntrybehavior. No equivalent tests remain elsewhere, including regression coverage for bare extension IDs and hook filtering. Preserve those unaffected tests in focused test files.
import {
buildCompositionFromCli,
} from "../composition/artifact-cli.mjs";
import { computePipelineFastPath } from "../composition/pipeline-fast-path.mjs";
plugins/spec-kit-copilot-wizard/extensions/speckit-wizard-canvas/project-scanner.mjs:132
- Starting the scan with empty composition breaks the existing phase gate in
canvas-runtime/snapshot-builder.mjs:113-116, which checksscan.composition.extensionsbeforesnapshot()overlaysinst.cachedComposition. Consequently,taskstoissuesremains gated even when its provider extension is installed. Apply the cached CLI composition before building the snapshot, or move this gate to a post-overlay source.
// Composition data comes from `runFastComposition` (CLI-driven, see
// composition/artifact-cli.mjs) and is applied via `overlayCachedComposition`
// after this scan runs. Start empty so the overlay step has a clean base.
const composition = { presets: [], extensions: [] };
plugins/spec-kit-copilot-wizard/extensions/speckit-wizard-canvas/composition/artifact-cli.mjs:350
- This filter removes the CLI-derived command row, and the appended synthetic hook row has a newly constructed stack. If a project or preset overrides an extension's hook command, the replacement reports the extension as active and loses the resolver's actual winning/hidden layers. Reclassify the original artifact as a hook while preserving its stack and description instead of replacing it.
// Strip extension-provided command artifacts whose name matches a
// declared hook command from the same extension. The hook artifact
// above replaces them.
const filtered = artifacts.filter((artifact) => {
plugins/spec-kit-copilot-wizard/extensions/speckit-wizard-canvas/composition/artifact-cli.mjs:108
- The upstream artifact contract sets
presetId/presetNameto null for extension layers and identifies their owner throughsourceId. Passing those fields through means extension rows never satisfy thepresetIdchecks later in this module, and UI lookups such asphase-runtime.js:793-796cannot associate extension commands with their extension. Translate extensionsourceIdinto the wizard's ownership shape (or migrate all consumers tosourceId) before summarizing/rendering.
layer: layer.layer == null ? "core" : layer.layer,
presetId: layer.presetId ?? null,
presetName: layer.presetName ?? null,
sourceId: layer.sourceId ?? null,
plugins/spec-kit-copilot-wizard/extensions/speckit-wizard-canvas/canvas-runtime/composition-apply.mjs:302
- This return shape removed
stage2Needed, but the unchanged refresh dispatcher still tests!fast.stage2Needed(canvas-runtime/dispatch.mjs:100). For a novel command or stack directive, the missing field isundefined, so the dispatcher incorrectly treats the run as complete and never invokes LLM pipeline inference. Preserve the existing result contract or update the dispatcher in the same change.
return { ok: true, reason, pipelineFastPath: fastPath.canSynthesize };
plugins/spec-kit-copilot-wizard/extensions/speckit-wizard-canvas/extension.mjs:212
runFastCompositioncatches CLI failures and returns{ ok: false }, so thisawaitnever enters the catch block. The code then marks composition successful and immediately marks boot ready, hiding the overlay; therefore an old CLI lackingartifact listis not surfaced as described. Inspect the returned result and expose its failure through a persistent boot or composition error state.
try {
await runFastComposition(inst, { reason: "boot" });
await snapshot(inst);
tracker.ok("composition");
- Files reviewed: 19/20 changed files
- Comments generated: 0 new
- Review effort level: Balanced
|
Superseded by #18, which contains the same commits on a branch created directly in this repo off |
Warning
Blocked by github/spec-kit#4305.
Do not merge until #4305 lands and a
specify-clirelease containingspecify artifact list --jsonis published to PyPI. Once that ships, bumpthe version floor note in
skills/speckit-cli-setup/SKILL.mdand un-draft.Summary
Move the wizard's Composition tab off direct filesystem inspection and onto a
single
specify artifact list --jsoncall. The CLI returns one row perartifact carrying the full composition
stack: [...], which is everything thewizard needs to render commands, templates, scripts, and hooks.
Depends on
specify artifactcommand spec-kit#4305 — addsspecify artifact(list/info) with deterministiccontribution IDs and per-row stack. This PR does not work at runtime
without it.
What changes
composition/artifact-cli.mjs(new sole source of composition): callsspecify artifact list --jsononce at boot, maps rows into the wizard'sartifact / preset / extension shapes, and produces the composition summary.
project-scanner.mjs: no longer reads.specify/{presets,extensions}.jsonfor composition. Starts empty and lets
overlayCachedCompositionapply theCLI-derived data after the scan.
composition/pipeline-fast-path.mjs: decides between the deterministicpipeline (canonical spine +
replace-only overrides) and the LLM pipeline(
prompts/composition.mjs::inferPipeline) needed when an extension adds anon-canonical command or uses
wrap/prepend/append.tracker steps so the user sees which phase is running.
test/artifact-cli.integration.test.mjs)and its fixture — that layer belongs in the spec-kit repo alongside the CLI
it exercises. Unit tests here inject a fake runner so CI is green regardless
of which
specify-cliversion is installed.Verification
207/207 unit tests pass (unit tests inject a fake
specifyrunner).Playwright DOM-diff matrix — captured the wizard side panel from a main-branch
plugin variant vs. this branch across five scenarios:
copilot-sub-agentspresetpirate-full-presetagent-contextextension onlyEach scenario snapshots five surfaces (Composition → Commands / Templates /
Scripts / Hooks, plus top-level Phases). 25/25 pairs are byte-identical
after normalizing internal
[ref=...]handles. No user-visible regressions.Runtime behavior when the CLI is too old
Right now the wizard will surface an error and empty composition if the
installed
specify-clilacksartifact list --json. That's acceptable whilethis PR is draft; when we un-draft, the floor version bump in
speckit-cli-setupguarantees users get a compatible CLI.