From db4762e6a5dc22a116812cdbcb180ffc93bb257d Mon Sep 17 00:00:00 2001 From: Ryan Dombrowski Date: Wed, 12 Aug 2026 13:29:18 -0400 Subject: [PATCH] docs: commit the architectural rationale and the Gateway acceptance corpus MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The founding thesis, the invariants, and the deliberate non-goals existed only in session transcripts; the plan file that held the original proposal was overwritten, and docs/COMPOSER.md advertised IMPLEMENTATION_LOG.md as Composer history although that log ends before Composer existed. - docs/ARCHITECTURE.md — the document someone joining in a year needs: the problem, the UI-over-existing-seams thesis, the core loop, why the AI proposes and deterministic machinery decides, the product model, why flows sit above surfaces, the browser/agent boundary, nine invariants, and what Composer deliberately does not do. - acceptance/gateway-corpus/ — the twelve prompts verbatim and immutable, the paraphrase set, the expected classification with each remaining item attributed to vocabulary/runtime/product-model, and the harness. Live model calls, so deliberately NOT in CI; the README carries the operational rules learned the hard way (pace it, sample more than once, discard adapter failures) and evidence/ is git-ignored. The harness is now path-portable. - docs/COMPOSER.md + README.md point at the current record instead of the historical one. Co-Authored-By: Claude Fable 5 --- README.md | 2 +- acceptance/gateway-corpus/.gitignore | 1 + acceptance/gateway-corpus/EXPECTED.md | 33 ++ acceptance/gateway-corpus/README.md | 41 +++ acceptance/gateway-corpus/corpus.mjs | 294 +++++++++++++++++ acceptance/gateway-corpus/harness.mjs | 366 ++++++++++++++++++++++ acceptance/gateway-corpus/paraphrases.mjs | 55 ++++ docs/ARCHITECTURE.md | 128 ++++++++ docs/COMPOSER.md | 10 +- 9 files changed, 926 insertions(+), 4 deletions(-) create mode 100644 acceptance/gateway-corpus/.gitignore create mode 100644 acceptance/gateway-corpus/EXPECTED.md create mode 100644 acceptance/gateway-corpus/README.md create mode 100644 acceptance/gateway-corpus/corpus.mjs create mode 100644 acceptance/gateway-corpus/harness.mjs create mode 100644 acceptance/gateway-corpus/paraphrases.mjs create mode 100644 docs/ARCHITECTURE.md diff --git a/README.md b/README.md index 3ceaf55..9cc47e4 100644 --- a/README.md +++ b/README.md @@ -138,7 +138,7 @@ AI proposal (hosted | local | scripted) ← the only seam that varies → registry renderer (native + wireframe fallback) ``` -The proposal provider differs; **the deterministic authority does not**. Hosted proposals come from managed Claude via the AI Gateway; local proposals go through the agent to your endpoint; scripted replays the intent's worked example — all through identical gates, in your browser for browser projects, or against files on disk for repository projects. Contract-time emission (catalogs from contracts) and runtime rendering (AG-UI events → A2UI processor → registry) are the same machinery the Studio exhibit demonstrates. Deeper docs: [docs/COMPOSER.md](docs/COMPOSER.md), [docs/renderer-abstraction.md](docs/renderer-abstraction.md), [docs/AUDIT.md](docs/AUDIT.md). +The proposal provider differs; **the deterministic authority does not**. Hosted proposals come from managed Claude via the AI Gateway; local proposals go through the agent to your endpoint; scripted replays the intent's worked example — all through identical gates, in your browser for browser projects, or against files on disk for repository projects. Contract-time emission (catalogs from contracts) and runtime rendering (AG-UI events → A2UI processor → registry) are the same machinery the Studio exhibit demonstrates. Deeper docs: [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) (why the product is shaped this way — the thesis, the invariants, what Composer deliberately does not do), [docs/COMPOSER.md](docs/COMPOSER.md) (what runs where), [docs/renderer-abstraction.md](docs/renderer-abstraction.md), [docs/AUDIT.md](docs/AUDIT.md). The twelve-prompt enterprise acceptance corpus, its expected classification, and a reproducible harness live in [acceptance/gateway-corpus](acceptance/gateway-corpus/README.md). ## The Studio (second app) diff --git a/acceptance/gateway-corpus/.gitignore b/acceptance/gateway-corpus/.gitignore new file mode 100644 index 0000000..48a58e6 --- /dev/null +++ b/acceptance/gateway-corpus/.gitignore @@ -0,0 +1 @@ +evidence/ diff --git a/acceptance/gateway-corpus/EXPECTED.md b/acceptance/gateway-corpus/EXPECTED.md new file mode 100644 index 0000000..3dee575 --- /dev/null +++ b/acceptance/gateway-corpus/EXPECTED.md @@ -0,0 +1,33 @@ +# Expected classification + +The state of each prompt against the current product. **Solved** means the requested interface is achievable as a governed artifact. **Partial** means a real version builds but a named part of the request is not representable. **Gap** means the vocabulary to express it honestly does not exist, and Composer says so rather than faking it. + +Runtime caveats are called out because they are the product's stated boundary: Composer composes and governs *representations*; it does not bind live data or execute workflows. + +| # | Prompt | Status | Owner of what remains | +|---|---|---|---| +| 1 | Service Catalog Explorer | **Partial** | Design-system vocabulary — a dependent selector needs option lists that react to another control; `Select.options` is a static literal and the change handlers are dropped props. What builds is a static configuration form | +| 2 | Filing Directory Browser | **Partial** | Design-system vocabulary — no combobox / type-ahead / query vocabulary. The shadcn contract's own `select.whenNotToUse` names the missing component | +| 3 | Estimate Workspace | **Solved** | — | +| 4 | Create Estimate | **Solved** as a flow | Runtime — the post-create read-back is live data | +| 5 | Convert Estimate to Project | **Solved** as a flow | Runtime — same | +| 6 | Direct Project Creation | **Solved** as a flow | Runtime — same | +| 7 | Project Workspace | **Solved** | — (line-item tables render natively; the renderer defect that once emptied shadcn tables is fixed and regression-guarded) | +| 8 | Mutation Confirmation | **Solved** for one state | Product model — a reusable *multi-variant* composition (calm vs destructive from one definition) is parameterized reuse, deliberately deferred | +| 9 | Operation Progress | **Gap** | Design-system vocabulary + one governance rule — indeterminate progress is the request's core demand and the shadcn rule requires a determinate value; Astryx has no progress vocabulary and refuses honestly | +| 10 | File / Artifact Card | **Partial** | Runtime — metadata, states, and a download *action* are representable; downloading is not | +| 11 | Gateway Error / Recovery | **Gap** | Design-system vocabulary — field-level error binding does not exist, and the alert action slot is still dropped by the profile pending upstream work | +| 12 | Full demo composition | **Solved** as a flow | Runtime — read-backs and context carriage between steps | + +**Tally: 6 solved, 4 partial, 2 gaps.** At the start of the investigation that produced this corpus, none of the twelve was achievable as intended. + +## How to read a failure + +- **A gate failure (S1/S2/S3)** is the model producing something the design system does not allow. Expected occasionally; the repair loop gets a bounded number of tries. +- **An emit refusal** is the surface being contract-legal but unrepresentable — a declared casualty, a dissolution rule, a missing join key. Since the pipeline-trust milestone these ride the repair loop with the refusal text as the instruction, so a terminal refusal means repair genuinely could not fix it. +- **A planning vocab-gap** is the honest one: the planner said the request needs a capability the design system does not have, and no generation was attempted. +- **An adapter error** is infrastructure, not product. Re-measure. + +## Keeping this current + +Update this table when a milestone changes a verdict, and say what changed it. Do not soften a classification to make a release look better — the corpus's only value is that it has never been graded generously. diff --git a/acceptance/gateway-corpus/README.md b/acceptance/gateway-corpus/README.md new file mode 100644 index 0000000..65f2fe4 --- /dev/null +++ b/acceptance/gateway-corpus/README.md @@ -0,0 +1,41 @@ +# The Gateway acceptance corpus + +Twelve prompts describing a real enterprise workflow product — an IP-services Gateway: service configuration, entity directories, estimates, estimate→project conversion, project workspaces, confirmation boundaries, operation progress, artifact cards, error recovery, and one end-to-end lifecycle. + +**This is the honest measuring stick for "can Composer build real enterprise software?"** It is not a demo script and not a unit-test fixture. It was written by the product owner as an acceptance suite before the product could satisfy any of it, and it has driven four milestones of measured improvement since. + +## Rules + +1. **The prompts are immutable.** Do not rewrite, simplify, shorten, or "fix" them. Their awkward parts are the point — real requests are not tidy. If a prompt cannot be satisfied, that is a finding about the product, never a reason to edit the corpus. +2. **The goal text is exactly the body of each prompt.** Titles, tool names, and the author's side-notes are metadata and are never pasted into the product. +3. **New prompts are additions, never replacements.** Preserve the original twelve and their numbering. +4. **Do not use the corpus to justify new vocabulary on its own.** A gap it reveals is evidence for a decision, not the decision. + +## What is here + +| File | What it is | +|---|---| +| `corpus.mjs` | The twelve prompts verbatim, with their metadata | +| `EXPECTED.md` | The current per-prompt classification, with the first blocker and which layer owns it | +| `harness.mjs` | A reproducible runner: drives the real pipeline per prompt × design system and writes one evidence file per cell | +| `paraphrases.mjs` | Reworded versions of four prompts, used to prove an improvement generalized beyond the exact strings | + +## Running it + +The harness makes **live model calls** through the hosted gateway (or a local model). It is therefore **not part of CI** and never should be: it costs real inference, it is nondeterministic, and a red result usually means "the model had a bad day," which is not a signal CI can act on. + +```bash +node acceptance/gateway-corpus/harness.mjs all +``` + +Evidence lands in `acceptance/gateway-corpus/evidence/` (git-ignored), one JSON file per prompt × design system, containing the plan, every generation attempt with its gate results, the emitted output, raw validator findings, and renderer coverage. + +Practical notes learned the hard way: + +- **Run it sequentially and paced.** After a few hundred calls in a day the shared provider begins refusing build-paced traffic while still answering slow probes; bursts make cells fail with adapter errors that say nothing about the product. The harness paces itself and retries, and it is resumable — existing evidence files are skipped. +- **Generation is stochastic.** One sample per cell is what a user experiences; two or three samples are what a *conclusion* needs. Composition-heavy prompts flip between runs. +- **An adapter failure is not a product result.** Discard and re-measure those cells rather than reporting them. + +## Provenance + +Delivered by the product owner on 2026-08-10 with the instruction: *"Treat these prompts as immutable acceptance tests. Do not rewrite them. Do not simplify them. Investigate why they succeed or fail."* The corpus has since been the acceptance basis for the pipeline-trust, generation-quality, and flow-composition milestones. diff --git a/acceptance/gateway-corpus/corpus.mjs b/acceptance/gateway-corpus/corpus.mjs new file mode 100644 index 0000000..2633ae0 --- /dev/null +++ b/acceptance/gateway-corpus/corpus.mjs @@ -0,0 +1,294 @@ +// The twelve Gateway MCP prompts — VERBATIM acceptance corpus (owner, 2026-08-10). +// "Treat these prompts as immutable acceptance tests. Do not rewrite them. Do not +// simplify them. Investigate why they succeed or fail." +// body = the goal text fed to Composer, exactly as delivered. title/tool = metadata only. + +export const CORPUS = [ + { + n: 1, + title: "Service Catalog Explorer", + tool: "get_service_catalog", + body: `A service catalog explorer for configuring an IP services request. +Start with a prominent Service Type selector showing the available +services by human-readable name. After a service is selected, show a +dependent Workflow selector populated with the workflows available for +that service. +Below that, organize the remaining available configuration options +such as jurisdiction, source locale, country, and translation quality +into a clean form. Disabled options should remain visible but clearly +unavailable. Show descriptions or useful metadata as contextual help +when available. +As selections are made, show a compact summary of the current service +configuration so the user can understand exactly what they have +chosen. +The interface should feel like the beginning of an estimate or project +creation workflow rather than a generic settings form.`, + }, + { + n: 2, + title: "Filing Directory Browser", + tool: "get_filing_directory", + body: `A searchable filing directory browser for selecting people, +organizations, agents, applicants, case managers, and other +filing-related entities. +Provide a directory type selector followed by a searchable list or +combobox of matching entries. +Each result should emphasize the entity's name, with its directory +kind, relevant contact information, and location or address shown as +secondary information when available. +Selecting an entry should open or reveal a compact detail view so the +user can confidently verify the entity before using it in a filing, +estimate, or project workflow. +Support loading, no-results, unavailable-entry, and selected states. +The interface should feel like selecting a trusted business entity +from an enterprise legal-services directory, not choosing a value from +a basic dropdown.`, + }, + { + n: 3, + title: "Estimate Workspace", + tool: "get_estimate", + body: `An estimate detail workspace for an IP services estimate. +At the top, show the estimate reference or identifier, current status, +and the most important service context. +Below it, organize the estimate into clear sections for service +details, jurisdictions or languages when available, pricing and +currency, important dates, and other meaningful estimate information +returned by the service. +Warnings should be visible without overwhelming the estimate. +If documents or estimate-related links are available, show them in a +separate documents area. +Provide a clear place for the next available action, especially when +the estimate can proceed into project creation. +The interface should make an estimate easy to review with a client or +colleague during a live demonstration without exposing raw API JSON.`, + }, + { + n: 4, + title: "Create Estimate", + tool: "create_estimate", + body: `A guided Create Estimate interface for an IP services workflow. +Begin with service configuration: Service Type, Workflow, jurisdiction +and other relevant service options supplied by the service catalog. +Organize the remaining estimate inputs into logical sections rather +than presenting one long technical form. +Before submission, show a review panel summarizing exactly what will +be sent: selected service, workflow, jurisdiction, client/reference +information, and other important request details. +Creating the estimate requires explicit confirmation. Show a calm +confirmation step explaining that a new Test 1 estimate will be +created, with Cancel and Confirm and Create actions. Confirmation must +never be preselected. +After creation succeeds, transition into the resulting estimate detail +view and clearly indicate that the newly created estimate was +successfully retrieved from Gateway. +The experience should tell one continuous story: +Configure service → enter estimate details → review → confirm → create +→ view the resulting estimate.`, + }, + { + n: 5, + title: "Convert Estimate to Project", + tool: "convert_estimate_to_project", + body: `A Convert Estimate to Project workflow. +Start with a summary of the selected estimate so the user can verify +which estimate will be converted. +Show the project information required for conversion in a focused +form, including relevant deadline, recipient, reference, +application/publication, or other fields when they are part of the +supplied request data. +Clearly separate information inherited from the estimate from +information the user is supplying for the new project. +Before conversion, show a review step explaining that this action will +create a project from the estimate. +Require explicit confirmation with Cancel and Confirm Conversion actions. +After conversion succeeds, transition directly to the resulting +Project detail interface. Make the new project identifier and status +prominent and show that the project was successfully read back from +Gateway. +The workflow should visually communicate: +Estimate → conversion details → review → confirmation → project +created → project verified.`, + }, + { + n: 6, + title: "Direct Project Creation", + tool: "create_project", + body: `A guided Create Project interface for creating an IP services project +directly without first creating an estimate. +Start with service and workflow configuration, followed by the +project-specific information required by the request. +Organize the form into clear sections such as Service Configuration, +Project Details, Client or Reference Information, Jurisdiction or +Language Details, and Timing or Delivery information when those values +are available. +Before submission, show a concise review of the project that will be created. +Require an explicit confirmation step. Explain that this will create a +new Test 1 project and that the operation should not be submitted +repeatedly if its result is uncertain. +After successful creation, transition to the resulting Project detail +view and prominently show the new project identity and status. +The interface should tell a continuous story: +Configure → enter project details → review → confirm → create → +retrieve and inspect the project.`, + }, + { + n: 7, + title: "Project Workspace", + tool: "get_project", + body: `A project detail workspace for an IP services project. +Create a strong project header showing the project identifier or +reference, current state, service/workflow context, and the most +important client or project information. +Below the header, organize information into clear sections: +- Project overview +- Service and jurisdiction details +- Important dates +- Line items +- Documents or artifacts when available +- Additional details +Make Project Line Items a major part of the interface. Present them as +a clean table or structured list showing the canonical service, target +language, jurisdiction, status, quantity, or other confirmed line-item +information supplied by the project. +Do not make raw extension data part of the primary interface. If +additional extension information is useful, place it behind an +Additional Details disclosure. +Warnings should be noticeable but secondary to the project itself. +If a next action is available, show it as a contextual project action. +The interface should feel like the main workspace someone would use to +understand the state and contents of a real IP services project.`, + }, + { + n: 8, + title: "Mutation Confirmation", + tool: null, + body: `A reusable confirmation interface for an MCP operation that will +create, convert, update, or otherwise change Gateway data. +Show a concise human-readable summary of what is about to happen and +the important resource or request details affected by the operation. +For normal creation operations, use a calm review-and-confirm +treatment rather than a dangerous-action warning. +For destructive or high-impact operations, use stronger warning +treatment when the tool indicates that it is destructive or +non-idempotent. +Clearly distinguish the two choices: +Cancel +Confirm and Continue +Confirmation must never be selected automatically. +If the operation should not be blindly retried, communicate that +clearly but concisely. +The user should be able to understand what they are approving without +seeing raw MCP arguments or API payloads.`, + }, + { + n: 9, + title: "Operation Progress", + tool: null, + body: `An operation progress interface for a long-running Gateway task. +Show the resource being processed, current operation state, and +elapsed time since the operation was accepted. +If a recommended polling interval is available, communicate when the +next status check will occur. +If expected webhook events are available, show them as secondary information. +Do not invent a completion percentage when the service has not +supplied one. Use an indeterminate progress treatment instead. +Clearly distinguish: +Processing +Processing files +Waiting +Timed out +Failed +Completed +A timeout should look different from a failed business operation. +If the service supplies a recommended next action, show it prominently +below the progress state. +The interface should feel like tracking a real business operation, not +merely displaying a loading spinner.`, + }, + { + n: 10, + title: "File / Artifact Card", + tool: "download_file (future)", + body: `A reusable document and artifact card for files associated with +estimates, projects, orders, or other Gateway resources. +Make the filename the primary element. +Show available metadata such as document type, media type, and file +size beneath it. Do not show empty metadata labels when those values +are unavailable. +Provide a clear Download or Open action when the artifact is retrievable. +Support a compact document-list layout where several artifact cards +appear together. +Include states for: +Available for download +Metadata available but file unavailable +Downloading +Download failed +The design should work for PDFs, Word documents, spreadsheets, and +unknown file types without assuming that MIME type or file extension +is always supplied. +The interface should look like a professional business-document +attachment, not a consumer cloud-storage tile.`, + }, + { + n: 11, + title: "Gateway Error / Recovery State", + tool: null, + body: `A reusable error and recovery interface for an MCP Gateway tool. +Present the human-readable error message first, followed by only the +technical information useful for recovery. +Support visually distinct states for: +Validation problem +Confirmation required +Resource not found +Gateway or dependency unavailable +Timeout +Authorization or configuration problem +General operation failure +When field-level issues are available, show them beside or beneath the +affected fields. +When the error is retryable, provide a Retry action. If a retry delay +is supplied, communicate it. +When another MCP tool is suggested as the recovery action, present +that as a clear next step. +Make correlation information available as secondary troubleshooting detail. +Never present raw stack traces, bearer tokens, credentials, or +sensitive upstream responses. +The interface should help the user recover rather than simply announce +that something failed.`, + }, + { + n: 12, + title: "The full demo composition", + tool: null, + body: `An end-to-end Gateway MCP demo workspace showing the lifecycle of an +IP services request. +The experience should guide the user through: +1. Explore the available service catalog. +2. Select a Service Type and dependent Workflow. +3. Configure and create an estimate. +4. Explicitly review and confirm creation. +5. Display the newly created estimate after it is read back from Gateway. +6. Convert that estimate into a project. +7. Explicitly review and confirm the conversion. +8. Display the newly created project after it is read back from Gateway. +9. Show the project's canonical line items and important project details. +10. Surface any relevant next action. +Use a clear workflow or step-based structure so the viewer always +understands where they are in the lifecycle: +Service → Estimate → Project +Preserve the resulting estimate and project context as the user moves +through the workflow rather than making each tool invocation feel like +an unrelated screen. +Read operations should feel immediate. +Write operations should always have an explicit review and +confirmation boundary. +Successful writes should visually transition into the canonical +resource returned by the subsequent read-back so the viewer can see +that the operation created a real Gateway resource. +Warnings and errors should appear contextually without destroying the +current workflow state. +This is a professional IP-services workflow demonstration, not an API +testing console. Do not expose raw MCP JSON as the primary experience.`, + }, +]; diff --git a/acceptance/gateway-corpus/harness.mjs b/acceptance/gateway-corpus/harness.mjs new file mode 100644 index 0000000..77a5c39 --- /dev/null +++ b/acceptance/gateway-corpus/harness.mjs @@ -0,0 +1,366 @@ +// Gateway MCP corpus evidence harness. Makes LIVE model calls — never run in CI. +// See README.md in this directory for the rules and the operational notes. +// Drives the REAL pipeline: composer-core plan.ts (native TS import) → dspack-gen +// runPipeline (browser subpath, product-identical orchestrator) → dspack-emit — +// with the model turn on the production Worker (/api/propose), exactly like a +// hosted build in the product (adapter cloned from apps/composer/app/hosted-build.ts:118-151). +// +// Usage: +// node harness.mjs one run (n=1..12, ds=shadcn|astryx) +// node harness.mjs all full corpus, sequential, paced +// GW_BASE=... overrides the gateway origin (default production). + +import { readFileSync, writeFileSync, mkdirSync, realpathSync } from "node:fs"; +import { createHash } from "node:crypto"; +import { createRequire } from "node:module"; +// GW_CORPUS selects an alternate corpus module (e.g. ./paraphrases.mjs); +// GW_PREFIX prefixes evidence filenames so alternate runs never collide. +const CORPUS_MODULE = process.env.GW_CORPUS ?? "./corpus.mjs"; +const PREFIX = process.env.GW_PREFIX ?? "run"; +const { CORPUS } = await import(CORPUS_MODULE); + +// Repo root, derived from this file's own location (acceptance/gateway-corpus/). +const REPO = new URL("../../", import.meta.url).pathname.replace(/\/$/, ""); +// GW_GEN / GW_EMIT point the harness at local package builds (post-fix evidence +// runs on exact merged code before npm publish); default = the app's installed deps. +const GEN = realpathSync(process.env.GW_GEN ?? `${REPO}/apps/composer/node_modules/@aestheticfunction/dspack-gen`); +const EMIT = realpathSync(process.env.GW_EMIT ?? `${REPO}/apps/composer/node_modules/@aestheticfunction/dspack-emit`); +const BASE = process.env.GW_BASE ?? "https://composer.aesthetic-function.com"; +const EVIDENCE = new URL("./evidence/", import.meta.url).pathname; +mkdirSync(EVIDENCE, { recursive: true }); + +const { runPipeline } = await import(`${GEN}/dist/browser.js`); +const { AdapterOutputError } = await import(`${GEN}/dist/adapters/types.js`); +const emitApi = await import(`${EMIT}/dist/index.js`); +const plan = await import(`${REPO}/packages/composer-core/src/plan.ts`); + +const req = createRequire(`${EMIT}/dist/index.js`); +const Ajv2020 = req("ajv/dist/2020.js").default; +const addFormats = req("ajv-formats").default; + +// Contracts + profiles exactly as Composer loads them (demo-data.ts:20-24). +// Native renderer name sets are the verified pre-merge registries +// (packages/shadcn-renderers/src/registry.tsx:29-44 minus Astryx-only names; +// packages/astryx-renderers/src/registry.tsx:23-36) — regression-locked by +// registry-parity.test.ts. +function loadDs(contractPath, profilePath, native) { + const contract = JSON.parse(readFileSync(contractPath, "utf8")); + const profileJson = JSON.parse(readFileSync(profilePath, "utf8")); + const profile = emitApi.loadProfile(profileJson); + return { contract, profileJson, profile, native: new Set(native) }; +} +const DS = { + shadcn: loadDs( + `${REPO}/apps/composer/shadcn-v3-project/shadcn-ui.dspack.json`, + `${REPO}/apps/composer/shadcn-v3-project/shadcn-v3.profile.json`, + ["Alert", "AlertDialog", "Badge", "Button", "Card", "Column", "Dialog", "Select", "Table", "Text", "TextField"], + ), + astryx: loadDs( + `${REPO}/apps/composer/astryx-project/astryx.dspack.json`, + `${REPO}/apps/composer/astryx-project/astryx.profile.json`, + ["AlertDialog", "Badge", "Button", "Card", "Column", "Dialog", "List", "MetadataList", "SelectableCard", "Table", "Text", "TextField"], + ), +}; + +const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); +const sha = (s) => createHash("sha256").update(s).digest("hex").slice(0, 16); + +// ---- gateway adapter — clone of hosted-build.ts:118-151, absolute base URL ---- +const HOSTED_ID = "hosted-ai:claude-haiku-4.5"; +let lastCall = 0; +const GAP_MS = 3000; // gentle inter-call spacing; behavior otherwise product-identical +async function pace() { + const wait = lastCall + GAP_MS - Date.now(); + if (wait > 0) await sleep(wait); + lastCall = Date.now(); +} +const gatewayCalls = []; +const gatewayAdapter = { + id: HOSTED_ID, + async generate(request) { + await pace(); + const t0 = Date.now(); + let res; + try { + res = await fetch(`${BASE}/api/propose`, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ system: request.system, messages: request.messages, jsonSchema: request.jsonSchema }), + signal: AbortSignal.timeout(120_000), + }); + } catch (e) { + gatewayCalls.push({ ms: Date.now() - t0, status: "network-error" }); + throw new AdapterOutputError(HOSTED_ID, `hosted AI request failed: ${e?.message ?? e}`); + } + if (!res.ok) { + let msg = `hosted AI endpoint returned ${res.status}`; + try { + const body = await res.json(); + if (body && typeof body.message === "string") msg = body.message; + } catch {} + gatewayCalls.push({ ms: Date.now() - t0, status: res.status }); + throw new AdapterOutputError(HOSTED_ID, msg); + } + const body = await res.json(); + gatewayCalls.push({ ms: Date.now() - t0, status: 200 }); + const { json, raw, model, usage } = body ?? {}; + return { json, raw: raw ?? JSON.stringify(json), model: model ?? "hosted-ai", ...(usage ? { usage } : {}) }; + }, +}; + +// ---- deterministic-score replication (plan.ts:121-160, constants copied exactly) ---- +const STOP_WORDS = new Set( + "a an the of for to and or with in on at your you i want need me my our create build make show give please that this it is are can new".split(" "), +); +const tokenize = (s) => s.toLowerCase().replace(/[^a-z0-9\s]/g, " ").split(/\s+/).filter((w) => w.length > 2 && !STOP_WORDS.has(w)); +function wordsMatch(a, b) { + if (a === b) return true; + const min = Math.min(a.length, b.length); + let i = 0; + while (i < min && a[i] === b[i]) i++; + return i >= 4; +} +function replicateScores(goal, contract) { + const intents = (Array.isArray(contract.intents) ? contract.intents : []).filter((i) => i && typeof i.id === "string"); + const goalTokens = tokenize(goal); + const rows = intents.map((it) => { + const hay = tokenize(`${it.name ?? ""} ${it.description ?? ""}`); + const covered = goalTokens.filter((g) => hay.some((w) => wordsMatch(g, w))); + return { intent: it.id, score: covered.length, covered: [...new Set(covered)] }; + }); + return { goalTokens, rows }; +} + +// ---- deep A3: same AJV config as dspack-emit ajv.ts:35-39, but FULL error objects ---- +function deepA3(catalog, surface) { + const ajv = new Ajv2020({ strict: false, allErrors: true, validateFormats: true }); + addFormats(ajv); + ajv.addSchema(catalog, catalog.$id); + const validateAny = ajv.getSchema(`${catalog.$id}#/$defs/anyComponent`); + if (!validateAny) return { error: "anyComponent schema not found" }; + const instances = emitApi.extractInstances(surface); + const failing = []; + for (const inst of instances) { + if (!validateAny(inst)) { + failing.push({ + component: inst.component, + id: inst.id, + errorCount: (validateAny.errors ?? []).length, + errors: (validateAny.errors ?? []).map((e) => ({ + instancePath: e.instancePath, schemaPath: e.schemaPath, keyword: e.keyword, params: e.params, message: e.message, + })), + }); + } + } + return { totalInstances: instances.length, failingInstances: failing }; +} + +// ---- what the user would see (validation.ts:141-146 join; build.ts:274-291 explosion) ---- +function reproduceUserFacing(report) { + const validations = report?.emitted?.validations ?? []; + const v091 = validations.find((v) => v.a2uiVersion === "0.9.1") ?? validations[0]; + const validateCells = (v091?.gates ?? []).filter((g) => !g.pass).map((g) => { + const msg = (g.errors ?? []).join("; ") || g.name; + return { gate: g.gate ?? g.name, name: g.name, chars: msg.length, preview: msg.slice(0, 240) }; + }); + let buildFailureRows = 0; + for (const v of validations) for (const g of v.gates ?? []) if (!g.pass) buildFailureRows += (g.errors ?? ["gate failed"]).length; + // refusal path: buildFailure's emitted.refusal branch (build.ts:266-272); in the + // browser, browserEmit catches EmitSurfaceError → an A3 "emit-surface" finding, + // acknowledged when classifySurfaceRefusal proves a declared casualty. + const refusal = report?.emitted?.refusal ?? null; + return { validateCells, buildFailureRows, refusal }; +} + +// components used per attempt (structured walk, mirrors extractInstances' spirit +// over dspack surfaces, which use "component" discriminators) +function surfaceComponents(node, out = new Set()) { + if (Array.isArray(node)) { for (const x of node) surfaceComponents(x, out); return out; } + if (node && typeof node === "object") { + if (typeof node.component === "string") out.add(node.component); + for (const v of Object.values(node)) surfaceComponents(v, out); + } + return out; +} + +function gateSummary(report) { + const att = report?.attempts ?? []; + const last = att.at(-1); + const s = {}; + for (const g of last?.gates ?? []) s[g.gate] = g.status; + const emitted = report?.emitted; + if (emitted?.validations) { + for (const v of emitted.validations) { + for (const g of v.gates ?? []) { + const key = `${g.gate}@${v.a2uiVersion ?? "?"}`; + s[key] = g.pass ? "PASS" : "FAIL"; + } + } + } + if (emitted?.refusal) s.refusal = emitted.refusal; + return { attempts: att.length, lastAttemptGates: s, outcome: report?.outcome }; +} + +// ---- one evidence run ---- +async function runOne(n, dsName, retriesLeft = 2, retryCount = 0) { + const p = CORPUS[n - 1]; + const d = DS[dsName]; + if (!p || !d) throw new Error(`bad args n=${n} ds=${dsName}`); + const t0 = Date.now(); + const ev = { + promptN: n, title: p.title, tool: p.tool, ds: dsName, base: BASE, + goalSha256_16: sha(p.body), goal: p.body, + startedAt: new Date().toISOString(), retryCount, + }; + + // 1) planning — deterministic + replicated scores + hosted (raw before reconcile) + const det = plan.planDeterministic(p.body, d.contract); + const scores = replicateScores(p.body, d.contract); + const top = scores.rows.reduce((b, r) => (r.score > (b?.score ?? -1) ? r : b), null); + ev.planning = { + deterministic: det, + replicatedScores: scores, + replicationConsistent: top?.intent === det.intent, + }; + try { + const request = plan.buildPlanRequest(p.body, d.contract); + const result = await gatewayAdapter.generate(request); + ev.planning.hostedRaw = result.json; + ev.planning.hosted = plan.reconcilePlan(result.json, d.contract, p.body); + ev.planning.pathUsed = "hosted"; + ev.planning.intentClamped = + typeof result.json?.intent === "string" && result.json.intent !== ev.planning.hosted.intent; + ev.planning.feasibleCoerced = result.json?.feasible === false && ev.planning.hosted.feasible === true; + } catch (e) { + ev.planning.hostedError = String(e?.message ?? e); + ev.planning.hosted = det; + ev.planning.pathUsed = "deterministic-fallback"; // planning.ts:27-31 product behavior + } + const gp = ev.planning.hosted; + + // 2) product behavior on infeasible plan: vocab-gap turn, NO generation (state.tsx:1034-1041) + if (gp.feasible === false) { + ev.vocabGap = { missingCapability: gp.missingCapability, reason: gp.reason }; + ev.outcome = "vocab-gap"; + ev.durationMs = Date.now() - t0; + save(ev); + return ev; + } + + // 3) generation through the real orchestrator (defaults: maxRepairs 2, both A2UI versions) + const events = []; + const result = await runPipeline({ + contract: d.contract, + intent: gp.intent, + prompt: gp.restated, + adapter: gatewayAdapter, + emitProfile: d.profile, + onEvent: (e) => events.push(e), + }); + ev.pipeline = { + exitCode: result.exitCode, + outcome: result.report?.outcome, + gateSummary: gateSummary(result.report), + attemptComponents: (result.report?.attempts ?? []).map((a) => ({ + index: a.index, + components: a.surface ? [...surfaceComponents(a.surface)].sort() : [], + })), + report: result.report, + events, + }; + + // gateway congestion → retry the whole run after backoff (evidence idempotent) + const adapterErr = (result.report?.attempts ?? []).map((a) => a.adapterError).filter(Boolean).join(" | "); + if (result.report?.outcome === "failed-adapter" && /busy|rate.?limit|429|503|capacity|unavailable|usable proposal/i.test(adapterErr) && retriesLeft > 0) { + console.log(` ~ gateway congested (${adapterErr.slice(0, 80)}) — backing off 90s, retrying run`); + await sleep(90_000); + return runOne(n, dsName, retriesLeft - 1, retryCount + 1); + } + + // 4) final surface + A2UI messages + const finalSurface = result.report?.attempts?.at(-1)?.surface ?? result.surface ?? null; + let messages = result.surfaceMessages ?? null; + if (!messages && finalSurface) { + try { + const es = emitApi.emitSurface(finalSurface, d.contract, { profile: d.profile }); + messages = es.messages; + ev.emitFallback = { warnings: es.warnings }; + } catch (e) { + ev.emitRefusal = String(e?.message ?? e); + } + } + ev.hasFinalSurface = Boolean(finalSurface); + ev.hasMessages = Boolean(messages); + + // 5) deep emit + A3 with full AJV error objects, both versions + if (messages) { + ev.deep = {}; + for (const v of ["0.9.1", "1.0"]) { + try { + const out = emitApi.transformFromJson(d.contract, { a2uiVersion: v, surface: { messages }, profile: d.profile }); + ev.deep[v] = { + pass: out.validation.pass, + gates: out.validation.gates, + ajv: deepA3(out.catalog, { messages }), + }; + } catch (e) { + ev.deep[v] = { error: String(e?.message ?? e) }; + } + } + // 6) renderer coverage (pre-merge native sets; wireframe fallback is NOT failure) + const used = [...new Set(emitApi.extractInstances({ messages }).map((i) => i.component))].sort(); + ev.renderer = { + used, + native: used.filter((c) => d.native.has(c)), + wireframe: used.filter((c) => !d.native.has(c)), + }; + } + + // 7) user-facing diagnostics reproduction + ev.userFacing = reproduceUserFacing(result.report); + ev.durationMs = Date.now() - t0; + save(ev); + return ev; +} + +function save(ev) { + const file = `${EVIDENCE}${PREFIX}-${String(ev.promptN).padStart(2, "0")}-${ev.ds}.json`; + writeFileSync(file, JSON.stringify(ev, null, 1)); + const gates = ev.pipeline?.gateSummary?.lastAttemptGates ?? {}; + const gateStr = Object.entries(gates).map(([k, v]) => `${k}:${v}`).join(" "); + console.log( + `#${ev.promptN} ${ev.ds}: plan=${ev.planning?.hosted?.intent}(${ev.planning?.pathUsed})` + + `${ev.vocabGap ? " VOCAB-GAP" : ` outcome=${ev.pipeline?.outcome} exit=${ev.pipeline?.exitCode}`}` + + `${ev.renderer ? ` wf=${ev.renderer.wireframe.length}/${ev.renderer.used.length}` : ""}` + + ` [${gateStr.slice(0, 120)}] ${Math.round((ev.durationMs ?? 0) / 1000)}s → ${file.split("/").pop()}`, + ); +} + +// ---- CLI ---- +const [, , a, b] = process.argv; +if (a === "all") { + const { existsSync } = await import("node:fs"); + for (const dsName of ["shadcn", "astryx"]) { + for (const p of CORPUS) { + const file = `${EVIDENCE}${PREFIX}-${String(p.n).padStart(2, "0")}-${dsName}.json`; + if (existsSync(file) && b !== "--force") { + console.log(`#${p.n} ${dsName}: evidence exists, skipping (resume mode)`); + continue; + } + try { + await runOne(p.n, dsName); + } catch (e) { + console.error(`#${p.n} ${dsName} CRASHED: ${e?.stack ?? e}`); + writeFileSync(`${EVIDENCE}${PREFIX}-${String(p.n).padStart(2, "0")}-${dsName}.crash.json`, + JSON.stringify({ promptN: p.n, ds: dsName, crash: String(e?.stack ?? e) }, null, 1)); + } + await sleep(Number(process.env.GW_RUN_GAP ?? 10_000)); + } + } + console.log(`gateway calls total: ${gatewayCalls.length}`); +} else if (a && b) { + await runOne(Number(a), b); + console.log(`gateway calls: ${gatewayCalls.length}`); +} else { + console.log("usage: node harness.mjs | node harness.mjs all"); +} diff --git a/acceptance/gateway-corpus/paraphrases.mjs b/acceptance/gateway-corpus/paraphrases.mjs new file mode 100644 index 0000000..39dd99c --- /dev/null +++ b/acceptance/gateway-corpus/paraphrases.mjs @@ -0,0 +1,55 @@ +// Paraphrase-generalization corpus — REWORDED versions of representative prompts +// (owner requirement: prove improvements generalize beyond the exact corpus strings). +// Same functional demands, deliberately different vocabulary, structure, and phrasing. +// Mapping: P1↔corpus#1 (dependent selection), P2↔corpus#6 (multi-step create), +// P3↔corpus#7 (detail workspace + disclosure), P4↔corpus#9 (operation progress). + +export const CORPUS = [ + { + n: 1, + title: "Offering picker (paraphrase of #1)", + tool: "get_service_catalog", + body: `Build a screen where a client assembles an intellectual-property service request. +The top control picks the offering (shown with friendly names, not codes). Once an offering is chosen, a second control lists only the process variants that belong to it. +Underneath, lay out the rest of the request options — country coverage, original language, target market, and quality tier — as a tidy form. Options that can't be used right now should stay on screen but read as inactive, and any explanatory notes should appear as inline help. +Keep a small running recap of everything picked so far, so the person always knows the current shape of their request. +It should read like step one of ordering professional services, not like an app's preferences page.`, + }, + { + n: 2, + title: "Engagement setup (paraphrase of #6)", + tool: "create_project", + body: `A start-to-finish screen for opening a new services engagement without a prior quote. +Lead with choosing the offering and its process variant, then collect the engagement's own details. +Split the inputs into digestible groups — offering setup, engagement basics, who it's for, countries and languages, and dates or delivery expectations, whenever those apply. +Before anything is sent, present a short recap of exactly what will be opened. +Ask for a deliberate go-ahead, explaining a sandbox engagement will be opened and that re-submitting on uncertainty is unsafe. +When it succeeds, land the person on the engagement's own page with its number and state shown large. +One unbroken thread: choose → fill in → recap → approve → open → inspect what was opened.`, + }, + { + n: 3, + title: "Matter overview page (paraphrase of #7)", + tool: "get_project", + body: `The main page someone keeps open to track a professional-services matter. +A strong top band: the matter's number, where it stands, which offering and process it uses, and the client facts that matter most. +Then well-labeled regions: a summary, the offering and country specifics, key dates, the billable line entries, any files, and a catch-all for everything else. +The line entries deserve the most room — a clean grid listing each entry's service, output language, country, state, and count. +Keep machine-ish extension payloads out of the main page; if some of it helps, tuck it behind an expandable "More details" area. +Cautions should be easy to spot but never louder than the matter itself. +When there's an obvious next move, offer it right on the page.`, + }, + { + n: 4, + title: "Long job tracker (paraphrase of #9)", + tool: null, + body: `A tracker for a slow back-office job. +Name the thing being worked on, the job's present phase, and how long it has been since the job was accepted. +If the service suggests when to check again, say when that next check happens. Mention any expected callback events quietly, as side information. +Never fabricate a percent-done figure when none was provided — show an open-ended in-progress treatment instead. +Give each phase its own unmistakable look: working, working through files, waiting, ran out of time, went wrong, finished. +Running out of time must not look like the job failing on its merits. +If the service recommends what to do next, make that recommendation prominent under the phase display. +It should feel like following a genuine piece of work, not staring at a spinner.`, + }, +]; diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md new file mode 100644 index 0000000..553d115 --- /dev/null +++ b/docs/ARCHITECTURE.md @@ -0,0 +1,128 @@ +# Composer — architectural rationale + +*This is the document a person joining this project needs. It explains why Composer is shaped the way it is, which decisions are load-bearing, and what it deliberately refuses to do. It is not a changelog; the phase-by-phase history lives in PR bodies and the release notes. For "what runs where," see [COMPOSER.md](COMPOSER.md); for setup, the [README](../README.md).* + +## 1. What Composer is + +Composer turns a plain-language goal into an interface built **only** from a design system's approved components, checked as it is built — and lets several of those interfaces be composed into a workflow. + +The problem it exists to solve: AI generates UI by guessing. It invents component names, fabricates props, hardcodes colors, and ignores the patterns a team has already agreed on. Every generated file then needs manual correction. The usual industry answer is a better prompt. Composer's answer is that **a design system is a contract, and generation should be checked against it mechanically**. + +## 2. The thesis: UI over seams that already existed + +The founding proposal's central claim was that Composer is *"UI over seams that already exist, not a new pipeline."* Three seams were already shipped code when Composer began: + +- **discovery** — `dspack-export`'s framework adapters extract components from a real codebase; +- **mapping** — `dspack-emit`'s `Profile` is pure data describing how contract components project onto A2UI; +- **rendering** — `a2ui-ingest`'s `Registry` maps catalog names to concrete visuals. + +Composer added an authoring and conversation layer over those seams. It has never added a pipeline of its own, and that rule has survived every milestone since — including the workflow layer (§6), which introduced zero pipeline changes. + +**Why this matters:** the ecosystem's failure mode, seen in earlier attempts, is a UI tool that grows its own private generation path and drifts away from the contract everything else validates against. Keeping Composer as *UI over seams* means the thing users see is the same thing the CLI, the MCP server, and CI see. + +There was also a measured reason. Full-vocabulary generation schemas exceeded the structured-output grammar ceiling in **72 of 72** hosted evaluation runs. Project-scoped vocabularies fit. Scoping was never an optimization — it is why the product works at all, and why it works on hardware users own. + +## 3. The core loop + +1. Pick a project (a design system plus your work). +2. Describe what you want, in your own words. +3. Composer infers the **governed context** — which of the design system's intents your goal belongs to — and says which one it chose. +4. `dspack-gen` proposes a surface, constrained by a schema built from that project's approved vocabulary. +5. Gates run visibly: **S1** structure, **S2** approved-vocabulary + containment, **S3** the design system's own rules. Failures become a bounded repair turn, not a silent retry. +6. `dspack-emit` compiles the contract into an A2UI catalog and the surface into A2UI messages, refusing anything it cannot represent honestly. Gates **A1/A2/A3** validate the emitted result. +7. The surface renders through the project's real registry. +8. You refine conversationally, or accept — an accepted surface becomes a governed worked example that seeds future generation. + +## 4. Why the AI proposes and deterministic machinery decides + +This is the load-bearing decision of the whole product. + +The model's output is a **proposal**. Every consequential judgment after it is deterministic: whether the vocabulary is approved (S2), whether the design system's rules hold (S3), whether the result can be represented at all (emit + A1/A2/A3), and whether it may be saved (the same gates run again at accept). A model can be wrong, slow, unavailable, or replaced; none of those events change what Composer will *accept*. + +Three consequences worth understanding before changing anything: + +- **The schema cannot express an unapproved component.** Generation is mechanically incapable of inventing vocabulary; S2 catches anything that slips through another path. +- **Refusals are first-class.** When the emitter cannot represent something, it says so with the reason, and that refusal is shown, not smoothed over. A declared casualty is a documented gap in the mapping, not a bug to route around. +- **Governance is never AI-authored.** Intents, rules, and every rationale are owner-written. The model may propose interfaces; it may not propose the standards those interfaces are judged by. + +Corollary: **scripted mode is first-class.** The entire experience — streaming, gates, rendering, accept — works with no model at all. If the AI layer disappeared tomorrow, Composer would still be a governed authoring tool. + +## 5. The product model + +- **Project** — an identity plus a source of vocabulary. Three sources: a packaged **reference** design system, an **imported** project file, or a connected **repository** (through the local agent). +- **Contract** — the design system as data: components, props, intents, rules, and worked examples. Owner-authored; the source of truth for what may be built. +- **Profile** — how the contract projects onto A2UI. Pure data, editable, and the reason a catalog is *derived, never edited*. +- **Surface** — one governed screen. Generated or hand-authored, always validated the same way. +- **Flow** — an ordered set of surfaces that tells one story (§6). +- **Catalog** — the emitted A2UI vocabulary. **Always derived** from contract + profile; there is deliberately no catalog editor, because an editable catalog is how contract drift starts. + +**Base + delta.** A project's working vocabulary is always *packaged base + the project's own authored delta*. Reference design systems are never mutated; your accepted work lands in a delta merged over the base on open. This is what lets Composer ship teaching material and user work in the same view without either pretending to be the other. + +## 6. Why flows live *above* surfaces + +Real enterprise requests are workflows: configure → review → confirm → result. The obvious implementation — teach the generator to emit one big multi-step surface — was measured and rejected. It produced surfaces that *described* a workflow (step labels as typography) without being one, and it pushed generation into exactly the compound-composition shapes the emitter refuses. + +So a flow is **project data, not contract data**: an ordered list of steps, each a *reference* to an existing governed surface. Nothing about a flow reaches the contract, the emitter, the protocol, or the renderer. Preview walks the steps; each step renders through the identical single-surface path. + +The consequences are the point: + +- Every governance guarantee is preserved automatically, because the pipeline never learns flows exist. +- The same surface can appear in many flows without duplication. +- Deleting every flow leaves every surface exactly as it was. +- Flows can be exported, versioned, diffed, and validated as plain data. + +Flow transitions are **view state**: a step may name emitted action names that advance the walk. No data moves between steps at runtime — continuity is authored into the surfaces themselves. This is a deliberate honesty boundary, not an oversight (§9). + +## 7. The browser/agent boundary + +Composer runs the **same deterministic pipeline** in two homes: + +- **Browser projects** — everything runs in the page: gates, emission, rendering, accept. Work persists in browser storage and travels through project export. Nothing about the user's machine is required. +- **Repository projects** — the local agent reads and writes real files in a connected repo. The contract, profile, and emitted output are files the team version-controls. + +The rule: **the agent is a bridge to your machine, never a privileged decision-maker.** It holds endpoints and credentials so the browser never does; it runs the same gates with the same verdicts. Where the hosted path needs a model it calls one Worker route for the model turn only — the pipeline stays client-side, and no project source leaves the browser. + +Anything implemented twice across this boundary is a liability. Where behavior must match, it belongs in `composer-core` with equivalence tests; the historical divergences found this way are why that rule now exists in writing. + +## 8. Invariants — the things that must not change casually + +1. The AI proposes; deterministic machinery decides. +2. Generation is scoped to the project's approved vocabulary. +3. The catalog is derived, never edited. +4. Reference design systems are never mutated by user work. +5. Governance (intents, rules, rationales) is owner-authored. +6. Refusals and casualties are shown honestly, with their reason. +7. Composer never generates component implementations. +8. Scripted mode keeps the product usable with no model. +9. Flows compose existing surfaces; they never become a new generation or rendering path. + +## 9. What Composer deliberately does not do + +Composer composes and governs **interface representations**. It does not: + +- bind those representations to live application or enterprise data; +- execute workflows, call external systems, or run MCP/Gateway tools; +- carry state between flow steps at runtime; +- generate component implementations (that is a separate, human-reviewed concern, deliberately kept off the fast path); +- manage accounts, teams, or collaboration. + +These are honest boundaries, not hidden features. Runtime binding is a coherent future chapter; pretending it exists today would make the product untrustworthy in exactly the dimension it competes on. + +## 10. Evolutions worth knowing about + +Several early constraints were deliberately reversed. Each loosened *access* while tightening *honesty* — a useful pattern to recognize before proposing the next reversal: + +| Originally | Today | Why | +|---|---|---| +| No hosted AI | Managed model through a governed gateway | Reachability without installing anything; gates still run client-side | +| Build requires the local agent | Browser-only Build works | The pipeline runs in the page; the agent is for real files | +| The user picks an intent explicitly | Goal-first inference, with the picker as an advanced override | People describe outcomes, not our taxonomy — and inference degrades to a deterministic classifier, never blocking | +| All state is files in the user's repo | Three project sources, two browser-local | Trying the product must not require a checkout | + +The invariants in §8 were never traded for any of these. + +## 11. Where the evidence lives + +- **The acceptance corpus** — a real twelve-prompt enterprise suite lives in `acceptance/gateway-corpus/`, with its expected classification and a reproducible harness. It is the honest measuring stick for whether the product actually builds enterprise software; it is deliberately **not** run in normal CI, because it makes live model calls. +- **Gates and refusals** — every run produces an audit report; failures name their cause and the layer that owns it. +- **Tests** — unit suites cover the deterministic layers; end-to-end suites cover the product paths; renderer parity tests assert that a design system cannot widen or narrow the vocabulary the contract defines, and that a required catalog prop cannot be silently ignored by a renderer. diff --git a/docs/COMPOSER.md b/docs/COMPOSER.md index 72911dc..dc43d00 100644 --- a/docs/COMPOSER.md +++ b/docs/COMPOSER.md @@ -131,6 +131,10 @@ user. Layer boundaries: [renderer-abstraction.md](renderer-abstraction.md). portability, lifecycle, hygiene), `e2e/composer-agent.spec.ts` + `e2e/composer-build.spec.ts` (repository projects against real files). -Historical record — how this was built, phase by phase, with the measurements -that drove each decision — is in -[IMPLEMENTATION_LOG.md](IMPLEMENTATION_LOG.md). +Why the product is shaped this way — the thesis, the invariants, the deliberate +non-goals — is in [ARCHITECTURE.md](ARCHITECTURE.md). The acceptance corpus that +measures whether it actually builds enterprise software lives in +[acceptance/gateway-corpus](../acceptance/gateway-corpus/README.md). + +[IMPLEMENTATION_LOG.md](IMPLEMENTATION_LOG.md) is a historical record of the +exhibit era and predates Composer; it contains no Composer phases.