Workflow Engine with Catalog System

## Summary

Add a workflow subsystem to Specify CLI that lets users define multi-step, resumable automation workflows in YAML. The CLI acts as the orchestrator — dispatching commands to CLI-based integrations, evaluating control flow, and pausing at human review gates. Workflows ship with their own catalog system mirroring the existing extension/preset catalog pattern.

## Motivation

Today, Spec-Driven Development steps (specify → plan → tasks → implement) are manually chained by the user. There's an informal `handoffs:` frontmatter concept in command templates but no actual execution engine. A workflow engine would:

- Automate multi-step SDD pipelines end-to-end
- Allow human checkpoints at critical decision points
- Enable community-shared automation recipes via catalogs
- Keep the CLI as the single driver, routing to whichever integration is configured
- Allow mixing integrations within a single workflow (e.g., Claude for spec, Gemini for planning)

---

## Workflow Definition (YAML)

```yaml
schema_version: "1.0"
workflow:
  id: "sdd-full-cycle"
  name: "Full SDD Cycle"
  version: "1.0.0"
  author: "github"
  description: "Runs specify → plan → tasks → implement with review gates"
  integration: claude              # workflow-wide default integration

requires:
  speckit_version: ">=0.15.0"
  integrations:
    any: ["claude", "gemini"]      # at least one must be available
    all: ["claude", "qwen"]        # all listed must be available (auto-inferred from step declarations)

inputs:
  feature_name:
    type: string
    required: true
    prompt: "Feature name"
  scope:
    type: string
    default: "full"
    enum: ["full", "backend-only", "frontend-only"]

steps:
  - id: specify
    command: speckit.specify
    integration: claude
    model: sonnet-4                 # fast model for initial spec drafting
    input:
      args: "{{ inputs.feature_name }}"
    output:
      spec_file: "{{ result.file }}"

  - id: review-spec
    type: gate
    message: "Review the generated spec before planning."
    show_file: "{{ steps.specify.output.spec_file }}"
    options: [approve, edit, reject]
    on_reject: abort

  - id: plan
    command: speckit.plan
    integration: gemini
    model: gemini-2.5-pro            # full power for planning
    options:                         # pass-through CLI flags
      thinking-budget: 32768
    input:
      args: "{{ steps.specify.output.spec_file }}"

  # --- if/then/else (inline nested steps) ---
  - id: check-scope
    type: if
    condition: "{{ inputs.scope == 'full' }}"
    then:
      - id: tasks-all
        command: speckit.tasks
        input:
          args: "{{ steps.plan.output.plan_file }}"
    else:
      - id: tasks-filtered
        command: speckit.tasks
        input:
          args: "{{ steps.plan.output.plan_file }} --scope {{ inputs.scope }}"

  # --- switch (inline nested steps) ---
  - id: route-by-task-count
    type: switch
    expression: "{{ steps.plan.output.task_count }}"
    cases:
      0:
        - id: notify-empty
          type: gate
          message: "No tasks generated. Review the plan."
          options: [abort, retry]
      1:
        - id: implement-single
          command: speckit.implement
          input:
            args: "{{ steps.tasks-all.output.task_list[0].file }}"
    default:
      - id: implement-parallel
        type: fan-out
        items: "{{ steps.tasks-all.output.task_list }}"
        max_concurrency: 3
        step:
          id: implement-task
          command: speckit.implement
          integration: "{{ item.preferred_integration | default('claude') }}"
          input:
            args: "{{ item.file }}"

  # --- fan-out (parallel) ---
  - id: implement-parallel
    type: fan-out
    items: "{{ steps.tasks-all.output.task_list }}"
    max_concurrency: 3
    step:
      id: implement-task
      command: speckit.implement
      integration: "{{ item.preferred_integration | default('claude') }}"
      input:
        args: "{{ item.file }}"

  # --- fan-in (join) ---
  - id: collect-results
    type: fan-in
    wait_for: [implement-parallel]
    output:
      summary: "{{ fan_in.results | map('result.status') }}"

  # --- while loop ---
  - id: test-loop
    type: while
    condition: "{{ steps.run-tests.output.exit_code != 0 }}"
    max_iterations: 5
    steps:
      - id: fix
        command: speckit.implement
        input:
          args: "--fix {{ steps.run-tests.output.failures }}"
      - id: run-tests
        type: shell
        run: "npm test"

  # --- do-while loop ---
  - id: review-cycle
    type: do-while
    condition: "{{ steps.human-check.output.choice == 'revise' }}"
    max_iterations: 3
    steps:
      - id: refine
        command: speckit.specify
        input:
          args: "--refine {{ steps.specify.output.spec_file }}"
      - id: human-check
        type: gate
        message: "Satisfied with the refined spec?"
        options: [approve, revise]
```

---

## Control Flow Primitives

| Primitive | `type:` value | Description |
|---|---|---|
| **Sequential** | _(default)_ | Steps execute in declaration order |
| **If/Then/Else** | `if` | Branch on expression over any prior step's `input` or `output`; `then:` and `else:` contain inline step arrays (arbitrarily nested) |
| **Switch** | `switch` | Multi-branch dispatch on expression; `cases:` map values → inline step arrays; `default:` fallback |
| **While Loop** | `while` | Repeat nested `steps:` while `condition:` is truthy; `max_iterations` safety cap |
| **Do-While Loop** | `do-while` | Like `while` but body executes at least once before condition is checked |
| **Fan-Out** | `fan-out` | Parallel dispatch over `items:` collection with `max_concurrency:` limit |
| **Fan-In** | `fan-in` | Join point; blocks until all `wait_for:` steps complete; aggregates results |
| **Gate** | `gate` | Pause for human review; options presented interactively; `on_reject:` controls abort/skip |
| **Shell** | `shell` | Run a local shell command (non-agent) |

### If/Then/Else

Branches based on a boolean `condition:` expression. Both `then:` and `else:` contain inline step arrays — full step definitions, not ID references. Branches can contain any step type, enabling arbitrary nesting.

```yaml
  - id: check-complexity
    type: if
    condition: "{{ steps.plan.output.task_count > 5 and steps.specify.input.args | contains('backend') }}"
    then:
      - id: split-tasks
        command: speckit.tasks
        input:
          args: "--split {{ steps.plan.output.plan_file }}"
      - id: implement-parallel
        type: fan-out
        items: "{{ steps.split-tasks.output.task_list }}"
        max_concurrency: 3
        step:
          id: impl
          command: speckit.implement
          input:
            args: "{{ item.file }}"
    else:
      - id: implement-sequential
        command: speckit.implement
        input:
          args: "{{ steps.plan.output.plan_file }}"
```

### Switch

Evaluates `expression:` once, matches the result against `cases:` keys (exact match, string-coerced). Falls through to `default:` if no case matches. Each case value is an inline step array — full step definitions supporting arbitrary nesting.

```yaml
  - id: route-by-review
    type: switch
    expression: "{{ steps.review-spec.output.choice }}"
    cases:
      approve:
        - id: plan
          command: speckit.plan
          input:
            args: "{{ steps.specify.output.spec_file }}"
      edit:
        - id: re-specify
          command: speckit.specify
          input:
            args: "--refine {{ steps.specify.output.spec_file }}"
        - id: re-review
          type: gate
          message: "Review the updated spec"
          options: [approve, reject]
      reject:
        - id: log-rejection
          type: shell
          run: "echo 'Spec rejected' >> .specify/workflows/runs/current/log.txt"
    default:
      - id: abort-workflow
        type: gate
        message: "Unknown review choice. Abort?"
        options: [abort]
        on_reject: abort
```

### While Loop

Evaluates `condition:` before each iteration. If falsy on first check, the body never runs. `max_iterations` is required as a safety cap.

```yaml
  - id: retry-tests
    type: while
    condition: "{{ steps.run-tests.output.exit_code != 0 }}"
    max_iterations: 5
    steps:
      - id: fix
        command: speckit.implement
        input:
          args: "--fix {{ steps.run-tests.output.failures }}"
      - id: run-tests
        type: shell
        run: "npm test"
```

### Do-While Loop

Body executes at least once, then `condition:` is checked. Continues while condition is truthy.

```yaml
  - id: review-cycle
    type: do-while
    condition: "{{ steps.human-check.output.choice == 'revise' }}"
    max_iterations: 3
    steps:
      - id: refine
        command: speckit.specify
        input:
          args: "--refine {{ steps.specify.output.spec_file }}"
      - id: human-check
        type: gate
        message: "Satisfied with the refined spec?"
        options: [approve, revise]
```

### Fan-Out (Parallel Execution)

Iterates over `items:` and dispatches the nested `step:` template for each item, up to `max_concurrency:` at a time. Each iteration has access to `{{ item }}` and `{{ item.<field> }}`.

```yaml
  - id: implement-parallel
    type: fan-out
    items: "{{ steps.tasks-all.output.task_list }}"
    max_concurrency: 3
    step:
      id: implement-task
      command: speckit.implement
      integration: "{{ item.preferred_integration | default('claude') }}"
      input:
        args: "{{ item.file }}"
```

### Fan-In (Join)

Blocks until all referenced steps complete. Aggregates their results into `fan_in.results` for downstream access.

```yaml
  - id: collect-results
    type: fan-in
    wait_for: [implement-parallel]
    output:
      summary: "{{ fan_in.results | map('result.status') }}"
```

### Gate (Human Review)

Pauses execution and presents an interactive prompt. The user's choice is stored in `output.choice`. `on_reject:` controls behavior when the reject/abort option is selected.

```yaml
  - id: review-spec
    type: gate
    message: "Review the generated spec before planning."
    show_file: "{{ steps.specify.output.spec_file }}"
    options: [approve, edit, reject]
    on_reject: abort   # abort | skip | retry
```

### Shell

Runs a local shell command directly (no agent integration). Captures exit code and stdout/stderr.

```yaml
  - id: run-tests
    type: shell
    run: "npm test"
```

### Nesting

All control flow constructs accept **inline step arrays** — full step definitions, not ID references. This means any construct can be nested inside any other to arbitrary depth. The engine uses a recursive executor.

| Construct | Child slot(s) | Accepts inline steps? |
|---|---|---|
| `if` | `then:`, `else:` | Yes — inline step arrays |
| `switch` | `cases.<value>:`, `default:` | Yes — inline step arrays |
| `while` | `steps:` | Yes — inline step array |
| `do-while` | `steps:` | Yes — inline step array |
| `fan-out` | `step:` | Yes — single inline step (can itself be a control flow step) |

**Example — `if` inside `while`, with a nested `fan-out`:**

```yaml
  - id: fix-loop
    type: while
    condition: "{{ steps.run-tests.output.exit_code != 0 }}"
    max_iterations: 5
    steps:
      - id: classify-failures
        type: shell
        run: "python classify_failures.py"

      - id: route-fix
        type: if
        condition: "{{ steps.classify-failures.output.count > 3 }}"
        then:
          - id: parallel-fix
            type: fan-out
            items: "{{ steps.classify-failures.output.failures }}"
            max_concurrency: 3
            step:
              id: fix-one
              command: speckit.implement
              input:
                args: "--fix {{ item.file }}"
        else:
          - id: single-fix
            command: speckit.implement
            input:
              args: "--fix {{ steps.classify-failures.output.failures | join(' ') }}"

      - id: run-tests
        type: shell
        run: "npm test"
```

**Scoping rules for nested steps:**
- Nested steps write to the same flat `steps.<id>` namespace — IDs must be unique across the entire workflow
- Inner steps can reference any outer step's `input`/`output` (lexical scoping outward)
- Loop iterations overwrite the previous iteration's step outputs (access the latest via `steps.<id>.output`)

---

## Per-Step Integration, Model & Options Override

Each step can specify which CLI integration executes it, which model to use, and pass-through CLI options.

### `integration:`

**Resolution order** (first match wins):
1. `integration:` on the step itself
2. `integration:` at workflow top level
3. Integration from `.specify/init-options.json` (project default)

### `model:`

**Resolution order** (first match wins):
1. `model:` on the step itself
2. `model:` at workflow top level
3. Integration's default model (no flag passed — CLI uses its own default)

### `options:`

Freeform key-value dict of pass-through CLI flags. Applied after `integration:` and `model:` are resolved. Merged with workflow-level `options:` (step-level keys win on conflict).

```yaml
workflow:
  id: "multi-agent-pipeline"
  name: "Multi-Agent Pipeline"
  version: "1.0.0"
  integration: claude                # workflow-wide default
  model: sonnet-4                    # workflow-wide default model
  options:                           # workflow-wide default CLI flags
    max-tokens: 8000

steps:
  - id: specify
    command: speckit.specify         # → claude, sonnet-4, max-tokens=8000 (all inherited)

  - id: plan
    command: speckit.plan
    integration: gemini              # → gemini (step override)
    model: gemini-2.5-pro            # → gemini-2.5-pro (step override)
    options:
      thinking-budget: 32768         # gemini-specific flag

  - id: implement
    command: speckit.implement
    model: opus-4                    # → claude (inherited), opus-4 (step override)
    options:
      max-tokens: 16000              # overrides workflow-level 8000
```

Expressions are supported for dynamic selection:

```yaml
  - id: pick-model
    type: if
    condition: "{{ steps.plan.output.task_count > 10 }}"
    then:
      - id: heavy-implement
        command: speckit.implement
        model: opus-4
    else:
      - id: light-implement
        command: speckit.implement
        model: sonnet-4
```

IDE-based integrations (Copilot, Cursor, Windsurf, etc.) are excluded from workflow dispatch — workflows target CLI integrations only.

---

## Step Context Model

Every step records its **resolved integration**, **resolved model**, **resolved options**, **resolved input**, and **output** into the run state. All subsequent steps and all control flow expressions can reference any of these:

```
steps.<step-id>.integration          # which integration ran the step
steps.<step-id>.model                # which model was used (null if default)
steps.<step-id>.options.<key>        # resolved pass-through CLI flags
steps.<step-id>.input.<field>        # what was sent to the step
steps.<step-id>.output.<field>       # what the step produced
```

Example state after two steps complete:

```json
{
  "steps": {
    "specify": {
      "integration": "claude",
      "model": "sonnet-4",
      "options": { "max-tokens": 8000 },
      "input": { "args": "new login flow" },
      "output": { "file": ".specify/specs/login-flow.md", "exit_code": 0, "duration_s": 14.2 }
    },
    "plan": {
      "integration": "gemini",
      "model": "gemini-2.5-pro",
      "options": { "thinking-budget": 32768 },
      "input": { "args": ".specify/specs/login-flow.md" },
      "output": { "plan_file": ".specify/plans/login-flow-plan.md", "task_count": 7, "exit_code": 0 }
    }
  }
}
```

This enables downstream decisions based on upstream inputs, outputs, integrations, and models.

---

## Expression Language

Sandboxed Jinja2 subset (no file I/O, no imports):

| Category | Examples |
|---|---|
| **Step data** | `{{ steps.<id>.input.<field> }}`, `{{ steps.<id>.output.<field> }}` |
| **Step integration** | `{{ steps.<id>.integration }}` |
| **Step model** | `{{ steps.<id>.model }}` |
| **Step options** | `{{ steps.<id>.options.<key> }}` |
| **Workflow inputs** | `{{ inputs.x }}` |
| **Fan-out item** | `{{ item }}`, `{{ item.<field> }}` |
| **Fan-in results** | `{{ fan_in.results }}` |
| **Comparisons** | `==`, `!=`, `>`, `<`, `>=`, `<=`, `in`, `not in` |
| **Boolean logic** | `and`, `or`, `not` |
| **Filters** | `| map(...)`, `| join(', ')`, `| default('...')`, `| contains('...')` |
| **Literals** | strings, numbers, booleans, lists |

---

## Resume & State Persistence

### Run State Directory

```
.specify/workflows/
├── runs/
│   └── <run-id>/
│       ├── state.json          # current step, all step outputs, gate decisions
│       ├── inputs.json         # resolved input values
│       └── log.jsonl           # append-only execution log
```

### Resume Behavior

- Every step transition persists to `state.json` **before** the next step begins
- `specify workflow resume <run-id>` picks up from the last incomplete step
- Gate steps serialize their pending state; resume re-presents the gate prompt
- Loop steps track their current iteration count for correct resume
- Fan-out steps track per-item completion for partial resume

### Run Lifecycle

```
created → running → paused (at gate) → running → completed
                  → failed (step error) ──→ resume → running
                  → aborted (gate reject or user cancel)
```

---

## CLI Commands

```
specify workflow run <workflow>       # run from installed workflow or local YAML path
specify workflow resume <run-id>      # resume a paused/failed run
specify workflow status [<run-id>]    # show run status (Rich table)
specify workflow list                 # list installed workflows
specify workflow add <source>         # install from catalog/URL/local path
specify workflow remove <id>          # uninstall a workflow
specify workflow search <query>       # search catalogs
specify workflow info <id>            # show workflow details and step graph
specify workflow catalog list         # list configured catalog sources
specify workflow catalog add <url>    # add a catalog source
specify workflow catalog remove <n>   # remove a catalog source by index
```

---

## Catalog System

Mirrors the existing extension/preset catalog pattern exactly.

### Resolution Order

```
1. SPECKIT_WORKFLOW_CATALOG_URL env var (overrides all)
2. .specify/workflow-catalogs.yml (project-level)
3. ~/.specify/workflow-catalogs.yml (user-level)
4. Built-in: workflows/catalog.json + catalog.community.json
```

### Catalog Entry Schema

```json
{
  "schema_version": "1.0",
  "workflows": [
    {
      "id": "sdd-full-cycle",
      "name": "Full SDD Cycle",
      "version": "1.0.0",
      "description": "Complete specify → plan → tasks → implement pipeline with review gates",
      "url": "https://github.com/github/spec-kit/releases/download/v1.0.0/sdd-full-cycle-1.0.0.zip",
      "tags": ["sdd", "full-cycle"],
      "min_speckit_version": "0.15.0"
    }
  ]
}
```

### Catalog Configuration File

```yaml
catalogs:
  - name: "default"
    url: "https://raw.githubusercontent.com/github/spec-kit/main/workflows/catalog.json"
    priority: 1
    install_allowed: true
    description: "Official workflows"
  - name: "community"
    url: "https://raw.githubusercontent.com/github/spec-kit/main/workflows/catalog.community.json"
    priority: 2
    install_allowed: false
    description: "Community-contributed workflows (discovery only)"
```

Caching follows existing pattern: 1-hour TTL, SHA256-hashed cache files in `~/.cache/spec-kit/`.

---

## Integration Dispatch

The workflow engine dispatches each step to its resolved integration:

1. **Resolve integration**: step `integration:` → workflow `integration:` → project default from `.specify/init-options.json`
2. **Resolve model**: step `model:` → workflow `model:` → omit flag (use integration default)
3. **Merge options**: workflow `options:` ← step `options:` (step wins on conflict)
4. **Validate**: integration exists in `INTEGRATION_REGISTRY`, `requires_cli` is satisfied, CLI tool is in PATH
5. **Construct CLI invocation**: e.g., `claude --model opus-4 --max-tokens 16000 /speckit.specify "feature X"`
6. **Capture structured output**: exit code, file paths written, stdout → stored as step `output`

### Validation Timing

- **Static `integration:` values**: validated before run starts (fail fast)
- **Expression-based `integration:` values**: validated at step execution time
- `requires.integrations.all` is auto-inferred from static step declarations during validation

---

## Step Types & Extensibility

### Architecture

Step types follow the same registry pattern as integrations. Every step type — built-in or extension-provided — implements `StepBase` and registers in `STEP_REGISTRY`.

```python
# src/specify_cli/workflows/base.py
class StepBase:
    type_key: str  # matches "type:" in workflow YAML

    def execute(self, context: StepContext) -> StepResult: ...
    def validate(self, config: dict) -> list[str]: ...  # schema errors
    def can_resume(self, state: dict) -> bool: ...
```

### Source Layout

Core step types ship as individual packages under `src/specify_cli/workflows/steps/`. Each package is self-contained and uses the identical `StepBase` interface that extensions use — core steps have no special privileges.

```
src/specify_cli/workflows/
├── __init__.py                # STEP_REGISTRY, auto-discovery
├── base.py                    # StepBase, StepContext, StepResult
├── engine.py                  # WorkflowEngine (executor, state machine)
├── expressions.py             # Sandboxed Jinja2 evaluator
├── steps/
│   ├── __init__.py            # scans subdirs, imports and registers StepBase subclasses
│   ├── command/               # default step — dispatches to an integration
│   │   └── __init__.py
│   ├── gate/                  # human review gate
│   │   └── __init__.py
│   ├── if_then/               # if/then/else branching
│   │   └── __init__.py
│   ├── switch/                # multi-branch dispatch
│   │   └── __init__.py
│   ├── while_loop/            # while loop
│   │   └── __init__.py
│   ├── do_while/              # do-while loop
│   │   └── __init__.py
│   ├── fan_out/               # parallel dispatch
│   │   └── __init__.py
│   ├── fan_in/                # join point
│   │   └── __init__.py
│   └── shell/                 # local shell command
│       └── __init__.py
```

### Core Step Reference Implementation

Each built-in step type is a minimal `StepBase` subclass:

```python
# src/specify_cli/workflows/steps/if_then/__init__.py
from specify_cli.workflows.base import StepBase, StepContext, StepResult

class IfThenStep(StepBase):
    type_key = "if"

    def execute(self, context: StepContext) -> StepResult:
        condition = self.evaluate(context, self.config["condition"])
        branch = self.config["then"] if condition else self.config.get("else", [])
        return StepResult(next_steps=branch)
```

### Extension-Provided Step Types

Extensions can ship custom step types alongside commands and workflows:

```yaml
# extension.yml
provides:
  step_types:
    - type_key: "deploy"
      module: "steps/deploy"      # relative to extension directory
```

At extension install time, the module is loaded and the `StepBase` subclass registers into `STEP_REGISTRY`. Extensions can also override built-in step types (e.g., replace `gate` with a Slack-integrated gate) — last-registered wins, with a warning.

### Security Model

- **Workflow YAML** is declarative — no code execution, safe to download from catalogs
- **Custom step types** require installing an extension (explicit user action) — same trust boundary as existing extension scripts
- The sandboxed expression engine prevents YAML-level code injection

---

## Workflow Packaging & Distribution

### Source Layout (in spec-kit repo)

Workflow definitions live in a top-level `workflows/` directory, each in its own subdirectory — the same pattern as `presets/scaffold/` and `presets/self-test/`:

```
workflows/
├── catalog.json                    # official catalog (installable)
├── catalog.community.json          # community catalog (discovery only)
├── speckit/                        # core SDD full-cycle workflow
│   └── workflow.yml
├── speckit-quick/                  # lightweight: specify → implement (no gates)
│   └── workflow.yml
├── speckit-review/                 # specify → plan → gate → tasks (review-focused)
│   └── workflow.yml
```

### Release Pipeline

CI packages each workflow directory into a zip artifact, mirroring how presets and extensions are released via `.github/workflows/scripts/create-release-packages.sh`:

```bash
for workflow_dir in workflows/*/; do
  id=$(basename "$workflow_dir")
  zip -r ".genreleases/spec-kit-workflow-${id}-${VERSION}.zip" "$workflow_dir"
done
```

Catalog entries point to release artifact URLs:

```json
{
  "id": "speckit",
  "name": "Full SDD Cycle",
  "version": "1.0.0",
  "url": "https://github.com/github/spec-kit/releases/download/v1.0.0/spec-kit-workflow-speckit-1.0.0.zip"
}
```

### Distribution Flow

```
Source                    Release                   User
workflows/speckit/   →   zip artifact          →   specify workflow add speckit
  workflow.yml            via GitHub Actions         → .specify/workflows/speckit/workflow.yml
```

### Auto-Install During `specify init`

`specify init` already installs core commands and presets. Workflows slot into the same flow:

- The `speckit` workflow is auto-installed by default
- `--workflow <id>` flag selects an alternative or `--no-workflow` to skip
- Uses the same catalog resolution and manifest tracking as presets

### Extension-Provided Workflows

Extensions can bundle complete workflows alongside commands and step types:

```yaml
# extension.yml
provides:
  commands:
    - name: "speckit.myext.deploy"
      file: "commands/deploy.md"
  step_types:
    - type_key: "deploy"
      module: "steps/deploy"
  workflows:
    - id: "myext-deploy-pipeline"
      file: "workflows/deploy-pipeline.yml"
```

When installed, extension workflows register into `workflow-registry.json` and become runnable via `specify workflow run myext-deploy-pipeline`.

### Full Lifecycle

| Stage | Mechanism | Example |
|---|---|---|
| **Author** | Create `workflows/<id>/workflow.yml` in repo or extension | `workflows/speckit/workflow.yml` |
| **Package** | CI zips each workflow dir into release artifact | `spec-kit-workflow-speckit-1.0.0.zip` |
| **Catalog** | `catalog.json` references zip URLs | Auto-updated in release workflow |
| **Discover** | `specify workflow search sdd` | Searches all configured catalogs |
| **Install** | `specify workflow add speckit` | Downloads, extracts to `.specify/workflows/speckit/` |
| **Auto-install** | `specify init` installs core workflow | Same as preset auto-install |
| **Run** | `specify workflow run speckit` | Engine loads `workflow.yml`, executes steps |
| **Uninstall** | `specify workflow remove speckit` | Hash-verified removal via manifest |

---

## Installation Layout

### Installed in User's Project

```
.specify/
├── workflows/
│   ├── workflow-registry.json     # installed workflows + metadata (mirrors preset-registry.json)
│   ├── speckit/
│   │   └── workflow.yml           # installed from catalog or local
│   ├── speckit-quick/
│   │   └── workflow.yml
│   └── runs/                      # execution state (separate from definitions)
│       └── <run-id>/
│           ├── state.json
│           ├── inputs.json
│           └── log.jsonl
```

---

## `requires:` Section

```yaml
requires:
  speckit_version: ">=0.15.0"
  integrations:
    any: ["claude", "gemini"]      # at least one must be available (for default dispatch)
    all: ["claude", "qwen"]        # all listed must be available
                                   # (auto-inferred from static step integration: declarations)
  step_types: ["deploy"]           # custom step types that must be installed (from extensions)
```

---

## Implementation Plan

| Phase | Scope | Details |
|---|---|---|
| **1. `StepBase` & registry** | `base.py`, `STEP_REGISTRY`, auto-discovery in `steps/__init__.py` | Define `StepBase`, `StepContext`, `StepResult`; scan `steps/` subdirs to register built-in types |
| **2. Core step types** | `steps/command/`, `steps/shell/`, `steps/gate/` | Implement as standard `StepBase` packages — validates the extensibility model from day one |
| **3. Schema & validation** | YAML schema for `workflow.yml`, validate with jsonschema | Define all step types, control flow primitives, input/output schemas |
| **4. Expression engine** | Sandboxed Jinja2 evaluator (`expressions.py`) | Variable interpolation, conditions, filters; no file I/O or imports |
| **5. Sequential executor** | `engine.py` — basic step-by-step runner | Command steps, shell steps, input/output capture, integration resolution |
| **6. State machine** | Persist/resume with `state.json` | Write-ahead state before each step; resume from last incomplete step |
| **7. Gate handler** | Interactive prompts via Rich | Serialize pending gate state; re-present on resume |
| **8. Control flow steps** | `steps/if_then/`, `steps/switch/`, `steps/while_loop/`, `steps/do_while/` | Each as a `StepBase` package; evaluate conditions from step context model; enforce max_iterations |
| **9. Fan-out / fan-in steps** | `steps/fan_out/`, `steps/fan_in/` | `concurrent.futures` or `asyncio`; per-item tracking for partial resume |
| **10. Integration dispatcher** | Map command → CLI invocation per integration | Resolution order, validation, stdout/stderr capture |
| **11. CLI commands** | `specify workflow run\|resume\|status\|list\|add\|remove\|search\|info\|catalog` | Typer command group mirroring preset/extension pattern |
| **12. Catalog system** | Reuse `CatalogManager` pattern | catalog.json, resolution stack, caching, search |
| **13. Manifest tracking** | Reuse `IntegrationManifest` pattern | Hash-based install/uninstall tracking |
| **14. Workflow packaging** | Release pipeline + auto-install during `specify init` | Zip packaging, `--workflow` flag, catalog entry generation |
| **15. Extension hook for step types** | `provides.step_types` in `extension.yml` | Load extension-provided `StepBase` subclasses at install time |
| **16. Built-in workflows** | Ship `speckit`, `speckit-quick`, `speckit-review` | Source in `workflows/`, packaged and cataloged |

---

## Non-Goals (v1)

- Visual workflow editor / GUI
- Remote or distributed execution
- Workflow-to-workflow nesting (future consideration)
- IDE-based integration dispatch
- Workflow versioning/migration (workflows are self-contained; no state migration between versions)

---

## Labels

`enhancement`, `workflow`, `new-subsystem`


Primitive	`type:` value	Description
Sequential	(default)	Steps execute in declaration order
If/Then/Else	`if`	Branch on expression over any prior step's `input` or `output`; `then:` and `else:` contain inline step arrays (arbitrarily nested)
Switch	`switch`	Multi-branch dispatch on expression; `cases:` map values → inline step arrays; `default:` fallback
While Loop	`while`	Repeat nested `steps:` while `condition:` is truthy; `max_iterations` safety cap
Do-While Loop	`do-while`	Like `while` but body executes at least once before condition is checked
Fan-Out	`fan-out`	Parallel dispatch over `items:` collection with `max_concurrency:` limit
Fan-In	`fan-in`	Join point; blocks until all `wait_for:` steps complete; aggregates results
Gate	`gate`	Pause for human review; options presented interactively; `on_reject:` controls abort/skip
Shell	`shell`	Run a local shell command (non-agent)

Construct	Child slot(s)	Accepts inline steps?
`if`	`then:`, `else:`	Yes — inline step arrays
`switch`	`cases.<value>:`, `default:`	Yes — inline step arrays
`while`	`steps:`	Yes — inline step array
`do-while`	`steps:`	Yes — inline step array
`fan-out`	`step:`	Yes — single inline step (can itself be a control flow step)

Category	Examples
Step data	`{{ steps.<id>.input.<field> }}`, `{{ steps.<id>.output.<field> }}`
Step integration	`{{ steps.<id>.integration }}`
Step model	`{{ steps.<id>.model }}`
Step options	`{{ steps.<id>.options.<key> }}`
Workflow inputs	`{{ inputs.x }}`
Fan-out item	`{{ item }}`, `{{ item.<field> }}`
Fan-in results	`{{ fan_in.results }}`
Comparisons	`==`, `!=`, `>`, `<`, `>=`, `<=`, `in`, `not in`
Boolean logic	`and`, `or`, `not`
Filters	`
Literals	strings, numbers, booleans, lists

Stage	Mechanism	Example
Author	Create `workflows/<id>/workflow.yml` in repo or extension	`workflows/speckit/workflow.yml`
Package	CI zips each workflow dir into release artifact	`spec-kit-workflow-speckit-1.0.0.zip`
Catalog	`catalog.json` references zip URLs	Auto-updated in release workflow
Discover	`specify workflow search sdd`	Searches all configured catalogs
Install	`specify workflow add speckit`	Downloads, extracts to `.specify/workflows/speckit/`
Auto-install	`specify init` installs core workflow	Same as preset auto-install
Run	`specify workflow run speckit`	Engine loads `workflow.yml`, executes steps
Uninstall	`specify workflow remove speckit`	Hash-verified removal via manifest

Phase	Scope	Details
1. `StepBase` & registry	`base.py`, `STEP_REGISTRY`, auto-discovery in `steps/__init__.py`	Define `StepBase`, `StepContext`, `StepResult`; scan `steps/` subdirs to register built-in types
2. Core step types	`steps/command/`, `steps/shell/`, `steps/gate/`	Implement as standard `StepBase` packages — validates the extensibility model from day one
3. Schema & validation	YAML schema for `workflow.yml`, validate with jsonschema	Define all step types, control flow primitives, input/output schemas
4. Expression engine	Sandboxed Jinja2 evaluator (`expressions.py`)	Variable interpolation, conditions, filters; no file I/O or imports
5. Sequential executor	`engine.py` — basic step-by-step runner	Command steps, shell steps, input/output capture, integration resolution
6. State machine	Persist/resume with `state.json`	Write-ahead state before each step; resume from last incomplete step
7. Gate handler	Interactive prompts via Rich	Serialize pending gate state; re-present on resume
8. Control flow steps	`steps/if_then/`, `steps/switch/`, `steps/while_loop/`, `steps/do_while/`	Each as a `StepBase` package; evaluate conditions from step context model; enforce max_iterations
9. Fan-out / fan-in steps	`steps/fan_out/`, `steps/fan_in/`	`concurrent.futures` or `asyncio`; per-item tracking for partial resume
10. Integration dispatcher	Map command → CLI invocation per integration	Resolution order, validation, stdout/stderr capture
11. CLI commands	`specify workflow run\|resume\|status\|list\|add\|remove\|search\|info\|catalog`	Typer command group mirroring preset/extension pattern
12. Catalog system	Reuse `CatalogManager` pattern	catalog.json, resolution stack, caching, search
13. Manifest tracking	Reuse `IntegrationManifest` pattern	Hash-based install/uninstall tracking
14. Workflow packaging	Release pipeline + auto-install during `specify init`	Zip packaging, `--workflow` flag, catalog entry generation
15. Extension hook for step types	`provides.step_types` in `extension.yml`	Load extension-provided `StepBase` subclasses at install time
16. Built-in workflows	Ship `speckit`, `speckit-quick`, `speckit-review`	Source in `workflows/`, packaged and cataloged

Workflow Engine with Catalog System #2142

Description

Summary

Motivation

Workflow Definition (YAML)

Control Flow Primitives

If/Then/Else

Switch

While Loop

Do-While Loop

Fan-Out (Parallel Execution)

Fan-In (Join)

Gate (Human Review)

Shell

Nesting

Per-Step Integration, Model & Options Override

integration:

model:

options:

Step Context Model

Expression Language

Resume & State Persistence

Run State Directory

Resume Behavior

Run Lifecycle

CLI Commands

Catalog System

Resolution Order

Catalog Entry Schema

Catalog Configuration File

Integration Dispatch

Validation Timing

Step Types & Extensibility

Architecture

Source Layout

Core Step Reference Implementation

Extension-Provided Step Types

Security Model

Workflow Packaging & Distribution

Source Layout (in spec-kit repo)

Release Pipeline

Distribution Flow

Auto-Install During specify init

Extension-Provided Workflows

Full Lifecycle

Installation Layout

Installed in User's Project

requires: Section

Implementation Plan

Non-Goals (v1)

Labels

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions

`integration:`

`model:`

`options:`

Auto-Install During `specify init`

`requires:` Section