From 426c2b65773acf6aec1b26183bde11bafc1eaa91 Mon Sep 17 00:00:00 2001 From: Manfred Riem <15701806+mnriem@users.noreply.github.com> Date: Tue, 26 May 2026 17:51:43 -0500 Subject: [PATCH 1/5] Re-validate spec quality checklist after clarify updates spec After clarify modifies spec.md, the existing checklists/requirements.md (generated by specify) can become stale. Items like 'No [NEEDS CLARIFICATION] markers remain' may now pass, and newly added requirements aren't reflected in the checklist evaluation. Add step 8 to the clarify command that re-validates the spec quality checklist against the updated spec after each clarification session: - Check/uncheck items based on current spec state - Report before/after pass counts in the completion report - Skip silently if no checklist exists Fixes #2693 --- templates/commands/clarify.md | 17 ++++++++++++++++- 1 file changed, 16 insertions(+), 1 deletion(-) diff --git a/templates/commands/clarify.md b/templates/commands/clarify.md index d4eb0048a8..c9340807d1 100644 --- a/templates/commands/clarify.md +++ b/templates/commands/clarify.md @@ -197,6 +197,19 @@ Execution steps: 7. Write the updated spec back to `FEATURE_SPEC`. +8. **Re-validate Spec Quality Checklist** (if it exists): + - Check if `FEATURE_DIR/checklists/requirements.md` exists. + - If it does NOT exist, skip this step silently. + - If it exists: + 1. Read the checklist file. + 2. Re-evaluate each checklist item against the **updated** spec (the version just saved in step 7). + 3. For each item: + - If the item now passes (e.g., a `[NEEDS CLARIFICATION]` marker was resolved, a requirement was made unambiguous, an edge case was added), update the marker from `- [ ]` to `- [x]`. + - If the item previously passed and still passes, leave it as `- [x]`. + - If the item does not pass, leave it as `- [ ]`. + 4. Save the updated checklist file. + 5. Record the before/after pass counts for the Completion Report (e.g., "12/16 → 15/16 items passing"). + Behavior rules: - If no meaningful ambiguities found (or all potential questions would be low-impact), respond: "No critical ambiguities detected worth formal clarification." and suggest proceeding. @@ -248,6 +261,7 @@ Report completion (after questioning loop ends or early termination): - Number of questions asked & answered. - Path to updated spec. - Sections touched (list names). +- Spec quality checklist status (if `FEATURE_DIR/checklists/requirements.md` was re-validated): show before/after pass counts (e.g., "Spec Quality Checklist: 12/16 → 15/16 items passing") and list any items that changed from unchecked to checked. If any items remain unchecked, list them as areas needing attention. - Coverage summary table listing each taxonomy category with Status: Resolved (was Partial/Missing and addressed), Deferred (exceeds question quota or better suited for planning), Clear (already sufficient), Outstanding (still Partial/Missing but low impact). - If any Outstanding or Deferred remain, recommend whether to proceed to `__SPECKIT_COMMAND_PLAN__` or run `__SPECKIT_COMMAND_CLARIFY__` again later post-plan. - Suggested next command. @@ -255,5 +269,6 @@ Report completion (after questioning loop ends or early termination): ## Done When - [ ] Spec ambiguities identified and clarifications integrated into spec file +- [ ] Spec quality checklist re-validated against updated spec (if `checklists/requirements.md` exists) - [ ] Extension hooks dispatched or skipped according to the rules in Mandatory Post-Execution Hooks above -- [ ] Completion reported to user with questions answered, sections touched, and coverage summary +- [ ] Completion reported to user with questions answered, sections touched, checklist status, and coverage summary From 032610f69a41d0229736baa9dd39974ef4c1f3d4 Mon Sep 17 00:00:00 2001 From: Manfred Riem <15701806+mnriem@users.noreply.github.com> Date: Tue, 26 May 2026 17:59:26 -0500 Subject: [PATCH 2/5] Address review: scope to checkbox lines, use FEATURE_DIR path - Constrain re-validation to GitHub task-list checkbox lines only (- [ ] / - [x] outside code fences), ignoring headings, notes, and non-checkbox content - Define pass counts as checked/total checkbox items - Use FEATURE_DIR/checklists/requirements.md in Done When for consistency with the rest of the template --- templates/commands/clarify.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/templates/commands/clarify.md b/templates/commands/clarify.md index c9340807d1..8be519a78e 100644 --- a/templates/commands/clarify.md +++ b/templates/commands/clarify.md @@ -202,13 +202,14 @@ Execution steps: - If it does NOT exist, skip this step silently. - If it exists: 1. Read the checklist file. - 2. Re-evaluate each checklist item against the **updated** spec (the version just saved in step 7). - 3. For each item: + 2. Identify all GitHub task-list checkbox lines — lines matching `- [ ]` or `- [x]` outside of code fences. Ignore all other content (headings, notes, non-checkbox bullets, metadata). + 3. Re-evaluate each checkbox item against the **updated** spec (the version just saved in step 7). + 4. For each checkbox item: - If the item now passes (e.g., a `[NEEDS CLARIFICATION]` marker was resolved, a requirement was made unambiguous, an edge case was added), update the marker from `- [ ]` to `- [x]`. - If the item previously passed and still passes, leave it as `- [x]`. - If the item does not pass, leave it as `- [ ]`. - 4. Save the updated checklist file. - 5. Record the before/after pass counts for the Completion Report (e.g., "12/16 → 15/16 items passing"). + 5. Save the updated checklist file. + 6. Record the before/after pass counts as checked/total checkbox items for the Completion Report (e.g., "12/16 → 15/16 items passing"). Behavior rules: @@ -269,6 +270,6 @@ Report completion (after questioning loop ends or early termination): ## Done When - [ ] Spec ambiguities identified and clarifications integrated into spec file -- [ ] Spec quality checklist re-validated against updated spec (if `checklists/requirements.md` exists) +- [ ] Spec quality checklist re-validated against updated spec (if `FEATURE_DIR/checklists/requirements.md` exists) - [ ] Extension hooks dispatched or skipped according to the rules in Mandatory Post-Execution Hooks above - [ ] Completion reported to user with questions answered, sections touched, checklist status, and coverage summary From 2320542170ac0b3856b2f3878c05157db6c9ff63 Mon Sep 17 00:00:00 2001 From: Manfred Riem <15701806+mnriem@users.noreply.github.com> Date: Tue, 26 May 2026 18:04:09 -0500 Subject: [PATCH 3/5] Address review: handle regressions in checklist revalidation - Clarify that each checkbox is set based solely on current spec state, regardless of prior marker (checked->unchecked is possible) - Completion report now lists both newly passing items and regressions (checked->unchecked) so users see what became non-compliant --- templates/commands/clarify.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/templates/commands/clarify.md b/templates/commands/clarify.md index 8be519a78e..9311d42f9e 100644 --- a/templates/commands/clarify.md +++ b/templates/commands/clarify.md @@ -204,10 +204,9 @@ Execution steps: 1. Read the checklist file. 2. Identify all GitHub task-list checkbox lines — lines matching `- [ ]` or `- [x]` outside of code fences. Ignore all other content (headings, notes, non-checkbox bullets, metadata). 3. Re-evaluate each checkbox item against the **updated** spec (the version just saved in step 7). - 4. For each checkbox item: - - If the item now passes (e.g., a `[NEEDS CLARIFICATION]` marker was resolved, a requirement was made unambiguous, an edge case was added), update the marker from `- [ ]` to `- [x]`. - - If the item previously passed and still passes, leave it as `- [x]`. - - If the item does not pass, leave it as `- [ ]`. + 4. For each checkbox item, set its marker based solely on whether it passes against the current spec — prior state does not matter: + - If the item passes: set to `- [x]`. + - If the item does not pass: set to `- [ ]` (even if it was previously `- [x]` — spec edits can cause regressions). 5. Save the updated checklist file. 6. Record the before/after pass counts as checked/total checkbox items for the Completion Report (e.g., "12/16 → 15/16 items passing"). @@ -262,7 +261,7 @@ Report completion (after questioning loop ends or early termination): - Number of questions asked & answered. - Path to updated spec. - Sections touched (list names). -- Spec quality checklist status (if `FEATURE_DIR/checklists/requirements.md` was re-validated): show before/after pass counts (e.g., "Spec Quality Checklist: 12/16 → 15/16 items passing") and list any items that changed from unchecked to checked. If any items remain unchecked, list them as areas needing attention. +- Spec quality checklist status (if `FEATURE_DIR/checklists/requirements.md` was re-validated): show before/after pass counts (e.g., "Spec Quality Checklist: 12/16 → 15/16 items passing") and list any items that changed state — both newly checked (unchecked → checked) and any regressions (checked → unchecked). If any items remain unchecked, list them as areas needing attention. - Coverage summary table listing each taxonomy category with Status: Resolved (was Partial/Missing and addressed), Deferred (exceeds question quota or better suited for planning), Clear (already sufficient), Outstanding (still Partial/Missing but low impact). - If any Outstanding or Deferred remain, recommend whether to proceed to `__SPECKIT_COMMAND_PLAN__` or run `__SPECKIT_COMMAND_CLARIFY__` again later post-plan. - Suggested next command. From 28b8f8efa4b5ab75e09bc3df2e55bae928cf04bf Mon Sep 17 00:00:00 2001 From: Manfred Riem <15701806+mnriem@users.noreply.github.com> Date: Wed, 27 May 2026 06:16:02 -0500 Subject: [PATCH 4/5] Address review: case-insensitive checkboxes, preserve file verbatim - Accept [x], [X], and leading whitespace for nested task items - Explicitly state only the [ ]/[x] marker is toggled; all other file content (headings, metadata, notes, ordering, whitespace) must remain unchanged to avoid noisy diffs --- templates/commands/clarify.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/templates/commands/clarify.md b/templates/commands/clarify.md index 9311d42f9e..f851904d67 100644 --- a/templates/commands/clarify.md +++ b/templates/commands/clarify.md @@ -202,12 +202,12 @@ Execution steps: - If it does NOT exist, skip this step silently. - If it exists: 1. Read the checklist file. - 2. Identify all GitHub task-list checkbox lines — lines matching `- [ ]` or `- [x]` outside of code fences. Ignore all other content (headings, notes, non-checkbox bullets, metadata). + 2. Identify all GitHub task-list checkbox lines — lines matching `- [ ]`, `- [x]`, or `- [X]` (case-insensitive, tolerant of leading whitespace for nested items) outside of code fences. Ignore all other content (headings, notes, non-checkbox bullets, metadata). 3. Re-evaluate each checkbox item against the **updated** spec (the version just saved in step 7). 4. For each checkbox item, set its marker based solely on whether it passes against the current spec — prior state does not matter: - If the item passes: set to `- [x]`. - If the item does not pass: set to `- [ ]` (even if it was previously `- [x]` — spec edits can cause regressions). - 5. Save the updated checklist file. + 5. Save the updated checklist file. **Only toggle the `[ ]`/`[x]` marker portion of checkbox lines.** All other file content — headings, metadata, notes, line ordering, whitespace — must remain unchanged to avoid noisy diffs. 6. Record the before/after pass counts as checked/total checkbox items for the Completion Report (e.g., "12/16 → 15/16 items passing"). Behavior rules: From 668a7ee9c899e4e8c29132f05eb328711e84cc0d Mon Sep 17 00:00:00 2001 From: Manfred Riem <15701806+mnriem@users.noreply.github.com> Date: Wed, 27 May 2026 06:22:40 -0500 Subject: [PATCH 5/5] Address review: track per-item state, preserve marker case - Add explicit before-snapshot step to capture each item's prior marker state before re-evaluation - Compute three lists for the report: newly passing, regressions, and still unchecked - Only toggle markers whose checked/unchecked state actually changes; preserve existing case ([x]/[X]) when state is unchanged to avoid cosmetic diffs --- templates/commands/clarify.md | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/templates/commands/clarify.md b/templates/commands/clarify.md index f851904d67..8488557e74 100644 --- a/templates/commands/clarify.md +++ b/templates/commands/clarify.md @@ -203,12 +203,18 @@ Execution steps: - If it exists: 1. Read the checklist file. 2. Identify all GitHub task-list checkbox lines — lines matching `- [ ]`, `- [x]`, or `- [X]` (case-insensitive, tolerant of leading whitespace for nested items) outside of code fences. Ignore all other content (headings, notes, non-checkbox bullets, metadata). - 3. Re-evaluate each checkbox item against the **updated** spec (the version just saved in step 7). - 4. For each checkbox item, set its marker based solely on whether it passes against the current spec — prior state does not matter: - - If the item passes: set to `- [x]`. - - If the item does not pass: set to `- [ ]` (even if it was previously `- [x]` — spec edits can cause regressions). - 5. Save the updated checklist file. **Only toggle the `[ ]`/`[x]` marker portion of checkbox lines.** All other file content — headings, metadata, notes, line ordering, whitespace — must remain unchanged to avoid noisy diffs. - 6. Record the before/after pass counts as checked/total checkbox items for the Completion Report (e.g., "12/16 → 15/16 items passing"). + 3. For each checkbox line, record its current marker state (checked or unchecked) and item text into a before-snapshot list. + 4. Re-evaluate each checkbox item against the **updated** spec (the version just saved in step 7). + 5. For each checkbox item, update only if the checked/unchecked state actually changes: + - If the item now passes and was unchecked: change `[ ]` to `[x]`. + - If the item now fails and was checked: change `[x]`/`[X]` to `[ ]`. + - If the state is unchanged: leave the marker as-is (preserve existing case to avoid cosmetic diffs). + 6. Save the updated checklist file. **Only toggle the `[ ]`/`[x]` marker portion of checkbox lines whose state changed.** All other file content — headings, metadata, notes, line ordering, whitespace — must remain unchanged to avoid noisy diffs. + 7. Compare the before-snapshot with the current state to compute three lists for the Completion Report: + - **Newly passing**: items that changed from unchecked to checked. + - **Regressions**: items that changed from checked to unchecked. + - **Still unchecked**: items that remain unchecked. + 8. Record the before/after pass counts as checked/total checkbox items (e.g., "12/16 → 15/16 items passing"). Behavior rules: