Skip to content

docs(skills): tooling division of labor — MCP to query, YAML to edit, CLI to gate - #36

Open
jimisola wants to merge 3 commits into
mainfrom
docs/tooling-division-of-labor
Open

docs(skills): tooling division of labor — MCP to query, YAML to edit, CLI to gate#36
jimisola wants to merge 3 commits into
mainfrom
docs/tooling-division-of-labor

Conversation

@jimisola

@jimisola jimisola commented Aug 15, 2026

Copy link
Copy Markdown
Member

What & why

Adds a "Tooling Division of Labor" section to the reqstool-conventions reference, separating four roles across the three interfaces: query and check through the MCP tools, edit the YAML SSOT directly, gate with the CLI against a fresh full build.

The section was originally written around a limitation that has since been fixed. It was motivated by an empirical mismatch in a consuming project: a long-lived reqstool mcp local server reported a days-old snapshot (14 requirements, 0 implementations, 0 tests) against a reality of 55/55 PASS. Follow-up testing pinned the mechanism — the server loaded the YAML model and the generated annotation/test artifacts once at spawn, with no reload and no staleness detection. reqstool-client #438, released in 0.12.1, fixes exactly that: the server now records which local files it parsed (including files it looked for and did not find, and the concrete files each test_results glob matched), re-checks them before answering, and reloads on change.

With that fixed, "verify only with the CLI" no longer describes anything true about the data, so the guidance is rewritten rather than softened:

  • MCP status is a real check, not a browsing aid. get_status, get_requirement_status, and get_requirements_status all delegate to the same per-requirement verdict computation the CLI uses — reqstool-client #412 closed the last parallel path — so on the same inputs they agree. get_requirements_status is the recommended way to find what is still unimplemented or untested, rather than shelling out repeatedly.
  • The CLI is the gate, on its own merit. Not because MCP is the weaker number, but because a gate has to run where there is no agent and no MCP client, has to leave an artifact someone else can reproduce, and re-parses from a cold start rather than confirming the world-view the session has been operating under. Its exit code is the number of unmet requirements.
  • The build caveat survives the fix, and is the part worth keeping. The server re-reads generated artifacts, it does not produce them — and the freshness check does not sequence your build either, so a status call made before the build finishes returns a fresh, well-formed, wrong answer. Run a clean full build first and treat any completeness number as scoped to that build, whichever interface reads it.
  • Adds the diagnostic path: snapshot on get_status (built_at, tracked_files, warnings) shows what is actually being served — a test_results pattern matching no files is reported there rather than counted as zero tests — and refresh forces a reload, mainly needed for remote (git/maven/npm/pypi) sources, which are version-pinned and never watched.
  • Records the pre-0.12.1 behaviour for anyone pinned to an older server; against those, MCP status really is a browsing aid only.

check-skill-reference-docs.sh passes.

@jimisola jimisola self-assigned this Aug 15, 2026
…CLI to verify

Adds guidance to reqstool-conventions: use the MCP tools for lookups, edit
the YAML SSOT directly, and treat only the CLI status (against a fresh full
build) as a gate verdict. Records the current MCP limitations (startup-time
model load, no artifact ingestion) that make its status a browsing aid, not
a verification.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: jimisola <jimisola@jimisola.com>
reqstool-client #438 (released in 0.12.1) makes the MCP server re-check the
files it parsed before each request and reload on change, so the "no reload, no
staleness detection" limitation this section was written around no longer
holds. Rewrites the guidance accordingly: MCP for querying (its status tools
share the CLI's verdict computation), YAML for editing, CLI for the CI gate —
the gate framing now resting on the exit code rather than on MCP being
untrustworthy.

Keeps the build caveat, which the fix does not address: the server re-reads
generated artifacts, it does not produce them, so a clean full build is still
what makes a completeness number real. Adds the `get_status` snapshot field and
`refresh` as the way to inspect what is being served, and records the pre-0.12.1
behaviour for anyone pinned to an older server.

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>
@jimisola jimisola changed the title docs(skills): tooling division of labor — MCP to read, YAML to edit, CLI to verify docs(skills): tooling division of labor — MCP to query, YAML to edit, CLI to gate Aug 15, 2026
The previous wording ("verify only with the CLI") was inherited from when MCP
status could not be trusted, and reads as though it is still the weaker number.
It is not: get_status, get_requirement_status and get_requirements_status all
delegate to the same per-requirement verdict computation the CLI uses (#412
closed the last parallel path), so on the same inputs they agree.

Splits the two roles. MCP status is the check you run while working — including
get_requirements_status to find what is still unimplemented — and the CLI is the
gate, on its own merit: it runs where there is no MCP client, its exit code is
the number of unmet requirements, and it re-parses cold rather than confirming
the world-view the session has been operating under.

Also sharpens the build caveat: the freshness check does not sequence the build,
so a status call made before the build finishes returns a fresh, well-formed,
wrong answer.

Signed-off-by: Jimisola Laursen <jimisola@jimisola.com>
@jimisola

Copy link
Copy Markdown
Member Author

Independent verification of the 0.12.1 claims from a consuming project (skills-gateway, 55 requirements):

  • Reload-on-change: one server process, no respawn — get_status returned 55, a requirement was appended to requirements.yml, the next call returned 56.
  • Cold vs stale baseline: a pre-0.12.1 server left running for ~2 days served its spawn-time snapshot (14 requirements, 0 implementations) while the CLI said 55/55 PASS; a fresh spawn matched the CLI exactly (55 reqs, 54 with implementation, 59 tests).

The guidance text as now written matches observed behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant