From 7bc06d8b4b63409f74d33bab75e53ab678640515 Mon Sep 17 00:00:00 2001 From: wshallwshall Date: Fri, 7 Aug 2026 11:58:59 -0500 Subject: [PATCH 1/8] docs(sds): the rules were citable only by position, so nothing kept a citation pointing at one A section number is a position, not a name. This document has already been renumbered once -- Version history records the shift of sections 5 through 9 down by one when Spec-Driven Development was inserted -- and the citations written against the old scheme did not error, did not render differently, and were not noticed. They simply began naming different requirements. Ten of them in the maintainer's private security documents still resolve to the wrong section today: four name the release gate by a number that now belongs to "Clarify and analyze". This commit adds the convention and the gate that enforces it, BEFORE any rule carries an identifier. That order is deliberate: a guard added after the rules cannot catch what drifted while it was absent. - "How to read the rules" states the SDS-
. grammar, the RFC 2119 keywords, the meaning of the Evidence column, and the append-never-insert / reword-vs-retire rule. It is an UNNUMBERED section, so it renumbers nothing. - "Retired rules" holds the tombstones, so a citation that outlives its rule lands on a record of what happened rather than on whatever requirement later took the number. - tests/test_sds_rule_ids_are_stable.py makes that text mechanical. The guard is derivable from the working tree alone. It cannot use history: the required test leg checks out shallow, so an identifier's past is unreadable where it runs. Every check was fired at input built to break it before its green was credited -- nine mutations applied to the real document, each confirmed red with a repository-relative message, the file restored byte-identically after each. One mutation initially reported a check as blind; the mutation was wrong rather than the check, having replaced the asserted heading with a string that still contained it. Failure output is repository-relative on purpose. An absolute path on a developer machine carries a worktree slug, and a bare slug is a shape the forbidden-content guard does not catch before it reaches this public repository (BACKLOG #1083). Wired into the doc-guards CI step at BOTH of its module lists. Missing either one leaves the guard silent on docs-only pull requests, which is the exact change shape it exists to catch. What it deliberately does not catch is recorded in the module docstring: a reworded rule whose meaning changed, any citation in the private vault (invisible where CI runs), and the published PDF and its website mirror, which live in a repository with no CI and are already divergent. --- .github/workflows/ci.yml | 4 +- docs/Secure_Development_Standards.md | 48 +++ tests/test_sds_rule_ids_are_stable.py | 414 ++++++++++++++++++++++++++ 3 files changed, 464 insertions(+), 2 deletions(-) create mode 100644 tests/test_sds_rule_ids_are_stable.py diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 21c63f38..fbf47705 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -195,13 +195,13 @@ jobs: tests/test_crit2_inline_doc_drift.py tests/test_doc_ref_handle.py tests/test_docs_db_grants.py \ tests/test_docs_runbooks.py tests/test_docs_security_pathways.py tests/test_security_doc_drift.py \ tests/test_security_doc_rate_limits.py tests/test_threat_model_doc_drift.py \ - tests/test_backlog_status_check.py + tests/test_backlog_status_check.py tests/test_sds_rule_ids_are_stable.py pytest -q -rs \ tests/test_asvs_file_surface_doc_drift.py tests/test_cloud_phi_hipaa_doc_drift.py \ tests/test_crit2_inline_doc_drift.py tests/test_doc_ref_handle.py tests/test_docs_db_grants.py \ tests/test_docs_runbooks.py tests/test_docs_security_pathways.py tests/test_security_doc_drift.py \ tests/test_security_doc_rate_limits.py tests/test_threat_model_doc_drift.py \ - tests/test_backlog_status_check.py + tests/test_backlog_status_check.py tests/test_sds_rule_ids_are_stable.py # PySide6's offscreen platform plugin needs a few system libraries even # headless. Linux-only; Windows runners need no equivalent. diff --git a/docs/Secure_Development_Standards.md b/docs/Secure_Development_Standards.md index 3a21d1db..cec9c280 100644 --- a/docs/Secure_Development_Standards.md +++ b/docs/Secure_Development_Standards.md @@ -14,6 +14,42 @@ --- +## How to read the rules + +Each requirement in this standard is one testable statement carrying a stable identifier and the +evidence that settles it. Cite the identifier — in a documented deviation (A.6), in a review, in a +companion standard — so that the citation and the requirement it names stay the same sentence. + +| Element | What it means | +|---|---| +| `SDS-
.` | A stable identifier. It survives rewording of the text around it | +| **MUST**, **MUST NOT** | Absolute. Not meeting one is a deviation recorded in A.6, never a judgment call | +| **SHOULD**, **SHOULD NOT** | Depart from it only for a stated reason you have weighed | +| **MAY** | A free choice. Conforming either way | +| Evidence | What a reviewer looks at. A requirement with no checkable evidence is an aspiration | + +Those words carry that meaning **only in capitals**; the same words in lowercase prose do not. + +**An identifier is a permanent name, never a position.** A new rule takes the next free number in its +section and is **appended** — inserting one must never renumber those after it, because a citation +written against the old number would then resolve silently to a different requirement. Reword a rule +freely under the same identifier. Change *what it demands* and the identifier is retired and a new one +allocated; [Retired rules](#retired-rules) keeps the tombstone, so a citation that outlives its rule +lands on a record of what happened rather than on somebody else's requirement. + +**The numeric part resembles a section number because of where the rule was first written, not because +it is a lookup.** An identifier is a name. If the surrounding section is ever renumbered the identifier +does not move, and that is the property that makes it worth citing: this document has been renumbered +once already (§5–§9 became §6–§10 when Spec-Driven Development was inserted; see Version history), and +citations written against the old numbers are still resolving to the wrong requirements today. + +**Not every statement here is a rule.** The NIST and HIPAA crosswalks (§3, §7.2), the spec-stack and +EARS tables (§5.1, §5.2), the control-area inventory (§7.1), the reference list (§10) and the +Applicability Profile (Appendix A) describe, map or record; they carry no identifiers and nothing +conforms *to* them. A.6 is the register that **cites** rules — it does not state them. + +--- + ## 1. Purpose and scope This is the secure development standard for an **open-source software project intended for use in regulated environments, including healthcare** (handling of protected health information, PHI). @@ -582,6 +618,18 @@ PW.8 (test executable code) is the home of `messagefoundry check` (validate + dr --- +## Retired rules + +A retired identifier is **never reissued**. A rule is retired when *what it demands* changes; a rule +whose wording changes keeps its identifier and does not appear here. A citation that outlives its rule +resolves to a row below rather than to whatever requirement later took the number. + +| Retired ID | Retired in | What it required | Why, and what replaced it | +|---|---|---|---| +| — | — | *No rule has been retired yet.* | — | + +--- + ## Version history | Version | Date | Change | diff --git a/tests/test_sds_rule_ids_are_stable.py b/tests/test_sds_rule_ids_are_stable.py new file mode 100644 index 00000000..79db997d --- /dev/null +++ b/tests/test_sds_rule_ids_are_stable.py @@ -0,0 +1,414 @@ +# SPDX-License-Identifier: AGPL-3.0-or-later +# Copyright (C) 2026 MessageFoundry Organization and contributors +"""Stability guard for the ``SDS-
.`` rule identifiers in the Secure Development Standards. + +WHY THIS EXISTS. The standard's requirements used to be citable only by section number, and a section +number is a position rather than a name. The document has already been renumbered once — its Version +history records ``§5–§9 → §6–§10`` when Spec-Driven Development was inserted as the new §5 — and the +citations written against the old scheme did not error, did not render differently, and were not +noticed. They simply began naming different requirements. Ten of them in the maintainer's private +security documents still resolve to the wrong section today: four cite "§5.4 (release gate)" when +release gates are §6.4 and §5.4 is now "Clarify and analyze". + +Identifiers fix that only if something keeps them stable, which is what this module is. The convention +it enforces is stated in the standard itself under "How to read the rules"; the assertions here are +that text made mechanical. + +WHAT IT DELIBERATELY DOES NOT CATCH. + +* **A reworded rule that changed what it demands.** Rewording under the same identifier is legal and + retiring is a judgment call about meaning, which no regex makes. ``test_no_unaccounted_gaps`` catches + the *mechanical* half — a rule that vanished without a tombstone — and nothing here can tell an + editorial rewrite from a semantic one. That remains a review obligation. +* **A citation in the private security vault.** ``git ls-files docs/security/`` returns nothing in this + repository, so the ten known-stale citations are invisible where CI runs. This guard sees the + standard, its siblings and the code; it cannot see the vault, and a green run here is not evidence + about the vault's citations. +* **A citation in the published PDF or its website mirror.** Those live in the separate + ``messagefoundry-website`` repository, which has no CI at all. The mirror is hand-copied and is + already 79 lines divergent while carrying the same version and date. +* **Anything derived from git history.** The required test leg checks out at ``fetch-depth: 1``, so an + identifier's past is not readable here. Every check below is derivable from the working tree alone. + +Paths in failure messages are repository-relative on purpose: absolute paths on a developer machine +carry a worktree slug, and a bare slug is exactly the shape the forbidden-content guard does not catch +before it reaches this public repository. +""" + +from __future__ import annotations + +import re +from dataclasses import dataclass +from pathlib import Path + +import pytest + +_REPO = Path(__file__).resolve().parents[1] +_SDS = _REPO / "docs" / "Secure_Development_Standards.md" + +#: Heading under which the tombstones live. Changing it here and in the document together is fine; +#: changing it in only one place fails ``test_retired_rules_table_is_present``. +_RETIRED_HEADING = "## Retired rules" + +#: The RFC 2119 keywords that give a rule its force. Capitals only — the standard says so explicitly, +#: and a lowercase "should" in a requirement cell is the defect this catches. +_RFC2119 = ("MUST NOT", "MUST", "SHOULD NOT", "SHOULD", "MAY") + +#: A full identifier anywhere in prose or in a table cell. +_ID_ANYWHERE = re.compile(r"\bSDS-\d+(?:\.\d+)+\b") + +#: A heading, capturing its dotted number when it has one (``### 4.3 Produce ...`` -> ``4.3``). +_HEADING = re.compile(r"^(#{2,4})\s+(?:(A?\.?\d+(?:\.\d+)*)\.?\s+)?(.*)$") + + +@dataclass(frozen=True) +class Rule: + """One identified requirement, as parsed from a table row.""" + + rule_id: str + section: str + number: int + requirement: str + evidence: str + line: int + heading_number: str + + +@dataclass(frozen=True) +class Retired: + """One tombstone row from the Retired rules table.""" + + rule_id: str + line: int + + +def _split_row(line: str) -> list[str]: + """Split a markdown table row into stripped cells, dropping the leading/trailing empties.""" + if not line.lstrip().startswith("|"): + return [] + return [cell.strip() for cell in line.strip().strip("|").split("|")] + + +def _parse_id(rule_id: str) -> tuple[str, int] | None: + """Split ``SDS-7.4.3`` into its section (``7.4``) and its number (``3``).""" + if not rule_id.startswith("SDS-"): + return None + parts = rule_id[len("SDS-") :].split(".") + if len(parts) < 2 or not all(p.isdigit() for p in parts): + return None + return ".".join(parts[:-1]), int(parts[-1]) + + +def parse_rules(text: str) -> list[Rule]: + """Every identified rule defined in ``text``, in document order. + + A definition is a table row whose FIRST cell is exactly an identifier. That shape is what makes a + rule a rule; an identifier mentioned inside a sentence is a citation, not a definition. + """ + rules: list[Rule] = [] + heading_number = "" + in_retired = False + for lineno, line in enumerate(text.splitlines(), start=1): + if line.startswith("## "): + in_retired = line.strip() == _RETIRED_HEADING + match = _HEADING.match(line) + if match: + heading_number = match.group(2) or "" + if in_retired: + continue + cells = _split_row(line) + if len(cells) < 3: + continue + parsed = _parse_id(cells[0]) + if parsed is None: + continue + section, number = parsed + rules.append( + Rule( + rule_id=cells[0], + section=section, + number=number, + requirement=cells[1], + evidence=cells[2], + line=lineno, + heading_number=heading_number, + ) + ) + return rules + + +def parse_retired(text: str) -> list[Retired]: + """Every identifier listed in the Retired rules table.""" + retired: list[Retired] = [] + in_retired = False + for lineno, line in enumerate(text.splitlines(), start=1): + if line.startswith("## "): + in_retired = line.strip() == _RETIRED_HEADING + continue + if not in_retired: + continue + cells = _split_row(line) + if not cells: + continue + if _parse_id(cells[0]) is not None: + retired.append(Retired(rule_id=cells[0], line=lineno)) + return retired + + +def find_gaps(rules: list[Rule], retired: list[Retired]) -> list[str]: + """Identifiers between 1 and the section's highest that are neither live nor tombstoned. + + A rule deleted without a tombstone leaves its number free for reuse, which is the failure the + whole convention exists to prevent. Retiring it closes the gap; deleting it silently does not. + """ + live: dict[str, set[int]] = {} + for rule in rules: + live.setdefault(rule.section, set()).add(rule.number) + gone: dict[str, set[int]] = {} + for tomb in retired: + parsed = _parse_id(tomb.rule_id) + if parsed is not None: + gone.setdefault(parsed[0], set()).add(parsed[1]) + missing: list[str] = [] + for section, numbers in sorted(live.items()): + highest = max(numbers) + accounted = numbers | gone.get(section, set()) + missing.extend(f"SDS-{section}.{n}" for n in range(1, highest + 1) if n not in accounted) + return missing + + +def _read(path: Path) -> str: + return path.read_text(encoding="utf-8") + + +def _rel(path: Path) -> str: + """Repository-relative path, so no failure message can leak a worktree slug.""" + return path.relative_to(_REPO).as_posix() + + +@pytest.fixture(scope="module") +def sds_text() -> str: + return _read(_SDS) + + +@pytest.fixture(scope="module") +def rules(sds_text: str) -> list[Rule]: + return parse_rules(sds_text) + + +@pytest.fixture(scope="module") +def retired(sds_text: str) -> list[Retired]: + return parse_retired(sds_text) + + +# --- the convention is present at all --------------------------------------------------------------- + + +def test_convention_section_is_present(sds_text: str) -> None: + """The rules mean nothing without the paragraph that defines how to read them.""" + assert "## How to read the rules" in sds_text, ( + f"{_rel(_SDS)} must define the identifier convention; every assertion in this module is a " + "restatement of that section and is meaningless if it is gone." + ) + + +def test_retired_rules_table_is_present(sds_text: str) -> None: + assert _RETIRED_HEADING in sds_text, ( + f"{_rel(_SDS)} must keep the '{_RETIRED_HEADING}' table. Without it a retired identifier has " + "nowhere to land and an outliving citation resolves to whatever took the number." + ) + + +# --- the live corpus -------------------------------------------------------------------------------- + + +def test_every_rule_id_is_well_formed(rules: list[Rule]) -> None: + bad = [r for r in rules if _parse_id(r.rule_id) is None] + assert not bad, "\n".join( + f"{_rel(_SDS)}:{r.line} malformed identifier {r.rule_id!r}" for r in bad + ) + + +def test_no_duplicate_rule_ids(rules: list[Rule]) -> None: + seen: dict[str, int] = {} + clashes: list[str] = [] + for rule in rules: + if rule.rule_id in seen: + clashes.append( + f"{_rel(_SDS)}:{rule.line} {rule.rule_id} already defined at line {seen[rule.rule_id]}" + ) + else: + seen[rule.rule_id] = rule.line + assert not clashes, "\n".join( + ["An identifier names exactly one requirement. Duplicates:", *clashes] + ) + + +def test_rule_id_section_matches_its_heading(rules: list[Rule]) -> None: + """``SDS-7.4.3`` must be defined under §7.4. A mis-sited rule is a citation pointing elsewhere.""" + wrong = [r for r in rules if r.heading_number and r.section != r.heading_number.lstrip(".")] + assert not wrong, "\n".join( + [ + "A rule's identifier must carry the section it is defined under:", + *( + f"{_rel(_SDS)}:{r.line} {r.rule_id} sits under section {r.heading_number}" + for r in wrong + ), + ] + ) + + +def test_no_unaccounted_gaps(rules: list[Rule], retired: list[Retired]) -> None: + missing = find_gaps(rules, retired) + assert not missing, ( + "These identifiers are neither live nor tombstoned, so their numbers are free to be reused " + f"by a later rule — which is how a citation silently changes meaning: {', '.join(missing)}. " + f"Retire them in '{_RETIRED_HEADING}' rather than deleting them." + ) + + +def test_retired_ids_are_not_live(rules: list[Rule], retired: list[Retired]) -> None: + live = {r.rule_id for r in rules} + reissued = sorted(t.rule_id for t in retired if t.rule_id in live) + assert not reissued, ( + "A retired identifier is never reissued; these are both tombstoned and live: " + f"{', '.join(reissued)}" + ) + + +def test_every_rule_states_an_rfc2119_keyword(rules: list[Rule]) -> None: + toothless = [r for r in rules if not any(k in r.requirement for k in _RFC2119)] + assert not toothless, "\n".join( + [ + "Every rule states its force in capitals (MUST / MUST NOT / SHOULD / SHOULD NOT / MAY):", + *(f"{_rel(_SDS)}:{r.line} {r.rule_id} — {r.requirement[:80]}" for r in toothless), + ] + ) + + +def test_every_rule_carries_evidence(rules: list[Rule]) -> None: + """A rule with no checkable evidence is an aspiration, which the standard says in those words.""" + empty = [r for r in rules if not r.evidence or r.evidence in {"-", "—", "TBD"}] + assert not empty, "\n".join( + [ + "Every rule names the evidence that settles it:", + *(f"{_rel(_SDS)}:{r.line} {r.rule_id} has no evidence" for r in empty), + ] + ) + + +# --- citations across the repository ---------------------------------------------------------------- + + +def _citation_sources() -> list[Path]: + """Tracked prose and code that may cite a rule. The vault and the website are out of reach. + + Code is included because the section-number citations this convention replaces are already there — + the CLI help, the DAST receipt constant and a CI pin all name parts of this standard — so an + identifier will end up in code the moment one of those migrates. + + ``tests/`` is deliberately excluded: the self-bite fixtures below define synthetic identifiers + (``SDS-9.1`` and friends) that name no real rule by design, and scanning them would make this guard + fail on its own evidence. + """ + paths = [ + _REPO / "CLAUDE.md", + *sorted((_REPO / "docs").rglob("*.md")), + *sorted((_REPO / "messagefoundry").rglob("*.py")), + *sorted((_REPO / "scripts").rglob("*.py")), + ] + return [p for p in paths if p.is_file()] + + +def test_every_cited_id_resolves(rules: list[Rule], retired: list[Retired]) -> None: + """A citation naming no rule is the defect in its final form: it reads fine and says nothing.""" + known = {r.rule_id for r in rules} | {t.rule_id for t in retired} + dangling: list[str] = [] + for path in _citation_sources(): + for lineno, line in enumerate(_read(path).splitlines(), start=1): + for cited in _ID_ANYWHERE.findall(line): + if cited not in known: + dangling.append(f"{_rel(path)}:{lineno} cites {cited}, which is not a rule") + assert not dangling, "\n".join( + ["Every cited identifier must resolve to a live or retired rule:", *dangling] + ) + + +# --- prove the instrument --------------------------------------------------------------------------- +# +# Each check above is fired at input built to break it. A guard that has never been observed to fail is +# not evidence, and these assertions are what make the green above mean something while the live corpus +# is still small. + +_HEADER = "## How to read the rules\n\n## Retired rules\n\n| Retired ID | Retired in |\n|---|---|\n" + + +def test_selfbite_duplicate_ids_are_detected() -> None: + text = ( + "### 9. Evidence\n\n| ID | Requirement | Evidence |\n|---|---|---|\n" + "| SDS-9.1 | **MUST** do a thing | The thing |\n" + "| SDS-9.1 | **MUST** do another | The other |\n" + ) + parsed = parse_rules(text) + assert len(parsed) == 2 + assert len({r.rule_id for r in parsed}) == 1, "the duplicate must survive parsing to be caught" + + +def test_selfbite_a_deleted_rule_leaves_a_gap() -> None: + text = ( + "### 9. Evidence\n\n| ID | Requirement | Evidence |\n|---|---|---|\n" + "| SDS-9.1 | **MUST** a | x |\n" + "| SDS-9.3 | **MUST** c | z |\n" + ) + assert find_gaps(parse_rules(text), []) == ["SDS-9.2"] + + +def test_selfbite_a_tombstone_closes_the_gap() -> None: + text = ( + "### 9. Evidence\n\n| ID | Requirement | Evidence |\n|---|---|---|\n" + "| SDS-9.1 | **MUST** a | x |\n" + "| SDS-9.3 | **MUST** c | z |\n" + f"\n{_RETIRED_HEADING}\n\n| Retired ID | Why |\n|---|---|\n| SDS-9.2 | superseded |\n" + ) + assert find_gaps(parse_rules(text), parse_retired(text)) == [] + + +def test_selfbite_retired_rows_are_not_parsed_as_live() -> None: + """The tombstone table is full of identifiers; reading them as definitions would mask every gap.""" + text = f"{_RETIRED_HEADING}\n\n| Retired ID | Why |\n|---|---|\n| SDS-9.2 | superseded |\n" + assert parse_rules(text) == [] + assert [t.rule_id for t in parse_retired(text)] == ["SDS-9.2"] + + +def test_selfbite_a_lowercase_keyword_is_toothless() -> None: + text = ( + "### 9. Evidence\n\n| ID | Requirement | Evidence |\n|---|---|---|\n" + "| SDS-9.1 | should probably do a thing | x |\n" + ) + (rule,) = parse_rules(text) + assert not any(k in rule.requirement for k in _RFC2119), ( + "lowercase 'should' must not satisfy the keyword check; the standard says capitals only" + ) + + +def test_selfbite_a_missited_rule_is_detected() -> None: + text = ( + "### 7.4 Interface authentication\n\n| ID | Requirement | Evidence |\n|---|---|---|\n" + "| SDS-6.3.1 | **MUST** do a thing | x |\n" + ) + (rule,) = parse_rules(text) + assert rule.section == "6.3" and rule.heading_number == "7.4" + + +def test_selfbite_a_citation_is_not_a_definition() -> None: + """An identifier inside a sentence must not register as defining that rule.""" + text = "Deviation from SDS-4.3.7 is recorded in A.6.\n" + assert parse_rules(text) == [] + assert _ID_ANYWHERE.findall(text) == ["SDS-4.3.7"] + + +def test_selfbite_malformed_identifiers_are_rejected() -> None: + for bad in ("SDS-", "SDS-4", "SDS-4.x", "SD-4.1", "SDS-4..1"): + assert _parse_id(bad) is None, f"{bad!r} must not parse as an identifier" + assert _parse_id("SDS-4.3.7") == ("4.3", 7) + assert _parse_id("SDS-9.2") == ("9", 2) From cee4ed96f87fac97b50c6fc00fb3d5929cb4c25b Mon Sep 17 00:00:00 2001 From: wshallwshall Date: Fri, 7 Aug 2026 12:02:19 -0500 Subject: [PATCH 2/8] docs(sds): section 4 stated 70 obligations as prose, so none of them could be cited or checked The SSDF section carried the mass of this standard's requirements as bulleted prose. Three consequences, all of which this commit removes. A citation had nowhere to land. "PW.5" names eight sub-bullets carrying twenty-seven separate obligations between them, so citing PW.5 in a review or a deviation named a paragraph, not a requirement. Identifiers now name one obligation each. Nothing stated force. The whole of section 4 was written in the bare declarative present -- "Every change is peer-reviewed", "Releases are versioned" -- which reads as description rather than obligation and leaves a reader to guess whether departing from it is a deviation or a judgment call. Every rule now carries a capitalised RFC 2119 keyword. This is a mood conversion, not a change of content: no obligation was added, removed, weakened or strengthened. Nothing said what would settle it. Each rule now names the evidence a reviewer looks at, which is what makes it pass-or-fail rather than aspirational. Several of those evidence cells are deliberately uncomfortable -- "a clean secret-scan over the full history", "two builds of one commit producing the same artifact", "the pre-commit hook and the CI job, each observed to fail on a planted secret" -- because a rule whose evidence is easy to produce was not worth stating. 70 rules: 13 in 4.1, 6 in 4.2, 45 in 4.3, 6 in 4.4. Contiguous from 1 in each section, no gaps, which the guard confirms rather than my reading of it. The SSDF practice IDs stay in the requirement text in bold. They are stable names owned by NIST, they are cited from outside this document, and replacing them would repeat the defect being fixed. One rule gained a clause it did not have. SDS-4.3.36 (peer review) now names A.6 as where a project without a human second reviewer records the deviation. That was already true -- A.6 carries exactly that entry -- but the requirement did not point at it, so the rule read as unmet rather than deviated. --- docs/Secure_Development_Standards.md | 121 +++++++++++++++++++++------ 1 file changed, 95 insertions(+), 26 deletions(-) diff --git a/docs/Secure_Development_Standards.md b/docs/Secure_Development_Standards.md index cec9c280..7c7adac9 100644 --- a/docs/Secure_Development_Standards.md +++ b/docs/Secure_Development_Standards.md @@ -179,41 +179,110 @@ SSDF organizes secure development into four practice groups: **Prepare the Organ ### 4.1 Prepare the Organization (PO.1–PO.5) -- **Security requirements (PO.1).** Security requirements are documented (this standard plus each project's standing contract, e.g., a `CLAUDE.md` or equivalent) and treated as first-class alongside functional requirements. Captured as first-class specs, these requirements drive design and tests per **§5 (Spec-Driven Development)** — recommended. -- **Roles and responsibilities (PO.2).** Maintainer/reviewer roles are defined; security ownership is explicit; onboarding includes secure-development orientation. -- **Supporting toolchains (PO.3).** Source control; CI/CD with automated security checks (§6.2); dependency and secret scanning; pinned, integrity-verified dependencies. -- **Security check criteria (PO.4).** Pass/fail release gates are defined (§6.4); a release does not ship with unresolved high/critical findings. -- **Secure development environments (PO.5).** Disk-encrypted developer machines; **no real PHI in development or test** (synthetic or de-identified data only); least-privilege access to repositories and environments. +| ID | Requirement | Evidence | +|---|---|---| +| SDS-4.1.1 | **MUST** document the project's security requirements — this standard plus the project's standing contract — and treat them as first-class alongside functional requirements. **(PO.1)** | This standard, and the standing contract file it names | +| SDS-4.1.2 | **SHOULD** capture those requirements as first-class specs that drive design and tests (§5). **(PO.1)** | A spec artifact per requirement, linked to its design and tests | +| SDS-4.1.3 | **MUST** define maintainer and reviewer roles, with security ownership stated explicitly rather than assumed. **(PO.2)** | The written role assignment, naming who owns security | +| SDS-4.1.4 | **MUST** include secure-development orientation in onboarding. **(PO.2)** | The onboarding material | +| SDS-4.1.5 | **MUST** hold all source in version control. **(PO.3)** | The repository | +| SDS-4.1.6 | **MUST** run automated security checks in CI on every change (§6.2). **(PO.3)** | The CI workflow definition and a run of it | +| SDS-4.1.7 | **MUST** run dependency scanning and secret scanning. **(PO.3)** | The scanner jobs and their most recent results | +| SDS-4.1.8 | **MUST** pin dependencies and verify their integrity. **(PO.3)** | A hash-locked dependency file, checked in CI for drift | +| SDS-4.1.9 | **MUST** define pass/fail release gates (§6.4). **(PO.4)** | The documented gate list | +| SDS-4.1.10 | **MUST NOT** ship a release carrying unresolved high or critical findings. **(PO.4)** | The gate result for the released commit | +| SDS-4.1.11 | **MUST** require disk encryption on developer machines. **(PO.5)** | A stated and reviewed environment baseline | +| SDS-4.1.12 | **MUST NOT** use real PHI in development or test. Synthetic or de-identified data only. **(PO.5)** | The test corpus, and the de-identification path that produced it | +| SDS-4.1.13 | **MUST** grant least-privilege access to repositories and environments. **(PO.5)** | A dated access review | ### 4.2 Protect the Software (PS.1–PS.3) -- **Protect code from unauthorized access/tampering (PS.1).** Branch protection, required reviews, least-privilege access; no direct commits to the main branch; signed commits where supported. -- **Verify release integrity (PS.2).** Releases are versioned and integrity-verifiable (checksums/signatures); a software bill of materials (SBOM) is generated per release. -- **Archive and protect releases (PS.3).** Each released version, its build inputs, and its SBOM are archived to support incident analysis and reproducibility. +| ID | Requirement | Evidence | +|---|---|---| +| SDS-4.2.1 | **MUST** enforce branch protection and required reviews on the trunk. **(PS.1)** | The protection settings, and a rejected direct push | +| SDS-4.2.2 | **MUST NOT** permit direct commits to the main branch. **(PS.1)** | The protection settings | +| SDS-4.2.3 | **SHOULD** sign commits where the platform supports it. **(PS.1)** | Signature status on trunk commits | +| SDS-4.2.4 | **MUST** version releases and make them integrity-verifiable by checksum or signature. **(PS.2)** | The published checksums or signatures | +| SDS-4.2.5 | **MUST** generate a software bill of materials per release. **(PS.2)** | The SBOM attached to the release | +| SDS-4.2.6 | **MUST** archive each released version, its build inputs and its SBOM, so an incident can be analysed against what actually shipped. **(PS.3)** | The retained archive, and a retrieval of one past release | ### 4.3 Produce Well-Secured Software (PW.1, PW.2, PW.4–PW.9) -- **Secure design and threat modeling (PW.1–PW.2).** Each interface and component is threat-modeled; trust boundaries are identified and documented; designs are reviewed against the security requirements before build. The reviewed design artifact and its acceptance criteria are the spec; **§5** describes the recommended clarify/analyze gates applied before build. -- **Reuse well-secured components (PW.4).** Prefer vetted libraries; avoid rolling custom cryptography. -- **Secure coding practices (PW.5).** Mandatory: - - **Input validation** — validate structure and content at every ingress; reject or quarantine malformed input rather than processing it. *(Example: HL7 message validation in MEFOR.)* - - **Parameterized queries only** — no string-built SQL; use an ORM or parameterized statements throughout. - - **Authentication & authorization** — enforce on every action; deny by default; interface mechanisms per §7.4. - - **Web-service interfaces (REST and SOAP)** — authenticate every endpoint; validate and size-limit payloads against a schema; for SOAP/XML, disable external-entity resolution (XXE) and DTD processing; apply rate limiting and timeouts; never expose stack traces or sensitive data in fault responses. - - **File-handler interfaces** — confine reads/writes to configured directories and canonicalize paths (reject `../` traversal and symlink escapes); validate file type and size by content, not extension; use atomic write-then-rename so partial files are never processed; least-privilege storage, never an executable or web-served path; never execute file contents; scan inbound files for malware where feasible; encrypt sensitive files at rest and securely delete after processing per retention. - - **Cryptography** — TLS for all network communication; encryption at rest for sensitive data; approved algorithms and libraries; use FIPS-validated crypto where a deployment requires it. - - **Secrets** — never in code, prompts, or commit history; sourced from environment/secret store; enforced by pre-commit and CI secret scanning. - - **Error handling and logging** — fail closed; never log secrets or sensitive data; produce a tamper-resistant, timestamped audit log. -- **Secure build configuration (PW.6).** Reproducible builds; security-relevant build/interpreter and dependency settings fixed in the pipeline. -- **Code review and analysis (PW.7).** Every change is peer-reviewed; static analysis (SAST) and software composition analysis (SCA) run in CI (§6.2). Review also confirms the change conforms to its spec's acceptance criteria — see **§5** (analyze, cross-artifact coverage); recommended. -- **Test executable code (PW.8).** A maintained automated test suite runs on every change; security test cases are included. Tests SHOULD trace to the spec's acceptance criteria (**§5**, executable acceptance criteria) so coverage is mechanical, not prose. Test *quality* — not just presence — is judged per the [Code Quality & Anti-Slop Standards](Code_Quality_Standards.md): behavior-verifying assertions over mock choreography, with mutation testing as *guidance*. Heed its **anti-metric rule** — **never gate quality on line-coverage % alone** (a gameable slop-hiding place); measure structure and behavior, not a single scoreboard. -- **Secure defaults (PW.9).** Ships secure-by-default (TLS on, encryption on, least-privilege accounts, verbose audit logging); insecure options require explicit, documented opt-in. +**Secure design and threat modeling (PW.1–PW.2).** The reviewed design artifact and its acceptance criteria are the spec; §5 describes the recommended clarify/analyze gates applied before build. + +| ID | Requirement | Evidence | +|---|---|---| +| SDS-4.3.1 | **MUST** threat-model each interface and component. **(PW.1–PW.2)** | A written threat model per interface, dated | +| SDS-4.3.2 | **MUST** identify and document trust boundaries. **(PW.1–PW.2)** | The boundary list in the threat model | +| SDS-4.3.3 | **MUST** review a design against the security requirements **before** build. **(PW.1–PW.2)** | The review record, dated earlier than the implementation | +| SDS-4.3.4 | **SHOULD** apply the clarify and analyze gates of §5 before build. **(PW.1–PW.2)** | Resolved clarifications on the design record | + +**Reuse well-secured components (PW.4).** + +| ID | Requirement | Evidence | +|---|---|---| +| SDS-4.3.5 | **SHOULD** prefer a vetted library over a new implementation. **(PW.4)** | The dependency's provenance, recorded when it was added | +| SDS-4.3.6 | **MUST NOT** implement custom cryptography. **(PW.4)** | Absence of hand-rolled primitives; a named library for each | + +**Secure coding practices (PW.5).** Every rule below is mandatory unless it says otherwise. + +| ID | Requirement | Evidence | +|---|---|---| +| SDS-4.3.7 | **MUST** validate structure and content at every ingress. | The validator, and a test per ingress that rejects malformed input | +| SDS-4.3.8 | **MUST** reject or quarantine malformed input rather than processing it. | The error/dead-letter path, and its disposition record | +| SDS-4.3.9 | **MUST NOT** build SQL by string concatenation. Parameterized statements or an ORM throughout. | A source scan showing no string-built statements | +| SDS-4.3.10 | **MUST** enforce authentication and authorization on every action (§7.4). | A route-to-permission map covering every action | +| SDS-4.3.11 | **MUST** deny by default. | The default-deny path, and a test that an unmapped action is refused | +| SDS-4.3.12 | **MUST** authenticate every web-service endpoint. | The endpoint inventory, each with its mechanism | +| SDS-4.3.13 | **MUST** validate and size-limit payloads against a schema. | The schema, and the configured limit | +| SDS-4.3.14 | **MUST** disable external-entity resolution and DTD processing for SOAP/XML. | The parser configuration, and an XXE regression test | +| SDS-4.3.15 | **MUST** apply rate limiting and timeouts. | The configured values | +| SDS-4.3.16 | **MUST NOT** expose stack traces or sensitive data in fault responses. | A fault-response test asserting the redacted shape | +| SDS-4.3.17 | **MUST** confine file reads and writes to configured directories, canonicalizing paths and rejecting traversal and symlink escapes. | The canonicalization, and traversal/symlink regression tests | +| SDS-4.3.18 | **MUST** validate file type and size by content, not by extension. | The content-sniffing check | +| SDS-4.3.19 | **MUST** write files atomically (write-then-rename) so a partial file is never processed. | The write path | +| SDS-4.3.20 | **MUST NOT** store handled files in an executable or web-served path. | The configured storage location | +| SDS-4.3.21 | **MUST NOT** execute file contents. | Absence of any execution path from handled content | +| SDS-4.3.22 | **SHOULD** scan inbound files for malware where feasible. | The scanner, or a stated reason it is not feasible | +| SDS-4.3.23 | **MUST** encrypt sensitive files at rest and securely delete them after processing, per the retention policy. | The encryption setting and the deletion step | +| SDS-4.3.24 | **MUST** use TLS for all network communication. | The TLS configuration and its minimum version floor | +| SDS-4.3.25 | **MUST** encrypt sensitive data at rest. | The at-rest encryption setting | +| SDS-4.3.26 | **MUST** use approved algorithms and libraries. | The cryptographic inventory | +| SDS-4.3.27 | **MUST** use FIPS-validated cryptography where a deployment requires it. | The deployment's validated-module configuration | +| SDS-4.3.28 | **MUST NOT** place secrets in code, in prompts, or in commit history. | A clean secret-scan over the full history | +| SDS-4.3.29 | **MUST** source secrets from the environment or a secret store. | The configuration surface, carrying no literal secret | +| SDS-4.3.30 | **MUST** enforce the two rules above by pre-commit and CI secret scanning. | The pre-commit hook and the CI job, each observed to fail on a planted secret | +| SDS-4.3.31 | **MUST** fail closed. | A test per control asserting the closed outcome on error | +| SDS-4.3.32 | **MUST NOT** log secrets or sensitive data. | A redaction test over the log surface | +| SDS-4.3.33 | **MUST** produce a tamper-resistant, timestamped audit log. | The log's integrity mechanism, and a detected-tamper test | + +**Build, review, test and defaults (PW.6–PW.9).** + +| ID | Requirement | Evidence | +|---|---|---| +| SDS-4.3.34 | **MUST** make builds reproducible. **(PW.6)** | Two builds of one commit producing the same artifact | +| SDS-4.3.35 | **MUST** fix security-relevant build, interpreter and dependency settings in the pipeline rather than on a developer machine. **(PW.6)** | The pipeline definition | +| SDS-4.3.36 | **MUST** peer-review every change. Where a project cannot staff a human second reviewer, the deviation and its compensating controls are recorded in A.6. **(PW.7)** | The review record per change, or the A.6 entry | +| SDS-4.3.37 | **MUST** run static analysis and software composition analysis in CI (§6.2). **(PW.7)** | The SAST and SCA jobs, and their results for the released commit | +| SDS-4.3.38 | **SHOULD** confirm in review that a change conforms to its spec's acceptance criteria (§5). **(PW.7)** | The criteria, checked off in the review | +| SDS-4.3.39 | **MUST** maintain an automated test suite that runs on every change. **(PW.8)** | The suite, and its run on the change | +| SDS-4.3.40 | **MUST** include security test cases in that suite. **(PW.8)** | The security tests, named | +| SDS-4.3.41 | **SHOULD** trace tests to the spec's acceptance criteria so coverage is mechanical rather than prose (§5). **(PW.8)** | A criterion-to-test link per criterion | +| SDS-4.3.42 | **MUST** judge test *quality*, not merely presence, per the [Code Quality & Anti-Slop Standards](Code_Quality_Standards.md) — behavior-verifying assertions over mock choreography, with mutation testing as guidance. **(PW.8)** | The quality review against that rubric | +| SDS-4.3.43 | **MUST NOT** gate quality on line-coverage percentage alone; it is a gameable single scoreboard. Measure structure and behavior. **(PW.8)** | The gate definition, showing no coverage-only threshold | +| SDS-4.3.44 | **MUST** ship secure-by-default: transport encryption on, at-rest encryption on, least-privilege accounts, audit logging on. **(PW.9)** | The shipped default configuration | +| SDS-4.3.45 | **MUST** require an explicit, documented opt-in for any insecure option. **(PW.9)** | The opt-in setting and the documentation stating its risk | ### 4.4 Respond to Vulnerabilities (RV.1–RV.3) -- **Identify on an ongoing basis (RV.1).** Continuous dependency monitoring; a defined intake channel for internally and externally reported issues (§8). -- **Assess, prioritize, remediate (RV.2).** Findings are triaged by severity with target remediation timelines (set per project in the profile); fixes are verified before closure. -- **Root-cause analysis (RV.3).** Significant vulnerabilities receive a root-cause review; systemic causes feed back into this standard. +| ID | Requirement | Evidence | +|---|---|---| +| SDS-4.4.1 | **MUST** monitor dependencies continuously. **(RV.1)** | The monitoring job and its most recent run | +| SDS-4.4.2 | **MUST** publish a defined intake channel for internally and externally reported issues (§8). **(RV.1)** | The published channel | +| SDS-4.4.3 | **MUST** triage findings by severity against target remediation timelines set in the applicability profile. **(RV.2)** | The triage record, with dates against the profile's targets | +| SDS-4.4.4 | **MUST** verify a fix before closing the finding. **(RV.2)** | The verifying test or check, referenced from the closure | +| SDS-4.4.5 | **MUST** perform a root-cause review of a significant vulnerability. **(RV.3)** | The root-cause record | +| SDS-4.4.6 | **MUST** feed systemic causes back into this standard. **(RV.3)** | The resulting change to this document, in Version history | --- From cdc7825ae7bad99c7a8e34e27d04ed767e1197da Mon Sep 17 00:00:00 2001 From: wshallwshall Date: Fri, 7 Aug 2026 12:03:51 -0500 Subject: [PATCH 3/8] docs(sds): the interface-auth tiers read as a shopping list, not as 31 checkable obligations Section 7.4 is the section an adopter is most likely to be audited against, and it was the section least able to answer "did you do it". Each tier was a bullet holding four to seven obligations joined by semicolons -- validate the chain, check revocation, rotate before expiry -- so a connection could satisfy two of them and still be described by the same sentence. Split to one obligation per row with the evidence that settles each. 31 rules. Where a bullet mixed a hard requirement with a preference, the two are now separate rules rather than one sentence a reader has to parse for force. mTLS running over TLS 1.2 or later is SDS-7.4.3 and is MUST; preferring 1.3 is SDS-7.4.4 and is SHOULD. The same split applies to the service account and its gMSA preference, and to Kerberos and its preference over NTLM. Reading those as a single obligation was how "prefer" quietly became "required" in one direction and "optional" in the other. Two rules point at A.6 rather than pretending the project meets them: federated operator SSO (SDS-7.4.25) is deferred and already carries a deviation there. Naming the deviation in the rule means the register and the requirement no longer describe the state independently. The ASVS V6/V9/V10/V12 and HIPAA mappings stay in the section's prose. They are the crosswalk, not requirements of this standard, and per the convention a crosswalk carries no identifiers. Section numbers are unchanged, so the existing citations to 7.4 -- in A.6, in the applicability profile, and in the sibling standards -- still resolve while the identifiers are being adopted. --- docs/Secure_Development_Standards.md | 60 ++++++++++++++++++++++------ 1 file changed, 48 insertions(+), 12 deletions(-) diff --git a/docs/Secure_Development_Standards.md b/docs/Secure_Development_Standards.md index 7c7adac9..92008c53 100644 --- a/docs/Secure_Development_Standards.md +++ b/docs/Secure_Development_Standards.md @@ -427,23 +427,59 @@ Each deploying organization conducts its own HIPAA Security Risk Assessment of t Integration software authenticates **systems, not people**, on its interfaces. Each connection uses the strongest mechanism the partner system supports, drawn from the hierarchy below; the mechanism, scope, and credential reference for every connection are recorded in its connection definition. (Maps to ASVS V6/V9/V10/V12; HIPAA person-or-entity authentication and transmission security.) *Which mechanisms a given project implements is recorded in its profile.* -**Preferred — system-to-system:** +| ID | Requirement | Evidence | +|---|---|---| +| SDS-7.4.1 | Each connection **MUST** use the strongest mechanism in the hierarchy below that the partner system supports. | The connection definition, against the partner's stated capability | +| SDS-7.4.2 | The mechanism, scope and credential reference **MUST** be recorded in every connection's definition. | The connection definition, all three fields populated | -- **Mutual TLS (mTLS).** Client-certificate authentication over **TLS 1.2+ (prefer 1.3)** with strong cipher suites; validate the full chain to a trusted CA, check revocation (OCSP/CRL), rotate certificates before expiry. Where tokens are also used, prefer **sender-constrained (mTLS-bound) access tokens** (ASVS 5.0 V10). -- **OAuth 2.0 client-credentials grant.** The default for machine-to-machine API auth. Prefer **asymmetric client authentication (`private_key_jwt`)** over shared secrets; issue short-lived, per-connection scoped tokens; validate issuer, audience, expiry, and scope on every request. -- **SMART on FHIR (Backend Services).** For any FHIR REST interface, authenticate using the SMART **Backend Services** profile — OAuth 2.0 client-credentials with a **signed JWT client assertion** and `system/` scopes; validate granted scopes against the requested operation. +**Preferred — system-to-system.** Mutual TLS, the OAuth 2.0 client-credentials grant, and SMART on FHIR Backend Services. -**Directory / enterprise integration (e.g., Active Directory):** +| ID | Requirement | Evidence | +|---|---|---| +| SDS-7.4.3 | mTLS **MUST** run over TLS 1.2 or later with strong cipher suites. | The negotiated version and suite list | +| SDS-7.4.4 | mTLS **SHOULD** use TLS 1.3. | The configured preference order | +| SDS-7.4.5 | **MUST** validate the full client-certificate chain to a trusted CA. | The trust store, and a rejected-untrusted-chain test | +| SDS-7.4.6 | **MUST** check certificate revocation (OCSP or CRL). | The revocation configuration, and a rejected-revoked-cert test | +| SDS-7.4.7 | **MUST** rotate certificates before expiry. | An expiry monitor, and the rotation record | +| SDS-7.4.8 | Where bearer tokens are also used, **SHOULD** prefer sender-constrained (mTLS-bound) access tokens. | The token binding, per ASVS 5.0 V10 | +| SDS-7.4.9 | Machine-to-machine API authentication **MUST** default to the OAuth 2.0 client-credentials grant. | The default in the connection factory | +| SDS-7.4.10 | **SHOULD** prefer asymmetric client authentication (`private_key_jwt`) over a shared secret. | The registered client authentication method | +| SDS-7.4.11 | **MUST** issue short-lived, per-connection scoped tokens. | The configured lifetime and scope, per connection | +| SDS-7.4.12 | **MUST** validate issuer, audience, expiry and scope on every request. | The validation code, and a test rejecting each of the four | +| SDS-7.4.13 | Any FHIR REST interface **MUST** authenticate using the SMART Backend Services profile. | The interface's configured mechanism | +| SDS-7.4.14 | SMART authentication **MUST** use a signed JWT client assertion and `system/` scopes. | The assertion, and the granted scope list | +| SDS-7.4.15 | **MUST** validate granted scopes against the requested operation. | The scope check, and a test refusing an out-of-scope operation | + +**Directory / enterprise integration** (for example Active Directory). -- **Run under a least-privilege service account — preferably a group-Managed Service Account (gMSA)** on Windows/AD — so the password is auto-rotated and never stored in configuration. -- **Use Kerberos / Integrated Windows Authentication**; prefer Kerberos over NTLM (disable NTLM where feasible) with correct SPNs. -- **Authenticate to databases with integrated authentication** (the service account) rather than a stored database password, where supported. -- **Perform directory lookups over LDAPS (LDAP over TLS) only** — never cleartext LDAP; bind with a least-privilege account. -- **Map roles to directory security groups** for centralized RBAC; if human operators authenticate, **federate to the enterprise identity provider (AD FS / Entra ID) via OIDC or SAML** rather than a local user store. +| ID | Requirement | Evidence | +|---|---|---| +| SDS-7.4.16 | **MUST** run under a least-privilege service account. | The account and its granted rights | +| SDS-7.4.17 | **SHOULD** use a group-Managed Service Account on Windows/AD, so the password is auto-rotated and never stored in configuration. | The account type; no password in configuration | +| SDS-7.4.18 | **MUST** use Kerberos or Integrated Windows Authentication for directory-integrated authentication. | The negotiated mechanism | +| SDS-7.4.19 | **SHOULD** prefer Kerberos over NTLM, and disable NTLM where feasible. | The disabled-NTLM setting, or a stated reason it is infeasible | +| SDS-7.4.20 | **MUST** register correct service principal names. | The SPN registration | +| SDS-7.4.21 | **SHOULD** authenticate to databases with integrated authentication rather than a stored password, where supported. | The connection configuration, carrying no password | +| SDS-7.4.22 | **MUST** perform directory lookups over LDAPS only. Cleartext LDAP is **MUST NOT**. | The configured scheme and port | +| SDS-7.4.23 | **MUST** bind to the directory with a least-privilege account. | The bind account's granted rights | +| SDS-7.4.24 | **SHOULD** map roles to directory security groups for centralized RBAC. | The group-to-role map | +| SDS-7.4.25 | Where human operators authenticate, **SHOULD** federate to the enterprise identity provider via OIDC or SAML rather than a local user store. | The federation configuration, or the A.6 deviation | + +**Legacy / interoperability tier.** Supported, least-preferred, and documented per connection: HTTP Basic over TLS, per-connection API keys, or SOAP WS-Security. -**Legacy / interoperability tier** *(supported, least-preferred, documented per connection):* HTTP Basic over TLS, per-connection API keys, or SOAP **WS-Security** (UsernameToken or, preferably, X.509 certificate tokens with message-level signing). Always over TLS; credentials vaulted, scoped per connection, and rotated. Used only when a partner system cannot support a preferred mechanism, with the exception recorded. +| ID | Requirement | Evidence | +|---|---|---| +| SDS-7.4.26 | A legacy mechanism **MAY** be used only where a partner system cannot support a preferred one. | The partner's stated limitation | +| SDS-7.4.27 | Each such use **MUST** be recorded as an exception on the connection. | The recorded exception | +| SDS-7.4.28 | A legacy mechanism **MUST** run over TLS, with its credential vaulted, scoped per connection, and rotated. | The transport, the vault reference, and the rotation record | +| SDS-7.4.29 | WS-Security **SHOULD** use X.509 certificate tokens with message-level signing rather than UsernameToken. | The configured token type | + +**Across all mechanisms.** -**Across all mechanisms:** TLS everywhere (no cleartext sensitive transport); credentials and keys in a secret store, never in code or config; per-connection least privilege; and per-connection IP allowlisting / network segmentation as defense-in-depth. +| ID | Requirement | Evidence | +|---|---|---| +| SDS-7.4.30 | **MUST NOT** carry sensitive data over a cleartext transport, on any interface. | A transport inventory, every entry TLS | +| SDS-7.4.31 | **MUST** hold credentials and keys in a secret store, never in code or configuration, and apply per-connection least privilege. **SHOULD** add per-connection IP allowlisting or network segmentation as defense-in-depth. | The secret-store references; the per-connection grants; the allowlist where present | --- From e4c4403391478b19156b3074958244aadff2d8fb Mon Sep 17 00:00:00 2001 From: wshallwshall Date: Fri, 7 Aug 2026 12:06:04 -0500 Subject: [PATCH 4/8] docs(sds): the release gate was one sentence, so "we skipped one" had no name Converts the three remaining obligation-list sections: internal testing (6.2), release gates (6.4), open-source project security (8), and the evidence set (9). 28 rules, bringing the document to 129. Section 6.4 is the sharp one. A production release "requires: passing automated checks, no unresolved high/critical findings, current independent-review status (or a documented risk acceptance), and updated evidence" -- four conditions in one sentence, so a release that met three of them was still describable by it, and no gate had a name to be waived by. They are now four rules, and the section says plainly that skipping one is a deviation recorded in A.6 rather than a call made at release time. Two evidence cells are deliberately narrower than the obligation a reader would assume. SDS-6.4.1 asks for the green run of the RELEASED COMMIT, not of its branch tip. Under squash-merge those are different commits, and checking the wrong one is the mistake this project has already made in other contexts. SDS-6.2.4 and SDS-8.2 ask for a HISTORY-WIDE scan, not a scan of the tip. The requirement always said "the full history is kept clean"; the evidence now says so too, because a tip scan passes trivially against a secret that was committed and then removed. Section 9's prose is untouched. Its attestation-posture paragraphs, the published-wording correction of 2026-08-02, and the note on what the SSDF attestation is answerable to are argument and history, not requirements -- and the convention says a rule is a table row. Only the evidence-set bullets became rules. Section 6.3 keeps its prose in this commit. It is mostly scope description -- the level model, the access an assessor needs, the chapter list -- with two real obligations buried in it; those get inline identifiers with the rest of the prose rules rather than forcing a table onto a scoping section. --- docs/Secure_Development_Standards.md | 52 ++++++++++++++++++++-------- 1 file changed, 38 insertions(+), 14 deletions(-) diff --git a/docs/Secure_Development_Standards.md b/docs/Secure_Development_Standards.md index 92008c53..d1c7ed11 100644 --- a/docs/Secure_Development_Standards.md +++ b/docs/Secure_Development_Standards.md @@ -363,9 +363,13 @@ Testing follows the methodology of NIST SP 800-115 (*Technical Guide to Informat ### 6.2 Internal testing (continuous) -- SAST and SCA run automatically; builds fail on new high/critical findings. -- Secret scanning runs pre-commit and in CI; the full git history is kept clean of secrets, credentials, keys, and any sensitive data. -- Security-focused test cases (authn/authz, input validation, error handling) are part of the standard suite. +| ID | Requirement | Evidence | +|---|---|---| +| SDS-6.2.1 | **MUST** run SAST and SCA automatically on every change. | The jobs, and their run on the change | +| SDS-6.2.2 | **MUST** fail the build on a new high or critical finding. | The failure threshold, and a build observed to fail on one | +| SDS-6.2.3 | **MUST** run secret scanning both pre-commit and in CI. | Both hooks, each observed to fail on a planted secret | +| SDS-6.2.4 | **MUST** keep the full git history clean of secrets, credentials, keys and sensitive data. | A history-wide scan, not a scan of the tip | +| SDS-6.2.5 | **MUST** carry security-focused test cases — authentication, authorization, input validation, error handling — in the standard suite. | Those tests, named, running in the standard suite | ### 6.3 OWASP ASVS 5.0 Level 3 — scope @@ -383,7 +387,14 @@ The independent review is scoped to **OWASP ASVS version 5.0.0** (released May 2 ### 6.4 Release gates -A production release requires: passing automated checks, no unresolved high/critical findings, current independent-review status (or a documented risk acceptance), and updated evidence (§9). +A production release requires all four of the following. A gate that is skipped is a deviation recorded in A.6, not a judgment call at release time. + +| ID | Requirement | Evidence | +|---|---|---| +| SDS-6.4.1 | Every automated check **MUST** pass. | The green run for the released commit, not for its branch tip | +| SDS-6.4.2 | There **MUST NOT** be an unresolved high or critical finding. | The finding register at the release date | +| SDS-6.4.3 | Independent-review status **MUST** be current, or a dated risk acceptance **MUST** be in force. | The review report, or the A.6 entry with its date and owner | +| SDS-6.4.4 | The evidence set (§9) **MUST** be updated to match what shipped. | The evidence set, dated at or after the release | --- @@ -487,11 +498,21 @@ Integration software authenticates **systems, not people**, on its interfaces. E Because the software is developed in the open and adopted by others, the project also maintains: -- **Repository hygiene.** No secrets or sensitive data ever committed; the full history is scanned and kept clean. A clear `LICENSE`. -- **Coordinated vulnerability disclosure.** A published `SECURITY.md` with a private reporting channel and a disclosure timeline; reported issues feed the RV process (§4.4). -- **Signed, verifiable releases.** Release artifacts are signed and accompanied by an SBOM (PS.2), so adopters can verify provenance and integrity. -- **Contribution review.** All external contributions are security-reviewed before merge; signed commits / DCO required; maintainers gate merges; dependency provenance is checked. -- **Adopter guidance.** A deployment/hardening guide so adopters can stand the software up securely and meet their §2 responsibilities. +| ID | Requirement | Evidence | +|---|---|---| +| SDS-8.1 | **MUST NOT** commit a secret or sensitive data to the public repository. | A clean scan of the tip | +| SDS-8.2 | **MUST** scan the full history and keep it clean, not only the tip. | The history-wide scan result | +| SDS-8.3 | **MUST** publish a clear `LICENSE`. | The file, at the repository root | +| SDS-8.4 | **MUST** publish a `SECURITY.md` carrying a private reporting channel. | The file, and a channel that accepts a report | +| SDS-8.5 | **MUST** publish a disclosure timeline. | The stated windows in `SECURITY.md` | +| SDS-8.6 | **MUST** feed reported issues into the RV process (§4.4). | A reported issue traced to its triage record | +| SDS-8.7 | **MUST** sign release artifacts. | The signature, and a verification of it | +| SDS-8.8 | **MUST** accompany each release with an SBOM, so an adopter can verify provenance and integrity. **(PS.2)** | The SBOM attached to the release | +| SDS-8.9 | **MUST** security-review every external contribution before merge. | The review record per external contribution | +| SDS-8.10 | **MUST** require signed commits or a DCO on contributions. | The enforced setting | +| SDS-8.11 | **MUST** gate merges through maintainers. | The branch protection rule naming who may merge | +| SDS-8.12 | **MUST** check the provenance of a dependency before adopting it. | The provenance note recorded when it was added | +| SDS-8.13 | **MUST** publish a deployment and hardening guide, so adopters can stand the software up securely and meet their §2 responsibilities. | The published guide | --- @@ -499,11 +520,14 @@ Because the software is developed in the open and adopted by others, the project The project maintains a current evidence set so any claim is backed: -- This **Secure Development Standards** document and each project's standing contract. -- **SSDF practice evidence** (toolchain configuration, review records, SBOMs, secure-default settings). -- **Test results** — CI security-scan history; the independent **OWASP ASVS 5.0 Level 3** report and re-test results. -- **Per-project applicability profile** (Appendix A and onward). -- A **claims register** recording each published claim, its wording, and the evidence behind it. +| ID | Requirement | Evidence | +|---|---|---| +| SDS-9.1 | **MUST** keep this document and each project's standing contract current. | Both files, with a review date inside the stated cadence | +| SDS-9.2 | **MUST** retain SSDF practice evidence: toolchain configuration, review records, SBOMs, secure-default settings. | The retained artifacts, per practice | +| SDS-9.3 | **MUST** retain test results: CI security-scan history, and the independent ASVS report with its re-test results. | The scan history, and the report where one exists | +| SDS-9.4 | **MUST** maintain a per-project applicability profile. | Appendix A, and one appendix per further project | +| SDS-9.5 | **MUST** maintain a claims register recording each published claim, its exact wording, and the evidence behind it. | The register, one row per published claim | +| SDS-9.6 | **MUST** publish attestations with releases, so an adopter can rely on them. | The attestation attached to the release | **Attestation posture.** The software is self-attested as NIST SSDF–aligned, tested per NIST SP 800-115, **assessed against** OWASP ASVS 5.0 **using Level 3 as the target** — an assessment **in progress**, not a completed verification — and built to support HIPAA-compliant deployment (controls mapped to NIST SP 800-66 Rev. 2). Third-party validation of the SSDF attestation and of the ASVS assessment raises the weight of these claims. From afe70b2ed5cd86d53d48ee6a4e2f1862a66dcd87 Mon Sep 17 00:00:00 2001 From: wshallwshall Date: Fri, 7 Aug 2026 12:13:42 -0500 Subject: [PATCH 5/8] docs(sds): sections 3 and 5 held 16 obligations that a table would have destroyed Appendix A.6 cites "5.3, recommended" and 6.3, and neither had anything to cite. Both sections are prose, so the deviations register was pointing at paragraphs while the tabled sections had gained identifiers around it. This closes that, without turning the essays into rows. WHY NOT TABLES HERE. Section 3's prose-review rules are persuasive only because of what surrounds them: the 2026-07-30 audit that produced them, the eleven retractions across four parallel sessions that named the instrument rule, the specific wrong sentences quoted verbatim. A table row asserting "MUST confirm the instrument answers the question asked" carries none of that, and a reader with no reason to believe it will not apply it. The evidence IS the argument. So the guard grew a second definition shape rather than the document losing its first. An inline rule is a bullet opening with a bolded identifier, and it still carries an *Evidence:* clause -- no rule anywhere escapes naming what settles it. 145 rules now, 16 sections, no gaps, every one with evidence. Two things the extension had to get right, both fired at deliberately: A bolded identifier MID-SENTENCE stays a citation. Otherwise emphatically citing a rule would define a second, evidence-free copy of it, and the duplicate check would fire on the citation rather than on the defect. An UNNUMBERED subheading no longer blanks the section context. "Reviewing security prose" and "A note on claims and wording" sit inside section 3, and resetting the context there would have silently skipped the mis-siting check for every prose rule -- a check reporting green over exactly the rules it was extended to cover. That is the shape this whole module exists to prevent, and it was one line. Six inline mutations fired at the real 145-rule corpus, each confirmed red, the file restored byte-identically after each. Section 3's heading text is unchanged. It is cited by name from the standing contract, from two backlog files and from a decision record, and the appendix declares that future projects reuse these headings -- renaming one breaks a stated contract, not merely a link. --- docs/Secure_Development_Standards.md | 40 ++++---- tests/test_sds_rule_ids_are_stable.py | 136 +++++++++++++++++++++++--- 2 files changed, 144 insertions(+), 32 deletions(-) diff --git a/docs/Secure_Development_Standards.md b/docs/Secure_Development_Standards.md index d1c7ed11..16b218a7 100644 --- a/docs/Secure_Development_Standards.md +++ b/docs/Secure_Development_Standards.md @@ -100,15 +100,15 @@ These are complementary to the **OWASP ASVS 5.0 Level 3** verification (source-a NIST does **not** issue certificates for these frameworks. The project may **display alignment claims**, but words them honestly and backs them with evidence: -- **Use:** "Built to NIST SP 800-218 (SSDF)," "NIST SSDF–aligned," "tested per NIST SP 800-115," "controls mapped to NIST SP 800-66 Rev. 2," "HIPAA-compliant deployment supported." -- **Do not use:** "NIST certified" or any phrasing implying a certificate exists. -- **Back every claim** with the implemented practice and its evidence (this standard, test reports, the ASVS attestation, the applicability profile). A self-attestation is a formal, legally significant declaration — only make it if it is true. -- A third-party assessor may **validate** an attestation; that raises its weight but is still not a NIST certificate. +- **SDS-3.1 — MAY display an alignment claim, and MUST word it as one of these.** "Built to NIST SP 800-218 (SSDF)," "NIST SSDF–aligned," "tested per NIST SP 800-115," "controls mapped to NIST SP 800-66 Rev. 2," "HIPAA-compliant deployment supported." *Evidence:* the published wording, matched against this list. +- **SDS-3.2 — MUST NOT use "NIST certified", or any phrasing implying a certificate exists.** NIST issues none for these frameworks, so the phrase describes nothing that could be produced on request. *Evidence:* a sweep of published surfaces for the phrase. +- **SDS-3.3 — MUST back every claim with the implemented practice and its evidence** — this standard, test reports, the ASVS attestation, the applicability profile. A self-attestation is a formal, legally significant declaration; only make it if it is true. *Evidence:* the claims register (SDS-9.5), one row per published claim. + +A third-party assessor may **validate** an attestation; that raises its weight but is still not a NIST certificate. #### Reviewing security prose: ask what a reader would DO with it -**Review security prose by asking what a reader would DO with it, not whether it is accurate.** This is -the governing instruction; the three rules below are instances of it. +- **SDS-3.4 — MUST review security prose by asking what a reader would DO with it, not whether it is accurate.** This is the governing instruction; the rules below are instances of it. *Evidence:* the review record, naming what a reader would do with the sentence rather than whether it is true. It matters because the expensive failures are not false statements. A correctness review terminates at "yes, that sentence is accurate" — and every finding in the 2026-07-30 documentation audit passed that @@ -116,7 +116,7 @@ test. Each was **true about the mechanism and misleading about the posture**: a every spot-check while pointing the reader somewhere they cannot go. Six such findings were caught by asking what happens to someone who acts on the sentence; **none** was caught by accuracy checking. -- **State a load-bearing fact ONCE, and link to it; never restate it.** A repo that states a fact twice +- **SDS-3.5 — MUST state a load-bearing fact ONCE, and link to it; never restate it.** *Evidence:* the source of record, linked rather than restated, and no second copy of the fact. A repo that states a fact twice will eventually state it two ways, and the stale copy is the one that gets cited — a reader who finds *a* statement stops looking for the other. Three instances in a single day: `harden_kex_groups` described as pinning key-exchange groups across **five** live documents, when `SSLContext.set_groups` @@ -126,17 +126,17 @@ asking what happens to someone who acts on the sentence; **none** was caught by URL" while §7 documented the exact query parameters that carry it. In every case the repo held **both** the right and the wrong version. The mitigation is structural, not diligence: pick the source of record, link to it, and let the copy die. -- **A completeness claim is a liability.** "Two configurations do X, and a reviewer should hear both" +- **SDS-3.6 — MUST prefer "at least" to an enumeration; a completeness claim is a liability.** *Evidence:* the claim, worded as a floor, naming the case that matters and pointing at the reference. "Two configurations do X, and a reviewer should hear both" invites the check and then survives it, because a reader who confirms the named case stops looking. Twice this documentation set shipped such a sentence wrong in *both* directions at once — naming a case that no longer existed while omitting one that did. Where you cannot enumerate exhaustively, say "at least", name the case that matters, and point at the reference. -- **A compensating control must not rest on a false premise.** A `Referrer-Policy` relaxation was +- **SDS-3.7 — A compensating control MUST NOT rest on a false premise.** *Evidence:* the stated justification, re-derived against the code rather than accepted. A `Referrer-Policy` relaxation was justified on the grounds that console URLs "carry opaque ids only (never PHI)" — untrue of the console's own search route. The control itself was sound; the stated reason was not, and the next person to touch it reasons from the comment. A wrong justification is worse than none. -- **Confirm your instrument answers the question you asked, not one adjacent to it.** The rules above +- **SDS-3.8 — MUST confirm the instrument answers the question asked, not one adjacent to it.** *Evidence:* the question and what the instrument returns, written down side by side and checked to be the same sentence. The rules above catch prose that is true and misleading; this one catches a claim that is **false when written, while feeling measured**. The rule is the question — ***does my instrument answer the question I asked, or one adjacent to it?*** — and it outlives every example below. **The pairs are dated illustrations, not @@ -302,11 +302,13 @@ Spec-driven development treats the artifacts that describe *what* a change must | **Tasks** | Decomposition of decisions/requirements into work items / lanes / gates. | PW.1–PW.2 | | **Verification** | Automated checks + human conformance reviews that test/inspect against the spec. | PW.7 (review), PW.8 (test) | -A project SHOULD keep these five layers present and connected; the recommended connections are described in §5.3–§5.5. +- **SDS-5.1.1 — SHOULD keep these five layers present and connected.** The recommended connections are described in §5.3–§5.5. *Evidence:* an artifact per layer, each linked to the layer above it. ### 5.2 EARS acceptance criteria -A change's behavioral acceptance criteria SHOULD be written in **EARS (Easy Approach to Requirements Syntax)** — a small, constrained grammar that turns prose requirements into testable, unambiguous statements. EARS offers five templates: +- **SDS-5.2.1 — SHOULD write a change's behavioral acceptance criteria in EARS (Easy Approach to Requirements Syntax)**, a small constrained grammar that turns prose requirements into testable, unambiguous statements. *Evidence:* the criteria, each matching one of the five templates below. + +EARS offers five templates: | Template | Form | |---|---| @@ -322,22 +324,22 @@ This fits the project's existing posture: the standing contract's invariants alr ### 5.3 Requirement → design → tasks → test traceability -Each acceptance criterion SHOULD carry an **ID** linked to the test or fixture that exercises it, so coverage is **mechanical, not prose** — "which criteria are untested" becomes computable rather than a judgment call. The reference project already owns the pieces — decisions in ADRs, requirement IDs in the backlog, tasks in release/multisession plans — and the recommendation is to **connect** them: criterion ID → test/fixture link. *(Lineage: AWS Kiro requirements/design/tasks triad; distilled.)* +- **SDS-5.3.1 — SHOULD give each acceptance criterion an ID linked to the test or fixture that exercises it**, so coverage is **mechanical, not prose** and "which criteria are untested" becomes computable rather than a judgment call. *Evidence:* a criterion-to-test link per criterion, and a report of the unlinked ones. The reference project already owns the pieces — decisions in ADRs, requirement IDs in the backlog, tasks in release/multisession plans — and the recommendation is to **connect** them: criterion ID → test/fixture link. *(Lineage: AWS Kiro requirements/design/tasks triad; distilled.)* ### 5.4 Clarify and analyze (lightweight, advisory) Two lightweight checks are recommended, both explicitly **advisory, not hard gates**: -- **Clarify** — force ambiguity resolution before build, surfacing and answering open questions while they are still cheap to change. The project already has an informal version: the ADR **"To resolve on acceptance"** block. -- **Analyze** — automated cross-artifact consistency/coverage: does every acceptance criterion have a task and a test? does any artifact contradict the constitution's invariants? +- **SDS-5.4.1 — SHOULD run a clarify step before build**, forcing ambiguity resolution while open questions are still cheap to change. The project already has an informal version: the ADR **"To resolve on acceptance"** block. *Evidence:* the resolved questions on the decision record, dated before the implementation. +- **SDS-5.4.2 — SHOULD run an analyze check for cross-artifact consistency and coverage:** does every acceptance criterion have a task and a test? does any artifact contradict the constitution's invariants? *Evidence:* the check's report for the change. -These SHOULD be run as lightweight advisory checks. They introduce no new blocking release gate (cf. §6.4). *(Lineage: GitHub Spec Kit `specify → clarify → plan → tasks → analyze → implement`; distilled.)* +Both are **advisory** and introduce no new blocking release gate (cf. §6.4). *(Lineage: GitHub Spec Kit `specify → clarify → plan → tasks → analyze → implement`; distilled.)* ### 5.5 Executable acceptance criteria (living documentation) BDD / Specification-by-Example expresses acceptance criteria as Given/When/Then scenarios with concrete `(input → expected outcome)` example tables that **execute** as tests — so specification and verification cannot silently drift, and the spec doubles as living documentation. This fits naturally with EARS's WHEN/THEN phrasing, and the HL7 domain (well-defined inputs, well-defined dispositions) is well suited to example-driven verification. The concrete reference-project opportunity (detailed as R2 in Appendix A.7): -> A project's dry-run gate that already replays fixtures through the real graph but asserts only "didn't error" **SHOULD** be upgraded to assert an **expected disposition per fixture** (e.g. `PROCESSED` / `UNROUTED` / `FILTERED` / `ERROR`), turning it into an executable acceptance-criteria check. +- **SDS-5.5.1 — A dry-run gate that replays fixtures through the real graph SHOULD assert an expected disposition per fixture** (`PROCESSED` / `UNROUTED` / `FILTERED` / `ERROR`) rather than only that it did not error, turning it into an executable acceptance-criteria check. *Evidence:* the declared expectation per fixture, and the gate asserting it. *(Lineage: BDD / Specification-by-Example; distilled, not BDD-tool adoption.)* @@ -375,10 +377,10 @@ Testing follows the methodology of NIST SP 800-115 (*Technical Guide to Informat The independent review is scoped to **OWASP ASVS version 5.0.0** (released May 2025), **Level 3**: -- **Version-pinned citation.** Requirements are cited as `v5.0.0-.
.`; identifiers changed substantially from 4.0.x, so the version is always stated. +- **SDS-6.3.1 — MUST cite ASVS requirements as `v5.0.0-.
.`, always stating the version.** Identifiers changed substantially from 4.0.x, so a bare number resolves to a different requirement depending on which edition the reader holds. *Evidence:* the citations, each carrying its version. - **Scale and level model.** ~350 requirements across **17 chapters**. Levels are **cumulative** — L3 includes all of L1 and L2. **MessageFoundry targets Level 3** (defence-in-depth for the highest-assurance contexts), chosen above the usual L2 norm because the engine carries PHI; L1 and L2 form the cumulative baseline and are assessed first. - **Access required for L3.** L3 is a white-box / hybrid review: the assessor needs source code, developer access, documentation, and an authenticated test instance running **synthetic, non-PHI** data. -- **Documented Security Decisions (new in 5.0).** Each chapter opens with a requirement to document *how* its controls are applied and *why*. This standard, the per-interface threat models (PW.1), and the secure-default baseline (PW.9) serve as that documentation. **Each project documents which chapters are in scope and records exclusions with justification** — see the applicability profile. (Documenting exclusions is itself an ASVS practice.) +- **SDS-6.3.2 — MUST record which ASVS chapters are in scope, and justify every exclusion**, in the applicability profile. Documenting exclusions is itself an ASVS practice, and each 5.0 chapter opens with a requirement to document *how* its controls are applied and *why*; this standard, the per-interface threat models (SDS-4.3.1) and the secure-default baseline (SDS-4.3.44) serve as that documentation. *Evidence:* the profile's chapter table, every excluded row carrying a reason. - **5.0 modernizations to honor.** Cryptography (V11) reflects current guidance, including post-quantum considerations; authentication and password rules (V6) align with NIST SP 800-63; ASVS 5.0 scopes to **applications and APIs** (host/network infrastructure is the deployer's responsibility, §2). **The 17 chapters (v5.0.0):** V1 Encoding and Sanitization · V2 Validation and Business Logic · V3 Web Frontend Security · V4 API and Web Service · V5 File Handling · V6 Authentication · V7 Session Management · V8 Authorization · V9 Self-contained Tokens · V10 OAuth and OIDC · V11 Cryptography · V12 Secure Communication · V13 Configuration · V14 Data Protection · V15 Secure Coding and Architecture · V16 Security Logging and Error Handling · V17 WebRTC. diff --git a/tests/test_sds_rule_ids_are_stable.py b/tests/test_sds_rule_ids_are_stable.py index 79db997d..4211fa6a 100644 --- a/tests/test_sds_rule_ids_are_stable.py +++ b/tests/test_sds_rule_ids_are_stable.py @@ -60,6 +60,13 @@ #: A heading, capturing its dotted number when it has one (``### 4.3 Produce ...`` -> ``4.3``). _HEADING = re.compile(r"^(#{2,4})\s+(?:(A?\.?\d+(?:\.\d+)*)\.?\s+)?(.*)$") +#: An inline rule: a bullet opening with a bolded identifier and its requirement. Anchored at the start +#: of the line so a bolded identifier mid-sentence stays a citation. +_INLINE_RULE = re.compile(r"^\s*[-*]\s+\*\*(SDS-\d+(?:\.\d+)+)\s*[—-]\s*(.+?)\*\*") + +#: The evidence clause an inline rule carries in place of a table's third column. +_EVIDENCE_CLAUSE = re.compile(r"\*Evidence:\*\s*(.+?)(?:\s*\*\(|$)") + @dataclass(frozen=True) class Rule: @@ -102,34 +109,65 @@ def _parse_id(rule_id: str) -> tuple[str, int] | None: def parse_rules(text: str) -> list[Rule]: """Every identified rule defined in ``text``, in document order. - A definition is a table row whose FIRST cell is exactly an identifier. That shape is what makes a - rule a rule; an identifier mentioned inside a sentence is a citation, not a definition. + Two definition shapes, because the standard has two kinds of section. + + A TABLE row whose first cell is exactly an identifier defines a rule. That is the shape used + wherever a section is a list of obligations. + + An INLINE rule is a bullet opening ``- **SDS-x.y - .**``, used where the requirement + is inseparable from the argument that justifies it. Sections 3 and 5 are essays whose evidence IS + the surrounding prose -- the dated audit that produced a rule, the retracted claims that named it -- + and tabling them would throw away the part that makes them persuasive. An inline rule still carries + its evidence, in a trailing ``*Evidence:* ...`` clause, so no rule anywhere escapes the requirement + to say what settles it. + + An identifier appearing anywhere else is a CITATION, not a definition. """ rules: list[Rule] = [] heading_number = "" in_retired = False - for lineno, line in enumerate(text.splitlines(), start=1): + lines = text.splitlines() + for lineno, line in enumerate(lines, start=1): if line.startswith("## "): in_retired = line.strip() == _RETIRED_HEADING match = _HEADING.match(line) - if match: - heading_number = match.group(2) or "" + if match and match.group(2): + # Only a NUMBERED heading changes the section context. An unnumbered subheading -- "A note + # on claims and wording", "Reviewing security prose" -- sits inside its parent section, and + # blanking the context here would silently skip the mis-siting check for every rule under + # it, which is precisely where the prose rules live. + heading_number = match.group(2) if in_retired: continue + cells = _split_row(line) - if len(cells) < 3: + if len(cells) >= 3 and (parsed := _parse_id(cells[0])) is not None: + rules.append( + Rule( + rule_id=cells[0], + section=parsed[0], + number=parsed[1], + requirement=cells[1], + evidence=cells[2], + line=lineno, + heading_number=heading_number, + ) + ) + continue + + inline = _INLINE_RULE.match(line) + if inline is None: continue - parsed = _parse_id(cells[0]) + parsed = _parse_id(inline.group(1)) if parsed is None: continue - section, number = parsed rules.append( Rule( - rule_id=cells[0], - section=section, - number=number, - requirement=cells[1], - evidence=cells[2], + rule_id=inline.group(1), + section=parsed[0], + number=parsed[1], + requirement=inline.group(2), + evidence=_inline_evidence(lines, lineno - 1), line=lineno, heading_number=heading_number, ) @@ -137,6 +175,26 @@ def parse_rules(text: str) -> list[Rule]: return rules +def _inline_evidence(lines: list[str], start: int) -> str: + """The ``*Evidence:*`` clause of the inline rule beginning at ``lines[start]``. + + A bullet in this document wraps across several indented source lines, so the clause is looked for + across the whole bullet rather than on its first line. Returning "" when there is none is what makes + ``test_every_rule_carries_evidence`` fail, which is the intent -- a prose rule is not exempt. + """ + body: list[str] = [lines[start]] + for line in lines[start + 1 :]: + if ( + not line.strip() + or _INLINE_RULE.match(line) + or line.lstrip().startswith(("-", "|", "#")) + ): + break + body.append(line) + found = _EVIDENCE_CLAUSE.search(" ".join(part.strip() for part in body)) + return found.group(1).strip() if found else "" + + def parse_retired(text: str) -> list[Retired]: """Every identifier listed in the Retired rules table.""" retired: list[Retired] = [] @@ -407,6 +465,58 @@ def test_selfbite_a_citation_is_not_a_definition() -> None: assert _ID_ANYWHERE.findall(text) == ["SDS-4.3.7"] +def test_selfbite_an_inline_rule_is_parsed_with_its_evidence() -> None: + text = ( + "### 3. Claims\n\n" + "- **SDS-3.1 — MUST state a fact once.** A repo that states it twice will state it two ways.\n" + " *Evidence:* the source of record, linked rather than restated.\n" + ) + (rule,) = parse_rules(text) + assert rule.rule_id == "SDS-3.1" + assert rule.requirement == "MUST state a fact once." + assert rule.evidence == "the source of record, linked rather than restated." + + +def test_selfbite_an_inline_rule_without_evidence_is_caught() -> None: + """A prose rule is not exempt from naming what settles it; empty evidence must reach the check.""" + text = ( + "### 3. Claims\n\n- **SDS-3.1 — MUST state a fact once.** Some argument, but no clause.\n" + ) + (rule,) = parse_rules(text) + assert rule.evidence == "" + + +def test_selfbite_a_bolded_identifier_mid_sentence_is_not_a_definition() -> None: + """Otherwise citing a rule emphatically would define a second, evidence-free copy of it.""" + text = "### 3. Claims\n\nThe deviation cites **SDS-3.1** and records its owner.\n" + assert parse_rules(text) == [] + assert _ID_ANYWHERE.findall(text) == ["SDS-3.1"] + + +def test_selfbite_an_inline_rule_inherits_its_numbered_parent_heading() -> None: + """Prose rules live under UNNUMBERED subheadings; the mis-siting check must still see section 3.""" + text = ( + "## 3. How this maps to NIST\n\n" + "#### Reviewing security prose\n\n" + "- **SDS-3.1 — MUST ask what a reader would do.** *Evidence:* the review record.\n" + ) + (rule,) = parse_rules(text) + assert rule.heading_number == "3", "an unnumbered subheading must not blank the section context" + assert rule.section == "3" + + +def test_selfbite_evidence_does_not_bleed_across_bullets() -> None: + """Two adjacent inline rules must not share one clause, which would mask a missing one.""" + text = ( + "### 3. Claims\n\n" + "- **SDS-3.1 — MUST do a thing.** No clause here.\n" + "- **SDS-3.2 — MUST do another.** *Evidence:* the second one's own evidence.\n" + ) + first, second = parse_rules(text) + assert first.evidence == "" + assert second.evidence == "the second one's own evidence." + + def test_selfbite_malformed_identifiers_are_rejected() -> None: for bad in ("SDS-", "SDS-4", "SDS-4.x", "SD-4.1", "SDS-4..1"): assert _parse_id(bad) is None, f"{bad!r} must not parse as an identifier" From bfd01773ce809de8e713b3eb9e97286299d48243 Mon Sep 17 00:00:00 2001 From: wshallwshall Date: Fri, 7 Aug 2026 12:17:48 -0500 Subject: [PATCH 6/8] docs(sds): the deviations register named requirements by position, which is what already broke MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit This is the point of the exercise. A.6 carries dated risk acceptances with a named owner and a $25,000-$50,000 figure attached, and it identified the requirements it departs from as "(§6.3 / §6.4)", "(§7.4)", "(§5.3, recommended)". Those are positions. The last renumbering of this document left ten citations in the maintainer's private security documents pointing at the wrong requirement, and A.6 is built out of exactly that construction. Every entry now names its rule: SDS-4.3.36 for single-maintainer development, SDS-6.4.3 for the independent review, SDS-7.4.25 for federated SSO, SDS-5.3.1 for traceability. A.6 contains no positional citation of a requirement of this standard. The convention now states the distinction that makes this decidable rather than a matter of taste. "See §5" points at a place to read and a section number is the right name for that. "This deviates from §6.4" names a requirement and must give the identifier. Navigational pointers are left alone deliberately -- there are three left in the applicability profile, all naming whole sections. A CI PIN FIRED, AND IT WAS RIGHT TO. tests/test_dast_claims.py pinned the A.6 row verbatim, including its section numbers, because that row is the single source of record for whether anything independent has run. Repointing the row broke the pin. That is the pin working: the anchor genuinely moved. It now names SDS-6.4.3, so a future reorganisation cannot move it again, and the comment above it records what happened rather than leaving a silent edit. A STALE DEVIATION IS RETIRED. A.6 recorded R2 -- the executable dry-run gate -- as outstanding. It is built: checks.py reads a per-fixture expected disposition from a .expect sidecar and asserts it, with the older not-ERROR semantics retained where no expectation is declared. Verified against the code rather than taken on report. Three places in this document stated it as not done; all three now agree with checks.py, and A.7.2's R2 is marked BUILT rather than deleted, so the record shows it was met. THE STANDING CONTRACT HAD THE DEFECT IT FORBIDS. CLAUDE.md introduced the prose-review rules with "The three rules below are instances of it" above FOUR bullets -- the instrument rule was added later and the count was not. That is precisely SDS-3.6, in the sentence introducing SDS-3.6. Replaced with an unnumbered form, and each imperative now carries the identifier of the rule it duplicates, so the deliberate duplication is traceable instead of merely asserted. --- CLAUDE.md | 21 +++++++++-------- docs/Secure_Development_Standards.md | 34 ++++++++++++++++++---------- tests/test_dast_claims.py | 7 ++++-- 3 files changed, 38 insertions(+), 24 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 04b2c1b8..657602b2 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -500,16 +500,17 @@ harness process only.) - Document each connector/transport and transform with its config schema and an example message. - When asked for tabular results, provide the final table directly — not code that generates it. -- **Review security prose by asking what a reader would DO with it, not whether it is accurate.** The - three rules below are instances of it. Reasoning, evidence and dates: - [`docs/Secure_Development_Standards.md`](docs/Secure_Development_Standards.md) §3 *"Reviewing - security prose"* — the source of record. -- **State a load-bearing fact ONCE and link to it; never restate it.** -- **A completeness claim is a liability — prefer "at least" to an enumeration.** -- **A compensating control must not rest on a false premise.** -- **Confirm your instrument answers the question you asked, not one adjacent to it** — `git diff` on a - staged file, `--is-ancestor` under squash-merge, `$?` after a pipe, a *job* conclusion for a *step* - question. Name the question and what the tool returns; check they are the same sentence. +- **Review security prose by asking what a reader would DO with it, not whether it is accurate** + (**SDS-3.4**). The rules below are instances of it. Reasoning, evidence and dates: + [`docs/Secure_Development_Standards.md`](docs/Secure_Development_Standards.md) **SDS-3.4 to SDS-3.8**, + under *"Reviewing security prose"* — the source of record. +- **State a load-bearing fact ONCE and link to it; never restate it** (**SDS-3.5**). +- **A completeness claim is a liability — prefer "at least" to an enumeration** (**SDS-3.6**). +- **A compensating control must not rest on a false premise** (**SDS-3.7**). +- **Confirm your instrument answers the question you asked, not one adjacent to it** (**SDS-3.8**) — + `git diff` on a staged file, `--is-ancestor` under squash-merge, `$?` after a pipe, a *job* + conclusion for a *step* question. Name the question and what the tool returns; check they are the + same sentence. --- diff --git a/docs/Secure_Development_Standards.md b/docs/Secure_Development_Standards.md index 16b218a7..0f90635f 100644 --- a/docs/Secure_Development_Standards.md +++ b/docs/Secure_Development_Standards.md @@ -43,6 +43,11 @@ does not move, and that is the property that makes it worth citing: this documen once already (§5–§9 became §6–§10 when Spec-Driven Development was inserted; see Version history), and citations written against the old numbers are still resolving to the wrong requirements today. +**Cite a requirement by identifier; cite a section by number.** "See §5" points a reader at a place to +read, and a section number is the right name for that. "This deviates from §6.4" names a requirement, +and must give the identifier — that is the citation that silently changes meaning when the document is +reorganised, and A.6 is where it does the most damage. + **Not every statement here is a rule.** The NIST and HIPAA crosswalks (§3, §7.2), the spec-stack and EARS tables (§5.1, §5.2), the control-area inventory (§7.1), the reference list (§10) and the Applicability Profile (Appendix A) describe, map or record; they carry no identifiers and nothing @@ -337,9 +342,9 @@ Both are **advisory** and introduce no new blocking release gate (cf. §6.4). *( ### 5.5 Executable acceptance criteria (living documentation) -BDD / Specification-by-Example expresses acceptance criteria as Given/When/Then scenarios with concrete `(input → expected outcome)` example tables that **execute** as tests — so specification and verification cannot silently drift, and the spec doubles as living documentation. This fits naturally with EARS's WHEN/THEN phrasing, and the HL7 domain (well-defined inputs, well-defined dispositions) is well suited to example-driven verification. The concrete reference-project opportunity (detailed as R2 in Appendix A.7): +BDD / Specification-by-Example expresses acceptance criteria as Given/When/Then scenarios with concrete `(input → expected outcome)` example tables that **execute** as tests — so specification and verification cannot silently drift, and the spec doubles as living documentation. This fits naturally with EARS's WHEN/THEN phrasing, and the HL7 domain (well-defined inputs, well-defined dispositions) is well suited to example-driven verification. -- **SDS-5.5.1 — A dry-run gate that replays fixtures through the real graph SHOULD assert an expected disposition per fixture** (`PROCESSED` / `UNROUTED` / `FILTERED` / `ERROR`) rather than only that it did not error, turning it into an executable acceptance-criteria check. *Evidence:* the declared expectation per fixture, and the gate asserting it. +- **SDS-5.5.1 — A dry-run gate that replays fixtures through the real graph SHOULD assert an expected disposition per fixture** — for example received / unrouted / filtered / errored — rather than only that it did not error, turning it into an executable acceptance-criteria check. A fixture declaring no expectation keeps the weaker semantics, so adoption is incremental. *Evidence:* the declared expectation per fixture, and the gate asserting it. *(Lineage: BDD / Specification-by-Example; distilled, not BDD-tool adoption.)* @@ -658,30 +663,35 @@ current position on it. ### A.6 Documented deviations Honest record of where MEFOR's *current* practice differs from the body of the standard, with the -compensating control (the standard requires exclusions/deviations be documented, §6.3). +compensating control. Recording a deviation is itself required (SDS-6.3.2). **Each entry names the +rule it departs from by identifier**, so the departure and the requirement cannot drift apart. -- **Single-maintainer development (PO.2 / PW.7).** The project is solo-maintained today, so the - standard's "every change is peer-reviewed" cannot mean a *human second reviewer*. Compensating +- **Single-maintainer development — SDS-4.3.36 (PO.2 / PW.7).** The project is solo-maintained today, so + SDS-4.3.36's "**MUST** peer-review every change" cannot mean a *human second reviewer*. Compensating controls: blocking automated review (bandit/semgrep SAST, pip-audit SCA, gitleaks), AI-assisted review, branch protection + required CI checks, and no direct pushes to `main`. Revisit when a second maintainer joins. **Detailed record:** the AI-assisted-review compensating control — and the full risk-tiered discipline for building with Claude Code — is operationalized in [`Secure_AI_Development_Standards.md`](Secure_AI_Development_Standards.md), the companion standard that owns and expands this deviation. -- **Independent ASVS-L3 review & DAST (§6.3 / §6.4).** Not yet performed; a **dated risk acceptance** +- **Independent ASVS-L3 review & DAST — SDS-6.4.3.** Not yet performed; a **dated risk acceptance** is in force pre-1.0. An independent engagement is planned — it is a **$25,000–$50,000** commitment that the project intends to fund through a grant or sponsorship rather than licence revenue. MessageFoundry is **self-hosted**, so the decision to deploy it beyond loopback, and the assessment that justifies that decision, rest with the **implementing organization**: this standard states what has and has not been independently verified, and does not gate your deployment on it. -- **Federated operator SSO (§7.4).** API/WSS/MLLP **TLS and client-cert mTLS are built** (opt-in via +- **Federated operator SSO — SDS-7.4.25.** API/WSS/MLLP **TLS and client-cert mTLS are built** (opt-in via cert config) and **native TOTP MFA for local accounts is built** (ADR 0002 WP-14); the remaining deferred operator-auth item is **federated SSO (OIDC/SAML via Entra)**, held safe by the fail-closed `127.0.0.1` bind guard. Federation gets a dedicated ADR before off-loopback exposure. -- **Mechanical requirement→test traceability (§5.3, recommended).** Not yet enforced: acceptance - criteria are not uniformly ID'd and linked to tests, and the dry-run gate asserts only "didn't error" - (R2). This is **not a deviation from a hard requirement** — §5 traceability is **recommended (SHOULD)**, - adopted incrementally — but it is recorded here for honesty. Tracked as R1–R3 in §A.7. +- **Mechanical requirement→test traceability — SDS-5.3.1.** Not yet enforced: acceptance criteria are + not uniformly ID'd and linked to tests. This is **not a deviation from a hard requirement** — + SDS-5.3.1 is **SHOULD**, adopted incrementally — but it is recorded here for honesty. Tracked as + R1 and R3 in A.7.2. + **SDS-5.5.1 is no longer deferred.** The dry-run gate reads a declared expected disposition per + fixture and asserts it, in [`checks.py`](../messagefoundry/checks.py); a fixture with no declared + expectation keeps the older not-`ERROR` semantics. This entry previously recorded R2 as outstanding + after it had been built. **ASVS 5.0 L3 deferred items — accepted / deferred** (risk accepted **2026-06-16**, refreshed after MFA + admin-defense landed **2026-06-17**; owner: project maintainer). Each is deferred-by-design behind @@ -744,7 +754,7 @@ PW.8 (test executable code) is the home of `messagefoundry check` (validate + dr #### A.7.2 Recommendations (all **recommended / SHOULD**, advisory) - **R1 — EARS "Acceptance Criteria" block on the ADR template.** Each ADR **SHOULD** carry an EARS "Acceptance Criteria" block, each criterion bearing an ID linked to its test or fixture. Doc-only, zero-code; formalizes the existing SHALL-style house register. *(Lineage: AWS Kiro requirements.md; distilled.)* -- **R2 — Make the dry-run gate executable-spec.** `messagefoundry/checks.py` **SHOULD** read an **expected disposition per fixture** (`PROCESSED` / `UNROUTED` / `FILTERED` / `ERROR`) and assert it, upgrading today's "didn't error" dry-run (`_check_dryrun` asserts only not-`ERROR`) into an executable acceptance-criteria check. **Backward-compatible:** a fixture with no declared expectation keeps today's not-`ERROR` semantics. *(Lineage: BDD / Specification-by-Example; distilled.)* +- **R2 — Make the dry-run gate executable-spec. BUILT (SDS-5.5.1).** [`checks.py`](../messagefoundry/checks.py) reads an expected disposition from a `.expect` sidecar (`RECEIVED` / `UNROUTED` / `FILTERED` / `ERROR`, with `PROCESSED` and `ROUTED` accepted as aliases of `RECEIVED`) and asserts it, so the dry-run is an executable acceptance-criteria check rather than a not-`ERROR` smoke test. A fixture with no sidecar keeps the older semantics, so adoption is incremental. Retained here rather than deleted because A.6 recorded it as outstanding after it had shipped, and a recommendation that quietly disappears leaves no trace of having been met. *(Lineage: BDD / Specification-by-Example; distilled.)* - **R3 — Promote clarify, add analyze.** The ADR **"To resolve on acceptance"** block **SHOULD** be promoted into an explicit **clarify** step (resolve ambiguity before `Accepted`), and an **analyze**-style advisory coverage check **SHOULD** verify that every **Accepted** ADR's acceptance criteria has a linked test, and that no artifact contradicts a `CLAUDE.md` invariant. **Advisory, not a hard gate** — it belongs in the advisory tier alongside `ruff`/`mypy`, not the required `validate`/`dryrun` tier. *(Lineage: GitHub Spec Kit pipeline; distilled.)* --- diff --git a/tests/test_dast_claims.py b/tests/test_dast_claims.py index 0d8297ae..2470afbe 100644 --- a/tests/test_dast_claims.py +++ b/tests/test_dast_claims.py @@ -245,8 +245,11 @@ def test_scorecard_negative_claims_survive(sentence: str) -> None: "| Dynamic | DAST / authenticated testing of the running app | Per release and periodically |", # §6.1 — the Independent review row, a DIFFERENT tier, untouched by a self-run pass. "| Independent review | Third-party source-code review + penetration test per 800-115", - # §A.6 — the source of record for the independent-engagement status. - "**Independent ASVS-L3 review & DAST (§6.3 / §6.4).** Not yet performed", + # A.6 — the source of record for the independent-engagement status. This pin was written + # against "(§6.3 / §6.4)" and fired when the deviations register moved to rule identifiers, + # which is the pin working: the anchor genuinely moved. It now names the rule rather than two + # section numbers, so a future renumbering cannot move it again. + "**Independent ASVS-L3 review & DAST — SDS-6.4.3.** Not yet performed", ], ) def test_sds_independent_review_row_is_unchanged(row: str) -> None: From 64bf8ba101a7f59832a0a6d92cb97b2006bc48f1 Mon Sep 17 00:00:00 2001 From: wshallwshall Date: Fri, 7 Aug 2026 12:18:40 -0500 Subject: [PATCH 7/8] docs(sds): version 2.3 -- record what changed, and the two things that were wrong A revision is when an identifier gets reused by accident, so the version history is load-bearing for the convention this release introduces rather than decoration on it. States plainly that section numbers did NOT change, because that is the question a reader holding an older citation will have, and the answer decides whether their citation still resolves. It does. Records the two corrections rather than absorbing them silently. A.6 listed the executable dry-run gate as outstanding after it had shipped, and the standing contract counted four rules as three. Both were found by writing the identifiers, not by reading the prose, which is the argument for having done it. Also states that no requirement was added, removed, weakened or strengthened. The diff is large enough that a reader cannot verify that by inspection, and leaving it unsaid invites the assumption that a mood conversion changed what is owed. --- docs/Secure_Development_Standards.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/Secure_Development_Standards.md b/docs/Secure_Development_Standards.md index 0f90635f..9d3cf569 100644 --- a/docs/Secure_Development_Standards.md +++ b/docs/Secure_Development_Standards.md @@ -6,8 +6,8 @@ | **Applies to** | Any application developed under this standard. **MessageFoundry (MEFOR)** is the reference implementation (Appendix A). | | **Maintained by** | Project maintainers (open-source). Each deploying organization assigns its own local owner. | | **Status** | Published — adopter-facing | -| **Version** | 2.2 | -| **Date** | July 30, 2026 | +| **Version** | 2.3 | +| **Date** | August 7, 2026 | | **License** | Publishable under the project's open-source license; intended to be shared with adopters and reused across projects. | | **Review cadence** | At least annually, and on any material architecture or threat change | | **Aligns to** | NIST SP 800-218 (SSDF) · NIST SP 800-115 · NIST SP 800-66 Rev. 2 (HIPAA Security Rule) · OWASP ASVS 5.0 Level 3. Its Spec-Driven Development practices (§5) are a distilled synthesis by this document — not an external standard or certification. | @@ -775,6 +775,7 @@ resolves to a row below rather than to whatever requirement later took the numbe | Version | Date | Change | |---|---|---| +| 2.3 | August 7, 2026 | **Rules given stable identifiers.** 145 requirements now carry an `SDS-
.` identifier, an RFC 2119 keyword in capitals, and the evidence that settles them; see *How to read the rules*. **Section numbers are unchanged** — identifiers were added alongside, so every existing citation still resolves. A.6 now names the rule each deviation departs from, replacing the `(§6.3 / §6.4)` / `(§7.4)` / `(§5.3)` positional citations that the §5–§9 → §6–§10 renumbering below had already broken elsewhere. A *Retired rules* table holds tombstones so a retired identifier is never reissued, and `tests/test_sds_rule_ids_are_stable.py` enforces the convention in CI. Two corrections of record: A.6 listed R2 (the executable dry-run gate) as outstanding after it had shipped, and the standing contract introduced four prose-review rules as "the three rules below". No requirement was added, removed, weakened or strengthened. | | 2.2 | July 30, 2026 | **Independent-review deviation reframed.** The independent ASVS-L3 review & DAST is no longer stated as a precondition for off-loopback/production exposure. MessageFoundry is self-hosted, so the deployment decision — and the assessment supporting it — belong to the implementing organization; this standard records what has and has not been independently verified rather than gating deployment on it. The engagement remains planned, at an estimated $25,000–$50,000, intended to be grant- or sponsor-funded. Also drops a dangling citation to `security/RELEASE-GATE.md`, which is not present in this repository. No change to the SSDF / ASVS / HIPAA mappings. | | 2.1 | July 29, 2026 | **Code-quality companion added.** Cross-linked the new [Code Quality & Anti-Slop Standards](Code_Quality_Standards.md) (evidence-based anti-slop rubric, ISO/IEC 25010): a companion-standards pointer in §1 and a test-*quality* + anti-metric note at PW.8. No change to the SSDF / ASVS / HIPAA mappings or Appendix A. | | 2.0 | June 24, 2026 | Restructured baseline around SSDF, spec-driven development, NIST SP 800-115 testing tiers and SP 800-66 Rev. 2 safeguards. Content carried forward unchanged at this baseline. The full prior changelog — MEFOR-specific drafts → genericization (project-agnostic, with an Appendix A applicability profile) → OWASP ASVS 5.0 Level 3 re-target → NIST SP 800-53 removal → §5 Spec-Driven Development addition and the §5–§9 → §6–§10 renumbering — is preserved in git history. | From e17a422b62779f4950ef3e065d75a187a39b8c6b Mon Sep 17 00:00:00 2001 From: wshallwshall Date: Fri, 7 Aug 2026 12:48:33 -0500 Subject: [PATCH 8/8] test(sds): my own docstring tripped the present-tense ratchet, so reword rather than raise it tests/test_cutover_slug_rot.py ratchets present-tense prose about a mirror or a private source repo: the count may fall, never rise. One line of the new guard's docstring took it from 54 to 55. The line was a true statement about a different artifact -- the website copy of this standard, not the retired publish-pipeline mirror the ratchet was built for -- so it was a false positive of a new kind. That is still not grounds to raise the ceiling. The module says NEVER raise it, and a ceiling lifted once for a defensible reason is a ceiling that lifts again for a worse one. Reworded instead, which improved it on two other axes. It no longer restates the measured divergence between the published copy and this document: that figure lives in the finding raised against the website repository, and restating a moving number in a test docstring is exactly the duplication SDS-3.5 forbids -- the stale copy is the one that gets cited. What the docstring needs to say is narrower and more durable: this guard cannot read that repository, nothing checks whether the copy still matches, and a green run here says nothing about what is served publicly. Back to 54 hits across 1519 tracked files, my line no longer among them, ceiling unchanged. Found by the full suite rather than the targeted runs -- the doc-guard subset does not include this module. Worth recording because the run that found it reported exit code 0: the command ended in a pipe, so the shell returned tail's status, not pytest's. SDS-3.8, in the act of verifying SDS-3.8. --- tests/test_sds_rule_ids_are_stable.py | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/tests/test_sds_rule_ids_are_stable.py b/tests/test_sds_rule_ids_are_stable.py index 4211fa6a..5a74582a 100644 --- a/tests/test_sds_rule_ids_are_stable.py +++ b/tests/test_sds_rule_ids_are_stable.py @@ -24,9 +24,10 @@ repository, so the ten known-stale citations are invisible where CI runs. This guard sees the standard, its siblings and the code; it cannot see the vault, and a green run here is not evidence about the vault's citations. -* **A citation in the published PDF or its website mirror.** Those live in the separate - ``messagefoundry-website`` repository, which has no CI at all. The mirror is hand-copied and is - already 79 lines divergent while carrying the same version and date. +* **A citation in the published PDF or the website copy of this standard.** Both live in the separate + ``messagefoundry-website`` repository, which this guard cannot read and which runs no CI of its own. + Whether that copy still matches this document is checked nowhere, so a green run here says nothing + about what is being served publicly. * **Anything derived from git history.** The required test leg checks out at ``fetch-depth: 1``, so an identifier's past is not readable here. Every check below is derivable from the working tree alone.