From e5fc5121730941efc8509c9bc3aaabec64890689 Mon Sep 17 00:00:00 2001 From: "Jonathan D.A. Jewell" <6759885+hyperpolymath@users.noreply.github.com> Date: Fri, 7 Aug 2026 12:26:07 +0100 Subject: [PATCH 1/4] =?UTF-8?q?feat(debt):=20a=20Debtfile=20and=20a=20debt?= =?UTF-8?q?=20runner=20=E2=80=94=20measured=20debt=20that=20can=20only=20f?= =?UTF-8?q?all?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A GitHub issue is a boolean. Much of what this estate tracks in issues is an integer that changes every week. An audit of the 697 open issues on 2026-08-07 found what that costs: standards#309 claimed ~45 banned .py files measured 176 (~4x wrong, 10 weeks) standards#308 claimed ~12,000 PMPL headers measured 4,968 (59% paid, unrecorded) reposystem#138 claimed Nix in 63 repos measured 67 (grew, unnoticed) The purest case was .git-private-farm, where a bot minted the same measurement as a NEW ISSUE every run: rustsec 138, 136, 139, 156, 156, 155 — six issues carrying one number, plus three more for mirror drift. This adds a register that holds the integer, so the tracker can go back to holding booleans. .machine_readable/Debtfile.a2ml the register — same ### block shape as Mustfile.a2ml, so the parsers agree scripts/check-debtfile-structure.sh structural half — rejects a HOLLOW ENTRY (a number with no probe behind it) scripts/run-debtfile.sh executable half — re-measures every probe scripts/check-debt-ratchet.sh ceilings may fall, never rise silently .github/workflows/debt-measure.yml weekly re-measure; opens ONE PR, never an issue. That is the whole point. Not a contractile, despite the name fitting the convention exactly. The contractile CLI does not exist — reposystem/contractiles/cli/ is an empty directory and five of the six verbs have no runner. Only `must` executes, via hand-written bash, and that is the pattern copied here. docs/EXEMPTION-MECHANISMS.adoc already rejects declarations with no reader ("never read by anything. Reject"), which a seventh verb would be. Not a sixth exemption mechanism either. It absorbs two conventions the estate already invented: debt.a2ml (4 repos, ADR-002) and Exemption Layer 4b, which stores debt as a Markdown table inside .claude/CLAUDE.md carrying a count and an unblock condition, in 19 repos. Sibling to the exemption ratchet, governing a different population: check-exemption-ratchet.sh debt you EXCUSED — "did you quietly excuse more?" check-debt-ratchet.sh debt you MEASURED — "did the number go up?" Deletion is a violation. An entry leaves by reaching zero, not by being deleted — the same reasoning that rejects `**` in the migration ledger. Probe discipline, all of it learned the hard way while writing this: * NO FALLBACK. A probe that fails is a hard failure. A counter returning 0 on error is indistinguishable from zero debt, and zero is the state that passes. * Three candidate probes were written and rejected for manufacturing debt — two matched COMMENTS (usage examples in codeql-reusable.yml; a comment explaining why a script does not use python3), one counted the test files themselves. * The register must not count itself. The first live run breached on todo-fixme-markers (278 > 277) and pmpl-licence-headers (590 > 589) purely because writing the entries down created the strings they search for. Licensing debt is policy: flag-only. Measured and reported, never acted on, per the standing owner directive in .claude/CLAUDE.md. Tests prove each gate CAN fail — 14 + 16 + 12 cases, discovered automatically by self-test.yml, each mutation-tested. The runner suite was itself found to be a fake gate during authoring (piping into `expect` ran it in a subshell, so counters were discarded and a failure would still have exited 0) and fixed. Wired as job `debt-ratchet` in governance-reusable.yml, so every caller picks it up with no per-repo PR. Tier 🟡 CHECK — MUST NOT be a required status check. Adoption is opt-in: a repo with no Debtfile is not in violation. Co-Authored-By: Claude Opus 5 --- .github/workflows/actions.lock | 2 + .github/workflows/debt-measure.yml | 108 ++++++++ .github/workflows/governance-reusable.yml | 50 ++++ .machine_readable/Debtfile.a2ml | 82 ++++++ Justfile | 13 + docs/DEBTFILE-SPEC.adoc | 311 ++++++++++++++++++++++ docs/EXEMPTION-MECHANISMS.adoc | 23 ++ scripts/check-debt-ratchet.sh | 174 ++++++++++++ scripts/check-debtfile-structure.sh | 131 +++++++++ scripts/run-debtfile.sh | 168 ++++++++++++ scripts/tests/debt-ratchet-test.sh | 140 ++++++++++ scripts/tests/debtfile-structure-test.sh | 177 ++++++++++++ scripts/tests/run-debtfile-test.sh | 97 +++++++ 13 files changed, 1476 insertions(+) create mode 100644 .github/workflows/debt-measure.yml create mode 100644 .machine_readable/Debtfile.a2ml create mode 100644 docs/DEBTFILE-SPEC.adoc create mode 100755 scripts/check-debt-ratchet.sh create mode 100755 scripts/check-debtfile-structure.sh create mode 100755 scripts/run-debtfile.sh create mode 100755 scripts/tests/debt-ratchet-test.sh create mode 100755 scripts/tests/debtfile-structure-test.sh create mode 100755 scripts/tests/run-debtfile-test.sh diff --git a/.github/workflows/actions.lock b/.github/workflows/actions.lock index f74064c3..146ee1af 100644 --- a/.github/workflows/actions.lock +++ b/.github/workflows/actions.lock @@ -20,6 +20,8 @@ workflows: '.github/workflows/codeql-reusable.yml': - 'actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1' - 'github/codeql-action@f205ea1c3313d32999d8d6a48b4f6530d4437b38' + '.github/workflows/debt-measure.yml': + - 'actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1' '.github/workflows/deno-ci-reusable.yml': - 'actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1' - 'denoland/setup-deno@22d081ff2d3a40755e97629de92e3bcbfa7cf2ed' diff --git a/.github/workflows/debt-measure.yml b/.github/workflows/debt-measure.yml new file mode 100644 index 00000000..7a3f80a8 --- /dev/null +++ b/.github/workflows/debt-measure.yml @@ -0,0 +1,108 @@ +# SPDX-License-Identifier: MPL-2.0 +name: "📅 Debt measure" + +# Re-measure every Debtfile probe and record the result as ONE pull request. +# +# This workflow exists to replace a specific bad habit. In `.git-private-farm`, +# a bot minted the same measurement as a NEW ISSUE on every run: +# +# #104 rustsec: 138 open advisories (34 high/critical) +# #105 rustsec: 136 open advisories (34 high/critical) +# #107 rustsec: 139 open advisories (34 high/critical) +# #117 rustsec: 156 open advisories (54 high/critical) +# #119 rustsec: 156 open advisories (54 high/critical) +# #120 rustsec: 155 open advisories (53 high/critical) +# +# Six issues carrying one number. This workflow opens at most one PR and +# NEVER opens an issue. That is the whole point; do not add issue creation. +# +# 📅 PERIODIC per docs/CICD-SIGNAL-DISCIPLINE.adoc — MUST NOT be a required +# status check. + +on: + schedule: + # Mondays 06:17 UTC. Off the hour on purpose: the estate's crons cluster on + # :00 and saturate the Actions queue. + - cron: '17 6 * * 1' + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: debt-measure-${{ github.ref }} + cancel-in-progress: false + +jobs: + measure: + name: Measure and record + runs-on: ubuntu-latest + timeout-minutes: 15 + permissions: + contents: write + pull-requests: write + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + + - name: Re-measure the Debtfile + id: measure + run: | + set -uo pipefail + if [ ! -f .machine_readable/Debtfile.a2ml ]; then + echo "No Debtfile — nothing to measure." + echo "changed=no" >> "$GITHUB_OUTPUT" + exit 0 + fi + + # --write updates `count` and lowers any `ceiling` that has been paid + # down. It NEVER raises one. If debt grew past its ceiling the run + # exits 1 and the resulting file has count > ceiling — which the + # structure check rejects, so the PR opens RED. That is deliberate: + # growth is a human decision (pay it down, or raise the ceiling with a + # `Debt-exception:` trailer), never something a cron decides. + rc=0 + bash scripts/run-debtfile.sh --write | tee "$RUNNER_TEMP/report.txt" || rc=$? + echo "rc=$rc" >> "$GITHUB_OUTPUT" + + if git diff --quiet -- .machine_readable/Debtfile.a2ml; then + echo "No change in measured debt." + echo "changed=no" >> "$GITHUB_OUTPUT" + else + echo "changed=yes" >> "$GITHUB_OUTPUT" + fi + + - name: Open (or update) the single recording PR + if: steps.measure.outputs.changed == 'yes' + env: + GH_TOKEN: ${{ github.token }} + run: | + set -euo pipefail + BRANCH="chore/debt-measure" + git config user.name "github-actions[bot]" + git config user.email "41898282+github-actions[bot]@users.noreply.github.com" + + git checkout -B "$BRANCH" + git add .machine_readable/Debtfile.a2ml + # One -m per paragraph. A multi-line quoted string here would put its + # continuation lines at column 1, which ends the YAML block scalar and + # leaves the workflow unparseable — the estate already has a large + # population of workflows broken exactly this way. + git commit -m "chore(debt): re-measure Debtfile probes" \ + -m "Machine-written by .github/workflows/debt-measure.yml. Counts follow the probes; ceilings only ever fall. A ceiling that needs to RISE is not done here — raise it in a human PR with a Debt-exception: trailer." + git push -f origin "$BRANCH" + + BODY="$(printf 'Automated re-measurement of `.machine_readable/Debtfile.a2ml`.\n\n```\n%s\n```\n\nCeilings only fall here. If an entry breached its ceiling this PR is red on\npurpose — pay the debt down, or raise the ceiling in a human commit carrying\na `Debt-exception:` trailer naming the entry.\n\nSee `docs/DEBTFILE-SPEC.adoc`.\n' "$(cat "$RUNNER_TEMP/report.txt")")" + + if gh pr view "$BRANCH" --json number >/dev/null 2>&1; then + gh pr edit "$BRANCH" --body "$BODY" + else + gh pr create --head "$BRANCH" --base "${{ github.ref_name }}" \ + --title "chore(debt): re-measure Debtfile probes" --body "$BODY" + fi + + - name: Surface a breach without minting an issue + if: steps.measure.outputs.rc != '0' + run: | + echo "::error::Debtfile run reported a breach, a broken probe, or an expired acceptance." + echo "See the PR opened by this run. No issue has been created, by design." + exit 1 diff --git a/.github/workflows/governance-reusable.yml b/.github/workflows/governance-reusable.yml index 44909407..7f88cc46 100644 --- a/.github/workflows/governance-reusable.yml +++ b/.github/workflows/governance-reusable.yml @@ -1226,3 +1226,53 @@ jobs: rm -rf .standards-checkout bash "$RUNNER_TEMP/check-exemption-ratchet.sh" \ "${{ github.event.pull_request.base.sha }}" + + debt-ratchet: + name: Debt ratchet + runs-on: ${{ inputs.runs-on }} + timeout-minutes: 5 + permissions: + contents: read + # Sibling to exemption-ratchet, and pull-request-only for the same reason: + # the check compares ceilings against a BASE. The two govern different + # populations — the exemption ratchet watches debt you have EXCUSED, this + # one watches debt you have MEASURED. + if: ${{ github.event_name == 'pull_request' }} + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + repository: ${{ github.repository }} + ref: ${{ github.sha }} + # Full history: the ratchet reads the Debtfile at the base ref and + # scans commit messages between base and HEAD for a declared + # exception. A shallow clone has neither. + fetch-depth: 0 + + - name: Check out standards repo for shared scripts + uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + with: + repository: hyperpolymath/standards + ref: main + path: .standards-checkout + sparse-checkout: | + scripts + sparse-checkout-cone-mode: false + + - name: Debt ratchet + run: | + set -euo pipefail + # Stage the scripts OUT of the scanned tree and delete the checkout, + # so the ratchet only ever reads the CALLER's Debtfile — standards + # has one of its own and it is not this repository's. + cp .standards-checkout/scripts/check-debt-ratchet.sh \ + .standards-checkout/scripts/check-debtfile-structure.sh "$RUNNER_TEMP/" + rm -rf .standards-checkout + + # A repo with no Debtfile is not in violation — adoption is opt-in. + # But a repo that HAS one must have a well-formed one, or the ratchet + # would be comparing ceilings it could not parse. + if [ -f .machine_readable/Debtfile.a2ml ]; then + bash "$RUNNER_TEMP/check-debtfile-structure.sh" + fi + bash "$RUNNER_TEMP/check-debt-ratchet.sh" \ + "${{ github.event.pull_request.base.sha }}" diff --git a/.machine_readable/Debtfile.a2ml b/.machine_readable/Debtfile.a2ml new file mode 100644 index 00000000..e7b9955c --- /dev/null +++ b/.machine_readable/Debtfile.a2ml @@ -0,0 +1,82 @@ +# SPDX-License-Identifier: MPL-2.0 +# Debtfile — measured, tolerated shortfalls for this repository. +# Author: Jonathan D.A. Jewell +# +# A Mustfile asserts a BOOLEAN: the invariant holds, or the build fails. +# A Debtfile asserts a MONOTONE-DECREASING INTEGER: this much debt is +# tolerated, and it may never grow without saying so out loud. +# +# Run with: just debt-measure (scripts/run-debtfile.sh) +# Ratchet: scripts/check-debt-ratchet.sh [CI, on pull_request] +# Structure: scripts/check-debtfile-structure.sh +# Spec: docs/DEBTFILE-SPEC.adoc +# +# `count` and `ceiling` are MACHINE-WRITTEN by the weekly measure run. +# Do not hand-edit them to make a gate pass; that is the one thing this +# file exists to prevent. + +@abstract: +Measured technical debt for the hyperpolymath `standards` canon. Every entry +carries a probe that returns an integer, so the claim can be re-checked +instead of believed. Entries leave this file by reaching zero, not by being +forgotten. +@end + +## Documentation + +### docs-md-not-adoc +- description: Docs under docs/ still in Markdown; estate policy is AsciiDoc (.adoc) except the four GitHub-required .md files +- probe: git ls-files 'docs/**/*.md' 'docs/*.md' | grep -vEi '(SECURITY|CONTRIBUTING|CODE_OF_CONDUCT|CHANGELOG)\.md$' | wc -l +- count: 309 +- ceiling: 309 +- severity: low +- policy: remediable +- tri: substitute +- accepted-until: 2027-02-01 + +## Test coverage of the gates + +### gate-scripts-without-tests +- description: Scripts under scripts/ with no matching scripts/tests/-test.sh — a gate with no test has never been shown able to fail +- probe: n=0; for f in $(git ls-files 'scripts/*.sh'); do b=$(basename "$f" .sh); case "$b" in *-test) continue;; esac; if [ ! -f "scripts/tests/${b}-test.sh" ] && [ ! -f "scripts/tests/${b#check-}-test.sh" ] && [ ! -f "scripts/tests/${b#run-}-test.sh" ]; then n=$((n+1)); fi; done; echo "$n" +- count: 31 +- ceiling: 31 +- severity: high +- policy: remediable +- tri: eliminate +- accepted-until: 2026-12-01 + +## Hygiene + +### shell-scripts-missing-spdx +- description: Shell scripts under scripts/ without an SPDX-License-Identifier in the first three lines +- probe: n=0; for f in $(git ls-files 'scripts/*.sh'); do head -3 "$f" | grep -q 'SPDX-License-Identifier' || n=$((n+1)); done; echo "$n" +- count: 2 +- ceiling: 2 +- severity: medium +- policy: remediable +- tri: eliminate +- accepted-until: 2026-10-01 + +### todo-fixme-markers +- description: Files carrying a TODO or FIXME marker. Advisory — a marker is a note to self, not a defect +- probe: git grep -I -l -E '\b(TODO|FIXME)\b' -- . ':(exclude).machine_readable/Debtfile.a2ml' ':(exclude)docs/DEBTFILE-SPEC.adoc' | wc -l +- count: 277 +- ceiling: 277 +- severity: low +- policy: remediable +- tri: control +- accepted-until: 2027-06-01 + +## Licensing — FLAG ONLY + +### pmpl-licence-headers +- description: Files carrying a PMPL-1.0 identifier. PMPL is correct for palimpsest-license, palimpsest-plasma and consent-aware-http and WRONG everywhere else — but which is which is an owner ruling, never an automated sweep +- probe: git grep -l 'PMPL-1\.0' -- . ':(exclude).machine_readable/Debtfile.a2ml' ':(exclude)docs/DEBTFILE-SPEC.adoc' | wc -l +- count: 589 +- ceiling: 589 +- severity: medium +- policy: flag-only +- tri: control +- tracking: hyperpolymath/standards#308 +- accepted-until: 2027-01-01 diff --git a/Justfile b/Justfile index 3f444d12..a6e14eaf 100644 --- a/Justfile +++ b/Justfile @@ -66,6 +66,19 @@ mustfile-check path=".machine_readable/contractiles/must/Mustfile.a2ml": must-check path=".machine_readable/contractiles/must/Mustfile.a2ml": @bash scripts/run-mustfile.sh "{{path}}" +# Structural validation of the Debtfile (probe + ceiling + policy + expiry per entry) +debtfile-check path=".machine_readable/Debtfile.a2ml": + @bash scripts/check-debtfile-structure.sh "{{path}}" + +# Re-measure every Debtfile probe and compare against its ceiling (read-only) +debt-measure path=".machine_readable/Debtfile.a2ml": + @bash scripts/run-debtfile.sh "{{path}}" + +# Re-measure and WRITE BACK: updates count, lowers ceilings that were paid down. +# Never raises a ceiling — that needs a Debt-exception in the commit message. +debt-ratchet-down path=".machine_readable/Debtfile.a2ml": + @bash scripts/run-debtfile.sh --write "{{path}}" + # Install this repo's git hooks into .git/hooks/ (pre-commit guards) hooks-install: @bash hooks/install.sh diff --git a/docs/DEBTFILE-SPEC.adoc b/docs/DEBTFILE-SPEC.adoc new file mode 100644 index 00000000..553316a4 --- /dev/null +++ b/docs/DEBTFILE-SPEC.adoc @@ -0,0 +1,311 @@ +// SPDX-License-Identifier: MPL-2.0 += Debtfile Specification +Jonathan D.A. Jewell +v1.0.0, 2026-08-07 +:toc: +:toclevels: 3 + +== Why this exists + +A GitHub issue is a *boolean*: open or closed. A great deal of what this estate +tracks in issues is an *integer* that changes every week. + +An audit of the estate's 697 open issues on 2026-08-07 found the consequences: + +[cols="3,2,2,3"] +|=== +|Issue |Claimed |Measured 2026-08-07 | + +|`standards#309` |~45 banned `.py` files |*176* |wrong by ~4×, unchallenged for ten weeks +|`standards#308` |~12,000 PMPL headers |*4,968* |59% paid down, unrecorded +|`reposystem#138` |Nix in 63 repos |*67* |grew, unnoticed +|=== + +The purest form was `.git-private-farm`, where a bot minted the same +measurement as a *new issue* on every run — `rustsec: 138 advisories`, then +`136`, then `139`, then `156`, `156`, `155` — six issues, plus three more for +mirror drift. Nine issues that should have been two entries with a time series. + +A Debtfile holds the integer, so the issue tracker can go back to holding +booleans. + +== Relationship to the Mustfile + +[cols="1,2,2"] +|=== +| |`Mustfile.a2ml` |`Debtfile.a2ml` + +|Asserts |a boolean |a monotone-decreasing integer +|Means |"this invariant holds" |"this much shortfall is tolerated" +|On breach |the build fails |the ratchet fails +|Leaves the file when |never — invariants are permanent |the count reaches zero +|=== + +The two are deliberately parallel. `Debtfile.a2ml` uses the same `### ` +block shape as `Mustfile.a2ml`, so the parsers agree and a reader who knows one +knows the other. + +== Why it is not a contractile + +The name fits the contractile convention exactly — `Mustfile.a2ml`, +`Trustfile.a2ml`, `Bustfile.a2ml`, `Dustfile.a2ml`, `Intentfile.a2ml`. The +placement does not, for four reasons established 2026-08-07: + +. *The contractile CLI does not exist.* `reposystem/contractiles/cli/` — named + by both `must.ncl` and `CONTRACTILE-SPEC.adoc` — is an empty directory. Five + of the six verbs have no runner at all. +. *Only `must` executes*, and it does so via hand-written bash + (`check-mustfile-structure.sh`, `run-mustfile.sh`) that bypasses the missing + CLI. That is the pattern this specification copies. +. *`EXEMPTION-MECHANISMS.adoc` rejects declarations with no reader.* Its + anti-patterns section refuses `.hypatia-ignore` because it is "never read by + anything". A seventh contractile verb would be exactly that. +. *No existing verb fits.* `bust` is fault-injection and recovery drills; + `dust` is exnovation candidates; `intend` is roadmap intents. Debt straddles + `dust` and `intend` and is neither. + +If the contractile CLI is ever built, `Debtfile.a2ml` can be absorbed as the +`debt` verb without a rename. + +== What it replaces + +This is not a sixth mechanism. It is the normalisation of two the estate +already invented, plus the retirement of a bot behaviour. + +[cols="2,1,2"] +|=== +|Convention |Reach |Fate + +|`debt.a2ml` — `[[debt.should]]` / `[[debt.could]]`, ADR-002 +|4 repos +|migrated in + +|Exemption Layer 4b — a Markdown table inside `.claude/CLAUDE.md`, rows +`\| \| \| \| \|`, matched by regex +|19 repos +|migrated in + +|Bot-minted measurement issues +|`.git-private-farm` and others +|replaced +|=== + +== File format + +Location: `.machine_readable/Debtfile.a2ml`, one per repository. + +[source] +---- +### +- description: +- probe: +- count: +- ceiling: +- severity: critical | high | medium | low +- policy: remediable | flag-only +- tri: eliminate | substitute | control (optional, Safety Triangle) +- tracking: (optional) +- accepted-until: YYYY-MM-DD +---- + +`##
` headings group entries and are otherwise ignored. + +=== Fields + +`probe`:: A shell command emitting one non-negative integer on stdout. This is +the field that makes the entry a claim rather than folklore. See +<>. + +`count`:: The last measured value. Written by `run-debtfile.sh --write`. Hand- +editing this to make a gate pass is the one thing the mechanism exists to +prevent. + +`ceiling`:: The maximum tolerated. The runner lowers it automatically when debt +is paid down; nothing raises it except a human with a declaration. + +`policy`:: `remediable` — automation may open PRs against this. `flag-only` — +*measure and report only, never act*. Licensing debt is always `flag-only`, +per the standing owner directive in `.claude/CLAUDE.md`: licence changes are +manual, per-file and owner-only, and every prior bulk sweep scrambled +identifiers or reverted owner decisions. + +`accepted-until`:: An expiry. Once passed, the entry fails the runner even while +holding under its ceiling. Debt without an expiry is debt nobody revisits — the +697-issue pile is what that looks like. + +[[probe-discipline]] +== Probe discipline + +=== No fallback + +A probe that fails to run, exits non-zero, or prints anything but an integer is +a *hard failure*. There is deliberately no `|| echo 0`. + +A counter that returns 0 on error is indistinguishable from zero debt, and zero +is the state that passes. Every gate in this estate that ever lied did it this +way — the `just proof-check` recipe exiting 0 with no prover installed, the +linter that never parsed its input, the ledger counter whose `grep -c` returned +1 on an all-comments file and killed its own script mid-report. + +=== Shell and awk, not Python or a JS runtime + +Python is banned estate-wide. The JS runtime has already moved once +(Deno to Bun). Probes and runners are POSIX-ish shell. + +=== A naive probe manufactures debt + +Three candidate probes were written and rejected during this specification's own +authoring, each of which reported debt that did not exist: + +[cols="2,3"] +|=== +|Probe |What it actually matched + +|`uses:` lines without a 40-character SHA +|Usage examples inside `#` comments in `codeql-reusable.yml` + +|Scripts containing `python3` +|A comment in `check-exemption-ratchet.sh` explaining why it does *not* use python3 + +|Scripts with no matching `scripts/tests/-test.sh` +|The test files themselves, plus scripts whose test uses a shortened name +|=== + +Two of the three matched *comments*. This is the same failure class as the +Hypatia `unwrap` rule matching comments and the GitHub-Actions log grep matching +the echoed `run:` source. + +Rules that follow: + +* Prefer `git ls-files` (structural) over `git grep` (textual) wherever the + question can be asked structurally. +* A probe must respect the carve-outs the corresponding policy already grants. + `standards` has four `.ts` files and *zero* TypeScript debt, because all four + are in documented exemption classes. +* Validate a new probe against a hand-checked number before committing it. + +=== The register must not count itself + +A textual probe will match the Debtfile entry that describes it, and the +specification that documents it. Both files necessarily contain the very string +being searched for. + +This was not hypothetical: the first live run of this Debtfile reported +`todo-fixme-markers` at 278 against a ceiling of 277, and +`pmpl-licence-headers` at 590 against 589. Both breaches were caused by the act +of writing the entries down. Writing a `## Banned languages` heading into a +register is not a banned language. + +Every textual probe therefore excludes the register and its spec: + +[source] +---- +- probe: git grep -l 'PMPL-1\.0' -- . \ + ':(exclude).machine_readable/Debtfile.a2ml' \ + ':(exclude)docs/DEBTFILE-SPEC.adoc' | wc -l +---- + +=== Probes measure tracked files + +`git ls-files` and `git grep` read the index, not the working tree. A probe +therefore measures what is committed or staged. This is the correct behaviour +for a PR gate, but it means a brand-new untracked file is invisible until added. + +== The runner + +`scripts/run-debtfile.sh [--write] [path]` + +[cols="2,3"] +|=== +|Condition |Result + +|`measured > ceiling` |❌ BREACH — exit 1 +|`measured < ceiling` |✅ PAID — with `--write`, the ceiling follows it down +|`measured == 0` |🎉 RESOLVED +|`measured == ceiling` |· HOLDING +|probe fails / non-integer |💥 PROBEFAIL — exit 1 +|`accepted-until` passed |⏰ EXPIRED — exit 1 +|=== + +`--write` never raises a ceiling. Raising one is a deliberate act that must go +through the ratchet's declaration. + +== The ratchet + +`scripts/check-debt-ratchet.sh [path]` + +Sibling to `check-exemption-ratchet.sh`. The two govern different populations: + +* *exemption ratchet* — debt you have **excused** (baseline, ignores, + allowlists). "Did you quietly excuse more?" +* *debt ratchet* — debt you have **measured** (`Debtfile.a2ml`). "Did the + number go up?" + +Three rules: + +. *No silent raise.* A ceiling may fall freely; raising one needs a declaration. +. *No silent deletion.* An entry leaves by reaching zero, not by being deleted. + Deletion is how measured debt becomes invisible debt — the exact analogue of + the `**` wildcard the exemption ratchet rejects in the migration ledger. +. *Adding an entry is free.* Recording new debt is the behaviour to encourage; + it needs no ceremony. + +=== Declaration + +[source] +---- +Debt-exception: — +Debt-exception(): +---- + +A declaration naming no known entry is *rejected*, not treated as blanket +permission, because "unparseable" must never mean "allowed". This is the bug the +exemption ratchet shipped with and had to fix: one bare `Ratchet-exception:` +licensed growth in all four ledgers at once. + +A declaration is scoped to the *pull request*, not to a commit — the ratchet +scans every commit message in `base..HEAD`. + +== CI wiring + +Job `debt-ratchet` in `.github/workflows/governance-reusable.yml`, so every +caller repository picks it up on its next run with no per-repo PR. + +Pull-request-only, for the same reason as the exemption ratchet: the check needs +a base to compare against. + +Tier *🟡 CHECK* per `docs/CICD-SIGNAL-DISCIPLINE.adoc` — it MUST NOT be a +required status check. Promotion to 🔴 GATE is deliberate and comes only after +the measure cycle has run cleanly for two consecutive weeks. + +A repository with no `Debtfile.a2ml` is not in violation; adoption is opt-in. +A repository that has one must have a well-formed one. + +== Recipes + +[source] +---- +just debtfile-check # structural validation +just debt-measure # re-measure, read-only +just debt-ratchet-down # re-measure and write back (lowers ceilings only) +---- + +== Tests + +Three suites under `scripts/tests/`, discovered automatically by +`.github/workflows/self-test.yml`: + +* `debtfile-structure-test.sh` — 14 cases +* `run-debtfile-test.sh` — 16 cases +* `debt-ratchet-test.sh` — 12 cases + +Every suite exists to prove its gate *can fail*. This estate's recurring defect +is the gate that cannot. Each was mutation-tested during authoring: flipping a +single expectation must turn the suite red. + +== See also + +* `docs/EXEMPTION-MECHANISMS.adoc` — the four exemption ledgers and their ratchet +* `docs/CONTRACTILE-SPEC.adoc` — the contractile convention this borrows its shape from +* `docs/CICD-SIGNAL-DISCIPLINE.adoc` — the four gate tiers +* `scripts/count-ledger-entries.sh` — the no-fallback rule, stated for ledgers diff --git a/docs/EXEMPTION-MECHANISMS.adoc b/docs/EXEMPTION-MECHANISMS.adoc index 6c02de76..075ea3a1 100644 --- a/docs/EXEMPTION-MECHANISMS.adoc +++ b/docs/EXEMPTION-MECHANISMS.adoc @@ -20,8 +20,31 @@ pre-existing finding? Are you trying to make a finding GO AWAY for one specific PR only? -> currently unsupported. See "per-PR exemptions" below. + +Are you trying to record that a shortfall EXISTS, is tolerated for now, +and must never grow? + -> that is not an exemption. Use the per-repo .machine_readable/Debtfile.a2ml + See docs/DEBTFILE-SPEC.adoc. ---- +[NOTE] +==== +*Exemption is not debt.* The layers below record findings you have decided +*not to act on* — the finding is excused, and the ledger's job is to keep the +excuse visible. A `Debtfile.a2ml` entry records a shortfall you *do* intend to +act on, carrying a probe that re-measures it and a ceiling that may only fall. + +The two have their own ratchets, and they answer different questions: + +* `check-exemption-ratchet.sh` — "did you quietly excuse more?" +* `check-debt-ratchet.sh` — "did the number go up?" + +Layer 4b below (the `.claude/CLAUDE.md` exemption tables, which carry a *count* +and an *unblock condition*) is the case where the two got conflated. Those +tables are debt wearing an exemption's clothing, and they migrate to +`Debtfile.a2ml`. +==== + == Layer 1: Estate-wide bot write denylist **File:** `hyperpolymath/standards/.machine_readable/bot_exclusion_registry.a2ml` diff --git a/scripts/check-debt-ratchet.sh b/scripts/check-debt-ratchet.sh new file mode 100755 index 00000000..0cfd4373 --- /dev/null +++ b/scripts/check-debt-ratchet.sh @@ -0,0 +1,174 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: MPL-2.0 +# +# check-debt-ratchet.sh — a Debtfile's ceilings may fall, never rise silently. +# +# Sibling to scripts/check-exemption-ratchet.sh, which does the same job for +# the four exemption ledgers. The two govern different populations: +# +# exemption ratchet — debt you have EXCUSED (baseline, ignores, allowlists) +# "did you quietly excuse more?" +# debt ratchet — debt you have MEASURED (Debtfile.a2ml) +# "did the number go up?" +# +# Three rules: +# +# 1. NO SILENT RAISE. A ceiling may fall freely. Raising one needs a +# declaration in the commit message. +# +# 2. NO SILENT DELETION. An entry leaves the Debtfile by reaching zero, not +# by being deleted. Deleting an entry is how measured debt becomes +# invisible debt, and it is the exact analogue of the `**` wildcard the +# exemption ratchet rejects in the migration ledger: the file appears to +# hold steady while what it covers quietly changes. Deletion needs the +# same declaration a raise does. +# +# 3. NO UNDECLARED NEW CEILING ABOVE ZERO for an entry that did not exist. +# Adding an entry is honest and welcome — that is how debt gets recorded. +# It is allowed without ceremony. (Rule kept explicit so nobody "fixes" +# the gate by making declaration mandatory here; that would punish the +# one behaviour this file wants to encourage.) +# +# Escape hatch, per-entry, never blanket: +# +# Debt-exception: — +# Debt-exception(): +# +# ⚠ A declaration naming no known entry is REJECTED rather than treated as +# blanket permission, because "unparseable" must never mean "allowed". This +# is the bug the exemption ratchet shipped with and had to fix: a single bare +# `Ratchet-exception:` licensed growth in all four ledgers at once. +# +# Usage: check-debt-ratchet.sh [path/to/Debtfile.a2ml] +# Exit: 0 ok · 1 violation · 2 bad invocation + +set -uo pipefail + +if [ "$#" -lt 1 ] || [ "$#" -gt 2 ]; then + echo "usage: check-debt-ratchet.sh [path/to/Debtfile.a2ml]" >&2 + exit 2 +fi +BASE_REF="$1" +DEBT="${2:-.machine_readable/Debtfile.a2ml}" + +fail=0 +note() { printf ' %s\n' "$*"; } + +# Emit " " for every entry in the Debtfile at a given revision. +# A missing file yields nothing — that is a real state (the Debtfile has not +# been adopted yet), not an error. +ceilings_at() { + local ref="$1" blob + blob="$(git show "${ref}:${DEBT}" 2>/dev/null)" || return 0 + printf '%s\n' "$blob" | awk ' + { line = $0; sub(/^[ \t]+/, "", line) } + line ~ /^### / { id = substr(line, 5); next } + line ~ /^- ceiling: / && id != "" { + v = substr(line, 12); gsub(/[ \t\r]/, "", v) + print id, v; id = "" + } + ' +} + +# Does the pull request declare that a SPECIFIC entry may rise or vanish? +declared_for() { + local entry="$1" msgs + msgs="$(git log --format=%B "${BASE_REF}..HEAD" 2>/dev/null || true)" + printf '%s' "$msgs" | grep -iE '^Debt-exception' | grep -qF "$entry" +} + +before_list="$(ceilings_at "$BASE_REF")" +after_list="$(ceilings_at "HEAD")" + +if [ -z "$before_list" ] && [ -z "$after_list" ]; then + echo "Debt ratchet: no Debtfile at ${BASE_REF} or HEAD — nothing to ratchet." + exit 0 +fi + +echo "Debt ratchet — comparing ${DEBT} against ${BASE_REF}" + +# --- rules 1 and 2: walk what existed before --- +while read -r id before; do + [ -n "$id" ] || continue + after="$(printf '%s\n' "$after_list" | awk -v k="$id" '$1 == k { print $2; exit }')" + + if [ -z "$after" ]; then + if declared_for "$id"; then + note "OK (declared) ${id}: removed [Debt-exception present]" + else + note "DELETED ${id}: ceiling ${before} -> entry removed" + note " An entry leaves by reaching zero, not by deletion." + fail=1 + fi + continue + fi + + if [ "$after" -gt "$before" ]; then + if declared_for "$id"; then + note "OK (declared) ${id}: ${before} -> ${after} [Debt-exception present]" + else + note "RAISED ${id}: ceiling ${before} -> ${after}" + fail=1 + fi + elif [ "$after" -lt "$before" ]; then + note "LOWERED ${id}: ${before} -> ${after} <- debt paid down" + else + note "unchanged ${id}: ${before}" + fi +done </dev/null | grep -iE '^Debt-exception' || true)" +if [ -n "$decls" ]; then + while IFS= read -r d; do + [ -n "$d" ] || continue + matched=0 + while read -r id _; do + [ -n "$id" ] || continue + case "$d" in *"$id"*) matched=1; break;; esac + done <&2 + exit 2 +fi +if [ ! -f "$DEBT" ]; then + echo "error: Debtfile not found: $DEBT" >&2 + exit 2 +fi + +fail=0 +entries=0 +seen_ids=" " + +name="" probe="" count="" ceiling="" severity="" policy="" accepted="" + +note() { printf ' %s\n' "$*"; } +bad() { printf ' ❌ %s\n' "$*"; fail=1; } + +is_uint() { case "${1:-}" in ''|*[!0-9]*) return 1;; *) return 0;; esac; } + +validate() { + [ -n "$name" ] || return 0 + entries=$((entries + 1)) + + case "$seen_ids" in + *" $name "*) bad "duplicate id: '$name'" ;; + *) seen_ids="${seen_ids}${name} " ;; + esac + + # Hollow-entry rejection — the core rule. + if [ -z "$probe" ]; then + bad "'$name' has no '- probe:' — a number nothing re-measures is not debt, it is folklore" + fi + if [ -z "$ceiling" ]; then + bad "'$name' has no '- ceiling:' — without a ceiling the ratchet has nothing to hold" + fi + if [ -z "$count" ]; then + bad "'$name' has no '- count:'" + fi + + is_uint "$count" || { [ -n "$count" ] && bad "'$name' count '$count' is not a non-negative integer"; } + is_uint "$ceiling" || { [ -n "$ceiling" ] && bad "'$name' ceiling '$ceiling' is not a non-negative integer"; } + + if is_uint "$count" && is_uint "$ceiling" && [ "$count" -gt "$ceiling" ]; then + bad "'$name' count ($count) exceeds ceiling ($ceiling) — the file records more debt than it tolerates" + fi + + case "$severity" in + critical|high|medium|low) ;; + '') bad "'$name' has no '- severity:'" ;; + *) bad "'$name' severity '$severity' is not one of critical|high|medium|low" ;; + esac + + case "$policy" in + remediable|flag-only) ;; + '') bad "'$name' has no '- policy:' — say whether this may be auto-remediated or is flag-only" ;; + *) bad "'$name' policy '$policy' is not one of remediable|flag-only" ;; + esac + + if [ -n "$accepted" ]; then + case "$accepted" in + [0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]) ;; + *) bad "'$name' accepted-until '$accepted' is not an ISO date (YYYY-MM-DD)" ;; + esac + else + bad "'$name' has no '- accepted-until:' — debt without an expiry is debt nobody ever revisits" + fi +} + +reset_block() { name="$1"; probe=""; count=""; ceiling=""; severity=""; policy=""; accepted=""; } + +while IFS= read -r raw || [ -n "$raw" ]; do + line="${raw#"${raw%%[![:space:]]*}"}" + case "$line" in + '### '*) validate; reset_block "${line:4}" ;; + '- probe: '*) probe="${line#- probe: }" ;; + '- count: '*) count="${line#- count: }" ;; + '- ceiling: '*) ceiling="${line#- ceiling: }" ;; + '- severity: '*) severity="${line#- severity: }" ;; + '- policy: '*) policy="${line#- policy: }" ;; + '- accepted-until: '*) accepted="${line#- accepted-until: }" ;; + esac +done < "$DEBT" +validate # flush last block + +echo +if [ "$entries" -eq 0 ]; then + echo "❌ Debtfile structure FAILED: $DEBT declares no entries." >&2 + echo " An empty Debtfile and an unparsed Debtfile look identical from here," >&2 + echo " so this is treated as a defect rather than as zero debt." >&2 + exit 1 +fi + +if [ "$fail" -ne 0 ]; then + echo "❌ Debtfile structure FAILED ($entries entr(y|ies) examined)." >&2 + exit 1 +fi +echo "✅ Debtfile structure OK ($entries entries)." diff --git a/scripts/run-debtfile.sh b/scripts/run-debtfile.sh new file mode 100755 index 00000000..104632a6 --- /dev/null +++ b/scripts/run-debtfile.sh @@ -0,0 +1,168 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: MPL-2.0 +# +# run-debtfile.sh — EXECUTE the probes declared in a Debtfile.a2ml. +# +# This is the executable half of Debtfile checking; +# scripts/check-debtfile-structure.sh is the structural half. +# +# Each '### ' block declares a `- probe:` — a shell command that prints a +# single non-negative integer: how much of that debt exists right now. The +# runner re-measures every entry and compares against the declared `ceiling`. +# +# measured > ceiling ❌ debt grew past what this repo tolerates +# measured < ceiling ✅ paid down (with --write, the ceiling follows it down) +# measured == 0 🎉 resolved +# accepted-until passed ⏰ expired — the acceptance was time-boxed and the time is up +# +# ⚠ NO FALLBACK. A probe that fails to run, or prints anything other than an +# integer, is a HARD FAILURE. This is the single most important property of +# the whole mechanism. A probe that returns 0 on error is indistinguishable +# from zero debt, and zero is the state that passes — so a broken probe would +# silently report a clean repo. Every estate gate that has ever lied did it +# this way. See scripts/count-ledger-entries.sh for the same rule stated for +# the exemption ledgers. +# +# ⚠ A PROBE MUST RESPECT EXISTING CARVE-OUTS. Naive probes manufacture debt. +# During this file's own authoring, three candidate probes were rejected for +# counting comments and documentation examples as violations: +# * `uses:` without a 40-char SHA → matched usage examples in comments +# * scripts containing `python3` → matched a comment saying "not python3" +# * scripts with no matching test file → counted the test files themselves +# Validate a new probe against a hand-checked number before committing it. +# +# Usage: run-debtfile.sh [--write] [path/to/Debtfile.a2ml] +# --write update `count` in place, and lower `ceiling` to match when debt +# has been paid down (the auto-ratchet). Never raises a ceiling. +# Default path: .machine_readable/Debtfile.a2ml +# +# Exit: 0 all entries at or under ceiling · 1 a ceiling was breached, a probe +# failed, or an acceptance expired · 2 file missing / bad invocation + +set -uo pipefail + +WRITE=0 +DEBT="" +for arg in "$@"; do + case "$arg" in + --write) WRITE=1 ;; + -*) echo "usage: run-debtfile.sh [--write] [path/to/Debtfile.a2ml]" >&2; exit 2 ;; + *) if [ -n "$DEBT" ]; then echo "error: more than one Debtfile given" >&2; exit 2; fi; DEBT="$arg" ;; + esac +done +DEBT="${DEBT:-.machine_readable/Debtfile.a2ml}" + +if [ ! -f "$DEBT" ]; then + echo "error: Debtfile not found: $DEBT" >&2 + exit 2 +fi + +TODAY="$(date -u +%Y-%m-%d)" + +fail=0 ok=0 paid=0 resolved=0 breached=0 expired=0 broke=0 +name="" probe="" count="" ceiling="" severity="" policy="" accepted="" +declare -a NEW_COUNT_IDS=() NEW_COUNT_VALS=() NEW_CEIL_IDS=() NEW_CEIL_VALS=() + +is_uint() { case "${1:-}" in ''|*[!0-9]*) return 1;; *) return 0;; esac; } + +measure() { + [ -n "$name" ] || return 0 + + if [ -z "$probe" ]; then + printf ' ❓ NOPROBE [%-8s] %s\n' "${severity:-?}" "$name" + broke=$((broke + 1)); fail=1; return 0 + fi + + local out rc + out="$(bash -c "$probe" 2>/dev/null)"; rc=$? + out="$(printf '%s' "$out" | tr -d ' \t\n\r')" + + if [ "$rc" -ne 0 ] || ! is_uint "$out"; then + printf ' 💥 PROBEFAIL [%-7s] %s (exit %s, output %s)\n' "${severity:-?}" "$name" "$rc" "${out:-}" + printf ' a probe that cannot measure must never report zero\n' + broke=$((broke + 1)); fail=1; return 0 + fi + + # Time-boxed acceptance. + local is_expired=0 + if [ -n "$accepted" ] && [ "$accepted" \< "$TODAY" ]; then is_expired=1; fi + + if [ "$out" -gt "$ceiling" ]; then + printf ' ❌ BREACH [%-8s] %-34s %s > ceiling %s\n' "$severity" "$name" "$out" "$ceiling" + breached=$((breached + 1)); fail=1 + elif [ "$out" -eq 0 ]; then + printf ' 🎉 RESOLVED [%-8s] %-34s 0 (was %s)\n' "$severity" "$name" "$ceiling" + resolved=$((resolved + 1)) + elif [ "$out" -lt "$ceiling" ]; then + printf ' ✅ PAID [%-8s] %-34s %s (ceiling %s -> %s)\n' "$severity" "$name" "$out" "$ceiling" "$out" + paid=$((paid + 1)) + NEW_CEIL_IDS+=("$name"); NEW_CEIL_VALS+=("$out") + else + printf ' · HOLDING [%-8s] %-34s %s\n' "$severity" "$name" "$out" + ok=$((ok + 1)) + fi + + if [ "$is_expired" -eq 1 ]; then + printf ' ⏰ EXPIRED [%-8s] %-34s accepted-until %s has passed\n' "$severity" "$name" "$accepted" + printf ' re-accept with a new date and a reason, or pay it down\n' + expired=$((expired + 1)); fail=1 + fi + + if [ "$out" != "$count" ]; then + NEW_COUNT_IDS+=("$name"); NEW_COUNT_VALS+=("$out") + fi +} + +reset_block() { name="$1"; probe=""; count=""; ceiling=""; severity=""; policy=""; accepted=""; } + +echo "Debtfile: $DEBT" +echo +while IFS= read -r raw || [ -n "$raw" ]; do + line="${raw#"${raw%%[![:space:]]*}"}" + case "$line" in + '### '*) measure; reset_block "${line:4}" ;; + '- probe: '*) probe="${line#- probe: }" ;; + '- count: '*) count="${line#- count: }" ;; + '- ceiling: '*) ceiling="${line#- ceiling: }" ;; + '- severity: '*) severity="${line#- severity: }" ;; + '- policy: '*) policy="${line#- policy: }" ;; + '- accepted-until: '*) accepted="${line#- accepted-until: }" ;; + esac +done < "$DEBT" +measure # flush last block + +# --------------------------------------------------------------------------- +# --write: update `count` everywhere it moved, and lower `ceiling` where debt +# was paid down. A ceiling is NEVER raised here — raising one is a deliberate +# act that must go through the ratchet's commit-message declaration. +# --------------------------------------------------------------------------- +if [ "$WRITE" -eq 1 ] && { [ "${#NEW_COUNT_IDS[@]}" -gt 0 ] || [ "${#NEW_CEIL_IDS[@]}" -gt 0 ]; }; then + tmp="$(mktemp)" + cur="" + while IFS= read -r raw || [ -n "$raw" ]; do + line="${raw#"${raw%%[![:space:]]*}"}" + case "$line" in + '### '*) cur="${line:4}" ;; + '- count: '*) + for i in "${!NEW_COUNT_IDS[@]}"; do + if [ "${NEW_COUNT_IDS[$i]}" = "$cur" ]; then raw="- count: ${NEW_COUNT_VALS[$i]}"; break; fi + done ;; + '- ceiling: '*) + for i in "${!NEW_CEIL_IDS[@]}"; do + if [ "${NEW_CEIL_IDS[$i]}" = "$cur" ]; then raw="- ceiling: ${NEW_CEIL_VALS[$i]}"; break; fi + done ;; + esac + printf '%s\n' "$raw" + done < "$DEBT" > "$tmp" + mv "$tmp" "$DEBT" + echo + echo " ✍ wrote ${#NEW_COUNT_IDS[@]} count update(s), ${#NEW_CEIL_IDS[@]} ceiling reduction(s)" +fi + +echo +echo "Debtfile: $ok holding · $paid paid-down · $resolved resolved · $breached breached · $expired expired · $broke broken-probe" +if [ "$fail" -ne 0 ]; then + echo "❌ Debtfile run FAILED" >&2 + exit 1 +fi +echo "✅ Debtfile run passed (no ceiling breached, no probe broken, no acceptance expired)" diff --git a/scripts/tests/debt-ratchet-test.sh b/scripts/tests/debt-ratchet-test.sh new file mode 100755 index 00000000..563d16aa --- /dev/null +++ b/scripts/tests/debt-ratchet-test.sh @@ -0,0 +1,140 @@ +#!/usr/bin/env bash +# SPDX-License-Identifier: MPL-2.0 +# +# Proves check-debt-ratchet.sh CAN FAIL, and fails for the right reasons. +# +# Same standard as scripts/tests/exemption-ratchet-test.sh: this estate's +# recurring defect is the gate that cannot fail. A debt ratchet that always +# passed would license exactly the silent ceiling-creep it exists to prevent, +# while looking green. Every branch is exercised, in both directions. +set -euo pipefail +SCRIPT="$(cd "$(dirname "$0")/.." && pwd)/check-debt-ratchet.sh" +WORK="$(mktemp -d)"; trap 'rm -rf "$WORK"' EXIT +cd "$WORK"; git init -q .; git config user.email t@example.com; git config user.name T +mkdir -p .machine_readable + +write_debtfile() { # write_debtfile + cat > .machine_readable/Debtfile.a2ml <