Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/opencode-review-dispatch.yml
Original file line number Diff line number Diff line change
Expand Up @@ -652,11 +652,15 @@ jobs:
r-base \
r-cran-covr \
r-cran-testthat \
llvm-19 \
rustc \
util-linux \
vulkan-tools \
xz-utils \
&& rm -rf /var/lib/apt/lists/*
ENV LLVM_COV=/usr/bin/llvm-cov-19
ENV LLVM_PROFDATA=/usr/bin/llvm-profdata-19
RUN test -x "$LLVM_COV" && test -x "$LLVM_PROFDATA"
RUN curl --proto '=https' --tlsv1.2 -fsSLo /tmp/node-linux-x64.tar.xz \
https://nodejs.org/dist/v24.18.0/node-v24.18.0-linux-x64.tar.xz \
&& echo '55aa7153f9d88f28d765fcdad5ae6945b5c0f98a36881703817e4c450fa76742 /tmp/node-linux-x64.tar.xz' | sha256sum -c - \
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
name: OpenCode Rust Coverage Toolchain Quality CI

on:
pull_request:
paths:
- ".github/workflows/opencode-review-dispatch.yml"
- ".github/workflows/opencode-rust-coverage-toolchain-quality-ci.yml"
- "tests/test_opencode_rust_coverage_toolchain_contract.py"
- "docs/doctoring/opencode-rust-coverage-runtime-boundary.md"
- "CHANGELOG.md"

permissions:
contents: read

concurrency:
group: opencode-rust-coverage-toolchain-quality-${{ github.event.pull_request.number || github.ref }}
cancel-in-progress: true

jobs:
quality:
name: quality
runs-on: ubuntu-24.04
timeout-minutes: 15
env:
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: true
steps:
- name: Harden runner
uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920
with:
egress-policy: audit

- name: Checkout exact pull request head
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
with:
ref: ${{ github.event.pull_request.head.sha }}
fetch-depth: 0
persist-credentials: false

- name: Set up Python
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97
with:
python-version: "3.14"
cache: pip
cache-dependency-path: requirements-opencode-review-ci-hashes.txt

- name: Install exact hash-locked test tooling
run: >-
python -m pip install --disable-pip-version-check --require-hashes
-r requirements-opencode-review-ci-hashes.txt

- name: Run permanent LLVM runtime-boundary contract
run: |
set -euo pipefail
python -m pytest -q tests/test_opencode_rust_coverage_toolchain_contract.py
python -m compileall -q tests/test_opencode_rust_coverage_toolchain_contract.py
git diff --check "${{ github.event.pull_request.base.sha }}...${{ github.event.pull_request.head.sha }}"
71 changes: 71 additions & 0 deletions docs/doctoring/opencode-rust-coverage-runtime-boundary.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# OpenCode Rust coverage runtime boundary

## Decision

The central OpenCode review path treats Rust coverage tooling as part of the trusted evidence boundary. A coverage image is not accepted merely because `cargo-llvm-cov` is present: the image must also contain reviewed, versioned LLVM coverage executables, propagate their exact paths into the isolated runtime, and revalidate those executables before the first coverage invocation.

The reviewed tool paths are `/usr/bin/llvm-cov-19` and `/usr/bin/llvm-profdata-19`. They are intentionally explicit rather than discovered from `PATH`. The trusted image must install Debian `llvm-19`, bind those paths through `LLVM_COV` and `LLVM_PROFDATA`, verify both are executable before installing the pinned cargo-llvm-cov archive, pass the same literal constants through the networkless Docker boundary, and verify both executables again inside the isolated runtime.

This is fail-closed evidence plumbing. A missing, non-executable, changed, or unpropagated tool path invalidates Rust coverage evidence; it must not silently fall back to an ambient host binary, a network installer, an older head, or a generated merge-tree result.

## Root cause and product impact

DiskSage exact-head OpenCode review exposed a central infrastructure defect rather than a DiskSage product defect. The trusted coverage image could contain a pinned `cargo-llvm-cov` binary while the isolated runtime lacked an explicitly compatible `llvm-cov` / `llvm-profdata` pair. That makes successful local or predecessor-head coverage irrelevant to the current review run: the durable central reviewer must independently reproduce coverage in its own restricted execution boundary.

`cargo-llvm-cov` documents `LLVM_COV` and `LLVM_PROFDATA` as explicit overrides and states that the selected `llvm-cov` must be compatible with the LLVM version used by `rustc`. LLVM documents that `llvm-cov` consumes instrumentation-based coverage data and that raw profile data is converted for reporting through `llvm-profdata merge`. The two executables therefore form one compatibility-sensitive evidence chain rather than interchangeable utilities.

## Security boundary

The repair preserves the existing separation between untrusted pull-request source and trusted reviewer execution:

- the coverage runtime remains `--network=none`;
- repository or model credentials are not introduced into the coverage container;
- no Docker socket is exposed to pull-request code;
- the reviewed LLVM paths are immutable workflow-source constants, not contributor-controlled inputs;
- build-time validation catches a malformed trusted image before it can become review infrastructure;
- runtime validation catches propagation or execution-boundary drift before coverage is accepted;
- no `rustup component add`, package installation, or network fallback is permitted during the isolated pull-request measurement step;
- exact-head binding and stale-head refusal remain independent requirements; and
- coverage success remains evidence, not durable merge or release authorization.

The change does not alter the existing review-agent model credential contract or authorize `COPILOT_GITHUB_TOKEN`. It changes only the deterministic Rust coverage toolchain carried into the isolated evidence runtime.

## Verification contract

`tests/test_opencode_rust_coverage_toolchain_contract.py` is the permanent source-level regression contract. It requires, in order:

1. Debian `llvm-19` in the trusted image;
2. exact `LLVM_COV=/usr/bin/llvm-cov-19` and `LLVM_PROFDATA=/usr/bin/llvm-profdata-19` image bindings;
3. executable checks for both paths before the pinned cargo-llvm-cov archive;
4. literal propagation of both reviewed constants through `docker run` before the coverage image argument; and
5. a second executable check after the Docker boundary and before the first `cargo llvm-cov` invocation.

The contract deliberately checks ordering as well as presence so a dead comment, post-coverage assertion, or unrelated environment declaration cannot satisfy the gate.

## Rollback and migration

Rollback is a reviewed workflow change, not a runtime bypass. If the Rust toolchain later moves to a different LLVM major version, update the Debian package, both versioned paths, the build-time and runtime assertions, this doctoring record, and the regression contract together. First prove the new compatibility requirement with a failing test, then rerun the complete exact-current-head central CI, security, coverage, docstring, packaging, provenance, and review suite. Do not revert to unversioned `PATH` discovery merely to make a failing review green.

Repositories consuming the central reviewer require no migration. Standalone repository operation is unchanged; the change only makes central review evidence deterministic. Modular CWL services such as DiskSage, Naruon, contextual-orchestrator, and other consumers continue to call the same central review contract and receive no additional runtime authority.

## Standards and primary-source evidence

NIST SP 800-218 SSDF Version 1.1 remains the current final publication and recommends integrating secure development practices into the SDLC. NIST published SP 800-218 Rev. 1 / SSDF Version 1.2 as an Initial Public Draft on December 17, 2025; it is recorded here as draft evidence, not as a final standard. The implementation choice here is narrower than either publication: it makes one build/test toolchain reproducible and fail-closed and does not claim SSDF conformance or certification.

LLVM's current command documentation identifies `llvm-cov` as the coverage reporting tool and `llvm-profdata` as the profile-data utility used to merge instrumentation profiles. The cargo-llvm-cov project documents the `LLVM_COV` and `LLVM_PROFDATA` override variables and explicitly requires LLVM compatibility with the LLVM version used by `rustc`. These primary technical sources support binding and validating a reviewed compatible pair rather than relying on ambient discovery.

## APA 7th references

LLVM Project. (2026). *llvm-cov—Emit coverage information*. https://llvm.org/docs/CommandGuide/llvm-cov.html

LLVM Project. (2026). *llvm-profdata—Profile data tool*. https://llvm.org/docs/CommandGuide/llvm-profdata.html

Souppaya, M., Scarfone, K., & Dodson, D. (2022). *Secure Software Development Framework (SSDF) version 1.1: Recommendations for mitigating the risk of software vulnerabilities* (NIST Special Publication 800-218). National Institute of Standards and Technology. https://doi.org/10.6028/NIST.SP.800-218

Booth, H., Ogata, M., Kent, K., Souppaya, M., & Dodson, D. (2025). *Secure Software Development Framework (SSDF) version 1.2: Recommendations for mitigating the risk of software vulnerabilities* (Initial Public Draft, NIST Special Publication 800-218 Rev. 1). National Institute of Standards and Technology. https://csrc.nist.gov/pubs/sp/800/218/r1/ipd

Taiki Endo. (2026). *cargo-llvm-cov: Cargo subcommand to easily use LLVM source-based code coverage*. GitHub. https://github.com/taiki-e/cargo-llvm-cov

## Reference verification note

The LLVM command guides, cargo-llvm-cov primary repository documentation, NIST SP 800-218 Version 1.1 final publication, and the SP 800-218 Rev. 1 Version 1.2 Initial Public Draft were rechecked on August 7, 2026. The draft status of Version 1.2 is intentionally preserved so it is not misrepresented as a final international or U.S. government standard.
86 changes: 86 additions & 0 deletions tests/test_opencode_rust_coverage_toolchain_contract.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
"""Permanent contract for the trusted Rust LLVM coverage toolchain."""

from __future__ import annotations

import re
from pathlib import Path


_REPOSITORY_ROOT = Path(__file__).resolve().parents[1]
_WORKFLOW_PATH = _REPOSITORY_ROOT / ".github/workflows/opencode-review-dispatch.yml"
_LLVM_COV_PATH = "/usr/bin/llvm-cov-19"
_LLVM_PROFDATA_PATH = "/usr/bin/llvm-profdata-19"


def _workflow_text() -> str:
"""Return the authoritative OpenCode review-dispatch workflow text."""

return _WORKFLOW_PATH.read_text(encoding="utf-8")


def _all_positions(text: str, fragment: str) -> list[int]:
"""Return every start position of ``fragment`` in ``text``."""

return [match.start() for match in re.finditer(re.escape(fragment), text)]


def test_trusted_rust_coverage_image_provisions_verified_llvm_19_tools() -> None:
"""Require explicit compatible LLVM tools before cargo-llvm-cov installation."""

workflow = _workflow_text()

llvm_package = workflow.index("llvm-19")
llvm_cov_environment = workflow.index(f"ENV LLVM_COV={_LLVM_COV_PATH}")
llvm_profdata_environment = workflow.index(
f"ENV LLVM_PROFDATA={_LLVM_PROFDATA_PATH}"
)
llvm_cov_checks = _all_positions(workflow, 'test -x "$LLVM_COV"')
llvm_profdata_checks = _all_positions(workflow, 'test -x "$LLVM_PROFDATA"')
cargo_llvm_cov_archive = workflow.index(
"cargo-llvm-cov-x86_64-unknown-linux-musl.tar.gz"
)

assert len(llvm_cov_checks) >= 2
assert len(llvm_profdata_checks) >= 2
assert (
llvm_package
< llvm_cov_environment
< llvm_profdata_environment
< llvm_cov_checks[0]
< llvm_profdata_checks[0]
< cargo_llvm_cov_archive
)


def test_isolated_runtime_receives_reviewed_llvm_constants() -> None:
"""Require exact LLVM 19 path propagation through the Docker boundary."""

workflow = _workflow_text()
docker_run = workflow.index("docker run --rm")
llvm_cov_binding = workflow.index(
f"--env LLVM_COV={_LLVM_COV_PATH}", docker_run
)
llvm_profdata_binding = workflow.index(
f"--env LLVM_PROFDATA={_LLVM_PROFDATA_PATH}", docker_run
)
coverage_image = workflow.index('"$coverage_tool_image"', docker_run)

assert docker_run < llvm_cov_binding < llvm_profdata_binding < coverage_image


def test_isolated_runtime_revalidates_llvm_tools_before_coverage() -> None:
"""Require reviewed-path equality and executable checks before Rust coverage."""

workflow = _workflow_text()
docker_run = workflow.index("docker run --rm")
toolchain_start = workflow.index("ensure_rust_toolchain() {", docker_run)
toolchain_end = workflow.index("rust_coverage_manifests() {", toolchain_start)
toolchain = workflow[toolchain_start:toolchain_end]
cargo_coverage_invocation = workflow.index("cargo llvm-cov", toolchain_end)
llvm_cov_checks = _all_positions(workflow, 'test -x "$LLVM_COV"')
llvm_profdata_checks = _all_positions(workflow, 'test -x "$LLVM_PROFDATA"')

assert f'"${{LLVM_COV:-}}" != "{_LLVM_COV_PATH}"' in toolchain
assert f'"${{LLVM_PROFDATA:-}}" != "{_LLVM_PROFDATA_PATH}"' in toolchain
assert docker_run < llvm_cov_checks[-1] < cargo_coverage_invocation
assert docker_run < llvm_profdata_checks[-1] < cargo_coverage_invocation
Loading