-
Notifications
You must be signed in to change notification settings - Fork 0
fix(coverage): restore LLVM 19 tools on current main #794
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
+292
−0
Closed
Changes from all commits
Commits
Show all changes
29 commits
Select commit
Hold shift + click to select a range
c8bbf69
test(coverage): require compatible LLVM tools before Rust coverage
seonghobae 3994d31
fix(coverage): restore compatible LLVM 19 tooling on current main
seonghobae 748bc5d
test(coverage): require Git isolation in low-privilege wrappers
seonghobae ee471f5
ci(coverage): repair PR 794 Git isolation
seonghobae c8462fd
ci(coverage): trigger PR 794 Git-isolation repair
seonghobae 8c6e1c3
chore(ci): remove inactive PR 794 repair workflow
seonghobae 84a2d94
ci: stage exact-head PR 794 Git isolation repair
seonghobae 214319c
ci: trigger exact-head PR 794 Git isolation repair
seonghobae 8323b78
chore(ci): remove inactive PR 794 repair workflow
seonghobae 79e372e
ci: activate bounded PR 794 Git isolation repair
seonghobae 6494136
ci: trigger bounded PR 794 Git isolation repair
seonghobae 278b10d
ci: remove inactive PR 794 repair trigger
seonghobae bce00fb
ci: repair OpenCode wrapper Git isolation
seonghobae 5db416e
ci: retrigger bounded PR 794 Git isolation repair
seonghobae 86685e3
chore(ci): trigger verified PR 794 repair
seonghobae aa1bf3c
ci: simplify PR 794 verified repair runner
seonghobae 30265bb
fix(ci): use established workflow-capable repair token
seonghobae 938846a
chore(ci): run PR 794 repair on synchronization
seonghobae d556f7b
fix(ci): materialize exact-parent repair commit via Git data API
seonghobae e23fcd9
fix(ci): use scoped job token for Git object creation
seonghobae 752a976
fix(automation): complete exact-head one-shot repair
seonghobae d009992
fix(automation): publish exact-head repair artifact
seonghobae a887717
fix(automation): preserve hidden repair source and stable hashes
seonghobae 5cf306e
fix(coverage): isolate low-privilege Git configuration
seonghobae fca5600
test(coverage): require exact-head toolchain quality workflow
seonghobae b2073df
ci(coverage): verify OpenCode toolchain at exact PR head
seonghobae 1635420
test(coverage): require full exact-head repository quality gate
seonghobae 7f12a15
ci(coverage): run complete exact-head repository quality gate
seonghobae 6bd29cc
fix(coverage): make hash-locked install contract explicit
seonghobae File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
131 changes: 131 additions & 0 deletions
131
.github/workflows/opencode-coverage-toolchain-quality-ci.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,131 @@ | ||
| name: OpenCode Coverage Toolchain Quality | ||
|
|
||
| on: | ||
| pull_request: | ||
| branches: | ||
| - main | ||
| types: | ||
| - opened | ||
| - synchronize | ||
| - reopened | ||
| - ready_for_review | ||
| paths: | ||
| - ".github/workflows/opencode-review-dispatch.yml" | ||
| - ".github/workflows/opencode-coverage-toolchain-quality-ci.yml" | ||
| - "tests/test_opencode_llvm_coverage_current_main.py" | ||
| - "docs/doctoring/opencode-llvm-coverage-toolchain.md" | ||
| - "CHANGELOG.md" | ||
| - "requirements-opencode-review-ci-hashes.txt" | ||
| - "pyproject.toml" | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| concurrency: | ||
| group: opencode-coverage-toolchain-quality-${{ github.event.pull_request.number }} | ||
| cancel-in-progress: true | ||
|
|
||
| jobs: | ||
| exact-head-contract: | ||
| name: Exact-head toolchain contract | ||
| runs-on: ubuntu-24.04 | ||
| timeout-minutes: 10 | ||
| steps: | ||
| - name: Harden runner | ||
| uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0 | ||
| with: | ||
| egress-policy: audit | ||
|
|
||
| - name: Checkout exact pull-request head | ||
| uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| ref: ${{ github.event.pull_request.head.sha }} | ||
| fetch-depth: 1 | ||
| persist-credentials: false | ||
|
|
||
| - name: Refuse stale or merge-tree evidence | ||
| shell: bash --noprofile --norc -e -o pipefail {0} | ||
| run: | | ||
| test "$(git rev-parse HEAD)" = "${{ github.event.pull_request.head.sha }}" | ||
| test -z "$(git status --short)" | ||
|
|
||
| - name: Execute dependency-free coverage-toolchain contracts | ||
| shell: bash --noprofile --norc -e -o pipefail {0} | ||
| run: | | ||
| python3 - <<'PY' | ||
| import importlib.util | ||
| from pathlib import Path | ||
|
|
||
| test_path = Path("tests/test_opencode_llvm_coverage_current_main.py") | ||
| spec = importlib.util.spec_from_file_location("opencode_toolchain_contract", test_path) | ||
| if spec is None or spec.loader is None: | ||
| raise SystemExit("unable to load the OpenCode toolchain contract module") | ||
| module = importlib.util.module_from_spec(spec) | ||
| spec.loader.exec_module(module) | ||
|
|
||
| tests = sorted( | ||
| name | ||
| for name in vars(module) | ||
| if name.startswith("test_") and callable(getattr(module, name)) | ||
| ) | ||
| if not tests: | ||
| raise SystemExit("no OpenCode toolchain contract tests were discovered") | ||
| for test_name in tests: | ||
| getattr(module, test_name)() | ||
| print(f"PASS {test_name}") | ||
| PY | ||
| python3 -m compileall -q tests/test_opencode_llvm_coverage_current_main.py | ||
| git diff --check | ||
| test -z "$(git status --short)" | ||
|
|
||
| full-repository-quality: | ||
| name: Full repository test, coverage, and docstring gate | ||
| needs: exact-head-contract | ||
| runs-on: ubuntu-24.04 | ||
| timeout-minutes: 30 | ||
| steps: | ||
| - name: Harden runner | ||
| uses: step-security/harden-runner@bf7454d06d71f1098171f2acdf0cd4708d7b5920 # v2.20.0 | ||
| with: | ||
| egress-policy: audit | ||
|
|
||
| - name: Checkout exact pull-request head | ||
| uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| ref: ${{ github.event.pull_request.head.sha }} | ||
| fetch-depth: 1 | ||
| persist-credentials: false | ||
|
|
||
| - name: Refuse stale or merge-tree evidence | ||
| shell: bash --noprofile --norc -e -o pipefail {0} | ||
| run: | | ||
| test "$(git rev-parse HEAD)" = "${{ github.event.pull_request.head.sha }}" | ||
| test -z "$(git status --short)" | ||
|
|
||
| - name: Set up current stable Python | ||
| uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 | ||
| with: | ||
| python-version: "3.14" | ||
| cache: pip | ||
| cache-dependency-path: requirements-opencode-review-ci-hashes.txt | ||
|
|
||
| - name: Install hash-locked repository quality tooling | ||
| shell: bash --noprofile --norc -e -o pipefail {0} | ||
| run: python -m pip install --disable-pip-version-check --require-hashes -r requirements-opencode-review-ci-hashes.txt | ||
|
|
||
| - name: Run complete central test and branch coverage gate | ||
| shell: bash --noprofile --norc -e -o pipefail {0} | ||
| run: | | ||
| python -m coverage erase | ||
| python -m coverage run -m pytest tests -q | ||
| python -m coverage report | ||
|
|
||
| - name: Enforce complete production docstrings | ||
| shell: bash --noprofile --norc -e -o pipefail {0} | ||
| run: python -m interrogate --fail-under 100 scripts/ci | ||
|
|
||
| - name: Compile production and every repository test | ||
| shell: bash --noprofile --norc -e -o pipefail {0} | ||
| run: | | ||
| python -m compileall -q scripts/ci tests | ||
| git diff --check | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| # OpenCode LLVM coverage toolchain decision | ||
|
|
||
| ## Decision | ||
|
|
||
| The central OpenCode coverage image installs Debian Trixie's `llvm-19` package and explicitly exports: | ||
|
|
||
| ```text | ||
| LLVM_COV=/usr/bin/llvm-cov-19 | ||
| LLVM_PROFDATA=/usr/bin/llvm-profdata-19 | ||
| ``` | ||
|
|
||
| The image build fails unless both paths are executable. This is required because the image uses Debian-packaged `rustc` rather than a rustup-managed toolchain, so `llvm-tools-preview` is not an available installation path. | ||
|
|
||
| ## Evidence and compatibility boundary | ||
|
|
||
| `cargo-llvm-cov` documents `LLVM_COV` and `LLVM_PROFDATA` as the overrides to use when a Rust toolchain is installed outside rustup. It also requires the selected tools to be compatible with the LLVM version used by `rustc`. Its published compatibility table maps Rust 1.82–1.95 to LLVM 19–22. The central image therefore selects LLVM 19 as the lowest compatible family for its supported Rust range and keeps the two binary paths explicit rather than relying on an unversioned system default. | ||
|
|
||
| Debian Trixie publishes `llvm-19` from the `llvm-toolchain-19` source package. The workflow installs the package from the pinned Debian image repositories and verifies the exact versioned executable paths during image construction. | ||
|
|
||
| ## Observed regression | ||
|
|
||
| DiskSage pull request 133 exact head `b7f980d265713d5ffb84f744ce454589e3d410ea` passed its repository Test, Release, Security Scan, and SAST workflows. Central OpenCode run `31037491215`, job `92413313900`, then failed before Rust test execution with `failed to find llvm-tools-preview`. The failure reproduced the previously diagnosed central-toolchain defect rather than a DiskSage production-code failure. | ||
|
|
||
| The earlier LLVM repair had been merged into an intermediate feature branch rather than protected `main`; later branch consolidation therefore left the required workflow source without the four toolchain lines. This current-main repair is intentionally limited to restoring those lines, permanent regression contracts, this decision record, the changelog, and the exact-head quality workflow that executes both focused and repository-wide evidence. | ||
|
|
||
| ## Security and reproducibility contract | ||
|
|
||
| - Pull-request content cannot select another LLVM package or executable path. | ||
| - The coverage image definition remains default-branch controlled and is built from immutable workflow source. | ||
| - `LLVM_COV` and `LLVM_PROFDATA` are set together; partial configuration is rejected. | ||
| - Missing executables fail the image build before any pull-request coverage measurement starts. | ||
| - Every low-privilege coverage wrapper disables ambient system and global Git configuration before applying the single bounded `/work` safe-directory overlay. | ||
| - Both quality jobs check out `github.event.pull_request.head.sha`, refuse merge-tree or stale-head evidence, and preserve no repository credentials. | ||
| - The fast contract job installs no packages and evaluates no pull-request-selected dependency manifest. | ||
| - The full repository job installs only the repository's SHA-256 hash-locked quality requirements, then runs every test plus the configured 100% branch coverage and production docstring gates. | ||
| - The image digest, workflow commit SHA, pull-request head SHA, and coverage artifacts remain independently addressable evidence. | ||
| - CPU coverage is a correctness gate. GPU execution and parity tests remain separate domain-specific gates and are not represented by LLVM host coverage alone. | ||
|
|
||
| This design does not claim formal compliance with a software supply-chain standard. It establishes a narrow, auditable compatibility boundary for deterministic Rust coverage execution. | ||
|
|
||
| ## Durable exact-head verification | ||
|
|
||
| `.github/workflows/opencode-coverage-toolchain-quality-ci.yml` is the repository-owned acceptance path for this contract. It runs whenever the trusted coverage workflow, either quality workflow contract, this decision record, the hash-locked quality requirements, `pyproject.toml`, or the changelog changes. | ||
|
|
||
| The first job checks out the exact pull-request head SHA, verifies that Git materialized that SHA rather than GitHub's generated merge revision, discovers every dependency-free `test_` function in the focused contract module, compiles the module, and fails if the test process changes the worktree. It never installs packages. | ||
|
|
||
| Only after that job passes, a separate approved-environment job checks out and revalidates the same exact head, installs the repository-owned hash-locked quality toolchain, runs `pytest` across the complete `tests` directory under the configured 100% branch-coverage gate, enforces 100% production docstrings, and compiles all production CI modules and repository tests. This separation preserves a minimal early fail-closed contract while preventing focused tests from substituting for full repository acceptance. | ||
|
|
||
| ## Regression contract | ||
|
|
||
| The central workflow contract test must continue to prove that: | ||
|
|
||
| 1. `llvm-19` is installed in the coverage image; | ||
| 2. `LLVM_COV` names `/usr/bin/llvm-cov-19`; | ||
| 3. `LLVM_PROFDATA` names `/usr/bin/llvm-profdata-19`; | ||
| 4. the image build checks both paths before installing or invoking `cargo-llvm-cov`; | ||
| 5. the OpenCode approval path remains fail-closed when Rust coverage cannot run; | ||
| 6. all three low-privilege wrapper processes isolate system and global Git configuration before the safe-directory overlay; | ||
| 7. the focused quality job is exact-head bound, credential-free, and dependency-free; and | ||
| 8. the dependent full repository job is exact-head bound, uses the SHA-256 hash-locked quality toolchain, and runs the complete test, branch-coverage, docstring, and compilation gates. | ||
|
|
||
| ## References | ||
|
|
||
| Debian Project. (2026). *Details of package llvm-19 in trixie*. https://packages.debian.org/trixie/amd64/llvm-19 | ||
|
|
||
| Endo, T. (2026). *cargo-llvm-cov: Cargo subcommand to easily use LLVM source-based code coverage* [Computer software]. GitHub. https://github.com/taiki-e/cargo-llvm-cov |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,89 @@ | ||
| from pathlib import Path | ||
|
|
||
|
|
||
| _WORKFLOW = Path(".github/workflows/opencode-review-dispatch.yml") | ||
| _QUALITY_WORKFLOW = Path( | ||
| ".github/workflows/opencode-coverage-toolchain-quality-ci.yml" | ||
| ) | ||
|
|
||
|
|
||
| def test_opencode_coverage_image_provisions_compatible_llvm_tools_before_cargo_llvm_cov(): | ||
| """Require a compatible system LLVM pair before installing cargo-llvm-cov.""" | ||
| workflow = _WORKFLOW.read_text(encoding="utf-8") | ||
|
|
||
| llvm_install = workflow.index(" llvm-19 " + chr(92)) | ||
| llvm_cov_env = workflow.index("ENV LLVM_COV=/usr/bin/llvm-cov-19") | ||
| llvm_profdata_env = workflow.index("ENV LLVM_PROFDATA=/usr/bin/llvm-profdata-19") | ||
| llvm_check = workflow.index( | ||
| 'RUN test -x "$LLVM_COV" && test -x "$LLVM_PROFDATA"' | ||
| ) | ||
| cargo_llvm_cov_install = workflow.index( | ||
| "https://github.com/taiki-e/cargo-llvm-cov/releases/download/" | ||
| ) | ||
|
|
||
| assert llvm_install < llvm_cov_env < llvm_check < cargo_llvm_cov_install | ||
| assert llvm_install < llvm_profdata_env < llvm_check | ||
|
|
||
|
|
||
| def test_low_privilege_coverage_wrappers_isolate_ambient_git_configuration(): | ||
| """Require system and global Git isolation before the safe-directory overlay.""" | ||
| workflow = _WORKFLOW.read_text(encoding="utf-8") | ||
| boundaries = ( | ||
| ("run_and_capture", "run_r_package_testthat"), | ||
| ("run_r_package_testthat", "run_and_capture_advisory"), | ||
| ("run_and_capture_advisory", "trusted_git"), | ||
| ) | ||
|
|
||
| for wrapper_name, next_name in boundaries: | ||
| start = workflow.index(f" {wrapper_name}() {{") | ||
| end = workflow.index(f" {next_name}() {{", start) | ||
| wrapper = workflow[start:end] | ||
|
|
||
| no_system = wrapper.index("GIT_CONFIG_NOSYSTEM=1") | ||
| no_global = wrapper.index("GIT_CONFIG_GLOBAL=/dev/null") | ||
| safe_directory_count = wrapper.index("GIT_CONFIG_COUNT=1") | ||
|
|
||
| assert no_system < no_global < safe_directory_count | ||
| assert wrapper.count("GIT_CONFIG_NOSYSTEM=1") == 1 | ||
| assert wrapper.count("GIT_CONFIG_GLOBAL=/dev/null") == 1 | ||
|
|
||
|
|
||
| def test_opencode_toolchain_quality_workflow_is_exact_head_bound_and_offline(): | ||
| """Require the fast contract job to stay exact-head and dependency-free.""" | ||
| workflow = _QUALITY_WORKFLOW.read_text(encoding="utf-8") | ||
| start = workflow.index(" exact-head-contract:") | ||
| end = workflow.index(" full-repository-quality:", start) | ||
| fast_job = workflow[start:end] | ||
|
|
||
| assert "ref: ${{ github.event.pull_request.head.sha }}" in fast_job | ||
| assert "persist-credentials: false" in fast_job | ||
| assert ( | ||
| 'test "$(git rev-parse HEAD)" = ' | ||
| '"${{ github.event.pull_request.head.sha }}"' in fast_job | ||
| ) | ||
| assert "importlib.util.spec_from_file_location" in fast_job | ||
| assert "python3 -m compileall -q" in fast_job | ||
| assert "pip install" not in fast_job | ||
| assert "uv sync" not in fast_job | ||
|
|
||
|
|
||
| def test_opencode_toolchain_quality_runs_full_hash_locked_repository_suite(): | ||
| """Require a separate exact-head full-suite job with 100% quality gates.""" | ||
| workflow = _QUALITY_WORKFLOW.read_text(encoding="utf-8") | ||
| start = workflow.index(" full-repository-quality:") | ||
| full_job = workflow[start:] | ||
|
|
||
| assert "needs: exact-head-contract" in full_job | ||
| assert "ref: ${{ github.event.pull_request.head.sha }}" in full_job | ||
| assert "persist-credentials: false" in full_job | ||
| assert ( | ||
| 'test "$(git rev-parse HEAD)" = ' | ||
| '"${{ github.event.pull_request.head.sha }}"' in full_job | ||
| ) | ||
| assert "actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97" in full_job | ||
| assert 'python-version: "3.14"' in full_job | ||
| assert "--require-hashes -r requirements-opencode-review-ci-hashes.txt" in full_job | ||
| assert "python -m coverage run -m pytest tests -q" in full_job | ||
| assert "python -m coverage report" in full_job | ||
| assert "python -m interrogate --fail-under 100 scripts/ci" in full_job | ||
| assert "python -m compileall -q scripts/ci tests" in full_job |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.