ci: trial test-impact reporting without skipping tests

[Abiorh001]

Change

Report-only backend test-impact trial. No tests are skipped or selected.

Goal and planning context

Learn whether existing static dependencies provide useful impact candidates
before considering selective CI. Bounded record.
No automatic follow-up series: further work requires demonstrated practical value.

What changed and design

• Reuse the Git target helpers, module registry, import reader and lane catalogue.
• Report reverse-transitive candidates separately from conservative full-suite
  recommendations and explicit uncertainty reasons.
• Add one advisory JSON-report step to existing preflight. No new service or dependency.
• Preserve merged feat: validate exact contribution policy selections before guide binding #409's three CON owners and both test modules. The single merge
  conflict was the partition digest; recompute it for all 324 registered targets.

Scope and CI integrity

No product code, permissions, required jobs, test commands, coverage thresholds
or retry-artifact behavior changed. Only the new advisory step tolerates failure.
No tests removed, skipped or weakened. docs/roadmap_status.md was assessed;
no product capability, exposure or dependency changed, so no roadmap edit is needed.

Evidence

Head: 73bcacacf2b35db1e976769d430633e2e35488e8.
Base: a3e4c6969bd25bc6168c6dc46c073071ee8d4bcb (merged #409).

• 55 focused tests; Ruff, partition validation, Commitrail, links and stale scan pass.
• Backend run 34902969147
  passed seven lanes, preflight and aggregate. Agent Gates passed.
• Native evidence: 6,655/6,655 completed, zero skipped/deselected,
  94.50% global coverage. Manifest/completion multisets and evidence hashes reconcile.
• Tested merge 27bf7863 has the exact base/head parents and the same tree as
  this head (2ba4c7625cebf7cef632f7917c783350851fa846).
• Reporter succeeded in four seconds and recommends all 311 test modules.
  Full Backend wall time was 878.574 seconds; the timing target was not met.

Reviews

Earlier source reviews passed; reporter, workflow and tests are byte-identical.
The focused merge replay verifies CI/docs and test preservation on this head.
Its discriminating probe removes a CP06 owner and recomputes the digest: the
validator still rejects the deletion. Original missing-transitive-edge and
missing-required-preflight probes remain applicable to unchanged source.
/root/ci_repair_ci_docs closes CI integrity, documentation and focused
QA/test-delta merge preservation as PASS on this head after inspecting the
native hosted bundles, hashes and exact completed-node inventory. Review summaries
are advisory; private review receipts remain out of Git.

CodeRabbit completed a fresh substantive review of this exact head with no
actionable comments. No unresolved review threads.

Remaining risks and human focus

Shared imports, dynamic helpers and unmapped paths currently force full-suite
recommendations. Candidate lists are diagnostic, not safe-omission proof.
This does not solve CI runtime. Do not expand it unless observations justify
the cost. Inspect that report output cannot control test execution.

Human merge ownership

• The human understands the change and accepts its remaining risks.
• The human approves this specific PR for merge.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: b3bab284-2f54-4eec-acb9-77046cc0c036

📥 Commits

Reviewing files that changed from the base of the PR and between a3e4c69 and 73bcaca.

📒 Files selected for processing (8)

• .ci/behavior-ownership/partition.v1.json
• .commitrail/changes/ci-impact-shadow.md
• .github/workflows/backend.yml
• backend/scripts/behavior_ownership.py
• backend/scripts/test_impact_shadow.py
• backend/scripts/test_lane_catalogue.py
• backend/tests/test_test_impact_shadow.py
• docs/operations_backend_testing.md

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.

---

📝 Walkthrough

Walkthrough

The change adds a report-only backend test-impact analyser. It computes candidate tests from Git changes and static dependencies, falls back to the full suite when uncertain, runs as a non-blocking CI step, and preserves existing required lanes and coverage checks.

Changes

Test-impact shadow reporting

Layer / File(s)	Summary
Impact report engine backend/scripts/test_impact_shadow.py	Adds ownership detection, reverse-transitive dependency analysis, candidate test mapping, conservative fallback reasons, and JSON report generation.
CLI and CI integration backend/scripts/test_impact_shadow.py, .github/workflows/backend.yml	Validates the checked-out head and clean state, compares the requested Git range, reports errors visibly, and runs the analyser as a non-blocking workflow step.
Catalogue and ownership validation backend/scripts/test_lane_catalogue.py, backend/scripts/behavior_ownership.py, .ci/behavior-ownership/partition.v1.json	Registers the new focused test in shared foundation lanes and registers the analyser in the approved ownership partition.
Behavior tests and operational record backend/tests/test_test_impact_shadow.py, docs/operations_backend_testing.md, .commitrail/changes/ci-impact-shadow.md	Adds isolated repository, dependency, fallback, Git, and partition tests. Documents operation and records the trial scope and evidence.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant BackendWorkflow
  participant ImpactShadow
  participant Git
  participant ImportHelpers
  BackendWorkflow->>ImpactShadow: Run with base and head
  ImpactShadow->>Git: Resolve commits and changed paths
  ImpactShadow->>ImportHelpers: Load registry and static imports
  ImportHelpers-->>ImpactShadow: Owners and dependencies
  ImpactShadow-->>BackendWorkflow: Emit JSON recommendation

Loading

Suggested reviewers: durutheguru

Merge Risk: ⚪ Minimal · up to 73bca

The report step has the required Git history and handles expected helper failures with its diagnostic JSON output. No merge-blocking risk is established.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 41.18% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 17 functions across 4 files. (4 skipped: …	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly and concisely describes the main change: adding report-only test-impact reporting without changing test selection.
Description check	✅ Passed	The description is substantially complete and explains the goal, design, scope, CI integrity, evidence, reviews, remaining risks, and human merge ownership. It uses combined headings instead of severa…

Full details: Docstring Coverage

Explanation

Docstring coverage is 41.18% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 17 functions across 4 files. (4 skipped: 4 unsupported.)

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/ci-impact-shadow

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}