Skip to content

fix(engine): converge gate-verdict + finding-severity calibration onto reviewer-consensus's zero-weight + malformed-repo handling#6234

Closed
real-venus wants to merge 1 commit into
JSONbored:mainfrom
real-venus:fix/engine-calibration-converge-zero-weight-malformed-repo
Closed

fix(engine): converge gate-verdict + finding-severity calibration onto reviewer-consensus's zero-weight + malformed-repo handling#6234
real-venus wants to merge 1 commit into
JSONbored:mainfrom
real-venus:fix/engine-calibration-converge-zero-weight-malformed-repo

Conversation

@real-venus

Copy link
Copy Markdown
Contributor

Summary

Three sibling calibration modules share a composite-scorer pattern, but the two older ones diverged from the newest/correct reviewer-consensus-calibration.ts in two ways. This converges them:

  1. Zero-weight handling (gate-verdict-calibration.ts, finding-severity-calibration.ts): both did if (total <= 0) return DEFAULT_COMPOSITE_WEIGHTS, silently restoring the 45/35/20 blend when a caller explicitly zeroes every component. They now return the explicit zeroed weights, so an all-zero config reaches the objective-only fallback in the composite scorer -- matching reviewer-consensus.
  2. Malformed-repo preservation (gate-verdict-calibration.ts): the rejected-row sanitizer used normalizeRepoFullName(row.repoFullName) alone, which returns null for an already-rejected invalid_repo row and drops it. It now uses normalizeRepoFullName(...) ?? normalizeId(...) (matching the other two files), preserving the raw string.

reviewer-consensus-calibration.ts is unchanged -- it is the reference for both fixes.

Tests

  • Engine node:test regressions added to packages/loopover-engine/test/gate-verdict-calibration.test.ts (zero-weight + invalid_repo-survives) and .../finding-severity-calibration.test.ts (zero-weight), mirroring the equivalents reviewer-consensus-calibration.test.ts already has.
  • A root vitest suite (test/unit/engine-calibration-convergence.test.ts) exercises the same changed branches. The calibration modules are otherwise only node:test-covered (which does not feed Codecov), so this vitest suite is what covers the changed engine-src lines for codecov/patch -- including all three branches of the rewritten malformed-repo line (valid repo, invalid bad, non-string).

Validation

Safety

  • No secrets/private terms. Behavior change is scoped to malformed/all-zero input, converging two calibrators onto the already-correct third; legitimate-input scoring is unchanged.

Closes #6170

…o reviewer-consensus's zero-weight + malformed-repo handling
@real-venus real-venus requested a review from JSONbored as a code owner July 15, 2026 23:26
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 15, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 66.66667% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 95.59%. Comparing base (2af2da3) to head (61ad4cd).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
...oopover-engine/src/finding-severity-calibration.ts 0.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6234      +/-   ##
==========================================
+ Coverage   95.35%   95.59%   +0.23%     
==========================================
  Files         598      598              
  Lines       47102    47102              
  Branches    14988    14988              
==========================================
+ Hits        44913    45026     +113     
+ Misses       1464     1290     -174     
- Partials      725      786      +61     
Flag Coverage Δ
shard-1 44.02% <0.00%> (ø)
shard-2 36.67% <0.00%> (+0.01%) ⬆️
shard-3 32.44% <0.00%> (+0.21%) ⬆️
shard-4 34.51% <66.66%> (+0.10%) ⬆️
shard-5 31.04% <0.00%> (-0.58%) ⬇️
shard-6 45.17% <0.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...es/loopover-engine/src/gate-verdict-calibration.ts 39.05% <100.00%> (+36.68%) ⬆️
...oopover-engine/src/finding-severity-calibration.ts 30.38% <0.00%> (+28.17%) ⬆️

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 15, 2026
@loopover-orb

loopover-orb Bot commented Jul 15, 2026

Copy link
Copy Markdown

Caution

🛑 LoopOver review result - fixes required

Review updated: 2026-07-15 23:39:47 UTC

5 files · 1 AI reviewer · no blockers · CI failing · unknown

🛑 Suggested Action - Fix Blockers

Review summary
This PR converges gate-verdict-calibration.ts and finding-severity-calibration.ts with the already-correct reviewer-consensus-calibration.ts on two points: (1) explicit all-zero composite weights now return the zeroed object instead of silently substituting DEFAULT_COMPOSITE_WEIGHTS, and (2) the rejected-row sanitizer now falls back to normalizeId when normalizeRepoFullName returns null for an already-invalid repo string, preserving the row instead of dropping it. Both changes are narrow, mirror the reference module's existing pattern exactly, and are backed by matching node:test (engine) and vitest (root, Codecov-visible) assertions that exercise the zero-weight fallback and all three ternary branches (valid repo, invalid-but-string repo, non-string repo) of the rejected-row fix. The diff is internally consistent and the added tests actually assert on the new behavior rather than fabricating unreachable states.

Nits — 4 non-blocking
  • The root vitest suite (test/unit/engine-calibration-convergence.test.ts) duplicates almost verbatim the two node:test additions in the engine package; understandable given the Codecov/node:test gap noted in the description, but worth a one-line comment in one of the two locations pointing at the other so future readers don't assume drift is intentional.
  • gate-verdict-calibration.ts:286 line is now quite long with the nested ternary/`??`; consider extracting a small `resolveRepoFullName(value)` helper shared across the three calibration files to avoid re-copying this exact expression a fourth time if a new calibration module is added.
  • Consider factoring the shared `normalizeRepoFullName(...) ?? normalizeId(...)` and zero-weight-preservation logic into a common helper used by all three calibration modules, since the PR's own premise is that these three files should not diverge again.
  • Link/verify the fix(engine): backport reviewer-consensus-calibration.ts's zero-weight and malformed-repo handling to its two older siblings #6170 issue reference resolves the tracked bug so the issue can be closed by this PR.

CI checks failing

  • codecov/patch — 66.66% of diff hit (target 99.00%)

Decision drivers

  • ✅ Code review — No blockers (1 reviewer)
  • ✅ Gate result — Passing (No configured blocker found.)
Context & advisory signals — never blocks the verdict
Signal Result Evidence
Linked issue ✅ Linked #6170
Related work ⚠️ 2 scoped overlaps Top overlaps are listed below; lower-confidence bulk is hidden.
Change scope ❌ 8/20 High review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 139 registered-repo PR(s), 75 merged, 21 issue(s).
Contributor context ✅ Confirmed Gittensor contributor real-venus; Gittensor profile; 139 PR(s), 21 issue(s).
Improvement ✅ Minor risk: clean · value: minor · LLM: moderate
Linked issue satisfaction

Addressed
The diff makes both required source fixes exactly as specified: gate-verdict-calibration.ts's zero-weight branch now returns explicit zeroed weights instead of DEFAULT_COMPOSITE_WEIGHTS and its sanitizer uses normalizeRepoFullName(...) ?? normalizeId(...), and finding-severity-calibration.ts gets the same zero-weight fix; reviewer-consensus-calibration.ts is untouched as required. It also adds the

Review context
Contributor next steps
  • Start here: Review top overlaps.
  • Then work through the remaining 3 steps in the Signals table above.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @loopover ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @loopover chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://gittensory.aethereal.dev/docs/loopover-commands

🧪 Experimental — new and may change.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@loopover-orb

loopover-orb Bot commented Jul 15, 2026

Copy link
Copy Markdown

LoopOver is closing this pull request on the maintainer's behalf (CI is failing (codecov/patch)). This is an automated maintenance action — to pursue this change, please open a new pull request with the issues resolved. Closed PRs may be analyzed later to improve review accuracy, but they are not automatically reopened or re-reviewed.

@loopover-orb loopover-orb Bot closed this Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(engine): backport reviewer-consensus-calibration.ts's zero-weight and malformed-repo handling to its two older siblings

1 participant