Hit during the v1.15.1 release gate on miner-3 (2026-08-15). Filing rather than re-running to green, per the
#831 practice of fixing a flake at the harness.
What happened
e2e-real.sh verify failed with:
✗ doctor didn't confirm the MSR mod applied (#66)
✗ doctor's rdmsr register verification did not pass (#66) — see the doctor output above
E2E-REAL (verify): FAIL — 2 failed, 36 passed.
An immediate re-run of the same phase, same rig, same commit, no intervention: PASS — 38 check(s) passed,
with both MSR lines present. So it is intermittent, not a real MSR fault — the rig was healthy throughout
(doctor run by hand reports MSR mod applied — XMRig set the 'ryzen_17h' preset and
MSR registers verified via rdmsr (4/4 match)).
What the evidence shows
In the failing run doctor's output goes straight from msr kernel module loaded to CPU governor — the
entire MSR block is absent, not failing:
✓ kernel lockdown: none — MSR writes are permitted
✓ msr kernel module loaded
✓ CPU governor = performance <- the two MSR lines should be between these
That block is guarded by [ -n "$log_file" ] && [ -f "$log_file" ] (rigforge.sh, in doctor, around the
"Resolve the worker's xmrig.log once" comment), where log_file="$wr/xmrig.log" and
wr=$(_worker_root_from_config). So on the failing run that guard was false — either wr came back empty
or the log path didn't resolve. _worker_root_from_config returns empty when $CONFIG_JSON is missing.
Ruled out:
I could not pin the trigger before the state was gone, hence this issue rather than a speculative patch.
Why it matters
Both assertions are bad (hard-fail) and they abort the release gate, so a transient makes the gate lie
about a healthy rig — and the natural reaction is to re-run until green, which is exactly how a real MSR
regression would get waved through.
Suggested direction
- Have the verify phase assert the guard's inputs first (resolve + report the log path) so the failure
says "couldn't resolve xmrig.log" instead of "MSR mod not applied" — a skipped block and a failed check
should not look identical.
- Consider making
doctor distinguish "MSR unverifiable (no log)" from "MSR not applied"; today the e2e
can only grep for the success string, so absence and failure are the same signal.
Incidental
xmrig.log on miner-3 is 122 MB (with a 121 MB .1 beside it) and _msr_log_status awk-scans the whole
file on every doctor. minsize 50M + daily rotation is letting it run well past the threshold. Tailing the
scan (the MSR line is written at miner start, and only the last one matters) would make doctor cheaper on
every invocation.
Hit during the v1.15.1 release gate on miner-3 (2026-08-15). Filing rather than re-running to green, per the
#831 practice of fixing a flake at the harness.
What happened
e2e-real.sh verifyfailed with:An immediate re-run of the same phase, same rig, same commit, no intervention: PASS — 38 check(s) passed,
with both MSR lines present. So it is intermittent, not a real MSR fault — the rig was healthy throughout
(
doctorrun by hand reportsMSR mod applied — XMRig set the 'ryzen_17h' presetandMSR registers verified via rdmsr (4/4 match)).What the evidence shows
In the failing run
doctor's output goes straight frommsr kernel module loadedtoCPU governor— theentire MSR block is absent, not failing:
That block is guarded by
[ -n "$log_file" ] && [ -f "$log_file" ](rigforge.sh, indoctor, around the"Resolve the worker's xmrig.log once" comment), where
log_file="$wr/xmrig.log"andwr=$(_worker_root_from_config). So on the failing run that guard was false — eitherwrcame back emptyor the log path didn't resolve.
_worker_root_from_configreturns empty when$CONFIG_JSONis missing.Ruled out:
msr register values for "ryzen_17h" preset have been set successfullyat 23:04:52; verify's doctor ran at 23:06:03._msr_log_statusawk-scans for the LASTmatch, so the data was there.
/etc/logrotate.d/xmrigusescopytruncate, soxmrig.logis never absent.log_fileis derived from_worker_root_from_configparsing
config.json, nothing to do with the API readers that PR touched.I could not pin the trigger before the state was gone, hence this issue rather than a speculative patch.
Why it matters
Both assertions are
bad(hard-fail) and they abort the release gate, so a transient makes the gate lieabout a healthy rig — and the natural reaction is to re-run until green, which is exactly how a real MSR
regression would get waved through.
Suggested direction
says "couldn't resolve xmrig.log" instead of "MSR mod not applied" — a skipped block and a failed check
should not look identical.
doctordistinguish "MSR unverifiable (no log)" from "MSR not applied"; today the e2ecan only grep for the success string, so absence and failure are the same signal.
Incidental
xmrig.logon miner-3 is 122 MB (with a 121 MB.1beside it) and_msr_log_statusawk-scans the wholefile on every
doctor.minsize 50M+ daily rotation is letting it run well past the threshold. Tailing thescan (the MSR line is written at miner start, and only the last one matters) would make
doctorcheaper onevery invocation.