The paired MDE on manual_gold is 0.0063, measured -- and Run A's curve declines after epoch 6 (#135) - #138
The paired MDE on manual_gold is 0.0063, measured -- and Run A's curve declines after epoch 6 (#135)#138jonfroehlich wants to merge 9 commits into
Conversation
…does not help (#135) Answers the prerequisite #135 put before the 1,675-3,350 GPU-hour spend, from committed data alone: no cluster time, no .model_cache, no GPU, no network. Two results decide the question. Pairing is worth 2-3x and it is free. Both checkpoints are scored on the same 1,000 panoramas against the same ground truth, so the pano-to-pano difficulty that dominates the unpaired noise cancels. Measured on manual_gold, the s.e. of a difference drops 1.8-3.6x, and the MDE on max-F1 goes from 0.0117 unpaired to a bracket of 0.0043-0.0135 paired. The quantity that governs it is the discordance between the two detectors, not n*p*(1-p), which is why the 0.01 tie bar -- an unpaired bar -- is the wrong instrument for an epoch-vs-epoch comparison. Pooling is worth 7%. Ten splits hold 6,560 instances to manual_gold's 3,919 and the MDE moves 0.0117 -> 0.0109; the nine city splits alone are worse than manual_gold alone (0.0219). Not a lever, and it would mix an independently-labeled GT with nine RampNet-anchored ones. Two things fall out along the way. Run A's plateau, re-read paired, is real on capability and not on calibration. Epochs 2 vs 6 are identical on max-F1 to four decimals and 3 vs 6 is unreadable even at the favourable end of the bracket -- so the #84 finding survives a sharper instrument. But the recall gaps at the fixed 0.30 point need 20-34% discordance to be unreadable against 2.1-9.1% observed, so the operating-point drift #84 described is measured, not hedged: the epoch-3 preference under the recall-first stance is a result rather than a judgment call. The 0.008 the tie bar was derived from is sqrt(0.25/3919), the binomial worst case at p=0.5. At RampNet's actual recall it is 0.0048, and the measured clustered value is 0.0055 -- so the bar was conservative by ~45%, and lands near the right unpaired answer for the wrong reason. Panorama clustering is real: design effect 1.23-1.88, which is why the bootstrap resamples panos and not instances. Instrument check, committed rather than noted: the RampNet-vs-RampNet pair straddles records.jsonl and op_cache, and #132's exclude_border artifact puts 30% of its disagreements at the seam against a 2.1% baseline. The two pairs drawn from one source read the baseline exactly, which is the control saying the diagnostic does not fire spuriously. The limit that actually binds is not the benchmark: this is panorama sampling variance only, and Run B is n=1. Nothing here or anywhere in the repo measures training-seed variance, so a gain below ~0.01 max-F1 would be measurable but not attributable to the anneal. Recorded next to the number, with the recommendation. Verified: metrics() under all-ones weights reproduces detection_eval.aggregate to 1e-12 on four (split, model) combinations spanning both GT schemes and both detection sources. Suite 1234 passed, 1 skipped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…#135) The #135 write-up bracketed the paired MDE at [0.0043, 0.0135] because the epoch-to-epoch discordance of two RampNet checkpoints has never been measured -- only bounded -- and said closing it would cost about a GPU-hour. It costs none. The 2026-08-17 scoring cache survived on makelab2 intact: 13 GB at run_a_84/evaluate_cache/heatmaps/, all EIGHT epochs single-pass, 1,000 panos each. Peak extraction from a cached heatmap is CPU-only numpy, so the measurement needs no model, no panorama images (they are not on that host anyway), no GPU and no network -- and it covers the whole curve rather than the two checkpoints the doc proposed. A separate script rather than a flag on stage_two/evaluate.py, deliberately: that evaluator produced every committed Stage 2 number and its heatmap cache key is <fingerprint>_<dataset>_<tta> and nothing else, so the cheapest way to guarantee this analysis cannot perturb either is to not touch it. What it does share is the part that must not diverge -- extract_peaks_from_heatmap, PEAK_MIN_DISTANCE and MODEL_HEATMAP_SIZE are imported from it, not copied. --threshold truncates the tail on purpose. Run A scored at --threshold 0.0, which keeps every local maximum: ~511,000 predictions over 1,000 panos, nearly all noise floor. The 0.05 default matches op_cache's floor, keeps files ~200 KB instead of ~40 MB, and sits far below everything this is used for (protocol point 0.30; Run A max-F1 lands at 0.268-0.582). The cost is that AP is not recoverable below it, which is stated in the file and beside the output. --verify re-scores each dump against manual_labels and checks it reproduces docs/data/run_a_84_manual_gold/summary.csv. That is the load-bearing part: it is what proves the dump is the same instrument that produced the committed curve rather than a second, similar one. Output lands in the published-detections shape, so benchmark_power_135.py and every other reader of benchmark/model_detections/ takes it with no special case. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…135) The write-up could only BRACKET the paired standard error for two Stage 2 checkpoints at [0.0043, 0.0135], because Run A's committed artifacts are aggregate PR curves and carry nothing per-panorama. That bracket is now a measurement, and it cost no GPU: the 2026-08-17 scoring cache survived on makelab2 with all eight epochs at 1,000 panos each, so dump_peaks_from_cache.py recovered the detections in 4 minutes with no model, no images and no network. MEASURED across all 28 pairs of Run A checkpoints on manual_gold: se(d max-F1) 0.0016-0.0029, median 0.0021; discordance 2.5-6.6%. For pairs >= 3 epochs apart -- the closest analogue to Run B vs Run A -- median se 0.0022, i.e. MDE 0.0063 at 80% power against the unpaired bar's 0.0117. The old bracket contained it and the working assumption (4-6% discordance => 0.006-0.009) was nearly exact. s.e. grows with epoch separation, 0.0016 at gap 1 to 0.0029 at gap 7, so a Run B checkpoint -- further from any Run A checkpoint than any pair here -- should be planned at se ~0.003, MDE ~0.008. That is an extrapolation of a measured trend and is flagged as one. AND IT CORRECTS #84's CURVE SHAPE. Each pair now has its own measured s.e. instead of one global bracket, and epochs 7 and 8 are measurably BELOW the plateau: 3 vs 7 at z=3.9, 3 vs 8 at 3.0, 6 vs 7 at 2.8. The unpaired 0.01 bar could not see it. So the curve is not "steps up once from 1 to 2 then flat" -- it is epoch 1 low, epochs 2-6 a genuine plateau (2 vs 6 identical to 4 dp, 3 vs 6 at z=1.4), then a measurable decline. A shallow inverted U. The headline is unchanged -- still no resolvable peak, 2-6 remain mutually indistinguishable -- but at constant LR the model does begin to LOSE capability after ~epoch 6. That sharpens what Run B is testing: not "does a longer budget add to a flat curve" but "does cosine decay arrest a decline that is now measured". It cuts both ways -- that decline is the classic signature of an LR left too high, which is what an anneal fixes; but a 30-epoch run whose decay does not bite could land BELOW epoch 3. It also makes the #84 amendment's 8-epoch cosine rung (~450 GPU-h, budget-matched) more attractive than it was, because B-A does not attribute on its own. Verification is the load-bearing part: re-scoring each dump against manual_labels reproduces the committed summary.csv max-F1 to 5e-9..4e-7 across all eight epochs. max-F1 is a property of the whole PR curve, so agreement at 1e-7 says the peak extraction is the same operation, not a similar one. The uniform 1.2-1.4e-4 offset on F1@0.30 is the committed table's own 0.005-grid downsampling, which its provenance note already documents. The dumps land in docs/data/run_a_84_detections/, NOT benchmark/model_detections/: roster.py asserts every file there belongs to a registered challenger leg (#122), and Run A's epochs are internal checkpoints, not entries in the RampNet-vs-VLM comparison. The suite caught the first attempt to put them there -- the registry working as intended. They are committed because the 13 GB cache they came from cannot be, which is stated beside them as the one input a clean clone cannot get. Suite 1236 passed, 1 skipped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
jonfroehlich
changed the title
Deep reviewRe-ran everything from the branch head (
So everything below is a labeling/interpretation problem sitting on top of an artifact that is exactly what the script produces. One of them makes a headline number wrong by ~2×. 1. High — the nine city bundles are hard-truncated at conf 0.55, so "the #54 operating point of 0.30" is not what was measured on 9 of 10 splits
Four consequences:
The repo already documents the truncation at Measured impact, re-running with
"Pooling is worth 7%" is understated by ~2× — it is ~14%. The conclusion holds (14% is still not a lever; POOLED cities alone stays far worse than manual_gold alone), but the number in the summary table and the PR body is wrong. Caveat on that check: The headline paired numbers are all 2. Medium — the "LOWER BOUND" section cannot bound what it claims
That matters because the module docstring assigns this section exactly that job ("…so they give the lower bound"). For max-F1 — the metric the headline bracket and the whole recommendation are stated in — it cannot. No published number is wrong (the real 0.0015 comes from 3. Medium — the doc's own reproducibility claim fails on its headline numberLine 281: "Every derived number in this document is in It errs safe (interpolating the three committed points to 4–6% gives about 0.0056–0.0068), but per the repo's replication rule it should be computed by the script, and that sentence should not stand as written. Related: the stated mechanism "the standard error tracks the discordance" does not hold on its own three points — 8.1% gives 0.0029 but 9.1% gives 0.0048, a 12% rise in discordance against a 66% rise in s.e. For max-F1 there is no McNemar form, as the doc itself says. 4. Medium-low — the summary table mixes F1 and max-F1 standard errors under a max-F1 headerThe headline table (and the PR body verbatim) is headed "s.e. on Numerically immaterial, but it is the one table people will quote, and "0.01 is very nearly the correct unpaired MDE of 0.0117" is stated about the wrong column. (The separate "1.8×–3.6× gain" claim is like-for-like — both sides F1.) 5. Medium-low — the #84 correction is not discoverable from #84's own docThe PR's best secondary result is that the epoch-3 preference is now a measured result rather than a judgment call. 6. One statistical cautionThe load-bearing assumption is hedged in one place and dropped in another. The max-F1 bracket is carefully "bracketed rather than point-estimated." But the recall table prints a flat "resolvable" for gaps needing 20–34% discordance, justified by "real pairs on this benchmark run 2.1–9.1%" — where those pairs are two cross-architecture YOLO comparisons and one seam-contaminated self-pair, none of them epoch-vs-epoch. Run A's own data suggests epoch-to-epoch calibration is unstable ( 7. Low
What holds upStating this because a defect list on its own misrepresents the PR:
The irony worth naming: finding 1 is exactly what "verify the instrument before judging" exists to catch, and the evidence was sitting in this PR's own committed JSON as 18 rows of Fixes for 1–7 to follow on this branch. 🤖 Generated with Claude Code (claude-opus-5[1m]) |
…safe schedule (#135) Run A measured that constant LR does not merely flatten -- epochs 7-8 are measurably BELOW the plateau (3v7 z=3.9, 3v8 z=3.0, 6v7 z=2.8). Drifting back up after settling is the signature of a learning rate left too high to settle with, which is what a decay fixes. So there is now something for an 8-epoch anneal to act on; before that measurement the curve looked flat and this rung would have had nothing to demonstrate against. It is also the only comparison that ATTRIBUTES. Run B is 30-60 epochs AND cosine, so B - A confounds length with schedule -- the #84 amendment says so explicitly. This rung holds the budget at Run A's 8 epochs and changes only the schedule, at ~480 GPU-h and ~34 h calendar against Run B's ~1,800 and ~5-7 days. It is not a substitute: an 8-epoch cosine decays fast and cannot show a benefit that needs length. It answers the narrower question for a quarter of the cost. THE SCHEDULE IS STATELESS AND THAT IS THE DESIGN. train.py's new lr_at_step computes the rate from the absolute step index, not from a scheduler object. Stage 2 runs on preemptible ckpt-all and resumes from latest_checkpoint.pth (Run A was requeued twice); a stateful scheduler restarts its decay from the peak on every requeue unless its state is ALSO serialized, turning a cosine into a sawtooth. That failure is silent -- the job completes and the loss curve looks plausible. Reading global_step, which is already checkpointed and already broadcast to every rank, makes it impossible by construction instead of by remembering one more field. tests/test_train_lr_schedule.py pins it by simulating a run chopped into segments by requeues and asserting the LR sequence matches an uninterrupted one. train.py now logs per-step LR to tensorboard, because a sawtooth is invisible in the loss curve and obvious there. Read that scalar before trusting the run. Default stays --lr-schedule constant, so a re-run of Run A or the paper recipe is unaffected; a test asserts that default rather than trusting it. No warmup: it would be a second change, and Run A was stable from step 0. The decision rule is pre-registered BEFORE launch, in docs/stage2_cosine_rung_135.md -- primary test (cosine ep8 vs Run A ep8, paired, max-F1, each pair against its own bootstrap s.e.), secondary mechanism test (is Run A's own ep3->ep8 decline arrested?), and the gate on whether Run B follows. A tie on both is explicitly NOT an automatic cancellation; it is a judgment call, recorded as one rather than dressed as a rule. What it cannot settle is stated in the same file: seed variance is unmeasured and this is n=1, so a difference below ~0.01 max-F1 is measured but not attributable. Suite 1243 passed, 1 skipped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…not assumed (#135) klone's train.py was byte-identical to main before the upload (ccce228b), so what runs is main plus the scheduler and nothing else -- its post-upload hash 051e3256 matches this branch exactly. The launcher landed as ASCII, not CRLF, which would have died with a bad-interpreter error. Dataset counted rather than stat'ed (300,126 entries under train/ = 150,063 pairs = Run A's 9,378 steps/epoch at world size 16), since the directory predates scrubbed's purge window. 270 GB free on /gscratch/makelab for a ~10 GB run. Env reused from Run A's durable copy, so the environment is identical to the arm being compared against. Fresh start confirmed at submit: no latest_checkpoint.pth in the new run dir, which is deliberately NOT Run A's -- pointing it there would have resumed Run A's weights and overwritten its artefacts. Recorded the first thing to check once it starts: the per-step LR scalar must fall smoothly from 1e-5. A sawtooth means a requeue reset the schedule, and that is invisible in the loss curve. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The cosine rung's whole premise is that `lr_at_step` is stateless, so a preemption cannot restart the decay from the peak. That was argued in a docstring and covered by a unit test of the *formula*; nothing checked what the running job actually applied after a resume. The failure mode is silent -- a sawtooth job completes and its loss curve looks fine -- so "it should be correct by construction" is exactly the claim that needs an instrument pointed at it. `scripts/analysis/check_lr_schedule_135.py` reads the per-step LR scalar train.py logs and asks four things a mis-resume would fail: is the merged series monotone, do overlapping incarnations agree on the same step index, what is the rate at each resume as a ratio of the peak (1.000000 is the sawtooth signature), and does every value match `lr_at_step` at that step. It reuses `stage2_train_cost.read_scalars` via `stage2_epoch_curve.read_scalars_by_file` -- stdlib only, no tensorboard install, and the per-file split is what makes incarnations visible at all. Two things worth naming: - **The off-by-one is real.** train.py computes `lr_at_step(n)` then logs it under step `n + 1`. Reading it without that shift makes a correct run look wrong by one step's worth of decay, so the script names a whole-series shift as a convention change rather than reporting it as a broken schedule. - **The lift out of train.py now has one home.** `tests/test_train_lr_schedule.py` had its own AST copy; it imports the script's `load_schedule` instead. That test covers the formula, this script covers where the formula was applied. The tests are mostly negative, on purpose: a checker that passed vacuously would be worse than none, because a PASS would be read as evidence. `test_a_sawtooth_resume_ fails` builds the exact defect and asserts it is caught -- note the teeth are only 0.7% deep 4,000 steps into a 75,024-step cosine, which is why this is a ratio and a monotonicity test rather than an eyeball. Validated against job 38640313's first incarnation (4,003 steps): 0 non-decreasing violations, max deviation 4.547e-13, reproducing the ad-hoc 49-step check that preceded it at three orders of magnitude more data. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…tions (#135) `sacct` collapses a requeued job to its LAST incarnation, so the count in these three docstrings was read off a view that cannot show the others. `sacct -D` on Run A's two job ids returns six incarnations, i.e. five requeues: 38541865 1s ran 8:12:19 REQUEUED 38541865 2m04s ran 0:59:35 PREEMPTED 38541865 2m26s ran 8:09:12 REQUEUED 38541865 2m01s ran 2:47:04 FAILED (its own error, not the scheduler) 38566413 16m02s ran 8:06:45 REQUEUED 38566413 2m03s ran 4:47:18 COMPLETED This strengthens rather than weakens the argument the docstrings are making: the stateless schedule had to survive five resume boundaries on the run that motivated it, not two. Comment-only. The copy of train.py under test on klone is untouched and still hashes to c359a45, matching the PROVENANCE.txt pin for job 38640313, so the run in flight is unaffected and its provenance still resolves. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ng everything (#135) The cosine rung spent 8h54m on 2026-08-19/20 advancing its durable checkpoint by zero steps, across nine allocations, four of which never logged a single training step. Two independent causes, both fixed here. **1. Resuming decoded every batch it meant to skip.** The fast-forward was for i, (images, target_heatmaps) in enumerate(train_loader): if epoch == start_epoch and i < batch_idx_in_epoch: continue which pulls each skipped batch all the way through the DataLoader -- reading and decoding a 2048x4096 panorama per skipped step -- and throws it away. Measured from event wall-times against sacct -D start times: fresh start (step 0) 4.6 min to first logged step resume from step 4,000 23.2, 23.4 min resume from step 9,000 29.6 min `ResumeSkipSampler` drops the *indices* instead, so the workers never fetch them. The batches that remain and their order are identical -- DistributedSampler's permutation is a pure function of (seed, epoch), so an islice off the front leaves the rest untouched. This is a speed fix, not a change to what the model sees, and `test_skipping_matches_the_old_discard_loop` is what holds that claim up: if it ever breaks, a resumed run trains on a different sample of the epoch than an uninterrupted one and nothing downstream would reveal it. **2. Checkpoints every 1,000 steps are 22 min apart.** Now `--checkpoint-interval-steps`, still defaulting to 1000 so every committed Stage 2 number is untouched; the rung's launcher passes 200 (~4.5 min). A test pins the default and asserts the launcher opts down explicitly rather than inheriting it. Together: a slice had to last ~45-52 min to bank anything, against observed slices of 36 s to 6 min. That is a livelock, not slow progress. Two traps worth naming, both silent if got wrong: - **The LR horizon must not move on resume.** `total_train_steps` now reads `train_sampler.epoch_length` (full, unskipped) rather than `len(train_loader)`, which shrinks on the epoch a resume lands in. Had it moved, a resumed run would decay over a shorter horizon than the one it is continuing and the schedule under test would quietly not be the pre-registered one. - **`batch_idx_in_epoch` is now `resume_offset + i + 1`.** With the skip moved into the sampler, `i` restarts at 0, so checkpointing bare `i + 1` would rewind further on every successive preemption. This is a latent defect in every preemptible Stage 2 run, not just this rung -- Run B would have hit it harder, being longer. It does not invalidate anything already measured: the LR verification across four resume boundaries stands on its own. Suite 1361 passed, 1 skipped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The red CI was a real matcher divergence, and the seam is worth 0.000264 to this analysisProvenance note first: this work landed inside commit fa02b37, whose message describes What was failingFailing since 2026-08-18, three runs, both Python versions. Not a flake. This branch predates
What it moved: one numberRe-scoring all eight committed epoch dumps with the wrapping matcher:
One prediction on one panorama now claims a ground-truth ramp across the seam instead of The conclusions hold, and sharpen slightly0.000264 is 4% of the 0.0063 paired MDE the recommendation rests on, and it moves epoch 7
So "epoch 1 low → 2–6 a genuine plateau → 7–8 measurably declining" stands, with 3 vs 7 now One thing this exposed that was not the seamThe max-F1 table was mixing matchers: point estimates read from Both matchers are now pinned. The historical curve is checked against the historical matcher at Spun outRe-scoring the whole roster to check the blast radius turned up three committed YOLO 🤖 Generated with Claude Code (claude-opus-5[1m]) |
1 participant
Answers the prerequisite #135 set before the 1,675–3,350 GPU-hour spend: can
manual_goldresolve the effect Run B might produce, and does pooling the ten splits raise it?No GPU time was spent. Run A was not re-run.
The answer
manual_goldmax-F1Pairing is worth ~2× and it is free. Both checkpoints are scored on the same 1,000 panoramas against the same GT, so pano difficulty cancels. What governs a paired comparison is the discordance between the two detectors — measured at 2.5–6.6% across Run A's epochs — not
n·p·(1−p).Pooling is worth 7% (0.0117 → 0.0109). The nine city splits alone are worse than
manual_goldalone (0.0219) despite more panoramas. #135's per-split instruction is right.The benchmark is not a reason to cancel Run B. At MDE 0.0063 measured — ~0.008 extrapolated to Run B's larger epoch separation — it resolves an effect the size of #51's annealed tail (+0.024 here, 8σ) with room to spare.
The bracket was closed for no GPU at all
An earlier revision could only bracket the paired s.e. at [0.0043, 0.0135] using stand-in pairs, because Run A's committed artifacts are aggregate PR curves. Then the 2026-08-17 scoring cache turned out to have survived on makelab2 — 13 GB, all eight epochs, 1,000 panos each — and peak extraction from a cached heatmap is CPU-only numpy. So the measurement needed no model, no panorama images (they are not on that host), no GPU and no network, took 4 minutes, and covered all 28 epoch pairs instead of the two originally proposed.
The old bracket contained the answer, and the working assumption ("4–6% discordance ⇒ 0.006–0.009") was nearly exact.
Verification is the load-bearing part. Re-scoring each dump against
manual_labels/reproduces the committedsummary.csv:max-F1 is a property of the whole PR curve, so agreement at 10⁻⁷ says the peak extraction is the same operation, not a similar one. The uniform 10⁻⁴ offset on F1@0.30 is the committed table's own 0.005-grid downsampling, which its provenance note already documents.
It corrects #84's curve shape
With each pair read against its own measured s.e. rather than a global bracket, epochs 7 and 8 are measurably below the plateau — 3 vs 7 at z = 3.9, 3 vs 8 at 3.0, 6 vs 7 at 2.8. The unpaired 0.01 bar could not see it.
So the curve is not "steps up once from epoch 1 to 2 and is then flat":
The headline is unchanged — still no resolvable peak; 2, 3, 4, 5, 6 remain mutually indistinguishable (2 vs 6 identical to 4dp, 3 vs 6 at z = 1.4). What changes is that at constant LR the model begins to lose capability after ~epoch 6.
That sharpens what Run B tests: not "does a longer budget add to a flat curve" but "does cosine decay arrest a decline that is now measured." It cuts both ways — that decline is the classic signature of an LR left too high, which is what an anneal fixes; but a 30-epoch run whose decay does not bite could land below epoch 3.
Recommendation
Run the 30-epoch arm; not 60 on spec. The plateau ends at 6, so the extra 30 epochs buy schedule shape rather than useful steps, and a 60-epoch cosine spends proportionally longer at the high LRs the epoch 7–8 decline is evidence against.
Pre-register three free amendments: read Run B against Run A paired on max-F1 with the tie bar replaced by the measured MDE (0.008 at Run B's separation); per-split, not pooled; attribute cautiously below ~0.01.
Also flagged, not relitigated: the #84 amendment's 8-epoch cosine rung (~450 GPU-h, budget-matched to Run A) is more attractive than it was, because constant-LR is now known to peak at ~3 and decline by 7, so that rung isolates schedule from length — the confound the amendment already noted B − A carries.
The limit that binds is not the benchmark
Panorama sampling variance only, and Run B is n = 1. Nothing measures training-seed variance, so a gain below ~0.01 max-F1 is measurable but not attributable to the anneal without a seed control that doubles the bill.
Notes
docs/data/run_a_84_detections/, notbenchmark/model_detections/—rampnet/roster.pyasserts every file there belongs to a registered challenger leg (#122), and these are internal checkpoints, not comparison entries. The suite caught the first attempt to put them there.docs/data/run_a_84_manual_gold/.exclude_borderartifact puts 30% of the RampNet-vs-RampNet pair's disagreements at the seam against a 2.1% baseline; the pairs drawn from one source read the baseline exactly.metrics()under all-ones weights reproducesrampnet.detection_eval.aggregateto 1e-12 on four (split, model) combinations. Suite 1236 passed, 1 skipped.Files
docs/stage2_run_b_power_135.md— the write-up, caveats beside the numbersscripts/analysis/benchmark_power_135.py— the analysis, seeded and deterministicscripts/analysis/dump_peaks_from_cache.py— cache → per-pano detections,--verifydocs/data/benchmark_power_135.json,docs/data/run_a_84_detections/— LF-pinnedtests/test_benchmark_power_135.py— 10 testsInput to the #135 decision; closes nothing on its own.
🤖 Generated with Claude Code (claude-opus-5[1m])