Skip to content

Multi-measure Perf support, BiB-only direction, and multiple-comparison correction - #37

Merged
mgraffg merged 2 commits into
mainfrom
develop
Aug 24, 2026
Merged

Multi-measure Perf support, BiB-only direction, and multiple-comparison correction#37
mgraffg merged 2 commits into
mainfrom
develop

Conversation

@mgraffg

@mgraffg mgraffg commented Aug 24, 2026

Copy link
Copy Markdown
Member

Summary

Closes #30, #32, #35, #36.

Test plan

  • pytest CompStats — 74 passed

mgraffg and others added 2 commits August 24, 2026 15:19
)

Perf now takes func (a callable or list of callables) plus a BiB default,
instead of separate score_func/error_func arguments. Direction still comes
from a callable's own .BiB tag when present (e.g. set by metrics.py's
.measure() factories), falling back to the constructor's BiB otherwise --
BiB remains the single source of truth, with no XOR-style dual-argument
split to keep in sync.

Fixes three call sites that had drifted out of sync by re-deriving
direction from `score_func is not None` instead of the actual .BiB:
Perf.statistic's sort order, Perf.__repr__'s label, and Perf.plot's
default value_name.

Updates all 26 metrics.py wrappers, utils.py's metrics_docs decorator,
tests (plus 3 new regression tests for the fixed call sites), and docs
accordingly. performance.py needs no changes -- its score_func is an
unrelated local variable.

Also normalizes whitespace across all .py files for PEP8 consistency:
trailing whitespace, missing EOF newlines, and spacing around annotated
defaults (e.g. `int=500` -> `int = 500`), applied via autopep8 restricted
to whitespace-only codes plus a manual trailing-whitespace/EOF-newline
pass. No logic changes; full test suite (74 tests) passes.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@mgraffg
mgraffg merged commit e52c8cd into main Aug 24, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Plan: Multi-measure API for Perf/Difference (e.g. macro-F1 + macro-recall together)

1 participant