Is your feature request related to a problem? Please describe.
I want to be able to give a prediction set with a coverage guarantee beyond the current calibration and TTA.
Describe the solution you'd like
-- Collect softmax on a held out calibration, compute non conformity scores (LAC:1-softmax), take the empirical quintile as threshold and at inference return score(y) <= threshold.
-- Then calibrate a single threshold that bounds an image level loss, put a per voxel uncertainty mask
Potentially:
inferers/conformal_predictor.py
metrics/conformal_risk.py
Describe alternatives you've considered
External libraries like TorchCP
Additional context
https://arxiv.org/pdf/2208.02814
Is your feature request related to a problem? Please describe.
I want to be able to give a prediction set with a coverage guarantee beyond the current calibration and TTA.
Describe the solution you'd like
-- Collect softmax on a held out calibration, compute non conformity scores (LAC:1-softmax), take the empirical quintile as threshold and at inference return score(y) <= threshold.
-- Then calibrate a single threshold that bounds an image level loss, put a per voxel uncertainty mask
Potentially:
inferers/conformal_predictor.py
metrics/conformal_risk.py
Describe alternatives you've considered
External libraries like TorchCP
Additional context
https://arxiv.org/pdf/2208.02814