Selective Prediction

The selective prediction module enables models to abstain from predictions when uncertain, providing risk-coverage tradeoffs.

Methods

SoftmaxThreshold(backbone)	Classical confidence-thresholding à la Chow (1957).
SelfAdaptiveTraining(backbone, num_classes)	Self-Adaptive Training for better calibration and selective prediction.
DeepGambler(backbone, num_classes)	Add an extra abstain logit and train with the gambler's loss:
SelectiveNet(backbone, num_classes[, ...])	Implementation of SelectiveNet (Geifman & El-Yaniv, 2019). The model outputs: * h(x): class logits * g(x): selection probability.

Metrics

aurc(sorted_conf, sorted_errors)

Area under the Risk-Coverage curve.

Visualization