incerto.calibration.adaptive_ece_score

incerto.calibration.adaptive_ece_score(logits, labels, n_bins=10, norm='l1')

Adaptive Expected Calibration Error (Nixon et al., 2019).

Uses equal-mass binning instead of equal-width binning, making it more robust to varying confidence distributions.

Reference:

Nixon et al., “Measuring Calibration in Deep Learning” (CVPR Workshops 2019)

Parameters:

• logits (Tensor) – Model logits (N, C)

• labels (Tensor) – True labels (N,)

• n_bins (int) – Number of bins

• norm (str) – Norm to use (‘l1’ or ‘l2’)

Return type:

float

Returns:

Adaptive ECE score