incerto.bayesian.uncertainty_quality

incerto.bayesian.uncertainty_quality#

incerto.bayesian.uncertainty_quality(uncertainties, errors)[source]#

Evaluate quality of uncertainty estimates.

Measures correlation between uncertainty and prediction error. Good uncertainty should be high when the model is wrong.

Parameters:
  • uncertainties (Tensor) – Predicted uncertainties (batch_size,)

  • errors (Tensor) – Binary error indicators (batch_size,) where 1=error, 0=correct

Return type:

Tuple[float, float]

Returns:

Tuple of (spearman_correlation, auroc)