incerto.calibration.HistogramBinningCalibrator

class incerto.calibration.HistogramBinningCalibrator(n_bins=10)

Bases: BaseCalibrator

Histogram binning calibration: bins predicted probabilities and uses empirical frequencies.

Parameters:

n_bins (int)

__init__(n_bins=10)

Parameters:

n_bins (int)

Methods

__init__([n_bins])
fit(logits, labels)	Fit the calibrator on a validation set.
load(path)	Load a calibrator from a file.
load_state_dict(state)	Load histogram binning state.
predict(logits)	Apply calibration to logits and return a Categorical distribution.
save(path)	Save calibrator state to a file.
state_dict()	Save histogram binning state.

__init__(n_bins=10)

Parameters:

n_bins (int)

fit(logits, labels)

Fit the calibrator on a validation set.

This method should learn any calibration parameters needed to map uncalibrated logits to calibrated probabilities.

Parameters:

• logits (Tensor) – Uncalibrated model outputs of shape (n_samples, n_classes). These should be raw logits before softmax.

• labels (Tensor) – True class labels of shape (n_samples,) with integer values in range [0, n_classes-1].

Returns:

For method chaining

Return type:

self

Note

The validation set used here should be separate from both training and test sets to avoid overfitting the calibration parameters.

predict(logits)

Apply calibration to logits and return a Categorical distribution.

This method applies the learned calibration to transform uncalibrated logits into calibrated probabilities.

Parameters:

logits (Tensor) – Uncalibrated model outputs of shape (n_samples, n_classes).

Return type:

Categorical

Returns:

A torch.distributions.Categorical distribution over calibrated probabilities. Access probabilities via .probs or sample via .sample().

Example

>>> calibrated_dist = calibrator.predict(test_logits)
>>> calibrated_probs = calibrated_dist.probs  # shape: (n_samples, n_classes)
>>> predictions = calibrated_dist.sample()     # shape: (n_samples,)
>>> log_probs = calibrated_dist.log_prob(labels)  # shape: (n_samples,)

state_dict()

Save histogram binning state.

Return type:

dict

load_state_dict(state)

Load histogram binning state.

Parameters:

state (dict)

Return type:

None