incerto.calibration.IsotonicRegressionCalibrator

class incerto.calibration.IsotonicRegressionCalibrator(out_of_bounds='clip')

Bases: BaseCalibrator

Multi-class isotonic regression calibration (per-class fitting).

Parameters:

out_of_bounds (str)

__init__(out_of_bounds='clip')

Parameters:

out_of_bounds (str)

Methods

__init__([out_of_bounds])
fit(logits, labels)	Fit the calibrator on a validation set.
load(path)	Load a calibrator from a file.
load_state_dict(state)	Load isotonic regression calibrators.
predict(logits)	Apply calibration to logits and return a Categorical distribution.
save(path)	Save calibrator state to a file.
state_dict()	Save isotonic regression calibrators.

__init__(out_of_bounds='clip')

Parameters:

out_of_bounds (str)

fit(logits, labels)

Fit the calibrator on a validation set.

This method should learn any calibration parameters needed to map uncalibrated logits to calibrated probabilities.

Parameters:

• logits (Tensor) – Uncalibrated model outputs of shape (n_samples, n_classes). These should be raw logits before softmax.

• labels (Tensor) – True class labels of shape (n_samples,) with integer values in range [0, n_classes-1].

Returns:

For method chaining

Return type:

self

Note

The validation set used here should be separate from both training and test sets to avoid overfitting the calibration parameters.

predict(logits)

Apply calibration to logits and return a Categorical distribution.

This method applies the learned calibration to transform uncalibrated logits into calibrated probabilities.

Parameters:

logits (Tensor) – Uncalibrated model outputs of shape (n_samples, n_classes).

Return type:

Categorical

Returns:

A torch.distributions.Categorical distribution over calibrated probabilities. Access probabilities via .probs or sample via .sample().

Example

>>> calibrated_dist = calibrator.predict(test_logits)
>>> calibrated_probs = calibrated_dist.probs  # shape: (n_samples, n_classes)
>>> predictions = calibrated_dist.sample()     # shape: (n_samples,)
>>> log_probs = calibrated_dist.log_prob(labels)  # shape: (n_samples,)

state_dict()

Save isotonic regression calibrators.

Return type:

dict

load_state_dict(state)

Load isotonic regression calibrators.

Parameters:

state (dict)

Return type:

None