incerto.bayesian.predictive_entropy

incerto.bayesian.predictive_entropy#

incerto.bayesian.predictive_entropy(predictions)[source]#

Compute predictive entropy for batched Bayesian predictions (total uncertainty).

This function is specialized for Bayesian deep learning where you have multiple posterior samples. For simple entropy of a single probability distribution, use incerto.core.entropy instead.

Computes: H[y|x] = -∑ p(y|x) log p(y|x) where p(y|x) is the predictive distribution averaged over the posterior.

Parameters:

predictions (Tensor) – Tensor of shape (num_samples, batch_size, num_classes) containing probability distributions from multiple posterior samples. Each sample represents p(y|x,θ) for a different parameter θ.

Return type:

Tensor

Returns:

Predictive entropy of shape (batch_size,) representing the total uncertainty for each input.

Example

>>> # Ensemble of 10 models, batch of 32, 5 classes
>>> predictions = torch.softmax(torch.randn(10, 32, 5), dim=-1)
>>> total_uncertainty = predictive_entropy(predictions)
>>> total_uncertainty.shape
torch.Size([32])

See also

  • mutual_information: For epistemic uncertainty

  • decompose_uncertainty: For full decomposition

Contents