incerto.llm.MutualInformation

incerto.llm.MutualInformation#

class incerto.llm.MutualInformation[source]#

Bases: object

Mutual information between predictions and model (aleatoric vs epistemic).

MI = E[H(y|x, θ)] - H(E[y|x, θ])

= Expected entropy - Entropy of expected distribution

High MI indicates epistemic uncertainty (model uncertainty).

__init__()#

Methods

__init__()

compute(logit_samples)

Compute mutual information.
static compute(logit_samples)[source]#

Compute mutual information.

Parameters:

logit_samples (List[Tensor]) – List of logit tensors from different samples

Return type:

Tensor

Returns:

Mutual information value

Contents