incerto.llm.MutualInformation

incerto.llm.MutualInformation#

class incerto.llm.MutualInformation[source]#

Bases: object

Mutual information between predictions and model (aleatoric vs epistemic).

MI = H(E[y|x, θ]) - E[H(y|x, θ)]

= Entropy of expected distribution - Expected entropy

High MI indicates epistemic uncertainty (model uncertainty).

__init__()#

Methods

__init__()

compute(logit_samples)

Compute mutual information.
static compute(logit_samples)[source]#

Compute mutual information.

Parameters:

logit_samples (List[Tensor]) – List of logit tensors from different samples

Return type:

Tensor

Returns:

Mutual information value

Contents