incerto.llm.SequenceEntropy

incerto.llm.SequenceEntropy#

class incerto.llm.SequenceEntropy[source]#

Bases: object

Aggregated entropy over the sequence.

Can use mean, sum, or max aggregation of token-level entropies.

__init__()#

Methods

__init__()

compute(logits[, mask, aggregation, dim])

Compute sequence entropy.
static compute(logits, mask=None, aggregation='mean', dim=-1)[source]#

Compute sequence entropy.

Parameters:

  • logits (Tensor) – Token logits of shape (batch, seq_len, vocab_size)

  • mask (Optional[Tensor]) – Optional mask for padding of shape (batch, seq_len)

  • aggregation (str) – How to aggregate (“mean”, “sum”, “max”)

  • dim (int) – Dimension to compute entropy over (default: -1)

Return type:

Tensor

Returns:

Sequence entropy of shape (batch,)

Contents