incerto.llm.SurprisalScore

incerto.llm.SurprisalScore#

class incerto.llm.SurprisalScore[source]#

Bases: object

Surprisal (negative log-probability) of generated tokens.

Also called information content. Higher values indicate the token was less expected by the model (higher uncertainty).

__init__()#

Methods

__init__()

compute(logits, token_ids[, dim])

Compute surprisal for specific tokens.
static compute(logits, token_ids, dim=-1)[source]#

Compute surprisal for specific tokens.

Parameters:

  • logits (Tensor) – Token logits of shape (batch, seq_len, vocab_size)

  • token_ids (Tensor) – Generated token IDs of shape (batch, seq_len)

  • dim (int) – Dimension to compute over (default: -1)

Return type:

Tensor

Returns:

Surprisal values of shape (batch, seq_len)

Contents