incerto.llm.NucleusSamplingUncertainty

incerto.llm.NucleusSamplingUncertainty#

class incerto.llm.NucleusSamplingUncertainty[source]#

Bases: object

Uncertainty for nucleus (top-p) sampling.

Analyzes the probability mass distribution to determine how concentrated or spread out the generation is.

__init__()#

Methods

__init__()

effective_vocabulary_size(logits[, p])

Number of tokens needed to cover p probability mass.

probability_mass_concentration(logits[, top_k])

Fraction of probability in top-k tokens.
static effective_vocabulary_size(logits, p=0.9)[source]#

Number of tokens needed to cover p probability mass.

Parameters:

  • logits (Tensor) – Token logits (…, vocab_size)

  • p (float) – Probability mass threshold (default: 0.9)

Return type:

int

Returns:

Effective vocabulary size

static probability_mass_concentration(logits, top_k=10)[source]#

Fraction of probability in top-k tokens.

Parameters:

  • logits (Tensor) – Token logits (…, vocab_size)

  • top_k (int) – Number of top tokens

Return type:

float

Returns:

Probability mass in top-k (0-1)

Contents