incerto.llm.BeamSearchUncertainty

incerto.llm.BeamSearchUncertainty#

class incerto.llm.BeamSearchUncertainty[source]#

Bases: object

Uncertainty estimation from beam search scores.

Beam search maintains multiple hypotheses with scores. The score distribution indicates uncertainty.

__init__()#

Methods

__init__()

compute_from_scores(beam_scores[, temperature])

Compute uncertainty from beam search scores.

diversity_among_beams(beam_sequences)

Measure diversity among beam search outputs.
static compute_from_scores(beam_scores, temperature=1.0)[source]#

Compute uncertainty from beam search scores.

Parameters:

  • beam_scores (Tensor) – Scores for each beam (num_beams,)

  • temperature (float) – Temperature for softmax (default: 1.0)

Returns:

  • entropy: Entropy over beam distribution

  • top_beam_prob: Probability of best beam

  • score_variance: Variance of beam scores

Return type:

Dictionary with

static diversity_among_beams(beam_sequences)[source]#

Measure diversity among beam search outputs.

Parameters:

beam_sequences (List[List[int]]) – List of token ID sequences from beams

Return type:

float

Returns:

Diversity score (0-1), higher = more diverse

Contents