incerto.llm.VerbosityBiasCorrection

incerto.llm.VerbosityBiasCorrection#

class incerto.llm.VerbosityBiasCorrection[source]#

Bases: object

Correct for the model’s tendency to be more confident on verbose outputs.

Some models produce higher probabilities when generating longer, more detailed responses, even if they’re not more accurate.

__init__()[source]#

Methods

__init__()

correct(length, confidence)

Apply verbosity bias correction.

fit(lengths, confidences)

Fit correction based on length-confidence relationship.
__init__()[source]#
fit(lengths, confidences)[source]#

Fit correction based on length-confidence relationship.

Parameters:

  • lengths (list[int]) – List of response lengths

  • confidences (list[float]) – List of confidence scores

correct(length, confidence)[source]#

Apply verbosity bias correction.

Parameters:

  • length (int) – Response length

  • confidence (float) – Original confidence score

Return type:

float

Returns:

Corrected confidence

Contents