incerto.llm.BidirectionalConsistency

class incerto.llm.BidirectionalConsistency

Bases: object

Check consistency by asking the question in different ways.

If the model gives different answers to equivalent questions, it indicates high uncertainty.

__init__()

Methods

__init__()
compute_consistency(answers)	Compute consistency across answers.
paraphrase_prompts(question)	Generate paraphrased versions of a question.

static paraphrase_prompts(question)

Generate paraphrased versions of a question.

Parameters:

question (str) – Original question

Return type:

list[str]

Returns:

List of paraphrased questions

static compute_consistency(answers)

Compute consistency across answers.

Parameters:

answers (list[str]) – List of answers to paraphrased questions

Return type:

float

Returns:

Consistency score (0-1), higher = more consistent