incerto.llm.VerbalizedConfidence

incerto.llm.VerbalizedConfidence#

class incerto.llm.VerbalizedConfidence[source]#

Bases: object

Ask the model to verbalize its confidence.

Prompt: “How confident are you in this answer? (0-100%)” Extract and return the confidence score.

__init__()#

Methods

__init__()

extract_percentage(response)

Extract a percentage value from text.

get_confidence_prompt(question, answer)

Generate a prompt to elicit confidence.
static extract_percentage(response)[source]#

Extract a percentage value from text.

Parameters:

response (str) – Text containing a percentage

Return type:

float | None

Returns:

Confidence value (0-1) or None if not found

static get_confidence_prompt(question, answer)[source]#

Generate a prompt to elicit confidence.

Parameters:

  • question (str) – The original question

  • answer (str) – The model’s answer

Return type:

str

Returns:

Prompt string

Contents