incerto.llm.PTrue

incerto.llm.PTrue#

class incerto.llm.PTrue[source]#

Bases: object

P(True) - asking the model the probability its answer is correct.

Prompt: “What is the probability that your answer is correct?”

__init__()#

Methods

__init__()

extract_probability(response)

Extract probability value from response.

get_ptrue_prompt(question, answer)

Generate P(True) prompt.
static get_ptrue_prompt(question, answer)[source]#

Generate P(True) prompt.

Parameters:

  • question (str) – The original question

  • answer (str) – The model’s answer

Return type:

str

Returns:

Prompt string

static extract_probability(response)[source]#

Extract probability value from response.

Parameters:

response (str) – Text containing a probability

Return type:

float | None

Returns:

Probability value (0-1) or None

Contents