incerto.llm.selective_accuracy

incerto.llm.selective_accuracy#

incerto.llm.selective_accuracy(predictions, targets, confidences, threshold)[source]#

Compute accuracy on high-confidence predictions.

Parameters:
  • predictions (Tensor) – Predicted classes/tokens (batch,)

  • targets (Tensor) – True classes/tokens (batch,)

  • confidences (Tensor) – Confidence scores (batch,)

  • threshold (float) – Confidence threshold for selection

Return type:

dict

Returns:

Dictionary with accuracy, coverage, and count