Conformal Prediction

The conformal prediction module provides distribution-free uncertainty quantification with finite-sample coverage guarantees.

Core Predictor

ConformalPredictor(predictor[, method, alpha])

Thin wrapper around a calibrated conformal predictor.

Classification Methods

inductive_conformal(model, calib_loader, alpha)	Classical Inductive Conformal Prediction (ICP) — Vovk, Gammerman, and Shafer, Algorithmic Learning in a Random World (2005).
mondrian_conformal(model, calib_loader, alpha)	Mondrian Conformal Prediction — Papadopoulos, Reliable Classification with Conformal Predictors (2008).
aps(model, calib_loader, alpha)	Adaptive Prediction Sets (APS) — Romano, Patterson, and Candes, NeurIPS 2020.
raps(model, calib_loader, alpha[, lam, k_reg])	Regularized APS (RAPS), Angelopoulos, Bates, Malik, and Jordan (ICLR 2021).

Regression Methods

jackknife_plus(model_fn, train_dataset, alpha)	Jackknife+ Intervals Barber, Candès, and Ramdas (Ann.
cv_plus(model_fn, train_dataset, folds, alpha)	Cross-Validation+ Intervals (CV+) Barber et al. (Ann.
conformalized_quantile_regression(...)	Conformalized Quantile Regression (CQR) Romano, Patterson, and Candes (NeurIPS 2019).

Metrics

empirical_coverage(y, sets)	Fraction of test examples where y_i ∈ Ŝ_i.
average_set_size(sets)
conditional_coverage(y, sets, groups)	Coverage conditioned on groups (e.g., class labels).

Visualization

plot_coverage_vs_alpha(alphas, coverages[, ...])	Plot empirical coverage versus the desired miscoverage rate alpha.
plot_set_size_hist(sets[, show, ax])	Plot a histogram of prediction-set sizes.

Utilities

compute_quantile(scores, alpha[, adjusted])	Compute the conformal quantile at level (1 - alpha).
prediction_set_from_scores(scores, threshold)	Convert conformity scores to prediction sets.
split_data(dataset[, cal_ratio, seed])	Split dataset into calibration and test sets.