Skip to main content
Back to top
Ctrl
+
K
incerto
incerto: Uncertainty Quantification for Machine Learning
User Guide
Overview
Installation
Quick Start
Which Method Should I Use?
Calibration Guide
Out-of-Distribution Detection Guide
Conformal Prediction Guide
Selective Prediction Guide
LLM Uncertainty Quantification Guide
Bayesian Deep Learning Guide
Active Learning Guide
Distribution Shift Detection Guide
API Reference
Calibration
incerto.calibration.BaseCalibrator
incerto.calibration.TemperatureScaling
incerto.calibration.VectorScaling
incerto.calibration.MatrixScaling
incerto.calibration.PlattScalingCalibrator
incerto.calibration.IsotonicRegressionCalibrator
incerto.calibration.HistogramBinningCalibrator
incerto.calibration.DirichletCalibrator
incerto.calibration.BetaCalibrator
incerto.calibration.IdentityCalibrator
incerto.calibration.LabelSmoothingLoss
incerto.calibration.FocalLoss
incerto.calibration.ConfidencePenalty
incerto.calibration.evidential_loss
incerto.calibration.get_uncertainty_from_evidence
incerto.calibration.TemperatureAwareTraining
incerto.calibration.ece_score
incerto.calibration.mce_score
incerto.calibration.classwise_ece
incerto.calibration.adaptive_ece_score
incerto.calibration.smooth_ece
incerto.calibration.brier_score
incerto.calibration.nll
incerto.calibration.plot_reliability_diagram
incerto.calibration.plot_smooth_reliability_diagram
incerto.calibration.plot_confidence_histogram
incerto.calibration.plot_calibration_curve
incerto.calibration.get_bin_stats
incerto.calibration.extract_confidences_and_predictions
incerto.calibration.logits_to_probs
Out-of-Distribution Detection
incerto.ood.OODDetector
incerto.ood.MSP
incerto.ood.MaxLogit
incerto.ood.Energy
incerto.ood.ODIN
incerto.ood.Mahalanobis
incerto.ood.KNN
incerto.ood.mixup_data
incerto.ood.mixup_criterion
incerto.ood.OutlierExposureLoss
incerto.ood.EnergyRegularizedLoss
incerto.ood.CutMix
incerto.ood.auroc
incerto.ood.fpr_at_tpr
incerto.ood.detection_accuracy
incerto.ood.plot_roc
incerto.ood.score_hist
incerto.ood.compute_threshold_at_tpr
incerto.ood.get_ood_predictions
incerto.ood.extract_features
Conformal Prediction
incerto.conformal.inductive_conformal
incerto.conformal.mondrian_conformal
incerto.conformal.aps
incerto.conformal.raps
incerto.conformal.jackknife_plus
incerto.conformal.cv_plus
incerto.conformal.conformalized_quantile_regression
incerto.conformal.empirical_coverage
incerto.conformal.average_set_size
incerto.conformal.conditional_coverage
incerto.conformal.plot_coverage_vs_alpha
incerto.conformal.plot_set_size_hist
incerto.conformal.ConformalPredictor
incerto.conformal.compute_quantile
incerto.conformal.prediction_set_from_scores
incerto.conformal.split_data
Selective Prediction
incerto.sp.BaseSelectivePredictor
incerto.sp.SoftmaxThreshold
incerto.sp.SelfAdaptiveTraining
incerto.sp.DeepGambler
incerto.sp.SelectiveNet
incerto.sp.make
incerto.sp.coverage
incerto.sp.risk
incerto.sp.aurc
incerto.sp.accuracy_coverage_curve
incerto.sp.plot_risk_coverage
incerto.sp.plot_accuracy_coverage
LLM Uncertainty
incerto.llm.TokenEntropy
incerto.llm.TokenConfidence
incerto.llm.TokenPerplexity
incerto.llm.SurprisalScore
incerto.llm.TopKConfidence
incerto.llm.SequenceProbability
incerto.llm.AverageLogProb
incerto.llm.NormalizedSequenceProb
incerto.llm.SequenceEntropy
incerto.llm.SequencePerplexity
incerto.llm.VarianceOfTokenProbs
incerto.llm.SelfConsistency
incerto.llm.LexicalSimilarity
incerto.llm.VarianceRatio
incerto.llm.PredictiveEntropy
incerto.llm.MutualInformation
incerto.llm.SemanticEntropy
incerto.llm.EnsembleDisagreement
incerto.llm.BeamSearchUncertainty
incerto.llm.NucleusSamplingUncertainty
incerto.llm.IDontKnowDetection
incerto.llm.ContrastiveDecoding
incerto.llm.VerbalizedConfidence
incerto.llm.PTrue
incerto.llm.SelfEvaluation
incerto.llm.BidirectionalConsistency
incerto.llm.TokenTemperatureScaling
incerto.llm.SequenceLengthCalibration
incerto.llm.VerbosityBiasCorrection
incerto.llm.HistogramBinning
incerto.llm.selective_accuracy
incerto.llm.calibration_error
incerto.llm.brier_score
incerto.llm.aur_c
incerto.llm.uncertainty_auc
incerto.llm.token_level_accuracy
incerto.llm.sequence_level_accuracy
incerto.llm.f1_score_tokens
incerto.llm.plot_token_uncertainty
incerto.llm.plot_confidence_vs_correctness
incerto.llm.plot_generation_diversity
incerto.llm.plot_semantic_clusters
incerto.llm.plot_risk_coverage_llm
incerto.llm.plot_uncertainty_distribution
incerto.llm.plot_length_vs_confidence
Bayesian Deep Learning
incerto.bayesian.BaseBayesianMethod
incerto.bayesian.MCDropout
incerto.bayesian.DeepEnsemble
incerto.bayesian.SWAG
incerto.bayesian.LaplaceApproximation
incerto.bayesian.VariationalBayesNN
incerto.bayesian.GaussianLinear
incerto.bayesian.ensemble_diversity
incerto.bayesian.uncertainty_quality
incerto.bayesian.disagreement
incerto.bayesian.negative_log_likelihood
incerto.bayesian.brier_score
incerto.bayesian.predictive_log_likelihood
incerto.bayesian.sharpness
incerto.bayesian.predictive_entropy
incerto.bayesian.mutual_information
incerto.bayesian.expected_calibration_error
incerto.bayesian.decompose_uncertainty
incerto.bayesian.compute_disagreement
incerto.bayesian.sample_from_posterior
incerto.bayesian.ensemble_predictions_to_distribution
Active Learning
incerto.active.BaseAcquisition
incerto.active.RandomAcquisition
incerto.active.EntropyAcquisition
incerto.active.LeastConfidenceAcquisition
incerto.active.MarginAcquisition
incerto.active.BALDAcquisition
incerto.active.VarianceRatioAcquisition
incerto.active.MeanSTDAcquisition
incerto.active.BatchBALDAcquisition
incerto.active.UncertaintySampling
incerto.active.DiversitySampling
incerto.active.CoreSetSelection
incerto.active.BadgeSampling
incerto.active.QueryByCommittee
incerto.active.split_labeled_unlabeled
incerto.active.compute_diversity_penalty
incerto.active.greedy_k_center
incerto.active.subsample_for_efficiency
incerto.active.active_learning_loop
Distribution Shift Detection
incerto.shift.BaseShiftDetector
incerto.shift.MMDShiftDetector
incerto.shift.EnergyShiftDetector
incerto.shift.KSShiftDetector
incerto.shift.ClassifierShiftDetector
incerto.shift.BBSDDetector
incerto.shift.LabelShiftDetector
incerto.shift.ImportanceWeightingShift
incerto.shift.energy_distance
incerto.shift.total_variation
incerto.shift.population_stability_index
incerto.shift.wasserstein_distance
incerto.shift.sliced_wasserstein_distance
incerto.shift.plot_feature_histograms
incerto.shift.plot_embedding_space
incerto.shift.plot_confidence_distributions
incerto.shift.plot_shift_severity
incerto.shift.plot_ks_statistics
Data Utilities
incerto.data.VisionDataset
incerto.data.MNIST
incerto.data.FashionMNIST
incerto.data.CIFAR10
incerto.data.CIFAR100
incerto.data.SVHN
incerto.data.OODBenchmark
incerto.data.get_ood_benchmark
incerto.data.MNIST_vs_FashionMNIST
incerto.data.CIFAR10_vs_CIFAR100
incerto.data.CIFAR10_vs_SVHN
incerto.data.MNIST_vs_NotMNIST
incerto.data.SubclassOOD
incerto.data.CorruptedDataOOD
incerto.data.create_dataloaders
incerto.data.create_balanced_dataloader
incerto.data.create_ood_dataloader
incerto.data.create_calibration_loaders
incerto.data.InfiniteDataLoader
incerto.data.get_dataloader_stats
incerto.data.split_dataset
incerto.data.filter_dataset_by_class
incerto.data.get_class_balanced_subset
incerto.data.compute_dataset_statistics
incerto.data.create_imbalanced_dataset
incerto.data.TransformDataset
incerto.data.LabelNoiseDataset
incerto.data.merge_datasets
incerto.data.subsample_dataset
Utilities
incerto.utils.ConvNet
incerto.utils.ResNet18
incerto.utils.MLP
incerto.utils.train_epoch
incerto.utils.evaluate
incerto.utils.seed_everything
incerto.utils.plot_training_curves
incerto.utils.plot_uncertainty_distribution
incerto.utils.plot_2d_classification
incerto.utils.get_logger
incerto.utils.setup_logging
incerto.utils.disable_logging
Examples
Selective Prediction
Bayesian Uncertainty
Active Learning
Distribution Shift Detection
Development
Contributing
Changelog
Repository
Suggest edit
Open issue
.rst
.pdf
Contents
Calibration Examples
Contents