Reference

Task & Metric Classes

Core classes for defining selective inference tasks and metrics

SelectiveInferenceTask: Wrap a trained LightningModule to attach selection results during inference.
SelectiveMetric: Evaluate a metric on full, selected, and rejected subsets.
RiskCoverageMetric: Build a risk-coverage curve from scores and per-sample errors.
RiskCoverage: Container for risk-coverage results.

Uncertainty scoring methods for embedding-based approaches

scores.CosineScore: Returns the KNN-distance based on the cosine distance to the nearest samples.
scores.EuclideanScore: Returns the KNN-distance based on the Euclidean distance to the nearest samples.
scores.MahalanobisScore: Returns the Mahalanobis distance to the training samples distribution.
scores.PCAScore: Returns uncertainty scores based on PCA reconstruction errors.
scores.PyODScore: Uncertainty scores based on detectors supplied by PyOD.
scores.utils.TensorPCA: Tensor-based PCA.

Uncertainty scoring methods based on model logits

Uncertainty scoring methods based on other principles

Abstract base classes

scores.UncertaintyScore: Abstract Base Class for Uncertainty Scores.
scores.EmbeddingScore: Base class for embedding-based uncertainty scores.
scores.KNNScore: Abstract base class for KNN distance-based uncertainty scores.
scores.LogitScore: Base class for logit-based uncertainty scores.

Helpers for progress and logging