## scores.UncertaintyScore Abstract Base Class for Uncertainty Scores. Usage ``` python scores.UncertaintyScore() ``` Uncertainty scores quantify the deviation of query samples from the training distribution. Low scores indicate likely inliers (samples similar to training), while high scores indicate likely outliers (samples deviating from training). Samples with scores exceeding the threshold are excluded from prediction. ## Attributes `trained: bool` Whether the score has been trained. Defaults to `False`. `train_required: bool` Whether training is required before scoring. Defaults to `False`. `cal_required: bool` Whether calibration is required before selecting. Defaults to `False`. `calibrated: bool` Whether the score has been calibrated. Defaults to `False`. `scores: torch.Tensor or None` Uncertainty scores of the calibration samples. Low scores indicate likely inliers, high scores indicate likely outliers. `threshold: torch.Tensor or None` Rejection threshold. Samples with scores higher than this value are excluded from prediction. `device: str` Device to which internal tensors are put. Defaults to `"cpu"`. `ident: str` String identifying the uncertainty score implementation. ## See Also [scores.EmbeddingScore](scores.EmbeddingScore.md#seapig.scores.EmbeddingScore) [scores.LogitScore](scores.LogitScore.md#seapig.scores.LogitScore) [scores.RandomScore](scores.RandomScore.md#seapig.scores.RandomScore) ## Attributes | Name | Description | |----|----| | [cal_required](#cal_required) | bool(x) -\> bool | | [calibrated](#calibrated) | bool(x) -\> bool | | [train_required](#train_required) | bool(x) -\> bool | | [trained](#trained) | bool(x) -\> bool | ------------------------------------------------------------------------ #### cal_required bool(x) -\> bool `cal_required: bool = ``False` Returns True when the argument x is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed. ------------------------------------------------------------------------ #### calibrated bool(x) -\> bool `calibrated: bool = ``False` Returns True when the argument x is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed. ------------------------------------------------------------------------ #### train_required bool(x) -\> bool `train_required: bool = ``False` Returns True when the argument x is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed. ------------------------------------------------------------------------ #### trained bool(x) -\> bool `trained: bool = ``False` Returns True when the argument x is true, False otherwise. The builtins True and False are the only two instances of the class bool. The class bool is a subclass of the class int, and cannot be subclassed. ## Methods | Name | Description | |----|----| | [fit()](#fit) | Fit an uncertainty score on training data. | | [get_threshold()](#get_threshold) | Get the current threshold value. | | [is_calibrated()](#is_calibrated) | Return boolean indicating if the score is already calibrated. | | [is_trained()](#is_trained) | Return boolean indicating if the score is already trained. | | [plot()](#plot) | Plot densities for uncertainty scores. | | [requires_calibration()](#requires_calibration) | Return boolean indicating if the score requires calibration. | | [requires_training()](#requires_training) | Return boolean indicating if the score requires training. | | [score()](#score) | Calculate the uncertainty score for a tensor of samples. | | [select()](#select) | Select samples for prediction based on their uncertainty score. | | [set_calibrated()](#set_calibrated) | Set a boolean that the score is already calibrated. | | [set_threshold()](#set_threshold) | Set a threshold based on a specific quantile on the available scores. | | [set_trained()](#set_trained) | Set a boolean that the score is already trained. | ------------------------------------------------------------------------ #### fit() Fit an uncertainty score on training data. Usage ``` python fit(*args, **kwargs) ``` `X` is used as training samples to fit the underlying method, while `Y` is an optional parameter that can be used to compute reference scores for the decision threshold (calibration set). Subclasses define the exact parameter signatures and accepted input modes (precomputed tensors or model + DataLoader). ------------------------------------------------------------------------ #### get_threshold() Get the current threshold value. Usage ``` python get_threshold() ``` ------------------------------------------------------------------------ #### is_calibrated() Return boolean indicating if the score is already calibrated. Usage ``` python is_calibrated() ``` ------------------------------------------------------------------------ #### is_trained() Return boolean indicating if the score is already trained. Usage ``` python is_trained() ``` ------------------------------------------------------------------------ #### plot() Plot densities for uncertainty scores. Usage ``` python plot(query_scores=None, bins=100) ``` By default, this method plots densities for the uncertainty scores. Optionally, it can also plot densities for `query_scores`. ##### Parameters `query_scores: torch.Tensor | None = None` A `torch.Tensor` representing query scores to include in the plot. Defaults to `None`. `bins: int = ``100` An `int` indicating the number of bins to use for density estimation. Defaults to `100`. ------------------------------------------------------------------------ #### requires_calibration() Return boolean indicating if the score requires calibration. Usage ``` python requires_calibration() ``` ------------------------------------------------------------------------ #### requires_training() Return boolean indicating if the score requires training. Usage ``` python requires_training() ``` ------------------------------------------------------------------------ #### score() Calculate the uncertainty score for a tensor of samples. Usage ``` python score(*args, **kwargs) ``` Returns scores where low values indicate likely inliers and high values indicate likely outliers. ------------------------------------------------------------------------ #### select() Select samples for prediction based on their uncertainty score. Usage ``` python select(*args, **kwargs) ``` Samples with scores lower than the threshold are selected for prediction, while samples with scores higher than the threshold are excluded. ##### Returns `dict[str, torch.Tensor]` A dict with keys `'score'` (raw uncertainty scores) and `'selected'` (boolean selection mask). ------------------------------------------------------------------------ #### set_calibrated() Set a boolean that the score is already calibrated. Usage ``` python set_calibrated() ``` ------------------------------------------------------------------------ #### set_threshold() Set a threshold based on a specific quantile on the available scores. Usage ``` python set_threshold(q=0.99) ``` Samples with scores higher than this threshold are excluded from prediction. ##### Parameters `q: float = ``0.99` Quantile in the interval `(0, 1)` used to compute the threshold from the stored calibration scores. Defaults to `0.99`. ##### Raises `ValueError` If no calibration scores are available yet. ------------------------------------------------------------------------ #### set_trained() Set a boolean that the score is already trained. Usage ``` python set_trained() ```