## scores.MutualInformationScore Mutual information (BALD) uncertainty score for ensembles / MC-dropout. Usage ``` python scores.MutualInformationScore( temperature=None, task="multiclass", ) ``` ## Methods | Name | Description | |----|----| | [fit()](#fit) | Fit the score on reference logits. | | [score()](#score) | Compute uncertainty scores for query logits. | | [select()](#select) | Select samples for prediction based on their uncertainty score. | | [set_threshold()](#set_threshold) | Set a threshold based on a specific quantile on the available scores. | | [get_threshold()](#get_threshold) | Get the current threshold value. | | [plot()](#plot) | Plot densities for uncertainty scores. | ------------------------------------------------------------------------ #### fit() Fit the score on reference logits. Usage ``` python fit( X=None, Y=None, temp_scale=False, model=None, loader=None, outdir=None, prefix=None, *args, **kwargs ) ``` This method supports two usage modes: 1. **Precomputed logits**: Supply logits directly via `X`, with optional labels via `Y` for temperature fitting. 2. **On-the-fly extraction**: Supply a `model` with a `.logits()` method and a `DataLoader` to extract logits automatically. You must use either logits OR model+loader, but not both. ##### Parameters `X: torch.Tensor or None = None` Reference logits. Shape depends on task (see class docstring). Required when not using `model` and `loader`. `Y: torch.Tensor or None = None` Optional labels for temperature fitting. Shape/type depends on task. `temp_scale: bool = ``False` Boolean indicating if temperature scaling is to be applied. Defaults to `False`. If set to `True` labels are required. `model: torch.nn.Module or None = None` Model with a `.logits(x)` method. Required when not using precomputed logits. `loader: DataLoader or None = None` DataLoader yielding batches for inference. Required when using `model`. `outdir: Path or str or None = None` Optional directory to save/load logits. Only used with `model` and `loader`. `prefix: str or None = None` Optional prefix for saved files. Only used with `model` and `loader`. ##### Notes Labels are required for temperature fitting to minimize NLL for the task. ------------------------------------------------------------------------ #### score() Compute uncertainty scores for query logits. Usage ``` python score(query_logits=None, model=None, loader=None, outdir=None, prefix=None) ``` This method supports two usage modes: 1. **Precomputed logits**: Supply query logits via `query_logits`. 2. **On-the-fly extraction**: Supply a `model` with an `.logits()` method and a `DataLoader` to extract logits automatically. You must use either logits (query_logits) OR model+loader, but not both. ``` python # Mode 1: Precomputed logits from seapig.scores import EntropyScore my_score = EntropyScore() scores = my_score.score(query_logits=test_logits) # Mode 2: On-the-fly extraction my_score = EntropyScore() scores = my_score.score(model=model, loader=test_dl) ``` ##### Parameters `query_logits: torch.Tensor = None` Logits for samples to score. Shape depends on task. `model: torch.nn.Module | None = None` A `torch.nn.Module` with an `.embed()` method. Required when not using `X`. `loader: DataLoader[Batch] | None = None` A `torch.utils.data.DataLoader` returning `torch.Tensor`s or dicts with the `"image"` key. Required when using `model`. `outdir: Path | None = None` A `pathlib.Path` pointing to a directory for saving/loading embeddings. Only used with `model` and `loader`. `prefix: str | None = None` A `str` used as filename prefix for saved embeddings. Only used with `model` and `loader`. ##### Returns `torch.Tensor` 1-D tensor of shape `(N,)`. Lower values indicate lower uncertainty. ------------------------------------------------------------------------ #### select() Select samples for prediction based on their uncertainty score. Usage ``` python select(query_logits=None, model=None, loader=None, outdir=None, prefix=None) ``` Samples with scores lower than the threshold are selected for prediction, while samples with scores higher than the threshold are excluded. ##### Parameters `query_logits: torch.Tensor = None` Logits for samples to select. Shape depends on task. `model: torch.nn.Module | None = None` A `torch.nn.Module` with an `.embed()` method. Required when not using `X`. `loader: DataLoader[Batch] | None = None` A `torch.utils.data.DataLoader` returning `torch.Tensor`s or dicts with the `"image"` key. Required when using `model`. `outdir: Path | None = None` A `pathlib.Path` pointing to a directory for saving/loading embeddings. Only used with `model` and `loader`. `prefix: str | None = None` A `str` used as filename prefix for saved embeddings. Only used with `model` and `loader`. ##### Returns `dict[str, torch.Tensor]` A dict with keys `'score'` (uncertainty scores) and `'selected'` (boolean mask where `True` means the sample is selected). ------------------------------------------------------------------------ #### set_threshold() Set a threshold based on a specific quantile on the available scores. Usage ``` python set_threshold(q=0.99) ``` Samples with scores higher than this threshold are excluded from prediction. ##### Parameters `q: float = ``0.99` Quantile in the interval `(0, 1)` used to compute the threshold from the stored calibration scores. Defaults to `0.99`. ##### Raises `ValueError` If no calibration scores are available yet. ------------------------------------------------------------------------ #### get_threshold() Get the current threshold value. Usage ``` python get_threshold() ``` ------------------------------------------------------------------------ #### plot() Plot densities for uncertainty scores. Usage ``` python plot(query_scores=None, bins=100) ``` By default, this method plots densities for the uncertainty scores. Optionally, it can also plot densities for `query_scores`. ##### Parameters `query_scores: torch.Tensor | None = None` A `torch.Tensor` representing query scores to include in the plot. Defaults to `None`. `bins: int = ``100` An `int` indicating the number of bins to use for density estimation. Defaults to `100`.