scores.LogitScore

Base class for logit-based uncertainty scores.

Usage

scores.LogitScore(
    temperature=None,
    task="multiclass",
    per_member=False,
)

Supports multiclass, binary (single/two-logit), and multilabel tasks. Handles temperature fitting and input normalization for all cases.

The per_member flag enables handling of logits that contain multiple stochastic members per sample (e.g. ensembles or MC-dropout). When True, score methods compute the metric for each member and return the mean across the member axis.

Parameters

temperature: float or None = None: Optional temperature to apply to logits. If None, no temperature scaling is applied until fit() is called.
task: ("multiclass", "binary", "multilabel") = "multiclass": Type of classification task. Determines score computation and temperature fitting loss.
per_member: bool = False: If True, logits are expected to have a member dimension (e.g. for ensembles or MC-dropout). Score methods will compute the score for each member and return the mean across members

Notes

Input shapes and label formats by task:

multiclass: logits (N, C), labels (N,) long
binary single-logit: logits (N,) or (N, 1), labels (N,) float/long
binary two-logit: logits (N, 2), labels (N,) long
multilabel: logits (N, C), labels (N, C) float

Examples

import torch
from seapig.scores.logits import SoftmaxScore
logits = torch.randn(4, 3)
score = SoftmaxScore()
score.score(logits)

Attributes

Name	Description
T	Temperature property.

T

Temperature property.

T: float

Methods

Name	Description
fit()	Fit the score on reference logits.
score()	Compute uncertainty scores for query logits.
select()	Select samples for prediction based on their uncertainty score.

fit()

Fit the score on reference logits.

Usage

Source

fit(
    X=None,
    Y=None,
    temp_scale=False,
    model=None,
    loader=None,
    outdir=None,
    prefix=None,
    *args,
    **kwargs
)

This method supports two usage modes:

Precomputed logits: Supply logits directly via X, with optional labels via Y for temperature fitting.
On-the-fly extraction: Supply a model with a .logits() method and a DataLoader to extract logits automatically.

You must use either logits OR model+loader, but not both.

Parameters

X: torch.Tensor or None = None: Reference logits. Shape depends on task (see class docstring). Required when not using model and loader.
Y: torch.Tensor or None = None: Optional labels for temperature fitting. Shape/type depends on task.
temp_scale: bool = False: Boolean indicating if temperature scaling is to be applied. Defaults to False. If set to True labels are required.
model: torch.nn.Module or None = None: Model with a .logits(x) method. Required when not using precomputed logits.
loader: DataLoader or None = None: DataLoader yielding batches for inference. Required when using model.
outdir: Path or str or None = None: Optional directory to save/load logits. Only used with model and loader.
prefix: str or None = None: Optional prefix for saved files. Only used with model and loader.

Notes

Labels are required for temperature fitting to minimize NLL for the task.

score()

Compute uncertainty scores for query logits.

Usage

Source

score(query_logits=None, model=None, loader=None, outdir=None, prefix=None)

This method supports two usage modes:

Precomputed logits: Supply query logits via query_logits.
On-the-fly extraction: Supply a model with an .logits() method and a DataLoader to extract logits automatically.

You must use either logits (query_logits) OR model+loader, but not both.

# Mode 1: Precomputed logits
from seapig.scores import EntropyScore
my_score = EntropyScore()
scores = my_score.score(query_logits=test_logits)

# Mode 2: On-the-fly extraction
my_score = EntropyScore()
scores = my_score.score(model=model, loader=test_dl)

Parameters

query_logits: torch.Tensor = None: Logits for samples to score. Shape depends on task.
model: torch.nn.Module | None = None: A torch.nn.Module with an .embed() method. Required when not using X.
loader: DataLoader[Batch] | None = None: A torch.utils.data.DataLoader returning torch.Tensors or dicts with the "image" key. Required when using model.
outdir: Path | None = None: A pathlib.Path pointing to a directory for saving/loading embeddings. Only used with model and loader.
prefix: str | None = None: A str used as filename prefix for saved embeddings. Only used with model and loader.

Returns

torch.Tensor: 1-D tensor of shape (N,). Lower values indicate lower uncertainty.

select()

Select samples for prediction based on their uncertainty score.

Usage

Source

select(query_logits=None, model=None, loader=None, outdir=None, prefix=None)

Samples with scores lower than the threshold are selected for prediction, while samples with scores higher than the threshold are excluded.

Parameters

query_logits: torch.Tensor = None: Logits for samples to select. Shape depends on task.
model: torch.nn.Module | None = None: A torch.nn.Module with an .embed() method. Required when not using X.
loader: DataLoader[Batch] | None = None: A torch.utils.data.DataLoader returning torch.Tensors or dicts with the "image" key. Required when using model.
outdir: Path | None = None: A pathlib.Path pointing to a directory for saving/loading embeddings. Only used with model and loader.
prefix: str | None = None: A str used as filename prefix for saved embeddings. Only used with model and loader.

Returns

dict[str, torch.Tensor]: A dict with keys 'score' (uncertainty scores) and 'selected' (boolean mask where True means the sample is selected).

Parameters

Notes

See Also

Examples

Attributes

T

Methods

fit()

Parameters

Notes

score()

Parameters

Returns

select()

Parameters

Returns