scores.EntropyClassWiseScore

Class-wise version of ~seapig.scores.logits.EntropyScore.

Usage

scores.EntropyClassWiseScore(**kwargs)

Methods

Name	Description
fit()	Fit a separate scorer for each class.
score()	Compute per-class uncertainty scores.
select()	Select samples below per-class thresholds.
set_threshold()	Set per-class thresholds based on the calibrated scores.
get_threshold()	Return the threshold for a specific class.
plot()	Forward the plotting method to the score implementation.

fit()

Fit a separate scorer for each class.

Usage

Source

fit(
    X=None,
    y=None,
    X_val=None,
    y_val=None,
    model=None,
    loaders=None,
    outdir=None,
    prefix=None,
    **kwargs
)

The method supports two mutually exclusive modes:

Tensor mode - pre-computed feature tensors Xand label y are provided directly.
Model mode - a torch.nn.Module together with a DataLoader is supplied; the underlying ModelExtractor extracts the required embeddings or logits.

Exactly one of these modes must be selected. If both or neither are provided a ValueErroris raised.

Parameters

X: torch.Tensor | None = None: Training tensors. X holds the feature representation required by the wrapped base_score_cls (embeddings for KNN-based scores or logits for logit-based scores). y contains class labels; a 1-D tensor for single-label classification or a 2-D binary matrix for multi-label tasks.
y: torch.Tensor | None = None: Training tensors. X holds the feature representation required by the wrapped base_score_cls (embeddings for KNN-based scores or logits for logit-based scores). y contains class labels; a 1-D tensor for single-label classification or a 2-D binary matrix for multi-label tasks.
X_val: torch.Tensor | None = None: Optional validation tensors used for calibration of per-class scorers.
y_val: torch.Tensor | None = None: Optional validation tensors used for calibration of per-class scorers.
model: torch.nn.Module | None = None: A torch.nn.Module whose forward method yields the representation needed by the scorer.
loaders: dict[str, DataLoader[torch.Tensor | dict[str, torch.Tensor]]] | None = None: Mapping of split names to DataLoader objects. At minimum a "train" loader is required; a "val" loader is used if validation tensors are not supplied directly.
outdir: Path | None = None: Destination directory and filename prefix for any intermediate files produced by the extractor.
prefix: Path | None = None: Destination directory and filename prefix for any intermediate files produced by the extractor.
**kwargs: Any: Additional keyword arguments forwarded to the concrete scorer’s fit method.

Raises

ValueError: If both tensor and model modes are specified or neither is.
RuntimeError: Propagated from the underlying scorer when training data are missing for a particular class.

score()

Compute per-class uncertainty scores.

Usage

Source

score(X=None, model=None, loader=None, outdir=None, prefix=None)

Mirrors the fit method in accepting either pre-computed tensors or a torch.nn.Module with a DataLoader. The appropriate representation (embeddings for KNN-based scorers or logits for logit-based scorers) is extracted via _make_extractor when a model is supplied.

Parameters

X: torch.Tensor | None = None: Tensor of shape (N, D) containing the features for which scores should be computed. Required in tensor mode.
model: torch.nn.Module | None = None: torch.nn.Module that produces the necessary representation.
loader: DataLoader[torch.Tensor | dict[str, torch.Tensor]] | None = None: DataLoader yielding the input data for modelwhen model is provided.
outdir: Path | None = None: Forwarded to the extractor for any intermediate files.
prefix: Path | None = None: Forwarded to the extractor for any intermediate files.

Returns

torch.Tensor: A (N, C) tensor where C is the number of discovered classes. Each column contains the scores for a particular class.

Raises

ValueError: If both tensor and model modes are specified or neither is.
RuntimeError: If fit has not been called before scoring.

select()

Select samples below per-class thresholds.

Usage

Source

select(X=None, model=None, loader=None, outdir=None, prefix=None)

The method first ensures that thresholds have been calibrated (by invoking set_threshold if necessary) and then computes the class-wise scores via score. A boolean mask of shape (N, C) is returned where True indicates that the score for a given sample and class falls below the corresponding threshold.

Parameters

X: torch.Tensor | None = None: Same semantics as score; either pre-computed tensors or a model with a DataLoader must be supplied.
model: torch.Tensor | None = None: Same semantics as score; either pre-computed tensors or a model with a DataLoader must be supplied.
loader: torch.Tensor | None = None: Same semantics as score; either pre-computed tensors or a model with a DataLoader must be supplied.
outdir: torch.Tensor | None = None: Same semantics as score; either pre-computed tensors or a model with a DataLoader must be supplied.
prefix: torch.Tensor | None = None: Same semantics as score; either pre-computed tensors or a model with a DataLoader must be supplied.

Returns

dict[str, torch.Tensor]: {"score": scores, "selected": mask} where scores is the (N, C) tensor of raw scores and mask is the boolean selection mask.

set_threshold()

Set per-class thresholds based on the calibrated scores.

Usage

Source

set_threshold(q=0.99)

The underlying scorer for each class provides its own set_threshold implementation (typically based on a quantile of the validation scores). This wrapper forwards the requested quantile q to each scorer, stores the resulting scalar threshold in self._thresholds and marks the wrapper as calibrated.

Parameters

q: float = 0.99: Quantile to use for threshold determination. 0.99 (default) selects the 99th percentile of the validation score distribution.

Raises

RuntimeError: If fit has not been called yet.
AssertionError: If a scorer fails to provide a threshold.

get_threshold()

Return the threshold for a specific class.

Usage

Source

get_threshold(id=None)

After calibration self._thresholds maps each class label to its scalar threshold tensor. If the wrapper has not been calibrated or id is None, None is returned; otherwise the threshold tensor for the requested class identifier is returned.

plot()

Forward the plotting method to the score implementation.

Usage

Source

plot(query_scores=None, bins=100)