scores.ClassWiseScore

Base class-wise wrapper for per-class uncertainty scoring.

Usage

scores.ClassWiseScore(
    base_score_cls, global_pca=None, aggregation="mean", **base_kwargs
)

The wrapper supports three usage modes:

Tensor mode - Directly provide pre-computed feature tensors X and label tensor y to fit() and optionally to score().
Model mode - Supply a torch.nn.Module and a DataLoader; the ModelExtractor extracts the required embeddings.
Full-matrix mode - When calling score(), setting full_matrix=True returns the raw (N, C) score matrix without aggregating over classes. This mode is useful when downstream code needs per-class scores.

Parameters

base_score_cls: type[sp.UncertaintyScore]: The concrete UncertaintyScore subclass to instantiate for each class.
base_kwargs: Any = {}: Keyword arguments passed to each base_score_cls instance.
global_pca: TensorPCA | None = None: Optional TensorPCA applied globally to inputs before scoring.
aggregation: str | Callable[[torch.Tensor, torch.Tensor], torch.Tensor] = "mean": Aggregation function or name ("mean", "max", "min") used to reduce per-class scores to a single scalar per sample in multi-label scenarios. Ignored in single-label mode.

Attributes

Name	Description
mode	Return the mode inferred during fitting.

mode

Return the mode inferred during fitting.

mode: ClassWiseMode

Raises

RuntimeError: If fit has not been called yet.

Methods

Name	Description
fit()	Fit a separate scorer for each class.
get_threshold()	Return the threshold for a specific class.
plot()	Forward the plotting method to the score implementation.
score()	Compute per-class uncertainty scores.
select()	Select samples below per-class thresholds.

fit()

Fit a separate scorer for each class.

Usage

Source

fit(
    ref=None,
    cal=None,
    model=None,
    loaders=None,
    outdir=None,
    prefix=None,
    **kwargs
)

The method supports two mutually exclusive modes:

Tensor mode - pre-computed feature tensors Xand label y are provided directly.
Model mode - a torch.nn.Module together with a DataLoader is supplied; the underlying ModelExtractor extracts the required embeddings.

Exactly one of these modes must be selected. If both or neither are provided a ValueErroris raised.

Parameters

ref: dict[str, torch.Tensor] | None = None: Training tensors. X holds the feature representation required by the wrapped base_score_cls. y contains class labels; a 1-D tensor for single-label classification or a 2-D binary matrix for multi-label tasks.
cal: dict[str, torch.Tensor] | None = None: Optional validation tensors used for calibration of per-class scorers.
model: torch.nn.Module | None = None: A torch.nn.Module whose forward method yields the representation needed by the scorer.
loaders: dict[str, DataLoader[torch.Tensor | dict[str, torch.Tensor]]] | None = None: Mapping of split names to DataLoader objects. At minimum a "train" loader is required; a "val" loader is used if validation tensors are not supplied directly.
outdir: Path | None = None: Destination directory and filename prefix for any intermediate files produced by the extractor.
prefix: Path | None = None: Destination directory and filename prefix for any intermediate files produced by the extractor.
**kwargs: Any: Additional keyword arguments forwarded to the concrete scorer’s fit method.

Raises

ValueError: If both tensor and model modes are specified or neither is.
RuntimeError: Propagated from the underlying scorer when training data are missing for a particular class.

get_threshold()

Return the threshold for a specific class.

Usage

Source

get_threshold(full_matrix=False)

After calibration self._thresholds maps each class label to its scalar threshold tensor. If the wrapper has not been calibrated, None is returned; otherwise the threshold dictionary with keys identifying the class label is returned.

plot()

Forward the plotting method to the score implementation.

Usage

Source

plot(query_scores=None, bins=100)

score()

Compute per-class uncertainty scores.

Usage

Source

score(
    query=None,
    model=None,
    loader=None,
    outdir=None,
    prefix=None,
    full_matrix=False,
    return_label=False
)

Mirrors the fit method in accepting either pre-computed tensors or a torch.nn.Module with a DataLoader. The appropriate embeddings for KNN-based scorers are extracted via _make_extractor when a model is supplied.

Parameters

query: dict[str, torch.Tensor] | None = None: Tensor of shape (N, D) containing the features for which scores should be computed. Required in tensor mode.
model: torch.nn.Module | None = None: torch.nn.Module that produces the necessary representation.
loader: DataLoader[torch.Tensor | dict[str, torch.Tensor]] | None = None: DataLoader yielding the input data for model when model is provided.
outdir: Path | None = None: Forwarded to the extractor for any intermediate files.
prefix: Path | None = None: Forwarded to the extractor for any intermediate files.
full_matrix: bool = False: If True, returns the full (N, C) score matrix without requiring label information. False by default.
return_label: bool = False: Boolean. Besides the scores also returns the labels. Useful for the select() method. False by default.

Returns

torch.Tensor: A (N, C) tensor where C is the number of discovered classes. Each column contains the scores for a particular class.

Raises

ValueError: If both tensor and model modes are specified or neither is.
RuntimeError: If fit has not been called before scoring.

select()

Select samples below per-class thresholds.

Usage

Source

select(
    query=None,
    model=None,
    loader=None,
    outdir=None,
    prefix=None,
    full_matrix=False
)

The method first ensures that thresholds have been calibrated (by invoking set_threshold if necessary) and then computes the class-wise scores via score. A boolean mask of shape (N, C) is returned where True indicates that the score for a given sample and class falls below the corresponding threshold.

Parameters

query: dict[str, torch.Tensor] | None = None: Same semantics as score; either pre-computed tensors or a model with a DataLoader must be supplied.
model: dict[str, torch.Tensor] | None = None: Same semantics as score; either pre-computed tensors or a model with a DataLoader must be supplied.
loader: dict[str, torch.Tensor] | None = None: Same semantics as score; either pre-computed tensors or a model with a DataLoader must be supplied.
outdir: dict[str, torch.Tensor] | None = None: Same semantics as score; either pre-computed tensors or a model with a DataLoader must be supplied.
prefix: dict[str, torch.Tensor] | None = None: Same semantics as score; either pre-computed tensors or a model with a DataLoader must be supplied.

Returns

dict[str, torch.Tensor]: {"score": scores, "selected": mask} where scores is the (N, C) tensor of raw scores and mask is the boolean selection mask.