Changelog

This changelog is generated automatically from GitHub Releases.

seapig v0.3.0

2026-09-22 · GitHub

Features

  • Class‑wise scoring – added class-wise scoring behavior for KNNScores, supporting per‑class thresholds and global PCA support.
  • TensorPCA – new mean_center argument, now allows to turn off mean centering.
  • Score API harmonization – unified ref, cal, query signatures for LogitScore, ClassWiseScore and EmbeddingScore.
  • SelectiveInferenceTask – now expects dict‑style model outputs from an extract() method.
  • ModelExtractor – extracts predictions via a extract() method; supports dict outputs and batch forwarding.
  • New scores – MutualInformationScore and PredictiveVarianceScore for ensemble / MC‑dropout uncertainty.
  • UncertaintyScore – set_threshold now uses nanquantile for NaN‑robust quantile computation.

Bug fixes

  • Fixed metric device placement in SelectiveMetric.
  • Updated SelectiveInferenceTask and ModelExtractor to use the proper predict contract.
  • Resolved label‑None handling in class‑wise logit scores.
  • Added jitter to Mahalanobis covariance to avoid factorisation failures.
  • Corrected random‑score dict contract and added multi‑label guard for class‑wise scores.

Refactor & Performance

  • Removed explicit griffe dependency from the build process.
  • Switched FAISS‑based HNSW index implementation (FAISS → FAISS‑CPU).

Documentation

  • README and quick‑start examples now use the predict interface.
  • Added API documentation for the new class‑wise and mutual‑information scores.
  • Updated embedding and model docstrings to reflect the new predict contract.

Tests & Typing

  • Migrated all type checks from mypy to ty.
  • Extended the test suite, coverage now at 100 %.

CI / Build / Chores

  • CI workflow updates and code‑cov fixes.
  • Dependabot configuration repaired.
  • Devcontainer refreshed with latest VS Code extensions.

Notes / Upgrade guidance

  • Scores now accept either a raw Tensor or a dict of Tensor’s as input to ref/cal arguments.
  • If the model+loader pattern is used, the model is expected to expose an extract() method that returns a dict of Tensors with required keys (e.g. “logit”, “embedding”, “label”, “prediction”)
  • The selective inference task‑wrapper now also expects the lightning module to expose an extract() method with a dictionary of required tensors as the output. Importantly, due to calculating error metrics, the dictionary MUST contain the prediction key.

seapig v0.2.0

2026-05-21 · GitHub

seapig v0.2.0

Features

  • add multi-metric support to RiskCoverageMetric and allow get_curve to return multiple curves (#157).
  • expose knn_search method to return distances and indices (#143).
  • rename kpn to offset and apply mean padding (#140).

Bug fixes

  • revert risk-coverage ordering to correct behavior (#157).

Refactor & Performance

  • refactor: switch to FAISS-based HNSW index in place of nmslib (#142).

Documentation

  • switch to “uncertainty” vocabulary (replacing “confidence”) to better reflect intent (#158).
  • add conda-forge references and render/readme improvements (#159).

Tests & typing

  • migration to use ty for typing instead of mypy (#150).
  • many new/improved tests covering RiskCoverageMetric, PCA/TensorPCA, logits, and L2 distance behavior to increase coverage to >99% (#148).

CI / Build / Chores

  • multiple CI/test workflow updates and codecov fixes

Notes / Upgrade guidance

  • Potential breaking change: indices serialzed to disk with version 0.1.0 are no longer supported and will result in errors.
  • This release collects numerous internal improvements (tests, typing, docs) in addition to notable API and backend changes.

seapig - Initial Zenodo Release

2026-05-03 · GitHub

Initial Release for Zenodo. See the original release notes.

seapig v0.1.0 - Initial Release

2026-05-03 · GitHub

A lightweight library for confidence‑based selective inference for deep models. seapig provides small, composable confidence scores operating on model embeddings (or logits), calibration utilities, and a PyTorch Lightning task + torchmetrics wrapper to evaluate selective systems.

Highlights

  • Fast, interpretable confidence scores based on latent representations and logits.
  • Calibration on an independent validation set to fix target coverage levels.
  • Select/abstain decision utilities that produce per-sample scores and binary selections.
  • Seamless integration with PyTorch Lightning via SelectiveInferenceTask and torchmetrics compatibility.
  • Small, dependency‑scoped core with optional extras for recommended features, docs, and development.

Main features

Quick install

  • PyPI: pip install seapig
  • Latest: pip install git+https://github.com/goergen95/seapig.git
  • Extras: seapig[suggested], seapig[dev], seapig[docs]

Docs & examples

Tests, quality & license

  • Tests included (pytest) and type hints present (py.typed).
  • CI and coverage badges maintained in repo.
  • License: MIT (see LICENSE)
  • Code of conduct: .github/CODE_OF_CONDUCT

Citation

  • CITATION.cff included for academic use.

Contact / contribution

  • Contributions welcome — follow the repository’s contributing guidelines and run make check (formatting, linting, tests) before submitting PRs.