Identifying Unknown Instances via Prediction Entropy

90 concepts64 papers2 proposals

When a machine learning model is deployed in the real world, it inevitably encounters the "open-world" problem: facing objects, categories, or data distributions it has never seen during training. Standard classifiers operating under a closed-world assumption are forced to map these novel inputs to one of their known categories, often leading to overconfident and incorrect predictions.

To build a reliable safety net, a model must "know what it doesn't know." A powerful and intuitive way to achieve this is by monitoring prediction entropy. Instead of treating a model's confusion as a failure mode, this research direction leverages the statistical flatness of a model's output distribution as a clear diagnostic signal to detect novel, out-of-distribution (OOD), or open-set instances.

The Intuition: Flatness as a Signal for Novelty

Consider a classifier trained to distinguish between a set of known classes. When presented with a familiar instance, the model concentrates its probability mass on a single category, resulting in a sharp, low-entropy distribution. However, when the model encounters an entirely novel category, it lacks a corresponding label. Rather than spiking on a single incorrect class, a well-regularized model distributes its probability mass across multiple known classes. This results in a uniform, high-entropy distribution.

Familiar Input (Low Entropy)        Novel Input (High Entropy)
       ▲                                   ▲
       │   █                               │   █   █   █   █
       │   █                               │   █   █   █   █
       └─► █ ◄─► ░ ◄─► ░                   └─► █ ◄─► █ ◄─► █ ◄─► █
        Class A  B     C                    Class A  B     C     D

This property allows entropy to serve as an effective routing mechanism. For instance, Uehara et al. utilize the entropy of a classifier's softmax output to identify unknown objects in an environment, triggering an autonomous system to ask humans context-aware questions for class acquisition. Similarly, Mandal et al. employ prediction entropy as a routing signal in generalized zero-shot action recognition, steering low-entropy samples to specialized seen-class classifiers and high-entropy samples to unseen-class models.

To make this distinction even sharper, models can be explicitly trained to maximize entropy when encountering known out-of-distribution samples. As demonstrated by Huan Ren et al., forcing uniform distributions on anomalous data creates a distinct bimodal entropy distribution across the dataset, making the boundary between known and unknown categories highly pronounced.

Threshold-Free Detection via Mixture Modeling

While high entropy signals novelty, setting a manual, hard-coded threshold to separate "known" from "unknown" is brittle and domain-sensitive. To resolve this, recent approaches apply statistical mixture models directly to the entropy distributions of unlabeled test data.

In unsupervised open-set graph domain adaptation, the authors of Towards Unsupervised Open-Set Graph Domain Adaptation via Dual Reprogramming employ a Beta Mixture Model (BMM) to model the entropy distribution of predictions. Because entropy values are bounded between 0 and 1 (when normalized), the Beta distribution is an ideal fit. The BMM automatically splits the data into two components—low-entropy knowns and high-entropy unknowns—yielding a clean, threshold-free probability estimate of whether a sample belongs to an unseen category.

Similarly, for resource-constrained scenarios, Schlachter et al. utilize a Gaussian Mixture Model (GMM) to track class clusters in feature space and compute the normalized Shannon entropy of the resulting cluster likelihoods. This statistical confidence measure effectively replaces complex auxiliary models, allowing systems to flag out-of-distribution samples purely based on how poorly they fit into the established mixture components.

Managing Class Asymmetry in Domain Adaptation

The utility of entropy-based novelty detection becomes critical in Universal Domain Adaptation (UDA) and Open-Set Domain Adaptation (OSDA). In these settings, a model must transfer knowledge from a labeled source domain to an unlabeled target domain, but the target domain often contains "private" classes not present in the source. Forcing the model to align these unknown target classes with source categories causes a phenomenon known as negative transfer, which degrades overall performance.

To prevent this, You et al. introduced a transferability criterion that pairs domain similarity with prediction uncertainty. By measuring the entropy of the target predictions, their network automatically identifies target-private samples and excludes them from the domain alignment process.

Target Data ──► Compute Prediction Entropy ──► [ Low Entropy ]  ──► Align with Source
                                           ──► [ High Entropy ] ──► Filter out (Unknown)

This filtering process can be dynamically integrated into optimization loops. For example, Liang et al. propose Universal Entropy Optimization (UEO) for fine-tuning vision-language models like CLIP on wild data. UEO uses initial confidence scores to dynamically weight adaptation: it minimizes entropy for high-confidence target samples (sharpening known classes) while maximizing entropy for low-confidence samples (pushing potential unknowns toward a uniform distribution).

Furthermore, entropy can act as a gating mechanism for calibration. Lin et al. use entropy-based gating to determine when to apply debiasing adjustments to pre-trained models, minimizing adjustments for low-entropy (likely known) categories and activating logit calibration for high-entropy (potentially novel) categories.

Extending Entropy to Open-Set Retrieval and Graph Spaces

The core mechanism of measuring the "flatness" of a distribution is not restricted to fixed-class softmax outputs; it generalizes effectively to open-set retrieval and metric learning spaces where explicit class labels do not exist.

In open-set retrieval tasks like person re-identification, a model evaluates queries against a dynamic gallery of individuals rather than a fixed set of training classes. To quantify uncertainty in this environment, Adachi et al. calculate the entropy of a pseudo-probability distribution derived from the cosine similarities between a query feature and its nearest neighbors. If a query matches a known individual, its similarity distribution will peak sharply around that identity. If the individual is new to the system, the similarities across the nearest neighbors will flatten out, yielding a high "re-id entropy" that flags the query as an unknown identity.

A similar principle applies to streaming data or graph structures. Jiaqian Ren et al. compute information entropy over the similarity distribution between a new data point and known class centers (reference landmarks) in a graph neural network. When a new social media event emerges, its uniform distance to existing event clusters creates a high-entropy signature, signaling to the system that it must allocate new representational capacity to capture an emerging category.

Key Foundational Work

Learning to Identify Unexpected Instances in the Test Set (2007) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_entropy_based_novelty_detection_da7245f2,
  title = {entropy-based novelty detection},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/entropy-based-novelty-detection/rdr_da7245f2dace4343bd8911c4d8d8196c},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_entropy_based_novelty_detection_da7245f2_v1,
  title = {entropy-based novelty detection},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/entropy-based-novelty-detection/rdr_da7245f2dace4343bd8911c4d8d8196c/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake