Discovering Novel Categories in Semi Supervised Learning

23 concepts23 papers

Traditional semi-supervised learning (SSL) operates under a strict closed-world assumption: the unlabeled data pool is assumed to share the exact same category distribution as the small labeled training set. In real-world deployments, this assumption routinely breaks down. Unlabeled data collected from the wild inevitably contains out-of-distribution samples or entirely novel categories.

When a standard SSL model encounters these unseen classes, its performance degrades significantly. As Alex Yuxuan Peng et al. demonstrated, hidden novel classes act as severe distractors, forcing the model to generate high-confidence but erroneous pseudo-labels. Instead of expanding the model's knowledge, the uncurated unlabeled data corrupts the existing decision boundaries.

To resolve this, a new paradigm shifts the objective from merely generalizing within a closed set to simultaneously classifying known categories and discovering entirely new ones.

The Core Mechanism: Dual-Objective Learning

To operate successfully in an open world, a model must execute two distinct behaviors at once: it must exploit supervised signals to anchor known classes, and it must use unsupervised clustering to discover the latent structure of unseen classes.

The foundational intuition behind this dual task is to combine a supervised classification loss with a pairwise similarity or contrastive clustering objective. For instance, Maria Brbic et al. implemented this via a geometric deep learning framework for single-cell data, using a supervised component for known cell types alongside a pairwise objective that groups unannotated data points into novel clusters.

By grounding the representative selection process in the actual distribution of the unlabeled target data—rather than relying solely on the source data, as argued by Suchen Wang et al.—the model naturally carves out new regions in the feature space for emerging classes.

The Learning Pace Dilemma

Why can't we simply train a standard neural network with both losses simultaneously? The primary challenge lies in an optimization imbalance known as the learning pace discrepancy. Labeled data provides a strong, immediate gradient that allows the model to master known classes rapidly. Unsupervised clustering, by contrast, is an emergent process that requires time to discover stable data structures.

If the model learns the known classes too quickly, the representation space collapses around those known classes before the novel categories can form distinct clusters. To fix this, Bo Ye proposed Learning Pace Synchronization (LPS), which introduces adaptive margins to intentionally slow down the learning of known classes while using contrastive clustering to accelerate the grouping of new ones. Similarly, Shengjie Niu et al. developed a hierarchical thresholding scheme to balance the conditional self-labeling speeds between known and novel categories, preventing the model's optimization from ignoring the slower discovery process.

Structuring the Representation Space

Without explicit geometric constraints, the features of different novel categories tend to jumble together, a phenomenon known as representation collapse. Researchers have tackled this by forcing the model's latent space to adopt rigid, well-separated structures.

The authors of Targeted Representation Alignment for Open-World Semi-Supervised Learning addressed this by leveraging "Neural Collapse." Instead of letting the geometry emerge organically, they force features to align with a pre-defined Simplex Equiangular Tight Frame (ETF)—a geometric structure that ensures the maximum possible mathematical separation between all class centers, both known and unknown. From a different angle, Yiyou Sun et al. introduced OpenCon, which uses prototype-based novelty detection paired with a specialized contrastive loss to dynamically push unknown class clusters away from known class boundaries.

To understand why this cross-class transfer works mathematically, Yiyou Sun et al. provided a graph-theoretic framework. They proved that open-world representation learning is equivalent to performing spectral decomposition on a data graph. This theoretical foundation explains exactly how the geometric structure learned from labeled data acts as a scaffolding that guides the discovery of entirely new, unlabelled topological clusters.

Alternative Perspectives: Filtering vs. Decomposition

While many frameworks focus on clustering novel classes, alternative paradigms treat the presence of unknown data differently. One approach is strict isolation: Yen‐Cheng Liu identified that open-set object detectors suffer from "semantic expansion" (mistakenly treating unknown objects as known ones with high confidence) and introduced an offline self-supervised filtering module to purge novel classes from the pseudo-labeling stream entirely.

Conversely, Jun-Yi Hang et al. argued that trying to explicitly filter or detect outliers introduces a fragile point of failure. Their framework, BDMatch, transforms the problem by decomposing the multi-class open-set problem into multiple simple binary tasks. In this setup, unknown outliers are not filtered out; instead, they serve as natural negative examples for every single known class, turning a disruptive force into a useful regularizer for the model's decision boundaries.

Key Foundational Work

Investigating the effect of novel classes in semi-supervised learning (2019) — contributed 1 concept elements to this direction.
Joint Representative Selection and Feature Learning: A Semi-Supervised Approach. (2019) — contributed 1 concept elements to this direction.
Semi-Supervised Streaming Learning with Emerging New Labels (2020) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_open_world_semi_supervised_learning_11c30a07,
  title = {open-world semi-supervised learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/open-world-semi-supervised-learning/rdr_11c30a071e8249e59bde09bdd2c60b5a},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_open_world_semi_supervised_learning_11c30a07_v1,
  title = {open-world semi-supervised learning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/open-world-semi-supervised-learning/rdr_11c30a071e8249e59bde09bdd2c60b5a/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}

See something wrong? Report a mistake