Modeling Inter-Class and Intra-Class Constraints in Novel Class Discovery

Wenbin Li, Zhichen Fan, Jing Huo, Yang Gao

Introduction

Novel Class Discovery (NCD) is a machine learning task where a model must leverage knowledge from a labeled dataset to automatically categorize a disjoint set of unlabeled images into novel classes. While existing methods have made strides in this area, they often suffer from two primary limitations. First, they frequently fail to explicitly model the disjoint nature of labeled and unlabeled sets, leading to overlapping feature representations. Second, they typically rely on rigid consistency constraints like Mean Squared Error (MSE) for data augmentation, which may not adequately capture semantic relationships under heavy transformations.

Proposed Method

To address these challenges, we introduce the Inter-class and Intra-class Constraints (IIC) framework. The architecture utilizes a shared ResNet-18 backbone ($E$) that branches into two specialized heads: $h$ for known labeled classes and $g$ for novel unlabeled classes.

The IIC architecture featuring shared backbone $E$ and specialized heads $g$ and $h$ for novel and known classes, respectively.

The core of the IIC framework is the application of symmetric Kullback-Leibler divergence (sKLD) to enforce two distinct constraints:

  1. Inter-class sKLD Constraint: This constraint explicitly maximizes the distance between the probability distributions of labeled and unlabeled samples. By enforcing this separation, the model learns an embedding space where known and novel categories are clearly distinguished.
  2. Intra-class sKLD Constraint: Rather than forcing exact value matches between an image and its augmented version (as in MSE), sKLD ensures that the distribution of predictions remains consistent. This "slacker" approach is more robust to heavy augmentations like color jittering, which alter low-level pixels but preserve high-level semantics.

To maintain computational efficiency, the sKLD loss is implemented using matrix multiplication, allowing for high-performance execution on GPUs even with large batch sizes.

Experimental Evaluation

The IIC method was evaluated on several standard NCD benchmarks, including CIFAR10, CIFAR100 (with 20 and 50 novel class splits), and a large-scale ImageNet subset. These datasets provide a range of difficulty levels and class distributions to test the model's robustness and scalability.

Results and Discussion

The proposed IIC framework established new state-of-the-art performance across all tested benchmarks. Notably, on the challenging CIFAR100-50 split, IIC achieved a 3.5% to 3.7% improvement in clustering accuracy over previous leading methods such as UNO and ComEx. On CIFAR10, the model reached near-perfect performance with 99.1% accuracy.

Beyond numerical metrics, t-SNE visualizations demonstrate the qualitative advantages of the IIC approach. Compared to baseline methods like UNOv2, IIC produces significantly tighter and more distinct clusters for novel classes, effectively reducing the "cloud" overlap between visually similar categories.

t-SNE visualization comparing UNOv2 (left) and the proposed IIC method (right) on the CIFAR10 dataset.

Furthermore, IIC showed superior performance in task-agnostic scenarios, where the model must classify images without knowing a priori if they belong to a known or novel class. This suggests that the inter-class constraint successfully helps the model maintain a more coherent global understanding of the feature space.

Conclusion

The IIC framework demonstrates that explicitly modeling the separation between known and novel classes, combined with robust distributional consistency constraints, significantly advances the state of Novel Class Discovery. The use of sKLD as a "slacker" constraint provides a more effective mechanism for handling data augmentations than traditional MSE. Future work may explore the integration of these constraints into Generalized Category Discovery (GCD) settings, where unlabeled data contains a mixture of both known and unknown categories.

Summary updated

Abstract

Novel class discovery (NCD) aims at learning a model that transfers the common knowledge from a class-disjoint labelled dataset to another unlabelled dataset and discovers new classes (clusters) within it. Many methods, as well as elaborate training pipelines and appropriate objectives, have been proposed and considerably boosted performance on NCD tasks. Despite all this, we find that the existing methods do not sufficiently take advantage of the essence of the NCD setting. To this end, in this paper, we propose to model both inter-class and intra-class constraints in NCD based on the symmetric Kullback-Leibler divergence (sKLD). Specifically, we propose an inter-class sKLD constraint to effectively exploit the disjoint relationship between labelled and unlabelled classes, enforcing the separability for different classes in the embedding space. In addition, we present an intra-class sKLD constraint to explicitly constrain the intra-relationship between a sample and its augmentations and ensure the stability of the training process at the same time. We conduct extensive experiments on the popular CIFAR10, CIFAR100 and ImageNet benchmarks and successfully demonstrate that our method can establish a new state of the art and can achieve significant performance improvements, e.g., 3.5%/3.7% clustering accuracy improvements on CIFAR100-50 dataset split under the task-aware/-agnostic evaluation protocol, over previous state-of-the-art methods. Code is available at https://github.com/FanZhichen/NCD-IIC.
BibTeX
@inproceedings{li2022modeling,
  author = {Wenbin Li and Zhichen Fan and Jing Huo and Yang Gao},
  title = {{Modeling Inter-Class and Intra-Class Constraints in Novel Class Discovery}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2022},
  doi = {10.1109/cvpr52729.2023.00336},
}
Figure