Introduction
Novel Class Discovery (NCD) is a machine learning task where a model must leverage knowledge from a labeled dataset to automatically categorize a disjoint set of unlabeled images into novel classes. While existing methods have made strides in this area, they often suffer from two primary limitations. First, they frequently fail to explicitly model the disjoint nature of labeled and unlabeled sets, leading to overlapping feature representations. Second, they typically rely on rigid consistency constraints like Mean Squared Error (MSE) for data augmentation, which may not adequately capture semantic relationships under heavy transformations.
Proposed Method
To address these challenges, we introduce the Inter-class and Intra-class Constraints (IIC) framework. The architecture utilizes a shared ResNet-18 backbone ($E$) that branches into two specialized heads: $h$ for known labeled classes and $g$ for novel unlabeled classes.
The core of the IIC framework is the application of symmetric Kullback-Leibler divergence (sKLD) to enforce two distinct constraints:
- Inter-class sKLD Constraint: This constraint explicitly maximizes the distance between the probability distributions of labeled and unlabeled samples. By enforcing this separation, the model learns an embedding space where known and novel categories are clearly distinguished.
- Intra-class sKLD Constraint: Rather than forcing exact value matches between an image and its augmented version (as in MSE), sKLD ensures that the distribution of predictions remains consistent. This "slacker" approach is more robust to heavy augmentations like color jittering, which alter low-level pixels but preserve high-level semantics.
To maintain computational efficiency, the sKLD loss is implemented using matrix multiplication, allowing for high-performance execution on GPUs even with large batch sizes.
Experimental Evaluation
The IIC method was evaluated on several standard NCD benchmarks, including CIFAR10, CIFAR100 (with 20 and 50 novel class splits), and a large-scale ImageNet subset. These datasets provide a range of difficulty levels and class distributions to test the model's robustness and scalability.
Results and Discussion
The proposed IIC framework established new state-of-the-art performance across all tested benchmarks. Notably, on the challenging CIFAR100-50 split, IIC achieved a 3.5% to 3.7% improvement in clustering accuracy over previous leading methods such as UNO and ComEx. On CIFAR10, the model reached near-perfect performance with 99.1% accuracy.
Beyond numerical metrics, t-SNE visualizations demonstrate the qualitative advantages of the IIC approach. Compared to baseline methods like UNOv2, IIC produces significantly tighter and more distinct clusters for novel classes, effectively reducing the "cloud" overlap between visually similar categories.
Furthermore, IIC showed superior performance in task-agnostic scenarios, where the model must classify images without knowing a priori if they belong to a known or novel class. This suggests that the inter-class constraint successfully helps the model maintain a more coherent global understanding of the feature space.
Conclusion
The IIC framework demonstrates that explicitly modeling the separation between known and novel classes, combined with robust distributional consistency constraints, significantly advances the state of Novel Class Discovery. The use of sKLD as a "slacker" constraint provides a more effective mechanism for handling data augmentations than traditional MSE. Future work may explore the integration of these constraints into Generalized Category Discovery (GCD) settings, where unlabeled data contains a mixture of both known and unknown categories.
Summary updated