TL;DR
This paper introduces ConOSR, a novel approach to Open Set Recognition (OSR) that leverages supervised contrastive learning to improve feature representations. By incorporating soft targets and mixup augmentation, ConOSR enhances the model's ability to distinguish known from unknown classes while maintaining high accuracy on known classes, outperforming existing state-of-the-art methods.
What Problem Does This Paper Address?
Traditional machine learning models often assume that the training and testing data come from the same set of classes (closed-set assumption). However, in real-world scenarios, it's common to encounter data from classes not seen during training. When faced with such "unknown" samples, standard classifiers tend to misclassify them into one of the known classes, leading to unreliable predictions. Open Set Recognition (OSR) aims to address this by developing models that can not only classify known classes accurately but also identify and reject samples from unknown classes. The challenge lies in achieving this while maintaining high performance on the known classes, as many existing OSR methods struggle with this trade-off. This paper focuses on improving OSR by enhancing the underlying representation learning capabilities of deep neural networks.
Methods
The core of ConOSR lies in its enhanced supervised contrastive learning approach. Here's a breakdown:
- Supervised Contrastive Learning (SupCon): This technique learns feature representations by pulling samples from the same class closer together (positive pairs) and pushing samples from different classes further apart (negative pairs) in the feature space.
- Soft Targets and Mixup: The authors introduce "SupCon-ST," an extension of SupCon that can handle "soft targets." This is crucial because they use data augmentation techniques like label smoothing (assigning a small probability to incorrect classes) and Mixup (linearly interpolating pairs of training examples and their labels). Mixup is particularly important as it generates "virtual examples" with ambiguous semantics, simulating unknown samples during training.
- Contrastive Representation Learning Step: In the first phase, ConOSR uses RandAugment (a strong data augmentation technique) to create multiple views of training data and Mixup to generate virtual examples. These augmented and mixed samples are then fed into a contrastive learning framework (SupCon-ST) to learn robust feature representations.
- Classifier Training Step: After learning good features, a lightweight classifier is trained on top of the learned representations. The rejection thresholds for unknown samples are then estimated from the training data.
Datasets
The paper evaluates ConOSR on several benchmark datasets:
- MNIST: A dataset of handwritten digits.
- SVHN: The Street View House Numbers dataset.
- CIFAR-10: A dataset of 10 common object classes.
- CIFAR+10 / CIFAR+50: Variations where CIFAR-10 is used for known classes and CIFAR-100 is used for unknown classes.
- TinyImageNet: A smaller version of ImageNet with 200 classes.
- Omniglot: A dataset of handwritten characters from various alphabets.
- Noise-MNIST: Synthetically generated noisy versions of MNIST.
- LSUN: A large-scale scene understanding dataset.
The experiments involve splitting these datasets to simulate OSR scenarios, where a subset of classes is designated as "known" and the remaining are treated as "unknown."
Key Insights and Ideas
- Contrastive Learning for OSR: The paper strongly suggests that improving representation learning through contrastive methods is a key to better OSR performance. Contrastive learning helps create more discriminative features, making it easier to detect when a sample lacks these discriminative features (i.e., is unknown).
- Soft Targets are Crucial: The ability to handle soft targets (via label smoothing and Mixup) is a significant advantage. This allows the model to learn from more nuanced training signals and better simulate the ambiguity of unknown samples during training.
- Mixup as a "Virtual Unknown": Using Mixup to create semantically vague examples is an effective way to expose the model to "unknown-like" data during training, improving its ability to reject true unknowns.
- Uniformity and Alignment: The analysis of contrastive learning properties highlights the importance of both "alignment" (positive pairs being close) and "uniformity" (features being spread out on the hypersphere). The proposed SupCon-ST method effectively balances these for OSR.
- Feature Focus: Class Activation Maps (CAMs) show that ConOSR's learned features are more focused on the essential parts of objects compared to plain CNNs, which are more sensitive to background or less discriminative regions. This focused representation aids in detecting the absence of expected features.
Key Results
- State-of-the-Art OSR Performance: ConOSR, particularly with the SupCon-ST enhancement, consistently outperforms existing OSR methods (like PROSER and ARPL) across various benchmark datasets, achieving higher AUROC scores. For instance, on TinyImageNet, ConOSR with SupCon-ST improves results by 2.7% over ARPL.
- Improved Closed-Set Accuracy: While OSR is the primary focus, ConOSR also demonstrates improved accuracy in traditional closed-set classification tasks compared to vanilla SupCon and plain CNNs, indicating that better representations benefit both scenarios.
- Robustness to Unknowns: The method shows strong performance in distinguishing known classes from diverse unknown samples, including noisy images and images from entirely different domains.
- Hyperparameter Sensitivity: The paper analyzes the impact of the rejection threshold parameter ($\lambda$), showing a trade-off between detecting unknown instances and maintaining accuracy on known classes. The default $\lambda=5$ provides good results.
Limitations and Future Work
- Computational Cost: The authors acknowledge that contrastive learning, especially with extensive data augmentation (RandAugment and Mixup), is computationally expensive and requires more GPU memory and training epochs compared to standard methods.
- Generalization to Novel Domains: While ConOSR excels at OSR, the learned features might not transfer as well to entirely novel domains as features learned by plain CNNs. This is attributed to the "focused" nature of contrastive features.
- Future Directions:
- Combining ConOSR with clustering methods to reduce computational cost and memory footprint.
- Extending OSR to lifelong learning scenarios.
- Investigating the decoupling modification for contrastive loss to potentially improve hard negative mining.
Summary updated