Introduction
Open-Set Recognition (OSR) is a critical challenge in computer vision where models must not only classify known categories (In-Distribution, ID) but also identify unknown categories (Out-of-Distribution, OOD) that were not present during training. Standard deep learning models often fail here, exhibiting overconfidence on unfamiliar data. This paper outlines the 2nd-place winning solution for the 2024 OOD-CV Workshop’s OSR challenge, utilizing a training-free approach based on the EVA-CLIP vision-language model.
Methodology
The proposed framework avoids expensive retraining by leveraging powerful pre-trained representations and post-hoc refinement techniques.
1. Base Model: EVA-CLIP
The core of the system is the eva_giant_patch14_336 model. As a massive vision-language model pre-trained on LAION-400M, EVA-CLIP provides highly robust and generalized features that are naturally better at distinguishing semantic shifts than models trained on narrower datasets.
2. Test-Time Augmentation (TTA)
To enhance stability, the system employs Test-Time Augmentation. Instead of a single inference pass, the model processes four versions of the input image—including random rotations, resized crops, horizontal flips, and color jitters. The final prediction is derived from the average of the resulting logits, which significantly reduces variance in OOD detection.
3. Rectified Activation (ReAct)
A key component of the pipeline is ReAct, a post-hoc technique applied to the model's penultimate layer. Neural networks often produce extreme activation values for OOD samples, leading to overconfident misclassifications. ReAct truncates these activations at a predefined threshold ($c$), effectively "clipping" the noise that leads to false positives.
4. Temperature Scaling
Finally, temperature scaling ($T=1.1$) is applied to the averaged logits before the Softmax operation. This helps calibrate the probability distribution, improving the separation between ID and OOD scores based on the maximum logit value.
Experimental Setup
The solution was evaluated on the Semantic Shift Benchmark (SSB).
- In-Distribution (ID): ImageNet-1k.
- Out-of-Distribution (OOD): Subsets of ImageNet-21k.
- Evaluation Splits: 'Easy' (100,000 images) and 'Hard' (99,000 images), testing the model's sensitivity to varying degrees of semantic similarity.
Results
The method achieved 2nd place in the competition, demonstrating the effectiveness of post-hoc methods on large-scale foundation models.
| Metric | Value | Rank |
|---|---|---|
| AUROC | 79.77 | 5th |
| FPR95 | 61.44 | 2nd |
| Accuracy (ACC) | 87.00% | - |
Ablation studies confirmed the importance of each component. ReAct alone improved AUROC by 3%, while TTA reduced the standard deviation of OOD detection from 2.1% to 1.2%, providing a much more stable decision boundary.
Conclusion
This work demonstrates that high-performance OOD detection does not necessarily require complex training regimes. By combining the generalized features of EVA-CLIP with simple yet effective post-hoc techniques like ReAct and TTA, the authors created a robust, efficient system. Future work will focus on automating parameter tuning for ReAct and optimizing TTA for real-time applications.
Summary updated