Solution for OOD-CV Workshop SSB Challenge 2024 (Open-Set Recognition Track)

Mingxu Feng, Dian Chao, Peng Zheng, Yang YangarXiv 2024 · DOI · arXiv

Introduction

Open-Set Recognition (OSR) is a critical challenge in computer vision where models must not only classify known categories (In-Distribution, ID) but also identify unknown categories (Out-of-Distribution, OOD) that were not present during training. Standard deep learning models often fail here, exhibiting overconfidence on unfamiliar data. This paper outlines the 2nd-place winning solution for the 2024 OOD-CV Workshop’s OSR challenge, utilizing a training-free approach based on the EVA-CLIP vision-language model.

Methodology

The proposed framework avoids expensive retraining by leveraging powerful pre-trained representations and post-hoc refinement techniques.

1. Base Model: EVA-CLIP

The core of the system is the eva_giant_patch14_336 model. As a massive vision-language model pre-trained on LAION-400M, EVA-CLIP provides highly robust and generalized features that are naturally better at distinguishing semantic shifts than models trained on narrower datasets.

2. Test-Time Augmentation (TTA)

To enhance stability, the system employs Test-Time Augmentation. Instead of a single inference pass, the model processes four versions of the input image—including random rotations, resized crops, horizontal flips, and color jitters. The final prediction is derived from the average of the resulting logits, which significantly reduces variance in OOD detection.

3. Rectified Activation (ReAct)

A key component of the pipeline is ReAct, a post-hoc technique applied to the model's penultimate layer. Neural networks often produce extreme activation values for OOD samples, leading to overconfident misclassifications. ReAct truncates these activations at a predefined threshold ($c$), effectively "clipping" the noise that leads to false positives.

Overview of the proposed OOD detection pipeline, featuring Test-Time Augmentation (TTA), the EVA-CLIP encoder, and the ReAct rectification process.

4. Temperature Scaling

Finally, temperature scaling ($T=1.1$) is applied to the averaged logits before the Softmax operation. This helps calibrate the probability distribution, improving the separation between ID and OOD scores based on the maximum logit value.

Experimental Setup

The solution was evaluated on the Semantic Shift Benchmark (SSB).

  • In-Distribution (ID): ImageNet-1k.
  • Out-of-Distribution (OOD): Subsets of ImageNet-21k.
  • Evaluation Splits: 'Easy' (100,000 images) and 'Hard' (99,000 images), testing the model's sensitivity to varying degrees of semantic similarity.

Results

The method achieved 2nd place in the competition, demonstrating the effectiveness of post-hoc methods on large-scale foundation models.

Metric Value Rank
AUROC 79.77 5th
FPR95 61.44 2nd
Accuracy (ACC) 87.00% -

Ablation studies confirmed the importance of each component. ReAct alone improved AUROC by 3%, while TTA reduced the standard deviation of OOD detection from 2.1% to 1.2%, providing a much more stable decision boundary.

Conclusion

This work demonstrates that high-performance OOD detection does not necessarily require complex training regimes. By combining the generalized features of EVA-CLIP with simple yet effective post-hoc techniques like ReAct and TTA, the authors created a robust, efficient system. Future work will focus on automating parameter tuning for ReAct and optimizing TTA for real-time applications.

Summary updated

BibTeX
@misc{feng2024solution,
  author = {Mingxu Feng and Dian Chao and Peng Zheng and Yang Yang},
  title = {{Solution for OOD-CV Workshop SSB Challenge 2024 (Open-Set Recognition Track)}},
  year = {2024},
  eprint = {2409.20277},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2409.20277},
}
Figure