Abstract
Deep neural networks often exhibit overconfidence when presented with out-of-distribution (OOD) data, posing significant safety risks in real-world deployments. While various post-hoc detection methods exist, their performance is highly sensitive to model architecture. This paper proposes ZODE (Zoo-based OOD Detection Enhancement), a framework that utilizes a "model zoo" to improve detection robustness. By applying the Benjamini-Hochberg statistical procedure to a diverse set of models, ZODE performs sample-aware model selection. Our approach significantly reduces false positive rates on standard benchmarks, demonstrating the power of collective intelligence in OOD detection.
Introduction
A major challenge in deploying deep learning systems in environments like autonomous driving or medical diagnostics is the "Out-of-Distribution" (OOD) detection problem. Models must identify when an input differs significantly from their training data to avoid making dangerous, overconfident predictions.
Existing state-of-the-art methods typically rely on a single pre-trained architecture. However, research indicates a "model selection dilemma": a model optimized for one type of OOD data (e.g., blurry textures) may fail on another (e.g., novel objects). The performance gap between the best and worst models on certain benchmarks can reach over 32%. Since the nature of OOD data encountered in the wild is unpredictable, picking a single model beforehand is inherently risky.
The ZODE Framework
To address this, we introduce ZODE, which moves away from the single-model bottleneck toward a "collective intelligence" approach. The framework consists of three main components:
- The Model Zoo: ZODE aggregates a diverse collection of models, including ResNets, DenseNets, and Swin Transformers, trained using various strategies (e.g., supervised and contrastive learning).
- P-Value Normalization: To compare scores across different architectures, ZODE converts raw model outputs into p-values based on a small validation set of in-distribution (ID) data.
- Sample-Aware Selection via Benjamini-Hochberg: To avoid the high false-positive rates associated with naive ensembles, ZODE employs the Benjamini-Hochberg (BH) procedure. This statistical method dynamically identifies "active" detectors for each specific test input.
Key Insights: The Power of Diversity
The success of ZODE stems from the inherent diversity of its model zoo. Different architectures perceive the world through different lenses; while one model might be sensitive to global shapes, another might focus on local textures.
By using p-values and the BH procedure, ZODE provides a statistically rigorous way to harness these diverse perspectives. It allows "specialist" models to flag specific OOD samples that a general-purpose model might miss, while maintaining a high True Positive Rate for in-distribution data.
Experimental Results
ZODE was evaluated on CIFAR10 and ImageNet-1K against a variety of OOD datasets (e.g., SVHN, LSUN, iNaturalist, SUN).
- CIFAR10: ZODE reduced the average False Positive Rate (FPR) from 11.07% to 3.83%, representing a 65.4% relative improvement.
- ImageNet: The average FPR dropped from 38.47% to 24.14%, a 37.25% relative improvement.
Crucially, ZODE remained robust across distribution shifts, adaptively switching to the most relevant models for each test case.
Conclusion and Future Work
ZODE demonstrates that a sample-aware ensemble of diverse models can significantly outperform single-model OOD detectors. While the computational cost of running a model zoo is higher, the gains in safety and reliability are substantial. Future research will focus on optimizing the zoo through model pruning and distributed computing to make ZODE more suitable for edge-device deployment.
Summary updated