Abstract
Deep neural networks often exhibit overconfidence when encountering Out-of-Distribution (OOD) data, posing significant risks in safety-critical applications. While activation-based detection methods have been proposed, they typically treat all penultimate layer neurons as equally relevant. We introduce Discriminability-Driven Channel Selection (DDCS), a test-time framework that identifies and utilizes only the most discriminative channels to enhance OOD detection. By rectifying anomalous activations and selecting channels that maximize in-distribution class separation, DDCS achieves state-of-the-art performance across multiple benchmarks without requiring model retraining.
Introduction
The ability to detect OOD samples—inputs that do not belong to any training class—is essential for the reliable deployment of machine learning models. Standard neural networks tend to assign high confidence to such samples, leading to potential failures in autonomous driving or medical diagnostics. Existing "activation-based" methods attempt to mitigate this by pruning or rectifying unusual neuron firings. However, these approaches often overlook the inherent variance in the utility of individual feature channels for OOD detection.
The Performance Gap in Feature Channels
A key observation of this work is that not all neurons in the final feature layer are equally effective at distinguishing between known and unknown data. Some channels provide a clear signal, while others introduce noise that degrades detection performance.
As shown in the figure above, individual channels exhibit significantly different OOD detection capabilities. This "performance gap" suggests that a global selection strategy could isolate high-quality features and improve overall robustness by discarding noisy or detrimental channels.
Methodology: Discriminability-Driven Channel Selection
The proposed DDCS framework operates in two main stages at test time:
1. Channel-level Anomalous activations Pre-rectifying (CAP)
Before evaluating channel importance, DDCS applies a mild rectification to activations. This step prevents extreme outliers from disproportionately influencing the discriminative scores. CAP combines local rectification, which clamps values to a high-density region, with global truncation to stabilize the feature space.
2. Discriminative Channel Selection
The core of DDCS is a selection mechanism based on In-Distribution (ID) training data. A discriminative score is calculated for each channel using two primary metrics:
- Low Inter-class Similarity: Channels should ideally produce distinct average values for different classes.
- High Inter-class Variance: Channels should exhibit a wide spread between the means of different classes.
Channels with the highest scores are deemed most "discriminative." At inference, only the top-$K$ channels are used to compute the final OOD score (e.g., Energy score). This approach is computationally efficient and requires no changes to the pre-trained model weights.
Experimental Results
DDCS was evaluated on standard OOD benchmarks using various backbones, including DenseNet-101, ResNet-50, and MobileNetV2.
Impact on OOD Score Distributions
The effectiveness of DDCS is most apparent when examining the separation between ID and OOD score distributions. By filtering out non-discriminative channels, the overlap between these distributions is significantly reduced.
As illustrated, the overlap between CIFAR-100 (ID) and iSUN (OOD) energy scores drops from 57.06% to 20.88% when DDCS is applied, leading to much more reliable detection and lower false positive rates.
Quantitative Performance
DDCS achieves state-of-the-art results across multiple benchmarks:
- CIFAR-10: Average FPR95 of 13.57%, outperforming the previous best method, LINe (14.75%).
- ImageNet-1K: On large-scale benchmarks, DDCS reduced average FPR95 to 19.36%, a substantial improvement over ReAct (31.43%) and LINe (20.69%).
Conclusion
Discriminability-Driven Channel Selection (DDCS) demonstrates that "less is more" in the context of OOD detection. By identifying a global set of high-quality, discriminative channels, the method clarifies the OOD signal and significantly reduces false positive rates. Because it is a test-time, feature-level approach, it is easily integrated into existing pipelines and generalizes well across different architectures and scoring functions. Future work will explore dynamic, input-dependent channel selection to further refine detection capabilities.
Summary updated