Abstract
We present a principled approach for detecting out-of-distribution (OOD) and adversarial samples in deep neural networks (DNNs). By modeling the internal feature distributions of a trained network using parametric probability distributions—specifically Gaussian and Gaussian Mixture Models (GMMs)—we derive uncertainty estimates that reliably discriminate in-distribution samples from anomalies. Our methodology demonstrates significant improvements in detection metrics (up to 12 percentage points in AUPR/AUROC) and provides the first reported reliable detection of white-box adversarial framing in video classifiers.
The Challenge of Neural Network Overconfidence
Deep neural networks are notoriously overconfident. Standard architectures often produce high-confidence predictions even when presented with inputs that look nothing like their training data (Out-of-Distribution) or inputs specifically engineered to fool them (Adversarial). The standard Softmax output provides only a relative ranking of classes, lacking an absolute measure of certainty. While Bayesian neural networks offer a solution, their high computational cost makes them impractical for many real-world applications. This research seeks a more efficient, "post-hoc" method to extract reliable uncertainty scores from standard, pre-trained networks.
A Generative Approach to Deep Features
Instead of relying solely on the final output layer, our approach analyzes "deep features"—the internal activations at various layers of the network. Once a network is trained, we pass the training data through it to fit parametric probability distributions to the features of each class.
To manage the high dimensionality of these features, we employ average pooling and Principal Component Analysis (PCA) to compress the data before modeling. During inference, we calculate the log-likelihood of a new sample's features against these learned distributions. A low likelihood serves as a "tell-tale" sign that the sample is OOD or adversarial.
Beyond Simple Gaussians: The GMM Advantage
Previous research often assumed that deep features follow a multivariate Gaussian distribution with a shared (tied) covariance matrix across all classes. We argue that this assumption is overly restrictive. Our work explores more flexible models:
- Separate Covariance: Each class is assigned its own unique Gaussian distribution.
- Gaussian Mixture Models (GMMs): Each class is modeled as a combination of multiple Gaussians, allowing for the capture of complex, non-convex shapes in the feature space.
As shown in Figure 6, a single Gaussian fails to capture the structure of complex feature manifolds, whereas a GMM provides a much more faithful representation. This precision is critical for internal layers where the data has not yet been linearized by the network.
Experimental Validation and Results
The methodology was validated across image (MNIST, CIFAR10) and video (UCF101) domains.
- Detection Performance: Switching from tied-covariance Gaussians to GMMs or separate-covariance models resulted in improvements of up to 12 percentage points in AUPR and AUROC for OOD detection.
- Classification Parity: Probabilistic models used for classification yielded accuracy comparable to standard Softmax, confirming they are faithful representations of the network's logic.
- Video Adversarial Detection: Our approach is the first to reliably detect white-box adversarial framing in video classifiers. While these attacks cause accuracy to plummet, the internal feature likelihoods remain a robust indicator of the attack.
Conclusion
Modeling deep features with flexible parametric distributions provides a powerful and efficient mechanism for anomaly detection in DNNs. While GMMs increase computational complexity, the significant gains in detection reliability justify their use, especially in safety-critical applications. Future work will explore how these distributions evolve during training and the use of large-scale datasets to inform prior distributions for specialized tasks.
Summary updated