Introduction
Neural networks are increasingly deployed in safety-critical environments, yet their reliability remains difficult to quantify. Standard metrics like accuracy provide a binary view of performance—correct or incorrect—but fail to capture the underlying probability of failure across a data distribution. Furthermore, the widely used softmax function is notorious for overconfidence; adversarial attacks can easily manipulate a model into predicting an incorrect class with near-certainty. This research proposes a probabilistic framework to quantify robustness by treating neural network outputs as random variables.
Methodology: Logits as Gaussian Distributions
The core innovation of this approach lies in the observation that "logits"—the raw, unnormalized values from the final layer—tend to follow a Gaussian (Normal) distribution. By modeling these outputs statistically, we can move beyond relative confidence scores toward absolute measures of significance.
Based on this Gaussian assumption, two primary metrics are introduced:
- Dataset-Centric Metric: This calculates the overlap between the probability density functions (PDFs) of correct and incorrect logits. Using the inclusion-exclusion principle, it provides a theoretical upper bound on the error rate for the entire distribution, identifying "statistical danger zones" where classes are likely to be confused.
- Sample-Centric Metric: For individual test points, the model calculates a two-sided p-value based on the distributions learned during training. This acts as a statistical significance test: if a logit is an outlier relative to training data, the p-value drops, signaling low confidence regardless of the softmax output.
Adversarial Robustness and Key Results
The framework was evaluated using a CNN trained on the MNIST dataset. While the model achieved 99% baseline accuracy, the dataset-centric probabilistic error was calculated at 25.63%, revealing a much higher latent risk than simple accuracy suggests.
The true strength of the p-value metric is revealed under adversarial pressure, such as Projected Gradient Descent (PGD) attacks. While both softmax and p-value metrics encounter errors under attack, the p-value metric is far more "honest" about its uncertainty.
As shown in the results, at a confidence threshold of > 0.99, the softmax metric produced 2,566 high-confidence errors. In contrast, the p-value metric produced only 24. This demonstrates that when the p-value metric claims high confidence, it is statistically justified, making it a much more reliable indicator for safety-critical applications.
Limitations and Future Outlook
While promising, this framework currently relies on the assumption that logits are Gaussian. While this holds for simpler models on MNIST, further research is required to validate this for complex architectures like Transformers or larger datasets like ImageNet. Additionally, calculating joint probabilities for many classes is computationally intensive. Future work aims to integrate these probabilistic metrics directly into the training process to minimize distribution overlap and create inherently more robust models.
Summary updated