Legitimate ground-truth-free metrics for deep uncertainty classification scoring

Arthur Pignet, Chiara Regniez, John Klein

Uncertainty Quantification (UQ) is essential for safety-critical AI applications, such as healthcare or autonomous driving, where knowing when a model is "guessing" is as important as the prediction itself. However, UQ is notoriously difficult to validate in production because there are no "true" probability labels for uncertainty in standard datasets. This research, presented at AISTATS 2025, bridges this "evaluation gap" by proving that specific metrics can objectively assess UQ performance without needing access to the underlying data distribution.

The Challenge of Evaluating Uncertainty

In standard classification, we compare a model's prediction to a ground truth label $y$. For uncertainty, no such label exists. While metrics like Expected Calibration Error (ECE) are common, they measure calibration (frequency matching) rather than a model's ability to rank its own trustworthiness. This paper focuses on the ranking ability: can a model correctly identify which inputs it is most likely to get wrong?

Methods: Grounding Metrics in Theory

The authors define two latent "ground truth" uncertainty measures:

  1. $\phi(x)$: The actual probability that the specific trained model will misclassify input $x$.
  2. $\varphi(x)$: The probability that the model's prediction disagrees with the optimal Bayes classifier.

UQ-AUC: Validating an Existing Standard

The authors prove that the widely used Uncertainty Quantification AUC (UQ-AUC) is a "legitimate" metric. Theorem 2.3 in the paper demonstrates that UQ-AUC is maximized if and only if the model's uncertainty score perfectly ranks the true misclassification probability $\phi(x)$. This provides the first rigorous theoretical justification for using UQ-AUC as a universal benchmark.

UQ-C-index: A Novel Robust Metric

The paper introduces the UQ-C-index, based on the concordance index. Unlike UQ-AUC, which relies on binary error labels, the UQ-C-index leverages the softmax gap (the distance between the model's confidence and 1). This metric is tied to $\varphi(x)$, rewarding models that align with optimal decision-making and filtering out "lucky guesses" where a model might be correct for the wrong reasons.

Experimental Validation

The theory was tested across four diverse domains to ensure broad applicability:

  • Synthetic Data: 2D Gaussians with mathematically known true probabilities.
  • CIFAR10-H: Image data with "soft" human labels capturing real-world ambiguity.
  • ReaL-ImageNet: Multi-label annotations for more accurate class uncertainty.
  • OASST2: A large-scale text dataset with human judgments for NLP tasks.

Key Results

The empirical findings strongly corroborate the theoretical proofs. Across all datasets, UQ-AUC and UQ-C-index showed near-perfect correlation with the hidden ground truth uncertainty.

Table 1: Pearson correlation between the proposed metrics and the hidden ground truth Kendall correlations across various datasets.

As shown in Table 1, the Pearson correlation often exceeds 95%, confirming that these metrics are reliable proxies for unobservable truths. Visual analysis further supports this linear relationship.

Figure 1: Scatter plots showing the linear relationship between the proposed metrics (UQ-AUC and UQ-C-index) and the Kendall correlation to hidden ground truth uncertainty across different datasets.

Conclusion and Future Work

By providing a mathematical "missing link," this work allows practitioners to validate model trustworthiness using only standard $(x, y)$ test pairs. While the current focus is on aleatoric uncertainty and binary classification extensions, future work aims to expand these metrics to native multi-class frameworks and out-of-distribution detection. These "legitimate" metrics offer a path toward safer, more transparent AI deployment in high-stakes environments.

Summary updated

BibTeX
@inproceedings{pignet2025legitimate,
  author = {Arthur Pignet and Chiara Regniez and John Klein},
  title = {{Legitimate ground-truth-free metrics for deep uncertainty classification scoring}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2025},
  url = {https://openreview.net/forum?id=t7J3l7RwGA},
}
Figure