Introduction
Interpreting chest X-rays (CXR) is a complex task characterized by significant inter-rater variability and inherent ambiguity. Standard deep learning models typically employ softmax classifiers that produce overconfident point estimates, failing to distinguish between aleatoric uncertainty (data noise) and epistemic uncertainty (model ignorance). This research introduces a framework that explicitly quantifies uncertainty using Subjective Logic to improve diagnostic reliability and model robustness.
Methodology
The proposed approach moves beyond standard probabilistic outputs by utilizing Subjective Logic and the Dempster-Shafer theory of evidence. Instead of a single probability, the model outputs evidence values ($e^+$ and $e^-$) that parameterize a Beta distribution.
Uncertainty ($\hat{u}$) is derived directly from the total evidence. When evidence is scarce, $\hat{u}$ approaches 1. The model is trained by optimizing the Bayes risk with a Beta-distributed prior, incorporating a Kullback-Leibler (KL) divergence term to regularize the model toward high uncertainty in the absence of conclusive evidence. This framework treats uncertainty as an orthogonal measure to the prediction, distinguishing between balanced evidence (ambiguity) and a lack of evidence (ignorance).
Experimental Setup
The framework was evaluated on two large-scale datasets: ChestX-Ray8 (112,120 images) and PLCO (185,421 images). To establish a ground truth, expert consensus sets were created where multiple board-certified radiologists reviewed test images to resolve label noise inherent in NLP-extracted reports.
Results
A key application of this framework is sample rejection. By identifying and "rejecting" the most uncertain cases for manual human review, the system's performance on the remaining automated cases improves significantly.
On the PLCO dataset, ROC-AUC increased from 0.86 to 0.91 by rejecting the 25% most uncertain cases. Furthermore, the model's uncertainty was found to be a strong indicator of label noise; uncertainty was significantly higher (0.41 vs 0.16) in cases where experts disagreed with the original NLP-derived labels.
Uncertainty-driven Bootstrapping
The authors also demonstrate that uncertainty can be used to clean training data. By filtering out the top 15% most uncertain training samples—often those with noisy or incorrect labels—the model's performance on clean test sets improved. For instance, Pleural Effusion AUC rose from 0.89 to 0.93 through this bootstrapping process. This strategy allows the model to become more robust against the noisy labels common in large medical datasets.
Conclusion
This work demonstrates that quantifying uncertainty is not just a safety feature but a tool for performance enhancement. By leveraging Subjective Logic, AI systems can identify their own limitations, effectively manage label noise, and optimize the human-AI diagnostic workflow. Future work may involve extending this to multi-label frameworks and exploring robust M-estimators to further refine the handling of uncertain samples.
Summary updated