Introduction
Deep learning has demonstrated significant potential in automating Tuberculosis (TB) detection from Chest X-rays (CXR), particularly in resource-constrained regions. However, high classification accuracy can be deceptive. Models often fall victim to "shortcut learning," where they rely on image artifacts—such as the specific characteristics of an X-ray machine—rather than clinical pathology. This study addresses the "reliability gap" by investigating how pre-training and class-balancing techniques can improve model interpretability and generalization.
Methodology
The researchers utilized a DenseNet-121 architecture and implemented three primary strategies to mitigate diagnostic bias:
- Proxy Task Pre-training: Models were first trained on the NIH-CXR14 dataset (112,120 images) featuring 14 general lung abnormalities. This provides the model with a broader radiological "vocabulary" before specializing in TB.
- Mixed Objective Optimization Network (MOON): To address class imbalance, the MOON loss-weighting technique was employed. Unlike standard oversampling, MOON adaptively re-weights losses during training to prevent bias toward the majority "healthy" class.
- Interpretability Quantification: Saliency mapping (Grad-CAM, HiResCAM, and Score-CAM) was used to visualize model attention. The authors introduced Proportional Energy as a metric to quantify how much of the model's focus falls within expert-defined ground-truth bounding boxes.
Experimental Setup
The study compared five model configurations based on their training pipeline:
- $M_U$ / $M_B$: Unbalanced or Balanced fine-tuning only.
- $M_{U,U}$ / $M_{U,B}$ / $M_{B,B}$: Combinations of Unbalanced/Balanced pre-training followed by Unbalanced/Balanced fine-tuning.
Performance was evaluated on the TBX11K target dataset and an external Shenzhen dataset to test real-world generalization.
Key Results
While all five models achieved a "perfect" AUROC of 1.00 on the internal target dataset, their performance diverged significantly when tested for generalization and interpretability.
As shown in the results table, the standard unbalanced model ($M_U$) exhibited the lowest interpretability and weaker generalization (0.79 AUROC on external data). In contrast, the fully balanced model ($M_{B,B}$) achieved the highest external AUROC (0.88) and the best alignment with human experts, nearly tripling the Proportional Energy scores of the baseline models.
Qualitative Analysis
Qualitative evaluation via saliency maps confirms the quantitative findings. The baseline model ($M_U$) frequently focused on irrelevant anatomical regions, such as the shoulders or lower abdomen, to make its predictions. The $M_{B,B}$ model, however, consistently localized its attention on the actual lung pathologies identified by radiologists.
Conclusion
This research demonstrates that accuracy alone is an insufficient metric for clinical AI. By combining proxy task pre-training with adaptive class balancing (MOON), developers can significantly reduce shortcut learning. Although Proportional Energy scores suggest room for further refinement, this methodology provides a robust framework for building more trustworthy and interpretable diagnostic tools in medical imaging.
Summary updated