Refining Tuberculosis Detection in CXR Imaging: Addressing Bias in Deep Neural Networks via Interpretability

Özgür Acar Güler, Manuel Günther, André Anjos

TL;DR

High classification accuracy in medical AI can be deceptive, often masking "shortcut learning" where models rely on image artifacts rather than clinical signs. This paper demonstrates that pre-training on a large-scale proxy task combined with class-balancing techniques (MOON) significantly improves a model's interpretability and its ability to generalize to new, unseen datasets without sacrificing performance.

What Problem Does This Paper Address?

Deep learning has shown immense promise in automating Tuberculosis (TB) detection from Chest X-rays (CXR), a critical need in regions with few radiologists. However, these models are often "black boxes." A model might achieve 100% accuracy on a specific dataset by "cheating"—for example, by learning to recognize the specific X-ray machine used in a hospital with high TB rates rather than the actual pathology in the lungs.

The authors tackle this "reliability gap." They argue that for a model to be truly trustworthy, it must not only get the right answer but also look at the right parts of the image—the same regions a human expert would use for diagnosis.

Methods

The researchers employed a DenseNet-121 architecture and explored several training strategies to mitigate bias:

  1. Proxy Task Pre-training: Instead of training only on TB data, they pre-trained models on a massive dataset of general lung abnormalities. This teaches the model a broader "vocabulary" of radiological signs.
  2. Mixed Objective Optimization Network (MOON): This is a loss-weighting technique designed to handle class imbalance. Unlike simple oversampling, MOON adaptively re-weights the loss for each class during training, ensuring the model doesn't become biased toward the majority class (usually "healthy" patients).
  3. Interpretability Evaluation: To measure if the model is "looking at the right things," they used saliency mapping techniques (Grad-CAM, HiResCAM, and Score-CAM). They quantified this using Proportional Energy, a metric that calculates how much of the model's "attention" falls within ground-truth bounding boxes provided by experts.

The researchers compared five different model configurations, varying whether pre-training and fine-tuning were "balanced" (using MOON) or "unbalanced" (standard training).

This table summarizes the five model configurations tested, showing the different combinations of pre-training and fine-tuning strategies.

Datasets

  • Proxy Dataset (NIH-CXR14): 112,120 images with 14 different image-level labels (e.g., pneumonia, fibrosis) used for initial pre-training.
  • Target Dataset (TBX11K): 11,702 images used for fine-tuning. Crucially, this dataset includes bounding boxes for TB-related radiological signs, allowing for interpretability testing.
  • External Dataset (Shenzhen): 662 images used strictly for evaluating how well the models generalize to data from a completely different source.

Key Insights and Ideas

  • Accuracy is a Poor Proxy for Trust: All five models achieved a "perfect" AUROC of 1.0 on the target test set. However, their internal reasoning varied wildly.
  • The "Balanced" Advantage: Naively training a classifier on the target dataset (Model $M_U$) resulted in the lowest interpretability. The model often focused on areas outside the lungs, like the armpits or the bottom of the image, to make its predictions.
  • Pre-training as a Debiasing Tool: Pre-training on a related but different task (general lung conditions) significantly reduced interpretation bias. It forced the model to learn features that are actually relevant to lung anatomy before it ever saw a TB case.
  • MOON is the Secret Sauce: Applying label balancing during both pre-training and fine-tuning (Model $M_{B,B}$) yielded the best alignment with human experts and the strongest generalization to the external Shenzhen dataset.

Key Results

The quantitative results highlight a clear trend: as models become more balanced and better pre-trained, their "real-world" utility increases.

The results show that while all models perform perfectly on the target set, the balanced pre-trained models (bottom rows) generalize much better to external data and show significantly higher Proportional Energy (interpretability).

Qualitatively, the difference is striking. In the figure below, the top row (a) shows the unbalanced model $M_U$ focusing on irrelevant areas. The bottom row (b) shows the best-performing model $M_{B,B}$ correctly identifying the regions of interest marked by experts (magenta boxes).

Saliency maps comparing the unbalanced model (top) vs. the balanced pre-trained model (bottom). Note how the balanced model's heatmaps align much better with the expert bounding boxes. Saliency maps for the balanced model.

Limitations and Future Work

Despite the improvements, the Proportional Energy scores remained relatively low (peaking at ~0.326). The authors note that this is partly due to the metric itself: radiological signs are rarely perfectly rectangular, so a model focusing perfectly on a lesion might still "miss" parts of a square bounding box.

Future work includes testing these models in actual clinical settings and using even more diverse, precisely annotated datasets to further refine the interpretability metrics. The authors also suggest that this methodology could be a blueprint for debiasing models in other medical imaging domains.

Summary updated

BibTeX
@inproceedings{guler2024refining,
  author = {Özgür Acar Güler and Manuel Günther and André Anjos},
  title = {{Refining Tuberculosis Detection in CXR Imaging: Addressing Bias in Deep Neural Networks via Interpretability}},
  booktitle = {Euvip},
  year = {2024},
  doi = {10.1109/euvip61797.2024.10772829},
}

Loading related content...

Figure