Abstract
Deep learning models for medical imaging typically require vast amounts of expert-annotated data, which is costly and time-consuming to obtain. This paper introduces two novel neural network architectures, CONAF and RAMAF, designed to leverage large-scale weakly labeled datasets alongside a small fraction of expert-annotated images. By utilizing "attention feedback," these models learn to focus on clinically relevant regions, resulting in improved detection of pulmonary lesions and significantly faster training convergence.
Introduction
The detection of lung nodules on chest radiographs is a critical yet challenging task in clinical practice. Small lesions are frequently obscured by anatomical structures such as ribs, the heart, and the diaphragm. While Deep Convolutional Neural Networks (DCNNs) have shown promise in image recognition, their performance in medical imaging is often bottlenecked by the requirement for thousands of manually segmented images.
Conversely, hospital archives contain millions of images with associated radiologist reports. While Natural Language Processing (NLP) can extract "weak labels" (e.g., presence or absence of a lesion) from these reports, they do not provide spatial localization. This study investigates whether a small amount of location data can be used to supervise the attention of a model trained primarily on weakly labeled data.
Methods
The researchers proposed two strategies to integrate "attention feedback" into the training process:
CONAF (Convolutional Networks with Attention Feedback)
CONAF employs a "soft attention" approach using a VGG13 backbone. The network generates a saliency map indicating regions of importance for classification. When a ground-truth bounding box is available, the model calculates a localization error by comparing the heatmap to the expert annotation. This error is back-propagated to force the model to focus on the correct anatomical regions.
RAMAF (Recurrent Attention Model with Annotation Feedback)
RAMAF utilizes a "hard attention" approach based on Reinforcement Learning (RL). Instead of processing the entire image, the model takes a sequence of "glimpses." A spatial reward is introduced: if a glimpse falls within an expert-annotated bounding box, the model receives an additional reward, effectively guiding its search strategy.
Datasets
The study utilized a dataset of 430,067 chest X-rays from Guy’s and St Thomas’ NHS Foundation Trust. Weak labels (Normal, Lesion, Others) were generated via NLP from clinical reports. A small subset of 2,196 images, containing 3,253 lesions, received expert bounding box annotations. This setup reflects a realistic clinical scenario where only ~9% of positive cases have detailed spatial data.
Results
The proposed models demonstrated significant improvements over standard weakly supervised methods:
- Accuracy: CONAF achieved an accuracy of 0.85 for "Lesion vs. Normal" and 0.76 for the more difficult "Lesion vs. All Others."
- Efficiency: RAMAF converged five times faster than standard recurrent attention models due to the guidance provided by spatial rewards.
- Localization: CONAF showed superior localization performance, with an average overlap of 0.43–0.45, compared to models relying solely on weak labels.
Discussion and Limitations
A key finding was the model's performance in the "Lesion vs. Others" category, which includes non-cancerous abnormalities that often mimic nodules. However, challenges remain regarding the detection of very small nodules (under 10mm), where accuracy dropped significantly. Future work may involve adaptive glimpse counts in RAMAF and addressing the inherent noise in NLP-generated labels.
Conclusion
By supervising the "attention" of neural networks with a minimal amount of expert data, the researchers demonstrated a path toward more efficient and accurate medical AI that can be trained on existing clinical archives.
Summary updated