Abstract
Locating diseases in chest X-ray images with few careful annotations saves large human effort. Recent works approached this task with innovative weakly-supervised algorithms such as multi-instance learning (MIL) and class activation maps (CAM), however, these methods often yield inaccurate or incomplete regions. One of the reasons is the neglection of the pathological implications hidden in the relationship across anatomical regions within each image and the relationship across images. In this paper, we argue that the cross-region and cross-image relationship, as contextual and compensating information, is vital to obtain more consistent and integral regions. To model the relationship, we propose the Graph Regularized Embedding Network (GREN), which leverages the intra-image and inter-image information to locate diseases on chest X-ray images. GREN uses a pre-trained U-Net to segment the lung lobes, and then models the intra-image relationship between the lung lobes using an intra-image graph to compare different regions. Meanwhile, the relationship between in-batch images is modeled by an inter-image graph to compare multiple images. This process mimics the training and decision-making process of a radiologist: comparing multiple regions and images for diagnosis. In order for the deep embedding layers of the neural network to retain structural information, we use the Hash coding and Hamming distance to compute the graphs, which are used as regularizers to facilitate training. By means of this, our approach achieves the state-of-the-art result on NIH chest X-ray dataset for weakly-supervised disease localization.
Introduction
Locating diseases in medical images typically requires "fine-grained" annotations—essentially, radiologists drawing bounding boxes around every abnormality. This is incredibly time-consuming and expensive. Consequently, most large-scale datasets only provide "weak" labels (e.g., image-level classification). Existing weakly-supervised localization (WSL) methods, such as Class Activation Maps (CAM), tend to focus only on the most "discriminative" part of a lesion, often resulting in incomplete or inaccurate regions. These methods fail because they treat image patches in isolation, ignoring the contextual clues radiologists use, such as comparing the left lung to the right lung or comparing the current patient to previous cases.
Methods
The authors propose GREN, which introduces two novel graph-based regularization modules to a standard ResNet-50 backbone to mimic the comparative decision-making process of radiologists.
Intra-image and Inter-image Knowledge Learning
The model uses a pre-trained U-Net to segment the left and right lung lobes. It then builds a graph to compare these two regions. Since many lung diseases are asymmetrical, this comparison helps the model identify subtle deviations from "normal" symmetry. Simultaneously, an inter-image module models relationships between different images within a training batch, encouraging the network to learn features that are consistent across different instances of the same disease.
Graph Regularization via Perceptual Hashing
Instead of building graphs directly from the noisy features of the deep network, GREN uses Hash coding and Hamming distance. Perceptual hashing captures the structural "fingerprint" of an image. By using these hashes as a fixed reference, the model forces its deep embedding layers to preserve essential structural information during training. The final objective function combines a standard Multi-Instance Learning (MIL) loss with these two graph-based regularizers.
Results
GREN achieved state-of-the-art (SOTA) performance on the NIH Chest X-ray dataset (112,120 images) across multiple evaluation settings.
At an Intersection over Union (IoU) threshold of 0.5, GREN achieved a mean accuracy of 0.54, significantly outperforming previous benchmarks like CAM (0.06) and other recent graph-based or attention-based models. The model proved robust even when trained on 100% unannotated images, demonstrating its effectiveness in true weakly-supervised scenarios. Qualitatively, GREN's predicted regions align much more closely with ground-truth bounding boxes compared to baseline models, which often produced fragmented heatmaps.
Conclusion
GREN demonstrates that modeling anatomical and cross-instance relationships is vital for accurate disease localization. While currently dependent on pre-trained segmentation and coarse anatomical blocks, future work will focus on more granular modeling (e.g., five distinct lobes) and improving detection for small targets like pulmonary nodules.
Summary updated