Abstract
Radiologists usually observe anatomical regions of chest X-ray images as well as the overall image before making a decision. However, most existing deep learning models only look at the entire X-ray image for classification, failing to utilize important anatomical information. In this paper, we propose a novel multi-label chest X-ray classification model that accurately classifies the image finding and also localizes the findings to their correct anatomical regions. Specifically, our model consists of two modules, the detection module and the anatomical dependency module. The latter utilizes graph convolutional networks, which enable our model to learn not only the label dependency but also the relationship between the anatomical regions in the chest X-ray. We further utilize a method to efficiently create an adjacency matrix for the anatomical regions using the correlation of the label across the different regions. Detailed experiments and analysis of our results show the effectiveness of our method when compared to the current state-of-the-art multi-label chest X-ray image classification methods while also providing accurate location information.
Introduction
When interpreting a chest X-ray (CXR), radiologists perform a structured search across specific anatomical zones, such as the lung bases, heart silhouette, or hilar regions. Most existing AI models treat the entire X-ray as a single global input, a "black box" approach that often ignores crucial anatomical context. This failure to capture how a finding in one region (e.g., fluid in the pleural space) relates to findings in another (e.g., an enlarged heart) limits both diagnostic accuracy and interpretability. AnaXNet addresses these gaps by explicitly incorporating anatomical awareness and modeling the inter-dependencies between different chest structures.
Methodology
AnaXNet (Anatomy-aware X-ray Network) is designed as a three-stage pipeline that mimics clinical reasoning:
- Detection Module: Utilizing a Faster R-CNN with a ResNet-50 backbone, the model identifies and extracts features from 18 specific anatomical regions of interest (ROIs), including the upper lung zones, mediastinum, and cardiac silhouette.
- Anatomy GCN: These 18 regions are represented as nodes in a graph. An adjacency matrix, constructed using the Jaccard similarity of disease co-occurrence, defines the connections between nodes. The Graph Convolutional Network (GCN) updates regional features by aggregating information from neighboring anatomical structures.
- Cross-Anatomy Attention: A non-local attention mechanism weights the importance of different anatomical features before final multi-label classification through fully connected layers.
Experimental Setup
The model was evaluated using the Chest ImaGenome dataset, derived from the MIMIC-CXR database. This dataset includes 217,417 frontal CXR images with annotations for 18 anatomical regions and 9 pathology labels. Unlike standard datasets, Chest ImaGenome links findings directly to specific bounding boxes, which is essential for training the detection and graph modules of AnaXNet.
Results and Discussion
AnaXNet demonstrated superior performance compared to several baselines, including standard global DenseNet (GlobalView) and CheXGCN (which uses graphs for labels but not anatomy).
- Classification Performance: AnaXNet achieved an average AUC of 0.93, significantly outperforming GlobalView (0.89) and CheXGCN (0.90). It showed exceptional strength in detecting Enlarged Cardiac Silhouette (0.99 AUC) and Pleural Effusion (0.96 AUC).
- Localization and Interpretability: Qualitative analysis reveals that AnaXNet provides more accurate localization than global models. While global models might correctly identify a disease based on irrelevant image regions, AnaXNet's attention remains anchored to the correct anatomical structures.
Conclusion
By treating human anatomy as a graph, AnaXNet leverages the constant physical relationships between organs to improve diagnostic accuracy. While the model's performance is currently dependent on the accuracy of the initial ROI detection and the quality of NLP-extracted labels, it represents a significant step toward more interpretable and anatomically-grounded AI in radiology. Future work will focus on expanding the pathology set and integrating the model into automated report generation systems.
Summary updated