Abstract
Pulmonary Embolism (PE) is a life-threatening condition where blood clots obstruct lung arteries. Rapid diagnosis via Computed Tomography Pulmonary Angiograms (CTPA) is essential but challenging due to the high volume of image slices. This paper introduces a two-stage, attention-based CNN-LSTM network designed to automate PE detection and characterization. By combining an EfficientNet-based spatial feature extractor with a Bidirectional LSTM for sequential context and a Multiple Instance Learning (MIL) attention mechanism, the model achieves an Area Under the Curve (AUC) of 0.95 on independent test data. Crucially, the model extends beyond binary classification to predict clinical features such as clot location, chronicity, and heart strain indicators, mirroring the comprehensive analysis performed by radiologists.
Introduction
Pulmonary Embolism (PE) remains a leading cause of preventable hospital mortality. Radiologists must manually review hundreds of CTPA slices to identify subtle clots, a process that is both time-intensive and prone to error under high workloads. Existing deep learning approaches often struggle with the 3D nature of CT data, either treating slices independently or requiring dense, pixel-level annotations for segmentation. Furthermore, clinical management of PE requires more than a simple positive/negative result; information regarding clot laterality (left, right, or central), chronicity (acute vs. chronic), and right ventricular strain is vital for treatment planning.
Methods
The proposed architecture employs a two-stage approach to mimic human diagnostic reasoning.
- Spatial Feature Extraction: An EfficientNet CNN backbone extracts features from individual slices. Input images are pre-processed into 3-channel RGB format using specific Hounsfield Unit (HU) windows (Lung, PE, and Mediastinal) to highlight relevant tissues.
- Sequential Reasoning: A Bidirectional LSTM (Bi-LSTM) processes the sequence of slice features, capturing the 3D continuity of vascular structures across the CT volume.
- Attention Mechanism: A Multiple Instance Learning (MIL) pooling layer acts as an attention module, weighting informative slices more heavily to generate study-level predictions.
- Multi-Head Output: The network utilizes dual output heads: $\psi_{image}$ for slice-level detection and $\psi_{study}$ for comprehensive study-level characteristics, including laterality and the Right Ventricular to Left Ventricular (RV/LV) diameter ratio.
- Label Consistency: A custom loss function and logic rules ensure logical consistency between slice-level and study-level predictions (e.g., a study cannot be positive if no individual slices are detected as positive).
Datasets
The model was trained and validated using the RSNA-STR PE (RSPECT) dataset, comprising over 12,000 CT studies. To evaluate generalizability, the researchers employed an independent, in-house test set of 106 studies from a separate clinical site, featuring different scanning protocols and equipment.
Results
The attention-based CNN-LSTM demonstrated robust performance across both validation and independent test sets.
Key results include:
- High Diagnostic Accuracy: The model achieved an AUC of 0.95 and an accuracy of 0.88 on the independent test set, significantly outperforming standard CNN baselines.
- Clinical Depth: The multi-head architecture successfully predicted secondary clinical findings like RV/LV ratio (AUC 0.957 for ratio > 1), which serves as a proxy for heart strain.
- Interpretability: t-SNE visualizations show clear separation between positive and negative studies in the feature space. Class Activation Maps (CAMs) further confirmed that the model focuses on actual intraluminal filling defects (clots) rather than image artifacts.
Discussion and Conclusion
By integrating spatial and sequential information with an attention mechanism, this network provides a clinically relevant tool for PE diagnosis. Its ability to generalize to external datasets suggests high potential for real-world deployment. While the model shows some limitations in pinpointing exact clot locations in complex bilateral cases, its overall performance in identifying high-risk features like heart strain offers significant value for prioritizing urgent cases. Future work will focus on end-to-end training and the integration of radiomic features to further enhance diagnostic precision.
Summary updated