EXACT-Net: EHR-guided lung tumor auto-segmentation for non-small cell lung cancer radiotherapy

Rui Zhang, PhD, Facmi, Famia, Fiahsi, Hamed Hooshangnejad, Xue Feng, Gaofeng Huang, Rui Zhang, Quan Chen, Kai DingarXiv 2024 · DOI · arXiv

Introduction

In the clinical management of non-small cell lung cancer (NSCLC), radiotherapy is a cornerstone treatment. However, its efficacy depends on precise tumor segmentation—the process of mapping the tumor's boundaries on medical images. Currently, this task is largely manual, performed by radiation oncologists, which is both time-consuming and prone to inter-observer variability. While automated AI models for lung nodule detection have proliferated, they frequently suffer from high false-positive rates, mistaking benign structures like blood vessels or lung borders for malignancies.

EXACT-Net addresses this challenge by integrating visual data from CT scans with unstructured textual data from Electronic Health Records (EHR). By leveraging the "ground truth" often already present in radiology and pathology reports (e.g., the specific lobe containing the tumor), EXACT-Net uses Large Language Models (LLMs) to provide a spatial prior that filters out erroneous AI detections.

Methodology: The Vision-Language Pipeline

The EXACT-Net framework consists of a multi-stage pipeline designed to bridge the gap between computer vision and clinical text.

1. Vision Component: Retina-UNet3D

The system utilizes a 3D architecture combining RetinaNet for detection and UNet3D for segmentation. This model is responsible for identifying lung lobes and potential nodules within 3D CT volumes.

2. Language Component: LLM-Guided Filtering

The core innovation lies in the use of GPT-3.5 Turbo in a zero-shot configuration. Through specialized prompt engineering, the LLM parses unstructured clinical notes to extract the tumor's anatomical location.

3. Integration and Spatial Filtering

The LLM's output serves as a spatial constraint. If the vision model detects multiple candidates across different lobes, the system automatically discards any detections that do not align with the lobe identified in the EHR.

The EXACT-Net framework overview, illustrating the three-stage process: initial organ segmentation, EHR parsing via LLM, and EHR-guided tumor auto-segmentation.

Experimental Results

The framework was evaluated using the public LIDC-IDRI dataset for base training and institutional data from Johns Hopkins for the EHR-guided validation.

Detection Performance

The impact of EHR guidance was profound. Without textual filtering, the vision model's detections matched the ground truth in only 20% of cases due to the high volume of false positives. With the integration of LLM-parsed data, this success rate jumped to 70%, representing a 250% increase in successful tumor detection.

Visual demonstration of the filtering process: The LLM identifies the "left upper lobe" from patient reports, allowing the system to correctly identify the True Positive (TP) while discarding False Positives (FP) in the right and left inferior lobes.

Segmentation Quality

The model achieved high Dice Coefficient Scores for lung lobes (up to 0.98). For the more complex task of nodule segmentation, it achieved a score of 0.67, which is considered a strong performance given the inherent difficulty of small-target segmentation in noisy CT data.

Discussion and Limitations

While EXACT-Net demonstrates the power of "anatomy-aware" AI, several areas for improvement remain. The EHR-guided validation was conducted on a small cohort of 10 patients, necessitating larger-scale studies to ensure robustness across varied clinical reporting styles. Additionally, the model faced challenges with nodules adjacent to the chest wall and in advanced Stage 4 cases where extensive lymph node involvement and tumor spread exceeded the localized training data.

Conclusion

EXACT-Net represents a significant step toward clinical-grade automated radiotherapy planning. By treating clinical text as a high-level spatial prior, the system effectively mitigates the false-positive problem inherent in medical imaging AI, paving the way for faster and more accurate treatment workflows.

Summary updated

BibTeX
@misc{fiahsi2024exact,
  author = {Rui Zhang, PhD, FACMI, FAMIA, FIAHSI and Hamed Hooshangnejad and Xue Feng and Gaofeng Huang and Rui Zhang and Quan Chen and Kai Ding},
  title = {{EXACT-Net: EHR-guided lung tumor auto-segmentation for non-small cell lung cancer radiotherapy}},
  year = {2024},
  eprint = {2402.14099},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2402.14099},
}
Figure