Abstract
Colorectal cancer (CRC) recurrence remains a significant clinical challenge, with approximately 30% of patients experiencing relapse post-surgery. Current monitoring methods, such as CEA blood markers and periodic imaging, often suffer from low sensitivity or high costs. This paper introduces Fusion with Multi-view Attention (FMA), a deep-learning framework that integrates heterogeneous data sources—static clinical records, longitudinal blood markers, and unstructured radiology reports. By employing specialized feature extraction heads and a multi-view attention mechanism, FMA achieves an AUROC of 0.97, significantly outperforming traditional clinical benchmarks and single-view models.
Introduction
Colorectal cancer is a leading cause of global mortality. For survivors, the threat of recurrence is substantial, yet early detection remains difficult. Standard clinical practices rely heavily on the Carcinoembryonic Antigen (CEA) blood marker, which is frequently criticized for its low sensitivity, and expensive imaging scans that carry radiation risks.
The primary data science challenge in improving prognosis is heterogeneity. A patient’s clinical history is a complex tapestry of static data (demographics, tumor stage), time-series data (longitudinal lab results), and unstructured text (radiology reports). Traditional AI models often focus on a single data modality, failing to capture the cross-view interactions essential for accurate clinical assessment.
The FMA Architecture
To address these challenges, the researchers developed the Fusion with Multi-view Attention (FMA) framework. This architecture processes diverse data streams through a two-stage process: specialized feature extraction followed by dynamic multi-view fusion.
Specialized Feature Extraction
Rather than using a uniform processing method, FMA utilizes three distinct "heads" tailored to specific data types:
- tab-cnn: Processes tabular clinical data by converting it into a high-dimensional space and using a Convolutional Neural Network (CNN) to identify local patterns.
- ts-transformer: A modified Transformer architecture for longitudinal lab results (e.g., CEA levels), utilizing "Convolutional Self-Attention" to focus on local temporal contexts.
- text-temporalattn: Leverages a pre-trained ClinicalBERT model to process sequences of radiology reports, using an attention mechanism to weigh the importance of specific reports over time.
Multi-view Fusion and Survival Modelling
The "brain" of the system is the Multi-view Attention Block. Instead of simple concatenation, this block uses an attention mechanism to dynamically weigh the importance of each data view for a specific patient. This mimics clinical intuition, where a clinician might prioritize a sudden spike in blood markers for one patient while focusing on subtle changes in imaging for another. Finally, the framework incorporates a Cox proportional hazards model, enabling the prediction of both the likelihood and the timing of recurrence.
Experimental Setup and Results
The study utilized a cohort of approximately 1,000 Stage I-III CRC patients, incorporating 65 clinical variables, longitudinal CEA levels (median 40 months), and unstructured radiology reports.
Key Performance Metrics
The FMA model demonstrated state-of-the-art performance, achieving:
- AUROC: 0.97
- Sensitivity: 0.90
- Specificity: 0.95
This represents a vast improvement over the standard CEA-only approach, which typically yields a sensitivity of approximately 0.5. Furthermore, sensitivity analysis confirmed that the multi-view approach is superior to any single-view or dual-view configuration. Notably, the text-based feature extractor (text-temporalattn) was the strongest individual component, suggesting that the nuanced language in radiology reports contains critical predictive signals often missed in structured data.
Conclusion and Future Directions
The FMA framework represents a significant step forward in personalized cancer prognosis. By effectively fusing heterogeneous longitudinal data, it provides a high-fidelity early warning system for CRC recurrence.
However, several areas for future work remain. The current study is retrospective and based on a single cohort; prospective validation is necessary to ensure robustness against "dataset drift." Additionally, while the multi-view attention weights offer some interpretability, further efforts are needed to make these deep-learning models fully transparent for clinical decision-making. Future iterations will also explore the generalizability of the FMA architecture to other cancer types and clinical domains.
Summary updated