Abstract
Foundation model (FM)-based multiple instance learning (MIL) demonstrates significant promise in predicting clinical outcomes for head and neck squamous cell carcinoma (HNSCC). This study shows that end-to-end MIL models achieve performance comparable or superior to traditional handcrafted radiomics, particularly enhancing prognostic accuracy for non-HPV+ patients.
Introduction
Predicting treatment response in head and neck squamous cell carcinoma (HNSCC) remains a clinical challenge, with 5-year survival rates varying widely between 30% and 70%. Current prognostic tools, such as TNM staging and HPV status, often fail to capture the intricate biological heterogeneity of these tumors. While traditional radiomics—the extraction of high-dimensional mathematical features from medical imaging—has been proposed as a solution, it is often hampered by the need for manual segmentation, lack of standardization, and labor-intensive feature engineering.
Deep learning offers an alternative, yet its application is frequently limited by the scarcity of large, annotated medical datasets. This research investigates the utility of foundation models—AI architectures pre-trained on massive, diverse datasets—integrated into a Multiple Instance Learning (MIL) framework to extract prognostic biomarkers directly from routine CT scans.
Methods
The study employed a Multiple Instance Learning (MIL) framework to process 3D CT data without requiring slice-by-slice annotations. In this "bag-of-instances" approach, the entire set of tumor-containing slices is treated as a single entity for patient-level prediction.
The workflow consists of three primary stages:
- Feature Extraction: Pre-trained foundation models (e.g., BioMedClip, ResNet50) serve as backbones to generate feature embeddings from CT slices.
- Aggregation: A MIL Transformer processes these embeddings to learn spatial and contextual relationships between slices, outputting a unified risk score.
- Endpoint Prediction: The models were trained to predict three critical clinical outcomes: 2-year overall survival (OS), locoregional control (LRC), and freedom from distant metastasis (FFDM).
Datasets and Patient Selection
The researchers evaluated their approach using a large-scale dataset comprising 2,485 patients across three distinct cohorts. The primary cohort, RADCURE (Toronto, Canada), was utilized for training (n=1,464) and internal testing (n=606). External validation was performed on the HN1 cohort (Maastricht, Netherlands; n=131) and the HN-PET-CT cohort (Canada; n=284).
Results and Discussion
The end-to-end MIL approach demonstrated robust predictive performance, often surpassing traditional methods. Key findings include:
- Dimensionality and Performance: 2D MIL models utilizing axial slices surprisingly outperformed more complex 3D or multiview architectures. This suggests that prognostic information in HNSCC is highly concentrated in axial cross-sections and that current 3D foundation models may require more specialized pre-training for head and neck anatomy.
- Comparison with Radiomics: The MIL models achieved 2-year test AUROCs of 0.75–0.84 for overall survival. Notably, in the HN-PET-CT cohort, the MIL model (AUROC 0.84) significantly outperformed traditional handcrafted radiomics (AUROC 0.68, p=0.012).
- Clinical Utility in non-HPV+ Subgroups: While HPV status is a strong prognostic indicator, the MIL scores provided the most substantial improvement to clinical baseline models in non-HPV+ patients. For this high-risk group, combining clinical data with MIL features increased the AUROC for OS and FFDM to 0.87.
- Risk Stratification: The models effectively stratified patients into high- and low-risk categories, yielding Hazard Ratios (HR) between 2.14 and 4.77.
Conclusion
This study validates the potential of foundation model-based MIL for automated, end-to-end clinical outcome prediction in HNSCC. By matching or exceeding the performance of labor-intensive radiomics, this approach simplifies clinical workflows and provides critical prognostic insights, particularly for non-HPV+ patients. Future research should focus on developing 3D foundation models specifically tailored for oncology and exploring the prognostic value of the peritumoral environment.
Summary updated