Abstract
Precision oncology for colorectal cancer (CRC) is currently limited by the time and cost associated with molecular assays for biomarkers like Microsatellite Instability (MSI). While deep learning has shown potential in predicting these markers from H&E pathology slides, traditional convolutional neural networks (CNNs) often struggle with global context and generalization across different clinical settings. This study introduces a fully transformer-based pipeline trained on a massive multicentric dataset of over 9,000 patients. The model achieves state-of-the-art performance, demonstrating high sensitivity and, crucially, clinical-grade accuracy on endoscopic biopsies even when trained solely on surgical resections. These findings suggest a path toward immediate, automated biomarker screening at the point of initial diagnosis.
Introduction
The identification of genetic biomarkers such as Microsatellite Instability (MSI) and mutations in BRAF or KRAS is essential for tailoring treatment in colorectal cancer. However, the current gold standard—molecular testing via PCR or next-generation sequencing—can take weeks to process. This delay can postpone the initiation of targeted therapies or immunotherapy.
Digital pathology offers a solution by using deep learning (DL) to predict molecular status directly from routine hematoxylin and eosin (H&E) stained slides. Despite this potential, existing DL methods face significant hurdles: they often fail to capture the broad spatial context of a tissue slide, generalize poorly to data from external institutions, and show a marked performance drop when applied to small endoscopic biopsies compared to larger surgical resections. This study addresses these limitations by leveraging the superior contextual modeling and generalization capabilities of transformer architectures.
Methods
The researchers developed a three-stage, fully transformer-based pipeline designed to move beyond the limitations of traditional CNN-based architectures.
1. Feature Extraction with CTransPath
Instead of standard CNN encoders, the pipeline utilizes CTransPath, a Swin Transformer-based encoder. This model was pre-trained using self-supervised learning on a diverse array of pathology images. This approach allows the encoder to learn rich, domain-specific visual features without the need for manual, pixel-level annotations by pathologists.
2. Patch Aggregation via Multi-Headed Self-Attention (MSA)
To process the thousands of small image "tiles" that make up a whole-slide image, the team employed a Transformer network with Multi-Headed Self-Attention. Unlike simple pooling methods (like mean or max pooling) used in previous models, MSA allows the model to relate every tile to every other tile across the entire slide. This enables the model to "contextualize" local features based on the global tissue environment.
3. Weakly Supervised Learning
The system was trained using only patient-level labels (e.g., MSI-high vs. Microsatellite Stable). This weakly supervised approach is highly scalable, as it does not require pathologists to manually outline tumor regions, relying instead on the model to identify the most predictive areas through its attention mechanism.
Experimental Design and Datasets
The study's scale is a key differentiator, utilizing 10 different patient cohorts totaling 9,048 patients. This includes public datasets (TCGA, CPTAC) and several large multicentric cohorts (DACHS, DUSSEL, Epi700, MECC, MUNICH, NLCS, QUASAR, and YCR-BCIP). A critical component of the evaluation was testing the model on a set of 1,502 patients from the YCR-BCIP cohort to assess performance on endoscopic biopsies—a common real-world clinical scenario where tissue samples are limited.
Results
State-of-the-Art Accuracy and Generalization
The transformer-based model demonstrated robust performance across all targets. For MSI prediction on surgical resections, the model achieved a sensitivity of 0.97 and a negative predictive value (NPV) of 0.99. This high NPV is particularly significant for clinical workflows, as it suggests the model could reliably "rule out" MSI-high status, allowing clinicians to focus molecular testing resources on a smaller subset of patients.
Furthermore, the model showed exceptional stability when tested on external cohorts. While traditional CNN-based approaches often see performance drops of up to 0.21 AUROC when moving to new data, the transformer model's performance varied by a maximum of only 0.08 AUROC.
Bridging the Biopsy Gap
One of the study's most significant breakthroughs was the model's performance on endoscopic biopsies. Despite being trained only on surgical resections, the model achieved an AUROC of 0.91 on biopsy samples. This suggests that the transformer architecture learns features that are fundamental to the underlying biology, rather than features specific to the type of tissue preparation.
Data Efficiency and Multi-Target Success
The architecture proved to be remarkably data-efficient, reaching an AUROC > 0.9 with as few as 250 patients. In comparison, previous state-of-the-art methods (such as AttentionMIL) required nearly 4,000 patients to reach similar performance levels. Beyond MSI, the model also successfully predicted other critical biomarkers, including BRAF (AUROC 0.86) and KRAS (AUROC 0.75).
Discussion and Explainability
To ensure the model was making decisions based on relevant biological features, the researchers used "attention rollout" maps. These visualizations confirmed that the model focused on morphologically significant areas, such as lymphocyte aggregates and mucinous regions—both of which are established clinical indicators of MSI. This alignment with known pathology increases the interpretability and trustworthiness of the system.
Despite these successes, the authors identified areas for further refinement. The current model does not explicitly incorporate the spatial coordinates of tiles (positional encoding), which could potentially improve context-awareness. Additionally, while the model is robust, the "generalization gap" caused by stain variability between labs remains a challenge that may require more advanced normalization techniques.
Conclusion
This large-scale study demonstrates that fully transformer-based models can overcome the primary hurdles facing digital biomarkers in colorectal cancer. By providing high-accuracy, clinical-grade predictions on both resections and biopsies, this technology offers a scalable way to accelerate precision oncology and improve patient outcomes through earlier, data-driven treatment decisions.
Summary updated