Abstract
Classical multiple instance learning (MIL) methods are often based on the identical and independent distributed assumption between instances, hence neglecting the potentially rich contextual information beyond individual entities. On the other hand, Transformers with global self-attention modules have been proposed to model the interdependencies among all instances. However, in this paper we question: Is global relation modeling using self-attention necessary, or can we appropriately restrict self-attention calculations to local regimes in large-scale whole slide images (WSIs)? We propose a general-purpose local attention graph-based Transformer for MIL (LA-MIL), introducing an inductive bias by explicitly contextualizing instances in adaptive local regimes of arbitrary size. Additionally, an efficiently adapted loss function enables our approach to learn expressive WSI embeddings for the joint analysis of multiple biomarkers. We demonstrate that LA-MIL achieves state-of-the-art results in mutation prediction for gastrointestinal cancer, outperforming existing models on important biomarkers such as microsatellite instability for colorectal cancer. Our findings suggest that local self-attention sufficiently models dependencies on par with global modules.
Introduction
In computational pathology, Whole Slide Images (WSIs) represent a significant data challenge due to their "gigapixel" scale. Analyzing these images typically requires Multiple Instance Learning (MIL), where a slide is decomposed into thousands of smaller image patches, or "tiles." Traditional MIL frameworks often treat these tiles as independent and identically distributed (i.i.d.) entities, effectively ignoring the spatial context and morphological relationships between neighboring cells and tissues.
While recent advancements have introduced Transformers to model these inter-tile dependencies via global self-attention, these methods face a fundamental scaling issue: quadratic computational complexity. As the number of tiles in a WSI grows, the memory and processing requirements for global attention become prohibitive. This research introduces LA-MIL (Local Attention Multiple Instance Learning), a framework that challenges the necessity of global modeling by demonstrating that local spatial context is often sufficient for accurate genetic mutation prediction.
The LA-MIL Framework
LA-MIL integrates Graph Neural Networks (GNNs) with Transformer architectures to create a spatially-aware model that remains computationally efficient. The pipeline consists of four primary stages:
- Feature Extraction: WSIs are tessellated into tiles, and a pretrained convolutional neural network (such as KimiaNet or DenseNet-121) extracts compact feature vectors for each tile.
- Spatial Graph Construction: To capture local context without the overhead of global modeling, the framework constructs a k-Nearest Neighbor (kNN) graph. This graph is based on the actual spatial coordinates of the tiles on the slide, ensuring that "attention" is only paid to physically adjacent regions.
- Local Attention Transformer: The model utilizes specialized Local Attention layers. Unlike standard Transformers where every tile attends to every other tile, LA-MIL restricts self-attention calculations to the $k$ neighbors defined by the spatial graph. This introduces a "spatial inductive bias," forcing the model to prioritize local tissue architecture.
- Multi-target Prediction: Rather than training isolated models for individual mutations, LA-MIL employs a multi-task learning strategy. A single backbone predicts multiple biomarkers simultaneously (e.g., MSI, TMB, and specific gene mutations). This is supported by a weighted Binary Cross-Entropy loss function designed to handle the class imbalance common in rare genetic mutations.
Experimental Results
The model was rigorously evaluated using two major cohorts from The Cancer Genome Atlas (TCGA): TCGA-CRC (594 colorectal cancer slides) and TCGA-STAD (440 stomach adenocarcinoma slides). The primary targets included clinical biomarkers like Microsatellite Instability (MSI) and Tumor Mutational Burden (TMB), alongside mutations in genes such as BRAF, KRAS, PIK3CA, and TP53.
Performance Benchmarks
LA-MIL achieved state-of-the-art (SOTA) results, particularly in predicting MSI in colorectal cancer. The model reached an AUROC of 0.85, a significant improvement over the previous benchmark of 0.77. Across the broader spectrum of genetic mutations, LA-MIL consistently matched or exceeded the performance of both instance-wise MIL and global Transformer-based methods.
The "Local is Enough" Hypothesis
A key insight from this study is that local self-attention performs on par with global modules. This suggests that the biological signals indicative of genetic alterations are primarily contained within local morphological structures—such as hyperchromatic nuclear chromatin or crowded glandular patterns—rather than long-range dependencies across distant areas of the slide.
Discussion and Future Work
By utilizing kNN graphs, LA-MIL bypasses the $O(N^2)$ complexity of standard Transformers, offering a computationally light alternative that scales effectively with varying slide sizes. Furthermore, the multi-task approach allows the model to learn shared representations, which appears to provide a synergistic boost to performance across multiple targets.
Despite these successes, challenges remain. The current model is limited to processing a single WSI per patient, whereas clinical practice often involves multiple slides. Additionally, while the model's attention maps highlight pathologically relevant regions, translating these "high-attention" zones into human-interpretable visual patterns for every specific gene mutation remains an ongoing area of research. Future work will focus on expanding this local attention hypothesis to other cancer types, such as lung and breast cancer, to further validate the universality of the LA-MIL approach.
Summary updated