TL;DR
This paper introduces a novel "Mutual Contrastive Low-rank Learning" (MCL) framework to improve glioma grading by jointly analyzing two types of whole slide images (WSIs): FFPE and frozen sections. By leveraging mutual learning, a normalized contrastive loss (NMC-loss), and a low-rank loss, the method effectively disentangles multi-modal representations, reduces intra-class variance, and enhances inter-class margins, leading to superior grading performance compared to single-modality or mixed-modality approaches.
What Problem Does This Paper Address?
Glioma grading, crucial for treatment planning, traditionally relies on manual histology assessment of whole slide images (WSIs). While automated grading using WSIs promises faster and more accessible diagnostics, current methods often face challenges. Specifically, models trained on formalin-fixed paraffin-embedded (FFPE) sections can be affected by artifacts introduced during tissue processing. Conversely, frozen sections, while preserving natural cellular morphology, can suffer from lower quality. Existing approaches typically focus on a single modality, potentially missing complementary information and introducing biases. This paper aims to overcome these limitations by developing a method that effectively integrates information from both FFPE and frozen sections to achieve more robust and accurate glioma grading.
Methods
The core of this work is the proposed Mutual Contrastive Low-rank Learning (MCL) scheme. It employs a parallel mutual learning strategy where two separate neural networks (Network $\alpha_1$ and $\alpha_2$) process FFPE and frozen sections, respectively, without weight sharing.
Key novel components include:
- Normalized Multi-modal Contrastive Loss (NMC-loss): This loss function is designed to disentangle complementary representations from different modalities (FFPE and frozen) within the same latent space. It uses layer normalization to handle the differing distributions of the two modalities, promoting better alignment and feature extraction.
- Low-rank (LR) Loss: This loss encourages the latent vectors from the same class to lie in a linear subspace by minimizing the matrix rank. It also promotes orthogonality between inter-class latent vectors. The goal is to reduce intra-class variance and increase inter-class margins, leading to more discriminative representations.
The overall optimization combines these losses with a standard classification loss (cross-entropy with Taylor Softmax for label smoothing) for each modality.
Datasets
The study utilized the Cancer Genome Atlas (TCGA) dataset, specifically focusing on glioblastoma (GBM) and lower-grade glioma (LGG) cases. The dataset comprised 499 patients with available FFPE and frozen sections. For pre-processing, WSIs were transformed, patched into 500x500 instances, and redundant patches were removed. The final dataset included 1,680,714 patches for FFPE and 483,886 for frozen sections. The dataset was split into 319 patients for training, 80 for validation, and 100 for testing. Data augmentation techniques were applied to increase sample size.
Key Insights and Ideas
- Synergy of Modalities: The paper strongly suggests that combining FFPE and frozen sections provides complementary information that is crucial for accurate glioma grading, outperforming single-modality approaches.
- Disentanglement is Key: The success of the MCL scheme hinges on its ability to disentangle representations. The NMC-loss is specifically designed to achieve this across modalities, while the LR loss further refines this by organizing the latent space.
- Addressing Intra-class Variance: Glioma, being a heterogeneous tumor type, presents significant intra-class variance. The LR loss is a novel way to tackle this by enforcing a structured latent space, which is often a challenge for standard contrastive methods.
- Layer Normalization for Heterogeneous Data: Using layer normalization instead of batch normalization in the contrastive loss is critical for handling the distinct distributions of FFPE and frozen sections, a common issue in multi-modal learning.
Key Results
The proposed MCL scheme achieved superior performance across various metrics (Accuracy, Precision, Recall) for both FFPE and frozen sections compared to single training and mixed training approaches.
- Overall Performance: MCL achieved an accuracy of 0.76 for FFPE and 0.74 for frozen sections, outperforming single training (0.71/0.70) and mixed training (0.70/0.68).
- Comparison with SOTAs: When combined with state-of-the-art methods like A-MIL, TransMIL, and CLAM, MCL consistently improved their performance, demonstrating its effectiveness in enhancing existing MIL frameworks. For instance, MCL+CLAM achieved 0.79 accuracy on FFPE and 0.75 on frozen sections.
- Loss Function Effectiveness: The paper demonstrated that the combination of NMC-loss and LR loss yielded the best results, outperforming other contrastive loss functions like KL-loss, marginal triplet loss, and NT-Xent loss.
- Visualization: UMAP visualizations showed that the MCL scheme effectively disentangled latent vectors, leading to more clustered representations for different tumor grades, indicating better separation and interpretability. Heatmaps also showed the model focusing on relevant proliferation regions.
Limitations and Future Work
- Temperature Hyper-parameter: The study noted that the training can collapse if the temperature hyper-parameter in the NMC-loss is set too low (below 0.05). While the models showed stable accuracy across different temperatures (0.1 to 1), further investigation into optimal temperature selection might be beneficial.
- Generalizability: While the paper suggests the scheme can be generalized to other MIL systems and cross-modality learning tasks, further validation on different datasets and cancer types would be valuable.
- Computational Cost: Mutual learning and contrastive methods can be computationally intensive. While not explicitly detailed as a limitation, future work could explore more efficient implementations or approximations.
Summary updated