TL;DR
This paper introduces DIOR-ViT, a novel Vision Transformer architecture that tackles cancer grading by not only classifying cancer into distinct categories but also learning the degree of difference between grades. This "differential ordinal learning" approach significantly improves accuracy and reliability in cancer classification, outperforming existing methods.
What Problem Does This Paper Address?
Cancer grading is a crucial step in determining prognosis and treatment. Traditionally, this is treated as a categorical classification problem, where each grade is an independent label. However, this overlooks the inherent ordinal nature of cancer grades: higher grades generally indicate more aggressive disease. Existing methods fail to leverage this ordering, potentially limiting their accuracy and robustness. The authors aim to bridge this gap by developing a model that explicitly learns the ordinal relationships and the magnitude of differences between cancer grades.
Methods
The core of the proposed method is the DIOR-ViT (Differential Ordinal Learning Vision Transformer). It's a multi-task learning framework built upon the Vision Transformer (ViT) architecture. DIOR-ViT comprises three main components:
- Feature Extractor (Tfeat): This component, based on ViT (specifically using ResNet50V2 for initial feature extraction), maps input pathology images into a high-dimensional feature space. This space is designed to retain the ordinal relationships between samples.
- Categorical Classifier (Tcat): This standard classifier takes the extracted features and predicts a discrete class label (e.g., cancer grade).
- Differential Ordinal Classifier (Tdiff): This is the novel component. It takes two feature vectors (from a "minuend" and a "subtrahend" sample) and predicts the difference in their ground truth ordinal labels. This allows the model to learn not just that grade A is different from grade B, but how much different they are.
A key innovation is the Negative Absolute Difference Log-Likelihood (LNAD) loss function, specifically designed for the differential ordinal classification task. This loss penalizes incorrect predictions on a logarithmic scale, providing a more nuanced learning signal than traditional regression losses.
Datasets
The authors evaluated their method on three types of cancer datasets:
- Colorectal Tissue Dataset: Publicly available, with four class labels (benign, well-differentiated, moderately-differentiated, poorly-differentiated).
- Prostate Tissue Dataset: Two public datasets were used, with four class labels (benign, grade 3, grade 4, grade 5 tumor).
- Gastric Tissue Dataset: A newly introduced large dataset containing over 100,000 tissue images with four histopathologic classes (benign, tubular well-differentiated, tubular moderately-differentiated, and tubular poorly-differentiated tumor).
Key Insights and Ideas
- Ordinality Matters: Explicitly modeling the ordered nature of cancer grades, rather than treating them as independent categories, significantly boosts classification performance.
- Differential Learning: Learning the magnitude of difference between grades, not just their relative order, provides a richer signal for the model. This is crucial because the difference between a high-grade and low-grade cancer is more significant than between two adjacent mid-grades.
- ViT as a Foundation: The Vision Transformer architecture proves effective in creating a feature space where ordinal relationships are well-preserved, making it suitable for this differential ordinal learning approach.
- LNAD Loss Effectiveness: The proposed LNAD loss function is critical for effectively optimizing the differential ordinal classification, outperforming conventional regression and ordinal cross-entropy losses.
Key Results
DIOR-ViT consistently outperformed nine competing models (including CNNs and other Transformers) across all three cancer types and evaluation metrics (Accuracy, F1-score, and Quadratic Weighted Kappa).
- Colorectal Cancer: DIOR-ViT achieved the best Accuracy (87.78%) and Kappa (0.942) on CTestI. On CTestII, it significantly outperformed others with an Accuracy gain of ≥ 5.26%.
- Prostate Cancer: DIOR-ViT achieved the best Accuracy (71.64%) on PTestI and significantly outperformed all models on PTestII (≥ 0.37% Acc).
- Gastric Cancer: DIOR-ViT was superior to all competing models, showing improvements of ≥ 0.47% Acc.
The paper also demonstrated that DIOR-ViT's performance was robust across datasets acquired from different institutes and scanned with different equipment.
Limitations and Future Work
- Model Complexity: DIOR-ViT, being based on ViT, is among the larger models in terms of parameters and FLOPs compared to CNN-based architectures. While training and inference times are comparable, further optimization for efficiency might be beneficial.
- Dataset Size: While the gastric dataset is large, the authors suggest that DIOR-ViT could benefit from even larger datasets, especially for Whole Slide Images (WSIs).
- Generalizability: The authors suggest that the differential ordinal learning framework could be applied to other problems and diseases where class labels possess ordering relationships, beyond cancer grading. Future work will focus on further developing and applying this method to a wider range of domains.
Summary updated