TL;DR
This paper demonstrates that you don’t always need to fine-tune massive deep learning models to get state-of-the-art results in medical imaging. By using pre-trained networks as "off-the-shelf" feature extractors and fusing them with traditional handcrafted features through a clever dissimilarity-based multi-view learning approach, the authors achieved a peak accuracy of 93% on breast cancer histology classification.
What Problem Does This Paper Address?
Deep learning has revolutionized image classification, but it has a "hunger" problem: it requires massive amounts of labeled data. In medical fields like histology (the study of microscopic tissue structure), datasets are often tiny. This paper tackles the High Dimensional Low Sample Size (HDLSS) problem.
When you only have a few hundred images, training a deep neural network from scratch is impossible, and even fine-tuning can lead to overfitting. Furthermore, researchers often struggle to combine different types of data—like "handcrafted" texture features and deep learning features—without the resulting feature vector becoming so large that it overwhelms the small number of available samples.
Methods
The authors propose a pipeline that treats different feature sets as different "views" of the same data:
- Feature Extraction (No Fine-Tuning): Instead of training, they use five deep learning models (ResNet-18, ResNet-152, ResNeXt, NASNet-A, and VGG16) pre-trained on ImageNet. They also extract traditional handcrafted features: PFTAS (morphological statistics) and GLCM (texture matrices).
- Random Forest Dissimilarity (RFD): For each feature group (or "view"), they train a Random Forest. They calculate a "dissimilarity" score between images based on how often they end up in different leaf nodes of the trees. If two images consistently land in the same leaf, they are considered similar.
- Multi-View Fusion (RFSVM): They average the dissimilarity matrices from all six views to create a single, joint similarity matrix.
- Classification: This joint matrix is used as a custom kernel for a Support Vector Machine (SVM). This approach, called RFSVM, effectively reduces the high-dimensional feature space into a manageable similarity-based representation.
Datasets
The study uses the ICIAR 2018 Grand Challenge on Breast Cancer Histology Images dataset.
- Composition: 400 H&E (Hematoxylin and eosin) stained microscopy images.
- Classes: The images are balanced across four categories: Normal, Benign, In situ carcinoma, and Invasive carcinoma.
- Protocol: The authors used a 75/25 train-test split, repeated 10 times to ensure statistical robustness.
Key Insights and Ideas
- Deep Learning Beats Handcrafted (Mostly): Even though the deep networks were trained on generic objects (like cats and cars in ImageNet), they significantly outperformed specialized handcrafted features for histology when used as feature extractors. ResNet-152 was the top individual performer.
- The Power of "Weak" Features: On its own, the handcrafted feature set was the worst performer (67% accuracy). However, when added to the deep learning ensemble, it boosted the overall accuracy. This proves that handcrafted features contain complementary information that deep networks might miss.
- Dissimilarity as a Dimensionality Reducer: By converting high-dimensional features (up to 31,855 dimensions) into a dissimilarity matrix (400x400), the authors bypassed the "curse of dimensionality" that usually plagues small datasets.
Key Results
The integration of multiple "views" showed a clear upward trend in performance:
- Handcrafted Features Only: 67.00% average accuracy.
- Best Single DL Model (ResNet-152): 79.30% average accuracy.
- Combining 5 DL Models (RFSVM-DL): 82.90% average accuracy.
- The Full Ensemble (RFSVM-All): 87.10% average accuracy, with a "best run" hitting 93.00%.
The model showed particularly high sensitivity (96%) for detecting invasive carcinoma, which is critical for clinical applications.
Limitations and Future Work
While the results are impressive, the authors note a few areas for improvement:
- Patch-level Information: To simplify the process, the authors rescaled whole images to fit the networks. Other state-of-the-art methods use "patch-wise" training (breaking images into smaller pieces), which might capture finer details.
- Computational Cost: Running six different feature extractors and building multiple Random Forests is computationally heavier than using a single model.
- Dataset Size: While 400 images is standard for this challenge, the authors suggest that testing on even larger, more diverse datasets would be necessary to validate the "multi-view" approach for clinical use.
Future work could involve exploring different ways to weight the "views" (e.g., giving more importance to ResNet-152 than the handcrafted features) rather than simple averaging.
Summary updated