TL;DR
This paper tackles the challenge of imbalanced data in medical image classification, specifically for colorectal cancer histology. They propose a novel "two-stage resampling" method that first oversamples images and then undersamples features to train convolutional neural networks more effectively, outperforming existing single-stage resampling techniques.
What Problem Does This Paper Address?
Data imbalance, where one class has significantly fewer samples than others, is a pervasive problem in machine learning, especially in medical imaging. This imbalance can lead to models that are biased towards the majority class, performing poorly on minority classes, which are often the most critical (e.g., identifying rare diseases). Traditional data-level resampling techniques like SMOTE (Synthetic Minority Over-sampling Technique) struggle with high-dimensional image data, often generating unrealistic synthetic images with artifacts. Undersampling, while simpler, discards valuable data, which is detrimental for training deep convolutional neural networks (CNNs) that require large datasets. This paper aims to bridge this gap by developing a resampling strategy that leverages the strengths of both oversampling and undersampling in a way that's suitable for CNNs.
Methods
The core contribution is a novel "two-stage resampling" approach.
- Stage 1: Image-Level Oversampling: The original imbalanced image dataset is first oversampled. The authors don't specify the exact oversampling method used in this stage in the abstract, but later experiments suggest SMOTE is a strong candidate. This oversampled dataset is then used for the initial training of a CNN.
- Stage 2: Feature-Level Undersampling: After the initial training, the network is used to extract high-level features from the original (not oversampled) dataset. These extracted features are then undersampled to create a balanced representation. This undersampled feature set is used to fine-tune the later layers (specifically, the fully-connected and softmax layers) of the CNN.
This two-stage approach aims to:
- Leverage a large amount of data for initial robust training (via oversampling).
- Mitigate the introduction of visual artifacts from image-level oversampling by performing the second stage resampling in the more abstract feature space.
- Avoid discarding too much data, which is a problem with direct undersampling.
The authors also explore variations, such as using SMOTE in the second stage or omitting the first stage entirely.
Datasets
The study uses a colorectal cancer histology dataset published by Kather et al. [14]. This dataset contains 5,000 histological images of human colorectal cancer, divided into eight tissue types. Each image has a dimensionality of 150x150 pixels. The original dataset was balanced, and artificial imbalance was introduced by random undersampling to create different imbalance ratios (IR) and types (linear, single majority, single minority, half minority).
Key Insights and Ideas
- SMOTE's limitations in image space: The paper highlights that SMOTE, while effective in general, can produce unrealistic synthetic images with artifacts when applied directly to high-dimensional image data. This is a critical observation for medical imaging where subtle details matter.
- The power of feature-level resampling: Performing undersampling in the high-level feature space, rather than directly on images, proves to be more effective. This suggests that the network learns to represent the data in a way that makes balancing at the feature level more robust and less prone to introducing noise or artifacts.
- Two-stage is better than one-stage: The proposed two-stage approach, combining image-level oversampling with feature-level undersampling, consistently outperforms single-stage resampling methods (applied either in image or feature space alone).
- Undersampling in feature space is crucial: The paper's results suggest that using undersampling in the feature space for fine-tuning is particularly beneficial, even outperforming a two-stage approach that uses SMOTE in both stages. This underscores the value of carefully balancing the learned representations.
- Accuracy is misleading for imbalanced data: The authors emphasize that classification accuracy can be a poor metric for imbalanced datasets, as it can mask poor performance on minority classes. They advocate for metrics like Average Accuracy (AvAcc), Class Balance Accuracy (CBA), and Geometric Average of Recall (MAvG).
Key Results
- The proposed two-stage resampling approach (specifically, SMOTE in the image space followed by RUS in the feature space) achieved the best performance across all considered metrics (AvAcc, CBA, MAvg) and imbalance scenarios.
- The two-stage approach with feature-level undersampling (TSSMOTE+RUS) achieved the highest average ranks in experiments, indicating its superior and consistent performance.
- Resampling directly in the image space (IS) showed improvement over the baseline, but was generally outperformed by feature-space (FS) and two-stage (TS) methods.
- Random undersampling (RUS) applied directly in the image space often led to worse performance than the baseline.
- The impact of imbalance was highly detrimental to performance on dedicated metrics (AvAcc, CBA, MAvg), even when classification accuracy showed little drop, reinforcing the need for appropriate evaluation metrics.
Limitations and Future Work
- Generalizability to natural images: The authors acknowledge that the success of SMOTE in the image space for this specific histological dataset might be due to its relatively small dimensionality and texture-based nature. They suggest further research is needed to confirm its usefulness on larger, higher-dimensional natural image datasets like ImageNet, where image interpolation artifacts might be more pronounced.
- Exploring other initial stage methods: While the paper focuses on oversampling in the first stage, they suggest exploring other methods that preserve a large amount of training data, such as class-balanced loss functions, as an alternative initial approach.
- Comparison with other advanced techniques: Future work could involve comparing their approach with generative adversarial networks (GANs) for data augmentation or other advanced imbalance-sensitive training protocols.
Summary updated