On Evaluating CNN Representations for Low Resource Medical Image Classification

Taruna Agrawal, Rahul Gupta, Shrikanth S. Narayanan

Introduction

In medical imaging, particularly gastrointestinal (GI) endoscopy, obtaining large-scale labeled datasets is a significant challenge. While deep learning models like Convolutional Neural Networks (CNNs) are the state-of-the-art for image classification, they typically require vast amounts of data. Researchers often employ transfer learning—fine-tuning models pre-trained on general datasets like ImageNet—to adapt to these low-resource scenarios. However, selecting the optimal architecture (e.g., VGG, ResNet, MobileNet) usually involves exhaustive trial-and-error. This paper proposes a "one-shot" metric to predict model performance using only training data, significantly reducing computational overhead.

Methodology

The study evaluates two primary feature extraction strategies for GI landmark classification:

  1. Hand-crafted Features: A "kitchen sink" baseline comprising color descriptors, Tamura texture features, and edge histograms.
  2. CNN Feature Extraction: High-level features extracted from five pre-trained architectures: VGGNet, ResNet50, Inception-V3, XceptionNet, and MobileNet.

The core contribution is a novel performance metric ($A$) designed to "foretell" model success without full testing. The process involves:

  • Dimensionality Reduction: Projecting high-dimensional features into 2D space via t-SNE.
  • Probabilistic Modeling: Representing each class as a Gaussian distribution in the 2D space.
  • Overlap Estimation: Calculating the probability of correct class identification based on these distributions. The final metric $A$ is the average of these probabilities across all classes.

Experimental Results

The researchers utilized the KVASIR dataset, consisting of 8,000 images across eight classes (anatomical landmarks and pathological findings). A 50/50 split was used for training and testing.

Predictive Power of the Metric

The proposed metric $A$ demonstrated an 87% correlation with actual test set accuracies. While the absolute values of $A$ did not perfectly match the accuracy percentages, the metric accurately ranked the relative performance of the different architectures.

Comparison of true test accuracy vs. the proposed estimate A across different feature representations.

Model Performance Insights

  • CNNs vs. Manual Features: In most instances, CNN-derived features outperformed hand-crafted ones, confirming that ImageNet-trained models capture relevant medical nuances.
  • The ResNet Anomaly: ResNet50 performed surprisingly poorly (61.1% accuracy), falling below the hand-crafted baseline (71.6%). This underscores the difficulty of architecture selection in specialized domains.
  • Top Performers: MobileNet (81.7%) and XceptionNet (80.8%) emerged as the strongest individual models, suggesting that lightweight architectures can be highly effective for niche medical tasks.

Visual Analysis of Feature Separability

The study found a strong link between the visual "separability" of features in t-SNE plots and final classification accuracy. Models with distinct, non-overlapping clusters achieved higher performance.

t-SNE projections of different feature representations (a-g) showing varying levels of class separability.

As seen in the projections, architectures like MobileNet and XceptionNet show clearer class boundaries compared to the more overlapping clusters of ResNet.

Conclusion

This research provides a practical tool for researchers working with small medical datasets. By using the proposed "one-shot" metric, one can quickly identify promising architectures without exhaustive cross-validation. Future work will explore applying this metric to intermediate CNN layers and refining projection techniques to better predict specific class-level confusion.

Summary updated

BibTeX
@inproceedings{agrawal2019evaluating,
  author = {Taruna Agrawal and Rahul Gupta and Shrikanth S. Narayanan},
  title = {{On Evaluating CNN Representations for Low Resource Medical Image Classification}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2019},
  doi = {10.1109/icassp.2019.8682397},
}
Figure