TRk-CNN: Transferable Ranking-CNN for image classification of glaucoma, glaucoma suspect, and normal eyes

Tae Joon Jun, Youngsub Eom, Dohyeun Kim, Cherry Kim, Ji-Hye Park, Hoang Minh Nguyen, Daeyoung KimExpert Systems with Applications 2019 · OpenReview

TL;DR

This paper introduces TRk-CNN, a novel deep learning architecture designed to improve the classification of medical images with highly correlated classes, such as the stages of glaucoma. By leveraging transferable weights from primitive binary classifiers, TRk-CNN effectively captures inter-class relationships, outperforming traditional multi-class and ranking-based CNNs on a glaucoma fundus image dataset.

What Problem Does This Paper Address?

Classifying medical images often presents a challenge when the different disease stages or conditions are not clearly distinct but rather exist on a continuous spectrum with high inter-class correlation. Traditional multi-class classification methods, like those using a softmax function, treat each class as independent, ignoring these crucial relationships. While Ranking-CNNs attempt to account for ordinal classes, they struggle to fully incorporate inter-class information into the final prediction. This is particularly relevant in diseases like glaucoma, where there are intermediate states (normal, suspect, glaucoma) that require nuanced classification for timely and effective treatment. The authors aim to bridge this gap by developing a method that explicitly models these inter-class dependencies.

Methods

The proposed TRk-CNN architecture consists of three main stages:

  1. Primitive Classification: This stage adapts the general Ranking-CNN approach. It employs $N-1$ binary sub-CNN models to classify $N$ ordinal classes. Each sub-CNN model is trained to determine if an input image belongs to a class greater than or equal to a reference class $k$. A key difference from standard Ranking-CNN is the absence of fully-connected layers at the top of these sub-CNNs, allowing for the extraction of Class Activation Maps (CAMs).
  2. ROI Extraction: CAMs generated from the primitive classification stage are combined to form a Region of Interest (ROI). This combination is not a simple summation; it incorporates a distance metric that accounts for the relationship between the predicted class and the reference class, aiming to weigh more relevant CAMs more heavily. This ROI is then combined with the original input image.
  3. Final Classification: The combined input (original image + ROI) is fed into a final classification stage. This stage uses a sophisticated classification process, including a fully-connected layer, to predict the final class. The architecture here also leverages the Ranking-CNN structure, starting from pre-trained weights from the primitive classification stage.

The backbone CNN used for the sub-classifiers is DenseNet-121, chosen for its ability to reuse features and reduce overfitting, which is beneficial for smaller datasets.

Datasets

The study utilizes a glaucoma fundus image dataset collected and labeled from Korea University Medical Center. This dataset contains images labeled into three classes: normal, glaucoma suspect, and glaucoma eyes. The authors state this is the first study to classify these three specific statuses of glaucoma fundus images.

Key Insights and Ideas

  • Inter-class relationships are crucial: The paper strongly emphasizes that for medical image classification problems with highly correlated classes (like disease progression), ignoring inter-class relationships leads to suboptimal performance.
  • Transferable CAMs for richer features: By extracting CAMs from primitive binary classifiers and combining them, TRk-CNN effectively transfers high-dimensional features that capture nuanced inter-class information, which is then used in the final classification.
  • ROI as a refined input: The ROI extraction step, which intelligently combines CAMs based on predicted class relationships, acts as a refined input that guides the final classifier more effectively than the raw image alone.
  • Glaucoma as a prime example: The authors highlight glaucoma as a disease where intermediate states are critical for early detection and treatment, making it an ideal testbed for their proposed method.

Key Results

TRk-CNN achieved an average accuracy of 92.96%, specificity of 93.33%, sensitivity for glaucoma suspect of 95.12%, and sensitivity for glaucoma of 93.98%. Compared to Ranking-CNN (Rk-CNN) and multi-class CNN (MC-CNN), TRk-CNN demonstrated superior performance across most metrics. Specifically, it was 8.04% and 9.54% higher in average accuracy than Rk-CNN and MC-CNN, respectively. Notably, it achieved a 26.83% higher sensitivity for glaucoma suspects compared to MC-CNN. The study also found that using the disc region of the fundus image yielded the best performance among the pre-processed regions (disc, extended disc, original), but an ensemble of models trained on all three regions achieved the highest overall performance.

Limitations and Future Work

  • Generalizability: While TRk-CNN is presented as a general model for ordinal classification, its effectiveness on other medical imaging tasks with similar inter-class correlations is yet to be fully explored.
  • Distance Metric Definition: The paper notes that the definition of the distance function used in ROI extraction is based on an assumption of linear relationship between classes. For some medical data, this relationship might not be linear, suggesting that the distance function could be adapted based on domain expertise.
  • Future Work: The authors express their intention to apply TRk-CNN to other medical images where disease states are continuous and increase in the positive class direction. They also suggest exploring alternative methods for combining input and ROI in the final classification step.

Summary updated

Abstract

In this paper, we proposed Transferable Ranking Convolutional Neural Network (TRk-CNN) that can be effectively applied when the classes of images to be classified show a high correlation with each other. The multi-class classification method based on the softmax function, which is generally used, is not effective in this case because the inter-class relationship is ignored. Although there is a Ranking-CNN that takes into account the ordinal classes, it cannot reflect the inter-class relationship to the final prediction. TRk-CNN, on the other hand, combines the weights of the primitive classification model to reflect the inter-class information to the final classification phase. We evaluated TRk-CNN in glaucoma image dataset that was labeled into three classes: normal, glaucoma suspect, and glaucoma eyes. Based on the literature we surveyed, this study is the first to classify three status of glaucoma fundus image dataset into three different classes. We compared the evaluation results of TRk-CNN with Ranking-CNN (Rk-CNN) and multi-class CNN (MC-CNN) using the DenseNet as the backbone CNN model. As a result, TRk-CNN achieved an average accuracy of 92.96%, specificity of 93.33%, sensitivity for glaucoma suspect of 95.12% and sensitivity for glaucoma of 93.98%. Based on average accuracy, TRk-CNN is 8.04% and 9.54% higher than Rk-CNN and MC-CNN and surprisingly 26.83% higher for sensitivity for suspicious than multi-class CNN. Our TRk-CNN is expected to be effectively applied to the medical image classification problem where the disease state is continuous and increases in the positive class direction.
BibTeX
@article{jun2019trk,
  author = {Tae Joon Jun and Youngsub Eom and Dohyeun Kim and Cherry Kim and Ji-Hye Park and Hoang Minh Nguyen and Daeyoung Kim},
  title = {{TRk-CNN: Transferable Ranking-CNN for image classification of glaucoma, glaucoma suspect, and normal eyes}},
  journal = {Expert Systems with Applications},
  year = {2019},
  url = {https://openreview.net/forum?id=DF9kN3tqwH},
}
Figure