TL;DR
This paper introduces TRk-CNN, a novel deep learning architecture designed to improve the classification of medical images with highly correlated classes, such as the stages of glaucoma. By leveraging transferable weights from primitive binary classifiers, TRk-CNN effectively captures inter-class relationships, outperforming traditional multi-class and ranking-based CNNs on a glaucoma fundus image dataset.
What Problem Does This Paper Address?
Classifying medical images often presents a challenge when the different disease stages or conditions are not clearly distinct but rather exist on a continuous spectrum with high inter-class correlation. Traditional multi-class classification methods, like those using a softmax function, treat each class as independent, ignoring these crucial relationships. While Ranking-CNNs attempt to account for ordinal classes, they struggle to fully incorporate inter-class information into the final prediction. This is particularly relevant in diseases like glaucoma, where there are intermediate states (normal, suspect, glaucoma) that require nuanced classification for timely and effective treatment. The authors aim to bridge this gap by developing a method that explicitly models these inter-class dependencies.
Methods
The proposed TRk-CNN architecture consists of three main stages:
- Primitive Classification: This stage adapts the general Ranking-CNN approach. It employs $N-1$ binary sub-CNN models to classify $N$ ordinal classes. Each sub-CNN model is trained to determine if an input image belongs to a class greater than or equal to a reference class $k$. A key difference from standard Ranking-CNN is the absence of fully-connected layers at the top of these sub-CNNs, allowing for the extraction of Class Activation Maps (CAMs).
- ROI Extraction: CAMs generated from the primitive classification stage are combined to form a Region of Interest (ROI). This combination is not a simple summation; it incorporates a distance metric that accounts for the relationship between the predicted class and the reference class, aiming to weigh more relevant CAMs more heavily. This ROI is then combined with the original input image.
- Final Classification: The combined input (original image + ROI) is fed into a final classification stage. This stage uses a sophisticated classification process, including a fully-connected layer, to predict the final class. The architecture here also leverages the Ranking-CNN structure, starting from pre-trained weights from the primitive classification stage.
The backbone CNN used for the sub-classifiers is DenseNet-121, chosen for its ability to reuse features and reduce overfitting, which is beneficial for smaller datasets.
Datasets
The study utilizes a glaucoma fundus image dataset collected and labeled from Korea University Medical Center. This dataset contains images labeled into three classes: normal, glaucoma suspect, and glaucoma eyes. The authors state this is the first study to classify these three specific statuses of glaucoma fundus images.
Key Insights and Ideas
- Inter-class relationships are crucial: The paper strongly emphasizes that for medical image classification problems with highly correlated classes (like disease progression), ignoring inter-class relationships leads to suboptimal performance.
- Transferable CAMs for richer features: By extracting CAMs from primitive binary classifiers and combining them, TRk-CNN effectively transfers high-dimensional features that capture nuanced inter-class information, which is then used in the final classification.
- ROI as a refined input: The ROI extraction step, which intelligently combines CAMs based on predicted class relationships, acts as a refined input that guides the final classifier more effectively than the raw image alone.
- Glaucoma as a prime example: The authors highlight glaucoma as a disease where intermediate states are critical for early detection and treatment, making it an ideal testbed for their proposed method.
Key Results
TRk-CNN achieved an average accuracy of 92.96%, specificity of 93.33%, sensitivity for glaucoma suspect of 95.12%, and sensitivity for glaucoma of 93.98%. Compared to Ranking-CNN (Rk-CNN) and multi-class CNN (MC-CNN), TRk-CNN demonstrated superior performance across most metrics. Specifically, it was 8.04% and 9.54% higher in average accuracy than Rk-CNN and MC-CNN, respectively. Notably, it achieved a 26.83% higher sensitivity for glaucoma suspects compared to MC-CNN. The study also found that using the disc region of the fundus image yielded the best performance among the pre-processed regions (disc, extended disc, original), but an ensemble of models trained on all three regions achieved the highest overall performance.
Limitations and Future Work
- Generalizability: While TRk-CNN is presented as a general model for ordinal classification, its effectiveness on other medical imaging tasks with similar inter-class correlations is yet to be fully explored.
- Distance Metric Definition: The paper notes that the definition of the distance function used in ROI extraction is based on an assumption of linear relationship between classes. For some medical data, this relationship might not be linear, suggesting that the distance function could be adapted based on domain expertise.
- Future Work: The authors express their intention to apply TRk-CNN to other medical images where disease states are continuous and increase in the positive class direction. They also suggest exploring alternative methods for combining input and ROI in the final classification step.
Summary updated