Predicting the progression of ophthalmic disease based on slit-lamp images using a deep temporal sequence network v1

Jiewei Jiang, Xiyang Liu, Lin Liu, Shuai Wang, Erping Long, Haoqing Yang, Fuqiang Yuan, Deying Yu, Kai Zhang, Liming Wang, Zhenzhen Liu, Dongni Wang, ChangZun Xi, Zhuoling Lin, Xiaohang Wu, Jiangtao Cui, Mingmin Zhu, Haotian Lin2018 · DOI

Abstract

Ocular images play an essential role in ophthalmology. Current research mainly focuses on computer-aided diagnosis using slit-lamp images; however, few studies have been done to predict the progression of ophthalmic disease. Exploring an effective approach for prediction can help clinicians plan treatment strategies and provide early warnings for patients. In this study, we present an end-to-end temporal sequence network (TempSeq-Net) to automatically predict the progression of ophthalmic diseases based on consecutive slit-lamp images. First, we comprehensively compare six potential combinations of three convolutional neural networks (CNNs) and long short-term memory (LSTM) or recurrent neural networks (RNNs) in terms of effectiveness and efficiency to obtain the optimal TempSeq-Net model. Second, we analyze the impacts of sequence lengths on the model’s performance, which helps evaluate their stability and validity and determine the appropriate range of sequence lengths.

Introduction: Moving Beyond Static Diagnosis

In modern ophthalmology, computer-aided diagnosis (CAD) has reached a high level of maturity, particularly in identifying diseases from single "snapshot" slit-lamp images. While identifying a condition is a vital first step, clinical management often hinges on understanding the trajectory of the disease. For chronic conditions, the most critical question for both patients and clinicians is: How fast is this progressing?

Predicting disease progression is significantly more complex than static diagnosis. It requires the capture of temporal changes—subtle shifts in tissue morphology or pathology that manifest over months or years. This paper addresses the current gap in automated tools for temporal prediction, transitioning from simple identification to prognostic forecasting by leveraging sequences of clinical images.

Methodology: The TempSeq-Net Architecture

The researchers developed TempSeq-Net, an end-to-end temporal sequence network designed to process chronological image data. The architecture employs a two-stage pipeline:

1. Spatial Feature Extraction

The model utilizes a Convolutional Neural Network (CNN) backbone to extract high-level spatial features from individual slit-lamp images. To identify the most effective architecture, the researchers benchmarked three industry-standard backbones:

  • AlexNet: A lightweight model suitable for environments with limited computational resources.
  • GoogLeNet: Known for its efficient use of parameters through inception modules.
  • ResNet-50: A deep residual network capable of capturing highly complex features.

2. Temporal Modeling

The spatial features extracted by the CNN are then fed into a sequence model. The study primarily focuses on Long Short-Term Memory (LSTM) networks, a specialized type of Recurrent Neural Network (RNN). Unlike standard feed-forward networks, LSTMs possess "memory" cells that allow them to retain information across a sequence. This capability is essential for recognizing patterns of change between consecutive medical check-ups, effectively treating a series of images like a video where the "meaning" is found in the transition between frames.

3. Optimization and Implementation

The study systematically compared six combinations of the aforementioned CNNs and LSTMs. The implementation was built using the Caffe deep learning framework, employing transfer learning from pre-trained models to enhance training efficiency and accuracy on medical datasets.

Data and Sequence Analysis

The research utilizes an ophthalmic sequence dataset consisting of consecutive slit-lamp images representing various chronological stages of disease. A key focus of the study was analyzing the sequence length—the number of consecutive images required for an accurate prediction.

The authors found that the "window" of time is a critical hyperparameter. If the sequence is too short, the model may fail to capture a clear trend; if it is too long, it may introduce noise or unnecessary computational overhead. By evaluating stability across different lengths, the researchers established an "appropriate range" that ensures valid predictions regardless of minor variations in clinical imaging frequency.

Key Insights and Results

The study yielded several important conclusions for the future of ophthalmic CAD:

  • Hybrid Architectures are Essential: Spatial-only models (standard CNNs) are insufficient for predicting progression. The integration of temporal modeling via LSTMs is necessary to derive a prognosis from the order and change of clinical observations.
  • Efficiency vs. Effectiveness: The comparison of different backbones provides a roadmap for clinical deployment. While deeper models like ResNet-50 offer high performance, lighter models like AlexNet provide viable alternatives for resource-constrained settings.
  • Automated Pipeline: TempSeq-Net successfully automates the transition from raw image input to progression prediction, removing the need for manual feature engineering and reducing the potential for human bias in prognostic assessment.

Limitations and Future Directions

While TempSeq-Net represents a significant step forward, several areas for improvement remain:

  • Framework Modernization: The current protocol relies on the Caffe framework. Future iterations would benefit from being ported to more modern libraries like PyTorch or TensorFlow, which offer better support for contemporary hardware and research modules.
  • Pathological Diversity: Future work should explore how the model handles a broader variety of ophthalmic diseases (e.g., distinguishing between the progression patterns of cataracts versus glaucoma) through multi-task learning.
  • Clinical Validation: The next phase of research involves moving from experimental validation to prospective clinical trials. Assessing how these "early warnings" impact real-world patient outcomes and treatment adjustments is the ultimate goal for integrating TempSeq-Net into standard ophthalmic care.

Code and data pointers for this research are available on GitHub at Ophthalmology-CAD/TempSeq-Net.

Summary updated

Abstract

Ocular images play an essential role in ophthalmology. Current research mainly focus on computer-aided diagnosis using slit-lamp images, however few studies have been done to predict the progression of ophthalmic disease. Therefore by exploring an effective approach of prediction can help to plan treatment strategies and to provide early warning for the patients. In this study, we present an end-to-endtemporal sequence network (TempSeq-Net) to automatically predict the progression of ophthalmic diseases based on consecutive slit-lamp images. First, we comprehensively compare six potential combinations of three convolutional neural networks and long short term memory (or recurrent neural network) in terms of effectiveness and efficiency, to obtain the optimal TempSeq-Net model. Second, we analyze the impacts of sequence lengths on model’s performance which help to evaluate their stability and validity and to determine the appropriate range of sequence lengths.
BibTeX
@misc{jiang2018predicting,
  author = {Jiewei Jiang and Xiyang Liu and Lin Liu and Shuai Wang and Erping Long and Haoqing Yang and Fuqiang Yuan and Deying Yu and Kai Zhang and Liming Wang and Zhenzhen Liu and Dongni Wang and ChangZun Xi and Zhuoling Lin and Xiaohang Wu and Jiangtao Cui and Mingmin Zhu and Haotian Lin},
  title = {{Predicting the progression of ophthalmic disease based on slit-lamp images using a deep temporal sequence network v1}},
  year = {2018},
  doi = {10.17504/protocols.io.qgzdtx6},
}
Figure