Morph-SSL: Self-Supervision With Longitudinal Morphing for Forecasting AMD Progression From OCT Volumes

Arunava Chakravarty, Taha Emre, Oliver Leingang, Sophie Riedl, Julia Mai, Hendrik P. N. Scholl, Sobha Sivaprasad, Daniel Rueckert, Andrew J. Lotery, Ursula Schmidt-Erfurth, Hrvoje BogunovicIEEE TMI 2024 · DOI

TL;DR

This paper introduces Morph-SSL, a novel self-supervised learning method that leverages longitudinal OCT scans to predict the progression of Age-Related Macular Degeneration (AMD). By learning to "morph" between scans from different visits, Morph-SSL effectively captures subtle structural changes, enabling more accurate forecasting of conversion to neovascular AMD (nAMD) and the development of a risk score for patient stratification.

What Problem Does This Paper Address?

Predicting the conversion of intermediate AMD (iAMD) to neovascular AMD (nAMD) is crucial for timely treatment and preventing vision loss. However, current methods lack reliable biomarkers, and the progression rate varies significantly between patients. While Optical Coherence Tomography (OCT) provides rich longitudinal data, manually labeling these scans for supervised deep learning is time-consuming and requires expert knowledge. This paper addresses the challenge of leveraging vast amounts of unlabeled longitudinal OCT data to build accurate predictive models for AMD progression.

Methods

The core innovation is Morph-SSL, a self-supervised learning (SSL) method designed for longitudinal data. It works by training an encoder-decoder network to predict the transformation (spatial deformation and intensity changes) required to "morph" an OCT scan from one visit into a scan from a subsequent visit of the same eye. This pretext task forces the model to learn feature representations that capture the temporal and structural changes indicative of disease progression.

Key aspects of the method include:

  • Longitudinal Morphing: The model learns to predict the parameters of a spatial deformation and an additive intensity transformation that maps an earlier OCT scan to a later one.
  • Smooth Feature Manifold: The learned features are constrained to form a smooth manifold, allowing for linear interpolation between features to generate intermediate OCT scans.
  • Encoder-Decoder Architecture: A specialized 3D convolutional neural network (CNN) architecture, utilizing Separable 3D Convolution Blocks (S3DConv) and concatenation-based skip connections, is employed to efficiently process OCT volumes.
  • Downstream Classification: After pre-training with Morph-SSL, the learned encoder is used to train a classifier to predict the cumulative distribution function (CDF) of the time to conversion (TTC) to nAMD using a sigmoidal function. This allows for continuous risk prediction.
  • Risk Score: A risk score is derived from the predicted CDF to stratify patients into low, moderate, and high-risk groups for nAMD conversion.

Datasets

  • SSL Dataset: 3570 unlabelled OCT scans from 399 eyes with at least 3 visits per eye were used for training Morph-SSL. Visits after the first anti-VEGF injection were removed to focus on the iAMD stage.
  • TTC Dataset: 343 eyes (2418 OCT volumes) were manually examined by clinical experts for the downstream Time-to-Conversion (TTC) task. This dataset was used for evaluating the predictive model.

Key Insights and Ideas

  • Morphing as a Pretext Task: The idea of using image morphing between longitudinal scans as a self-supervised pretext task is novel and effectively captures disease progression dynamics.
  • Smooth Feature Embeddings for Interpolation: Learning a smooth feature manifold allows for meaningful interpolation, which can potentially be used for data augmentation or visualizing future disease states.
  • Sigmoidal CDF for Continuous Risk Prediction: Modeling the time-to-conversion as a CDF with a sigmoidal function provides a continuous and interpretable risk score, overcoming limitations of discrete classification or regression.
  • Inadequacy of Handcrafted Biomarkers: The study highlights that current clinically known biomarkers are insufficient for accurately predicting nAMD conversion, motivating the use of DL models that learn prognostic features directly from OCT scans.

Key Results

  • Morph-SSL Performance: Morph-SSL features significantly improved performance on the TTC task compared to end-to-end training from scratch or fine-tuning standard 3D-CNNs.
  • AUC for Prediction: The Morph-SSL features achieved an AUC of 0.779 in predicting conversion to nAMD within 6 months.
  • Risk Stratification: The derived risk score effectively stratified eyes into low and high-risk groups, with statistically significant differences in survival curves.
  • Architectural Efficiency: The proposed S3DConv architecture reduced computation and parameters while maintaining or improving performance compared to full 3D convolutions.
  • Comparison to Other SSL Methods: Morph-SSL outperformed other state-of-the-art SSL methods, particularly in scenarios with limited labeled data or when features were used off-the-shelf.
  • Regression vs. Classification: Modeling TTC as a classification task using a sigmoidal CDF outperformed a regression-based approach.

Limitations and Future Work

  • Training Time: Training multiple 3D CNN networks was time-consuming, preventing an exhaustive search for optimal architectures, feature map sizes, and loss term weights.
  • Intra-Eye Consistency: The current model treats scans from different visits of the same eye as independent samples for the downstream task. While moderate consistency was observed (eCI = 0.73), future work could explore methods to explicitly enforce this constraint.
  • Interpolation Applications: The ability to interpolate in the learned feature space opens avenues for generating artificial training samples or visualizing disease progression with Recurrent Neural Networks.
  • Generalizability: Adapting Morph-SSL to other medical domains, such as forecasting cancer progression or dementia onset, is suggested as future work.

Summary updated

BibTeX
@article{chakravarty2024morph,
  author = {Arunava Chakravarty and Taha Emre and Oliver Leingang and Sophie Riedl and Julia Mai and Hendrik P. N. Scholl and Sobha Sivaprasad and Daniel Rueckert and Andrew J. Lotery and Ursula Schmidt-Erfurth and Hrvoje Bogunovic},
  title = {{Morph-SSL: Self-Supervision With Longitudinal Morphing for Forecasting AMD Progression From OCT Volumes}},
  journal = {IEEE Trans. Medical Imaging},
  year = {2024},
  doi = {10.1109/tmi.2024.3390940},
}
Figure