LSOR: Longitudinally-Consistent Self-Organized Representation Learning

Jiahong Ouyang, Qingyu Zhao, Ehsan Adeli, Wei Peng, Greg Zaharchuk, Kilian M. Pohl

Introduction

Deep learning has significantly advanced medical imaging analysis, yet many models function as "black boxes," providing predictions without clear interpretability. In neurology, understanding the trajectory of brain aging and disease progression is as critical as the diagnosis itself. While Self-Organizing Maps (SOMs) are traditionally used to visualize high-dimensional data by clustering it onto a 2D grid, they often struggle with the complexity of 3D brain MRIs and fail to capture clinically relevant longitudinal changes unless guided by labels.

To bridge this gap, the authors introduce LSOR (Longitudinally-Consistent Self-Organized Representation Learning). LSOR is a self-supervised framework designed to organize complex longitudinal brain MRI data into an interpretable 2D map, naturally stratifying patients by brain age and disease progression without requiring clinical labels during training.

Methodology

LSOR combines deep representation learning with three primary innovations to ensure stability and clinical relevance in a self-supervised setting:

  1. Soft Weighting and Stability: Traditional SOMs use "hard" assignments where a sample belongs to a single cluster, which can be unstable in high-dimensional spaces. LSOR employs a soft weighting scheme, a stop-gradient operator, and a commitment loss to prevent the model from collapsing into naive solutions where all images map to the same point.
  2. Longitudinal Consistency Regularization: This core novelty treats the change between two scans of the same subject as a "trajectory" in the latent space. The model forces these individual trajectories to align with a "reference trajectory" defined for their specific cluster, ensuring that the learned representation respects the temporal nature of aging.
  3. Exponential Moving Average (EMA) Updates: Reference trajectories are updated using EMA, allowing the model to iteratively refine its understanding of how a typical brain in a specific cluster evolves over time.

Experimental Setup

The model was evaluated using the ADNI-1 (Alzheimer’s Disease Neuroimaging Initiative) dataset, comprising 2,389 longitudinal T1-weighted MRIs from 632 individuals. The cohort included Normal Controls (NC), stable Mild Cognitive Impairment (sMCI), progressive Mild Cognitive Impairment (pMCI), and Alzheimer’s Disease (AD) patients. Images were preprocessed using standard clinical pipelines and downsampled to $64 \times 64 \times 64$ volumes for analysis.

Key Insights and Results

Emergent Clinical Stratification

Despite being trained in a completely self-supervised manner, the LSOR grid naturally organized itself according to clinical reality. One region of the grid consistently represented younger, healthier brains, while the opposite region captured older, diseased brains. By averaging MRIs at specific grid locations, the authors visually confirmed that the model captured physical hallmarks of neurodegeneration, such as ventricular enlargement and tissue atrophy.

Quantitative Performance

LSOR was tested on downstream tasks to validate the utility of its unsupervised representations:

  • Classification (sMCI vs. pMCI): The model achieved an AUC of 72.4, outperforming or matching existing state-of-the-art self-supervised methods like LSSL and LNE.
  • Cognitive Score Regression: In predicting ADAS-Cog scores, LSOR achieved the highest accuracy among compared methods with an $R^2$ of 0.32 and the lowest RMSE (6.31).
  • Interpretability: The coordinates on the SOM grid demonstrated extremely high correlations with chronological age (0.92), cognitive decline (0.94), and ADAS-Cog scores (0.91).

Conclusion and Future Work

LSOR provides a robust framework for mapping the "spectrum" of brain aging. Rather than a binary classification, it allows researchers to visualize where a patient sits on a progression map and how rapidly they are moving across it. While the $R^2$ of 0.32 in cognitive score prediction highlights the inherent noise in subjective clinical testing, the model's ability to extract high-level clinical features without labels is a significant step forward.

Future research may explore fine-tuning the encoders for specific tasks or incorporating multi-modal data—such as PET scans or genetic markers—to provide a more holistic view of neurodegenerative health.

Summary updated

BibTeX
@inproceedings{ouyang2023lsor,
  author = {Jiahong Ouyang and Qingyu Zhao and Ehsan Adeli and Wei Peng and Greg Zaharchuk and Kilian M. Pohl},
  title = {{LSOR: Longitudinally-Consistent Self-Organized Representation Learning}},
  booktitle = {International Conference on Medical Image Computing and Computer-Assisted Intervention},
  year = {2023},
  url = {https://openreview.net/forum?id=MpgjIIQzi8},
}
Figure