Learning-based inference of longitudinal image changes: Applications in embryo development, wound healing, and aging brain

Heejong Kim, Batuhan K. Karaman, Qingyu Zhao, Alan Q. Wang, Mert R. Sabuncu, Michael D. Weiner, Paul Aisen, Ronald Petersen, Clifford R. Jack, William Jagust, Susan Landau, Mónica Rivera Mindt, Ozioma C. Okonkwo, Leslie M. Shaw, Edward B. Lee, Arthur W. Toga, Laurel Beckett, Danielle Harvey, Robert C. Green, Andrew J. Saykin, Kwangsik Nho, Richard J. Perrin, Duygu Tosun, Perminder S. Sachdev, Erin Drake, Tom Montine, Cat Conti, Rachel L. Nosheny, Diana Truran Sacrey, Juliet Fockler, Melanie J. Miller, Winnie Kwang, Chengshi Jin, Adam Diaz, Miriam T. Ashford, Derek Flenniken, Adrienne Kormos, Michael S. Rafii, Rema Raman, Gustavo Jimenez‐Maggiora, Michael Donohue, Jennifer Salazar, Andrea Fidell, Virginia Boatwright, Justin Robison, Caileigh Zimmerman, Yuliana Cabrera, Sarah Walter, Taylor Clanton, Elizabeth Shaffer, Caitlin Webb, Lindsey Hergesheimer, Stephanie Smith, Sheila Ogwang, Olusegun Adegoke, Payam Mahboubi, Jeremy Pizzola, Cecily Jenkins, Naomi Saito, Kedir Adem Hussen, Hannatu Amaza, Mai Seng Thao, Shaniya Parkins, Omobolanle Ayo, Matt Glittenberg, Isabella Hoang, Kaori Kubo Germano, Joe Strong, Trinity Weisensel, Fabiola Magana, Lisa Thomas, Vanessa Guzmán, Adeyinka Ajayi, Joseph Di Benedetto, Sandra Talavera, Joel P. Felmlee, Nick C. Fox, Paul Thompson, Charles DeCarli, Arvin Forghanian-Arani, Bret Borowski, Calvin Reyes, Caitie Hedberg, Chad Ward, Christopher G. Schwarz, Denise Reyes, Jeff Gunter, John Moore-Weiss, Kejal KantarciProceedings of the National Academy of Sciences 2025 · DOI

TL;DR

This paper introduces LILAC, a novel machine learning framework that automatically identifies and quantifies significant longitudinal changes in medical images, bypassing the need for complex, custom preprocessing pipelines. By learning to predict temporal ordering or specific target variables from image pairs, LILAC effectively isolates relevant temporal signals across diverse applications like embryo development, wound healing, and brain aging.

What Problem Does This Paper Address?

Longitudinal imaging studies are crucial for understanding biological processes and disease progression over time. However, analyzing these datasets is challenging due to inherent variations in image acquisition (e.g., noise, field of view differences, slight motion) that can obscure the true biological changes of interest. Traditional methods often rely on hand-engineered pipelines to remove these "nuisance" variations, which can be time-consuming, prone to errors, and may inadvertently remove subtle but important signals. Existing deep learning approaches, while powerful, often focus on predicting clinical endpoints rather than directly quantifying and localizing the temporal changes themselves. This paper aims to bridge this gap by developing a method that can automatically learn to ignore irrelevant variations and focus on the meaningful temporal dynamics within longitudinal image sequences.

Methods

The core of LILAC is a Siamese convolutional neural network architecture. This means it uses two identical networks (sharing weights) to process pairs of longitudinal images (e.g., an image at time t1 and an image at time t2 from the same subject). These networks extract feature representations from each image. The key innovation lies in what happens next: the feature vectors from the two images are subtracted, and this "feature difference" is then fed into a bias-free fully connected layer. This design encourages the model to learn representations that capture the temporal change between the images, while the bias-free layer helps it to ignore constant differences between the two networks.

LILAC can be trained for different tasks:

  • LILAC-o (Temporal Ordering): The model learns to predict whether an image pair is in the correct temporal order (t1 before t2) or not. This is a self-supervised task that forces the model to learn about time-irreversible dynamics present in development, aging, or disease progression.
  • LILAC-t (Time Interval Prediction): The model is trained to predict the actual time difference between the two images. This is a regression task.
  • LILAC-s (Specified Change Prediction): The model is trained to predict a specific target variable (e.g., a clinical score change), potentially with additional metadata (like age or sex) to help disentangle different temporal processes. This is also a regression task.

To understand where LILAC is focusing its attention, the authors employ a modified Gradient-weighted Class Activation Mapping (Grad-CAM) technique. This allows them to generate saliency maps that highlight the image regions most influential in the model's predictions.

Datasets

The paper evaluates LILAC on four diverse real-world datasets:

  • Embryo Development Dataset: Time-lapse images of human embryos undergoing in vitro fertilization, annotated with 16 developmental phases. This dataset was used to test LILAC-o's ability to capture developmental dynamics.
  • Wound-Healing Assay Dataset: Time-lapse microscopy images of wound healing, used to assess LILAC-o's performance in capturing group-level differences between treated and control groups.
  • Aging Brain Dataset (OASIS-3): Longitudinal 3D T1-weighted MRI scans from healthy aging individuals. This dataset was used to evaluate LILAC-t's ability to predict time intervals (age progression).
  • Mild Cognitive Impairment (MCI) Brain Dataset (ADNI): Longitudinal 3D T1-weighted MRI scans from patients with MCI, along with clinical scores (CDRSB). This dataset was used to test LILAC-s for predicting clinical changes while controlling for confounding factors like aging and sex.

Key Insights and Ideas

  • Temporal Ordering as a Powerful Self-Supervised Task: The paper demonstrates that simply training a model to predict the temporal order of images is sufficient to capture fundamental, time-irreversible biological changes without explicit labels for those changes. This is a significant insight for leveraging unlabeled longitudinal data.
  • Pairwise Comparison is Key: The core idea of comparing image pairs and focusing on the difference in features is central to LILAC's success in ignoring static variations and highlighting dynamic changes.
  • Bias-Free FC Layer for Nuisance Removal: The specific architectural choice of a bias-free fully connected layer after feature subtraction is crucial for learning meaningful temporal differences, effectively filtering out irrelevant information.
  • Localization Reveals Biological Processes: The use of modified Grad-CAM maps provides valuable qualitative insights, showing how LILAC localizes specific biological events like cell division in embryos or cell migration in wound healing, and highlighting brain regions affected by aging or disease.
  • Flexibility Across Domains: LILAC's ability to be applied to vastly different imaging modalities (microscopy, MRI) and biological contexts (embryo, wound, brain) highlights its versatility.

Key Results

  • Embryo Development: LILAC-o achieved an AUC of 0.989 for temporal ordering, with predicted time differences strongly correlated (r=0.911) with actual phase changes. Grad-CAM localized cell division events.
  • Wound Healing: LILAC-o achieved an AUC of 0.9908 and predicted time differences correlated well (r=0.875) with actual time differences. Crucially, LILAC-o successfully distinguished between treatment and control groups, indicating faster healing in the treated group, outperforming a segmentation-based baseline.
  • Aging Brain: LILAC-t achieved a RMSE of 1.825 years for predicting time intervals, significantly outperforming a naive baseline and a single image regression baseline. Grad-CAM maps showed sparser, more individualized patterns compared to a baseline, suggesting better capture of individual aging variability.
  • MCI Brain: LILAC-s achieved a RMSE of 1.415 for predicting CDRSB change, outperforming a naive baseline and a single image regression baseline. When controlling for aging and sex, LILAC-s highlighted the temporal lobe as a key region for predicting clinical decline, demonstrating its ability to disentangle specific disease-related changes.

Limitations and Future Work

  • Data Quality and Quantity: Like all data-driven methods, LILAC's performance is sensitive to the quality and quantity of the training data, and potential biases within it.
  • Architectural Improvements: The authors suggest that more complex architectures, such as transformers, could further enhance LILAC's ability to capture fine-grained changes and improve localization.
  • Integrating Diverse Data: LILAC currently focuses on image pairs. Future work could explore integrating richer multimodal data (e.g., clinical reports, genetic data) to provide a more comprehensive understanding of longitudinal changes.
  • Single Image Predictions: The paper notes that LILAC features and predicted scores computed on single images were not explored but showed promise in preliminary analysis.
  • Failure Cases: LILAC-o struggled with images very close in phase (embryo) or time (wound healing). Regression tasks had failures with large target value differences or when one image in a pair looked significantly different. Grad-CAM's reliability is also contingent on prediction quality.

The authors believe LILAC offers a flexible, powerful, and easy-to-use framework for longitudinal image analysis with broad applications in both research and clinical settings, encouraging further exploration across various domains.

Summary updated

Abstract

Longitudinal imaging data are routinely acquired for health studies and patient monitoring. A central goal in longitudinal studies is tracking relevant change over time. Traditional methods remove nuisance variation with custom pipelines to focus on significant changes. In this work, we present a machine learning-based method that automatically ignores irrelevant changes and extracts the time-varying signal of interest. Our method, called Learning-based Inference of Longitudinal imAge Changes (LILAC), performs a pairwise comparison of longitudinal images in order to make a temporal difference prediction. LILAC employs a convolutional Siamese architecture to extract feature pairs, followed by subtraction and a bias-free fully connected layer to learn meaningful temporal image differences. We first showcase LILAC's ability to capture key longitudinal changes by simply training it to predict the temporal ordering of images. In our experiments, temporal ordering accuracy exceeded 0.98, and predicted time differences were strongly correlated with actual changes in relevant variables (Pearson Correlation Coefficient <i>r</i> = 0.911 with embryo phase change, and r = 0.875 with time interval in wound healing). Next, we trained LILAC to explicitly predict specific targets, such as the change in clinical scores in patients with mild cognitive impairment. LILAC models achieved over a 40% reduction in root mean square error compared to baseline methods. Our empirical results demonstrate that LILAC effectively localizes and quantifies relevant individual-level changes in longitudinal imaging data, offering valuable insights for studying temporal mechanisms or guiding clinical decisions.
BibTeX
@inproceedings{kim2025learning,
  author = {Heejong Kim and Batuhan K. Karaman and Qingyu Zhao and Alan Q. Wang and Mert R. Sabuncu and Michael D. Weiner and Paul Aisen and Ronald Petersen and Clifford R. Jack and William Jagust and Susan Landau and Mónica Rivera Mindt and Ozioma C. Okonkwo and Leslie M. Shaw and Edward B. Lee and Arthur W. Toga and Laurel Beckett and Danielle Harvey and Robert C. Green and Andrew J. Saykin and Kwangsik Nho and Richard J. Perrin and Duygu Tosun and Perminder S. Sachdev and Robert C. Green and Erin Drake and Tom Montine and Cat Conti and Michael W. Weiner and Rachel L. Nosheny and Diana Truran Sacrey and Juliet Fockler and Melanie J. Miller and Winnie Kwang and Chengshi Jin and Adam Diaz and Miriam T. Ashford and Derek Flenniken and Adrienne Kormos and Ronald Petersen and Paul Aisen and Michael S. Rafii and Rema Raman and Gustavo Jimenez‐Maggiora and Michael Donohue and Jennifer Salazar and Andrea Fidell and Virginia Boatwright and Justin Robison and Caileigh Zimmerman and Yuliana Cabrera and Sarah Walter and Taylor Clanton and Elizabeth Shaffer and Caitlin Webb and Lindsey Hergesheimer and Stephanie Smith and Sheila Ogwang and Olusegun Adegoke and Payam Mahboubi and Jeremy Pizzola and Cecily Jenkins and Laurel Beckett and Danielle Harvey and Michael Donohue and Naomi Saito and Adam Diaz and Kedir Adem Hussen and Ozioma C. Okonkwo and Mónica Rivera Mindt and Hannatu Amaza and Mai Seng Thao and Shaniya Parkins and Omobolanle Ayo and Matt Glittenberg and Isabella Hoang and Kaori Kubo Germano and Joe Strong and Trinity Weisensel and Fabiola Magana and Lisa Thomas and Vanessa Guzmán and Adeyinka Ajayi and Joseph Di Benedetto and Sandra Talavera and Clifford R. Jack and Joel P. Felmlee and Nick C. Fox and Paul Thompson and Charles DeCarli and Arvin Forghanian-Arani and Bret Borowski and Calvin Reyes and Caitie Hedberg and Chad Ward and Christopher G. Schwarz and Denise Reyes and Jeff Gunter and John Moore-Weiss and Kejal Kantarci},
  title = {{Learning-based inference of longitudinal image changes: Applications in embryo development, wound healing, and aging brain}},
  booktitle = {Proceedings of the National Academy of Sciences},
  year = {2025},
  doi = {10.1073/pnas.2411492122},
}
Figure