Abstract
Longitudinal datasets measured repeatedly over time from individual subjects, arise in many biomedical, psychological, social, and other studies. A common approach to analyse high-dimensional data that contains missing values is to learn a low-dimensional representation using variational autoencoders (VAEs). However, standard VAEs assume that the learnt representations are i.i.d., and fail to capture the correlations between the data samples. We propose the Longitudinal VAE (L-VAE), that uses a multi-output additive Gaussian process (GP) prior to extend the VAE's capability to learn structured low-dimensional representations imposed by auxiliary covariate information, and derive a new KL divergence upper bound for such GPs. Our approach can simultaneously accommodate both time-varying shared and random effects, produce structured low-dimensional representations, disentangle effects of individual covariates or their interactions, and achieve highly accurate predictive performance. We compare our model against previous methods on synthetic as well as clinical datasets, and demonstrate the state-of-the-art performance in data imputation, reconstruction, and long-term prediction tasks.
Introduction
In fields like medicine, psychology, and sociology, data rarely comes in neat, independent packages. Instead, researchers deal with longitudinal data: repeated measurements of the same subjects over time. Whether it’s a patient’s blood pressure over a decade or a user’s social media behavior over a year, these data points are deeply correlated.
High-dimensional longitudinal data presents significant challenges: it is often sparse (missing values), irregular (measurements at different times), and highly correlated. While Variational Autoencoders (VAEs) are excellent at compressing high-dimensional data into a low-dimensional latent space, they typically assume every data point is independent and identically distributed (i.i.d.). This means a standard VAE treats a patient's measurement at Year 1 and Year 5 as unrelated events, failing to capture the underlying temporal dynamics.
Methods: The L-VAE Architecture
The Longitudinal VAE (L-VAE) addresses these limitations by reimagining the latent space prior. Instead of a simple isotropic Gaussian, the L-VAE employs a multi-output additive Gaussian Process (GP) prior. This allows the model to capture structured correlations imposed by auxiliary covariate information (e.g., time, subject ID, sex).
Additive Gaussian Processes
The "additive" nature of the GP prior is a key innovation. It decomposes the latent dynamics into a sum of distinct effects:
- Shared Effects: Population-level trends that affect everyone (e.g., general aging).
- Individual Effects: Subject-specific variations (e.g., individual baselines).
- Interaction Effects: Interactions between specific traits (e.g., how sex influences aging trajectories).
Scalable Inference
Gaussian Processes typically scale at $O(N^3)$, which is prohibitive for large datasets. To ensure scalability, the authors derived a new KL divergence upper bound. By utilizing "inducing points"—representative landmarks in the data—they implemented a mini-batch training scheme that allows the model to scale linearly with the number of samples.
Results and Evaluation
The L-VAE was evaluated on three benchmarks: Rotated MNIST (toy rotation data), Health MNIST (simulated aging/sex effects), and the Physionet Challenge 2012 (real-world ICU records).
Predictive Performance and Disentanglement
The additive GP prior naturally "disentangles" different factors of variation. In the Health MNIST dataset, the model could separate the effects of age from individual identity. Quantitatively, L-VAE significantly outperformed state-of-the-art models like GP-VAE and GPPVAE.
On the Health MNIST dataset, L-VAE achieved a Mean Squared Error (MSE) of 0.025, a nearly 60% improvement over the standard GP-VAE (0.059). In clinical applications using the Physionet ICU data, L-VAE achieved higher AUROC scores for mortality prediction than competing methods like BRITS or GRUI-GAN, indicating that its latent representations are more clinically informative.
Conclusion
The L-VAE represents a significant advancement in deep generative modeling for longitudinal studies. By integrating additive Gaussian Processes into the VAE framework, it provides a principled way to handle temporal correlations and missing data while maintaining scalability. While the model requires careful kernel selection and tuning of inducing points, it offers a powerful tool for "Deep Biosignal" analysis, bridging the gap between flexible deep learning and structured statistical modeling.
Summary updated