Deep Latent Variable Model for Longitudinal Group Factor Analysis

Lin Qiu, Vernon M. Chinchilli, Lin LinarXiv 2020 · OpenReview

TL;DR

This paper introduces ITM-VAE, a novel deep generative model that disentangles latent representations from temporal multi-view data. By combining variational autoencoders with recurrent neural networks, ITM-VAE can effectively model complex time-dependent relationships across different data sources, offering interpretable insights into underlying dynamics.

What Problem Does This Paper Address?

Many real-world scientific and engineering problems involve data collected from multiple sources (multi-view data) over time. Examples include genomic data, financial time series, and video surveillance. Understanding these systems requires not only integrating information from diverse views but also accounting for their temporal evolution. Traditional methods often struggle to handle the complexity of time-dependent, multi-view data, especially when aiming for interpretable results. This paper addresses the challenge of learning disentangled, interpretable latent representations from such data, capturing both the inter-view dependencies and the temporal dynamics.

Methods

The core of the proposed method is the Interpretable Temporal Multi-view Variational Autoencoder (ITM-VAE). It builds upon the variational autoencoder (VAE) framework but incorporates a recurrent neural network (RNN) to model temporal dependencies.

Here's a breakdown of the key components:

  • Variational Recurrent Neural Network (VRNN): ITM-VAE integrates a VRNN to capture the temporal dynamics. The latent variable at each timestep ($z_t$) is modeled as dependent on the previous hidden state ($h_{t-1}$) of the RNN.
  • Multi-view Integration: The model explicitly handles multiple views of the data. Each view's data ($x^{(g)}$) is generated based on the shared latent variable ($z_t$) and the temporal state ($h_{t-1}$), with view-specific parameters.
  • Disentanglement and Interpretability: A key contribution is the focus on interpretable latent representations. This is achieved by introducing a sparsity-inducing prior on a set of learned weight matrices ($W^{(g)}$) that map latent variables to view-specific features. This encourages a column-wise sparsity, allowing for the identification of dominant latent factors for each view over time.
  • Timestep-wise Variational Inference: The authors propose an efficient inference scheme that optimizes the model's parameters in a timestep-wise manner, leveraging collapsed variational inference and proximal gradient descent to effectively learn the sparse weights.

Datasets

The ITM-VAE model was evaluated on three distinct datasets:

  • Artificial Data: A synthetic dataset of "one-bar" images was generated to visualize the model's ability to disentangle latent factors and reconstruct data.
  • Motion Capture Data: Data from CMU's motion capture database was used, representing human body poses over time. This dataset allowed for evaluating the model's ability to capture dynamic disentanglement and anatomical interpretations.
  • Metabolomic Data: A longitudinal study dataset of E. coli metabolic responses to different stress conditions was used. This dataset tested the model's performance on limited sample size studies, common in life sciences.

Key Insights and Ideas

  • Bridging Temporal and Multi-view Learning: The ITM-VAE successfully unifies temporal modeling (via RNNs) with multi-view learning, a crucial step for analyzing complex, real-world data.
  • Interpretable Latent Factors: The sparsity prior on the learned weights ($W^{(g)}$) is a powerful mechanism for achieving interpretable latent representations. It allows researchers to understand which latent factors are most influential for specific views and how these influences change over time.
  • Dynamic Relationships: The model can reveal how relationships between different views evolve dynamically, offering insights into the underlying processes. For instance, in motion capture, it can track how different body parts' movements relate to each other over time.
  • Generalization Beyond Training: The model demonstrates good generalization capabilities, performing well on unseen data and even on brisk walking trials that were significantly different from the training data.

Key Results

  • Lowest MSE: ITM-VAE consistently achieved the lowest Mean Squared Error (MSE) on test data across all three datasets compared to baseline models like VAE, CVAE, VRNN, and GP-VAE. For example, on artificial data, it achieved an MSE of 0.002 ± 0.001, significantly lower than others.
  • Effective Disentanglement: Visualizations of the learned weights ($W^{(g)}$) on artificial data showed clear disentanglement, where different latent dimensions corresponded to distinct groups (rows) in the generated images.
  • Meaningful Interpretations: In motion capture data, the learned latent factors showed intuitive anatomical interpretations, correlating with realistic body movements (e.g., foot, wrist, thorax changes).
  • Competitive Performance: On metabolomic data, ITM-VAE showed strong performance, with its learned factors aligning with known biological responses to stress conditions.

Limitations and Future Work

  • Scalability: While the model shows promise, the paper doesn't extensively discuss its scalability to extremely high-dimensional or very long temporal sequences.
  • Hyperparameter Tuning: The performance is sensitive to hyperparameters like the sparsity parameter ($\lambda$) and latent dimension ($k$), requiring careful tuning.
  • Future Work: The authors suggest further exploration of more complex temporal dependencies, investigating different sparsity priors, and applying the model to a wider range of scientific domains. The interpretability of the learned factors is highlighted as a key area for future biological or domain-specific analysis.

Summary updated

Abstract

In many scientific problems such as video surveillance, modern genomics, and finance, data are often collected from diverse measurements across time that exhibit time-dependent heterogeneous properties. Thus, it is important to not only integrate data from multiple sources (called multi-view data), but also to incorporate time dependency for deep understanding of the underlying system. We propose a generative model based on variational autoencoder and a recurrent neural network to infer the latent dynamics for multi-view temporal data. This approach allows us to identify the disentangled latent embeddings across views while accounting for the time factor. We invoke our proposed model for analyzing three datasets on which we demonstrate the effectiveness and the interpretability of the model.
BibTeX
@misc{qiu2020deep,
  author = {Lin Qiu and Vernon M. Chinchilli and Lin Lin},
  title = {{Deep Latent Variable Model for Longitudinal Group Factor Analysis}},
  year = {2020},
  eprint = {2005.05210},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2005.05210},
}
Figure