Abstract
Clinical patient records are an example of high-dimensional data that is typically collected from disparate sources and comprises of multiple likelihoods with noisy as well as missing values. In this work, we propose an unsupervised generative model that can learn a low-dimensional representation among the observations in a latent space, while making use of all available data in a heterogeneous data setting with missing values. We improve upon the existing Gaussian process latent variable model (GPLVM) by incorporating multiple likelihoods and deep neural network parameterised back-constraints to create a non-linear dimensionality reduction technique for heterogeneous data. In addition, we develop a variational inference method for our model that uses numerical quadrature. We establish the effectiveness of our model and compare against existing GPLVM methods on a standard benchmark dataset as well as on clinical data of Parkinson's disease patients treated at the HUS Helsinki University Hospital.
Introduction
In personalized medicine, disease stratification—identifying subgroups of patients with distinct clinical profiles—is crucial for targeted treatment. However, clinical records are inherently "heterogeneous," containing a mix of binary diagnostic codes, continuous lab results, and bounded percentages, often riddled with missing values. Traditional dimensionality reduction techniques like PCA or standard Gaussian Process Latent Variable Models (GPLVMs) typically assume data is homogeneous (e.g., all Gaussian). Forcing non-Gaussian data into these models leads to a loss of nuance and can fail to preserve local clinical similarities in the resulting latent space.
Methods: Shared GPLVM with Composite Likelihoods
The proposed model addresses these challenges through three primary innovations:
- Composite Likelihoods: Instead of a single likelihood function, the model assigns specific distributions (Gaussian, Binomial, Beta, Poisson, or Categorical) to each variable. These are connected to the underlying Gaussian Process via appropriate link functions (e.g., sigmoid or exponential), allowing the model to respect the data's natural structure.
- Numerical Quadrature: To handle the mathematical intractability of non-Gaussian variational inference, the authors employ Gauss-Hermite quadrature. This numerical integration technique is significantly faster and more stable than traditional Monte Carlo sampling for calculating the Evidence Lower Bound (ELBO).
- Neural Network Back-Constraints: Deep neural networks (MLPs) are used as recognition models to map high-dimensional data back into the latent space. This hybrid approach ensures local distance preservation and enables mini-batching, allowing the model to scale to large clinical datasets.
Results and Discussion
The model was evaluated on a Parkinson’s disease dataset from HUS Helsinki University Hospital (1,400 patients, 46 features) and a simulated heterogeneous MNIST benchmark.
The composite likelihood approach significantly outperformed standard Gaussian-only models in predictive log-likelihood. On the Parkinson's dataset, the model identified four distinct, stable patient clusters. Consensus clustering confirmed the statistical significance of these subgroups, while t-statistic analysis revealed contrasting clinical characteristics between them.
Key insights include:
- Likelihood Selection: Using appropriate distributions (e.g., Beta for percentages) provides a substantial boost in predictive performance.
- Efficiency: Numerical quadrature proved roughly twice as fast as sampling-based inference.
- Scalability: Back-constraints bridged the gap between GPs and VAEs, facilitating the analysis of large-scale heterogeneous data.
Conclusion
This framework provides a robust method for unsupervised disease stratification in messy, real-world clinical settings. Future work may extend this to longitudinal data to track patient progression across clusters or incorporate more complex likelihoods for specialized medical imaging or genomic data. While the model handles missing data effectively, further research into automated latent space centering could improve ease of use.
Summary updated