Probabilistic Latent Variable Models for Incomplete Data

This direction is split into other directions: latent variable imputation (expectation-maximization for missing data). Its last version stays readable and citable here.

Version 1 of 1158 papers174 conceptsfrozen 2026-10-04

In traditional machine learning, missing data is often treated as a nuisance to be scrubbed away. Common heuristics like mean imputation or discarding incomplete records (complete-case analysis) are standard practice, but they often introduce bias or destroy the very signals the model is trying to learn.

A more powerful perspective, pioneered by researchers like Zoubin Ghahramani and Michael I. Jordan, treats missing data not as noise, but as unobserved latent variables. By framing supervised learning as a joint density estimation problem, we can "fill in" missing values and learn model parameters simultaneously within a unified probabilistic framework.

The Intuition: Data Lives on a Manifold

Why does treating missingness as a latent variable work? The core intuition is that high-dimensional data rarely fills the entire space; instead, it lives on a lower-dimensional "manifold" or structure. Even if a specific feature is missing, the remaining observed features often provide enough context to project the data point onto that manifold.

In EM Algorithms for PCA and SPCA, the Expectation-Maximization (EM) algorithm is used to estimate principal components from incomplete datasets. Instead of guessing a value for a missing coordinate, the algorithm treats the coordinate as a hidden variable. In the "E-step," it finds the most likely value for that variable given the current principal components; in the "M-step," it updates the components to better fit the "completed" data. This iterative process ensures that the model captures the global correlation structure even when every single observation is partially missing.

From Linear Subspaces to Deep Generative Models

While PCA works for linear relationships, modern research has extended this logic to complex, non-linear data using deep generative models like Variational Autoencoders (VAEs).

In VAEs in the Presence of Missing Data, Alfredo Nazábal and colleagues demonstrate that a VAE can be trained on incomplete data by treating missingness as a "corruption process." By providing the model with an explicit mask of what is missing, the encoder learns to map partial observations to a latent space, and the decoder learns to reconstruct the full signal.

However, sampling from these models is not always straightforward. Vaidotas Simkus and Michael U. Gutmann point out that standard sampling methods can get "stuck" in the latent space. They propose algorithms like LAIR to navigate these complex structures, ensuring that the imputed values are consistent with the learned latent manifold rather than just being noisy guesses.

When Missingness is Informative

The most difficult challenge in this field is Missing Not At Random (MNAR) data, where the reason a value is missing is related to the value itself (e.g., high-income individuals refusing to report their salary).

Recent work has shifted toward treating the "missingness mask" as a primary data source rather than a byproduct. Jialei Chen and colleagues introduced GNR, a model that treats the data and the mask as two equal modalities. This allows the latent space to organize itself into distinct regions for different missingness patterns. Similarly, David K. Lim developed NIMIWAE, which uses Importance-Weighted Autoencoders to explicitly model the missingness mechanism, turning the absence of data into an informative signal for clinical predictions.

Applications Across Domains

This "latent variable" approach to missingness has proven robust across diverse fields:

  • 3D Reconstruction: Hyeonwoo Yu and Jean Oh use VAEs with category-specific priors to reconstruct 3D objects even when 70-90% of the data is lost during transmission.
  • Healthcare: In Readmission prediction via deep contextual embedding of clinical concepts, Cao Xiao and David Blei show that combining topic modeling (a latent variable approach) with sequential models makes predictions more resilient to the sparse nature of electronic health records.
  • Education: Sebastian Tschiatschek and Cheng Zhang use Partial VAEs to mine insights from massive, sparse student performance datasets, treating unobserved student-question interactions as latent variables to be inferred.

By moving away from ad-hoc imputation and toward principled probabilistic inference, these researchers have shown that we can learn effectively from the "holes" in our data, treating what we don't know as a vital part of the model itself.

Papers in this version

Showing 60 of 158 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_probabilistic_latent_variable_models_for_98cc7ca7,
  title = {Probabilistic Latent Variable Models for Incomplete Data},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/probabilistic-latent-variable-models-for-incomplete-data/rdr_98cc7ca78aff4cf685601957ab7630e0},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_probabilistic_latent_variable_models_for_98cc7ca7_v1,
  title = {Probabilistic Latent Variable Models for Incomplete Data},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/probabilistic-latent-variable-models-for-incomplete-data/rdr_98cc7ca78aff4cf685601957ab7630e0/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}