Joint Imputation and Classification for Incomplete Data

This direction is merged into another direction: joint imputation and clustering. Its last version stays readable and citable here.

Version 1 of 132 papers33 conceptsfrozen 2026-10-04

In real-world machine learning, data is rarely "clean." Sensors fail, patients miss appointments, and users leave survey questions blank. Traditionally, researchers handle these gaps using a two-stage pipeline: first, they "impute" the missing values using statistical techniques (like mean substitution or matrix completion), and then they feed the completed dataset into a classifier.

However, a growing body of research suggests that this decoupling is fundamentally flawed. When we impute data independently of the final task, we optimize for statistical plausibility (making the data look "normal") rather than predictive utility (preserving the signals that matter for the diagnosis). The emerging alternative is a unified, end-to-end framework where imputation and classification are learned simultaneously.

The Problem with Decoupling

The core intuition behind joint learning is that not all missing values are created equal. In a two-stage process, an imputer might fill a missing blood pressure reading with the population average. While this is statistically "safe," it might wash out the very signal a classifier needs to detect a rare heart condition.

As Uwe Dick, Peter Haider, and Tobias Scheffer argue in Learning from incomplete data with infinite imputations, traditional methods fail to account for how imputation errors propagate. By treating the distribution of missing values as a free parameter in a joint optimization problem, the model can find estimates that are specifically optimized for the downstream objective.

This "task-agnostic" nature of standard imputation is a recurring theme. Lifeng Shen and Qianli Ma note in End-to-End Time Series Imputation via Residual Short Paths that separating these stages prevents the model from leveraging the synergy between learning the data distribution and optimizing for a specific application.

Label-Aware Imputation

The most powerful advantage of joint learning is the introduction of "label-aware" supervision. In a joint framework, the gradients from the classification loss flow back into the imputation mechanism. This tells the imputer: "Fill the gap in a way that helps the classifier distinguish between Class A and Class B."

Several researchers have implemented this using Generative Adversarial Networks (GANs). For instance, Xinlu Zhang and Linda Ruth Petzold developed Classifier-GAIN in Multiple Organ Failure Prediction with Classifier-Guided Generative Adversarial Imputation Networks. By using the downstream prediction task to provide guidance, the model ensures that the recovered features are clinically relevant, not just statistically likely. Similarly, Rozhina Ghanavi and Ben Liang proposed the Generative Adversarial Classification Network (GACN) in Generative Adversarial Classification Network with Application to Network Traffic Classification, prioritizing the features most critical for accuracy.

Hybrid Generative and Discriminative Architectures

To handle the "messy" reality of clinical data, researchers often combine generative models (which learn the data's structure) with discriminative models (which learn the labels).

Pengtao Xie and Eric P. Xing demonstrate this in Medical Diagnosis From Laboratory Tests by Combining Generative and Discriminative Learning. They use a Variational Recurrent Neural Network (VRNN) to impute values while a diagnostic classifier simultaneously identifies disease patterns. This approach leverages the inherent relationship between a patient's underlying health state and the patterns of their missing data—a concept known as "informative missingness."

In the realm of time series, Visar Berisha and Jing Li introduced MAGIC in MAGIC: Multi-task Gaussian process for joint imputation and classification in healthcare time series, which uses multi-task Gaussian processes to ensure that the classification objective informs the estimation of missing values in irregular, sparse medical logs.

Geometric and Graph-Based Approaches

When data is missing across a population, we can look at the "geometry" of the dataset to fill the gaps. Gerome Vivar, Nassir Navab, and Seyed‐Ahmad Ahmadi pioneered Multigraph Geometric Matrix Completion (MGMC).

As detailed in Simultaneous imputation and classification using Multigraph Geometric Matrix Completion (MGMC), this framework treats the entire dataset as a matrix where rows are patients and columns are features. By representing the population as a graph, the model can "smooth" missing values based on similar patients while simultaneously predicting disease status. This geometric approach is particularly effective in neurodegenerative disease classification, where data is often sparse but patient similarities are strong.

Direct Classification Without Imputation

A more radical branch of this research suggests that we might not need to "fill in" the data at all. In Classification with Low Rank and Missing Data, Elad Hazan, Roi Livni, and Yishay Mansour introduce an algorithm that classifies incomplete data directly.

By assuming the underlying "clean" data resides in a low-rank subspace, they use a kernel-based method to bypass reconstruction entirely. This avoids the computational overhead of imputation and prevents the potential loss of linear separability that can occur when trying to force-fit values into missing slots.

Why It Works: The Intuition

The shift toward joint imputation and classification works because it acknowledges that missingness is information. In clinical settings, the fact that a specific test was not ordered can be as informative as the result of the test itself.

By unifying these tasks, models like Qian Dong's PATNet (PATNet: Propensity-Adjusted Temporal Network for Joint Imputation and Prediction) can explicitly model "observation bias." Instead of treating missing data as a nuisance to be removed, these end-to-end systems treat it as a feature to be leveraged, leading to more robust and accurate predictions in the messy, incomplete environments of the real world.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_joint_imputation_and_classification_for__1666c790,
  title = {Joint Imputation and Classification for Incomplete Data},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/joint-imputation-and-classification-for-incomplete-data/rdr_1666c790e93c4834906775fe5428154a},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_joint_imputation_and_classification_for__1666c790_v1,
  title = {Joint Imputation and Classification for Incomplete Data},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/joint-imputation-and-classification-for-incomplete-data/rdr_1666c790e93c4834906775fe5428154a/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}