Joint Optimization of Imputation and Downstream Tasks

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2207 papers312 conceptsfrozen 2026-10-04

In traditional machine learning pipelines, missing data is treated as a preprocessing problem. If a dataset has holes, we "fix" it first—using mean imputation, K-Nearest Neighbors, or iterative methods like MICE—and then feed the completed data into a model. This two-stage approach assumes that the best way to fill a gap is to make it look statistically similar to the observed data.

However, a growing body of research suggests this "impute-then-predict" strategy is fundamentally flawed. By decoupling imputation from the final goal, we risk filling gaps with values that are statistically plausible but functionally useless, or worse, introduce biases that propagate through the entire system. The emerging alternative is joint optimization: treating imputation and the downstream task (such as clustering or classification) as a single, unified learning problem.

The Fallacy of Task-Agnostic Imputation

The core intuition behind joint optimization is that the "correct" value for a missing attribute depends on what you intend to do with the data. In What’s a good imputation to predict with missing values?, Marine Le Morvan and Gael Varoquaux argue that the goal of imputation in a predictive context isn't to recover the "true" underlying value, but to find a value that minimizes the risk of the downstream model.

When imputation is performed independently, the errors made during the reconstruction phase are "blind" to the needs of the classifier. As Alexander Van Esbroeck demonstrates in the context of trauma patient evaluation, joint optimization allows the classification task to guide the imputation process. If a specific missing lab result is critical for identifying a high-risk patient, the joint model learns to estimate that value in a way that preserves the signal necessary for an accurate diagnosis, rather than just aiming for a low reconstruction error.

Clustering as a Guide for Completion

Multi-view clustering—where data comes from multiple sources like images, text, and audio—is particularly sensitive to missing values. If one "view" is missing for a sample, traditional methods might discard the sample or fill it with noise.

Recent frameworks like AMGC (Attribute-Missing Graph Clustering Network) and HSACC (Xiaojian Ding et al.) solve this by using the emerging cluster structure as a "prompt" for imputation. In these models, the clustering assignments and the missing features are updated iteratively. The intuition is simple: if the model thinks a data point belongs to "Cluster A," it should fill in the missing attributes using the patterns prevalent in Cluster A. This creates a virtuous cycle where better imputation leads to clearer clusters, and clearer clusters provide better guidance for imputation.

Similarly, Tingting Wu and Jiazheng Yuan in their work on LRKT-IMVC treat multiple views as a 3D kernel tensor. By enforcing a low-rank constraint across the entire tensor, they capture high-order correlations that allow the model to "borrow" structure from available views to fill the missing ones, ensuring the reconstructed data is specifically optimized for the grouping task.

End-to-End Learning in Time Series

Time series data, especially in healthcare, is notoriously "holy" (full of holes). Michael Moor and Karsten Borgwardt addressed this in their work on sepsis detection. By training an imputation layer and a sequence classifier simultaneously, their MGP-TCN model ensures that the features extracted from irregular time series are those most relevant to the target classification objective.

This end-to-end philosophy is also central to the CRLI framework by Qianli Ma. By using an adversarial strategy to ensure imputations are realistic while simultaneously optimizing a "cluster-friendly" latent space, the model prevents the loss of task-relevant information that typically occurs in decoupled pipelines. The result is a representation where groups are significantly more separable because the gaps were filled with the intent to separate them.

Overcoming Selection Bias

A significant challenge in missing data is "Missing Not At Random" (MNAR) scenarios, where the reason data is missing is related to the values themselves—for example, high-income individuals refusing to report salary. David K. Lim and Junier B. Oliva show that integrating imputation directly into the supervised learning objective prevents the propagation of biases that occur when these steps are handled independently.

By modeling the missingness mechanism alongside the predictive task—as seen in the GNM framework for graph-based learning—researchers can learn bias-correction weights during the feature extraction process. This ensures that the model doesn't just "guess" the missing values, but understands the systematic reasons why they are missing, leading to more robust and fair predictions.

The Representer Result and Practicality

One might worry that searching for an optimal distribution of missing values is computationally impossible. However, early theoretical work by Uwe Dick, Peter Haider, and Tobias Scheffer proved a "representer-style" result: even though the search space for possible imputations is infinite, the optimal solution for a joint optimization problem concentrates on a small, finite set of imputations.

This theoretical foundation justifies the practical, iterative algorithms we see today. Whether it is Jinsung Yoon’s Clairvoyance toolkit for medical pipelines or Kyongmin Yeo’s simultaneous imputation and forecasting, the research direction is clear: the most effective way to handle missing data is to stop treating it as a problem to be solved before the machine learning begins, and start treating it as a core part of the learning process itself.

Papers in this version

Showing 60 of 207 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 239 papers
  2. Version 1 · 2026-10-04 · 207 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_joint_imputation_and_clustering_983ea26f,
  title = {joint imputation and clustering},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/joint-imputation-and-clustering/rdr_983ea26f065d40018baa1d0d446d6d09},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_joint_optimization_of_imputation_and_dow_983ea26f_v1,
  title = {Joint Optimization of Imputation and Downstream Tasks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/joint-optimization-of-imputation-and-downstream-tasks/rdr_983ea26f065d40018baa1d0d446d6d09/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}