Joint Imputation and Task Optimization for Incomplete Data

394 concepts235 papers1 proposal

Real-world datasets are rarely pristine. Whether due to sensor failures in time-series tracking, unaligned perspectives in multi-view learning, or missing attributes in a social network graph, data scientists routinely confront the challenge of missing values.

The traditional remedy is a two-stage pipeline: first, impute the missing values using a statistical or generative model (such as mean imputation, MICE, or a GAN), and second, feed the completed data into a downstream model for clustering, classification, or forecasting.

While intuitive, this "impute-then-analyze" approach suffers from a fundamental flaw: the imputation model operates in a vacuum. It fills in blanks to maximize statistical plausibility or minimize reconstruction error, completely unaware of what the data will be used for. This decoupling leads to error propagation, feature drift, and the accidental erasure of critical downstream signals.

A rapidly growing body of research solves this by unifying data imputation and downstream tasks into a single, end-to-end optimization framework. By allowing the final objective to guide the data recovery process, these methods ensure that imputed values are not just plausible, but functionally optimal.

The Flaw of Independent Imputation

To understand why joint optimization works, we must first look at how independent imputation fails. When an imputation model minimizes reconstruction error alone, it prioritizes the global or "normal" distribution of the data. However, what is statistically typical is not always what is useful for a specific task.

Consider anomaly detection. In Unsupervised Anomaly Detection in The Presence of Missing Values, Feng Xiao et al. point out that independent imputation models—trained predominantly on normal data—will inadvertently "fix" anomalous data points. By mapping outliers back toward the normal distribution, the independent pre-processing step destroys the very signal the downstream model is trying to detect.

A similar problem occurs in regression and classification. As Marine Le Morvan et al. demonstrate in What’s a good imputation to predict with missing values?, decoupling imputation from prediction introduces unnecessary structural complexity. An independent imputation might create jagged, non-smooth boundaries in the feature space that make downstream regression incredibly difficult. By contrast, optimizing both steps simultaneously forces the system to learn an internal completion strategy that keeps the downstream target function smooth and predictable.

The Mutual Refinement Feedback Loop

The core mechanism of joint optimization is a symbiotic feedback loop: the emerging task structure informs data recovery, and better data recovery refines the task structure.

┌──────────────────────────────┐
│  Downstream Task Objective   │
│ (Clustering, Classification) │
└──────────────┬───────────────┘
               │ Guides
               ▼
┌──────────────────────────────┐
│   Data Imputation Module     │
│  (Fills in Missing Values)   │
└──────────────┬───────────────┘
               │ Refines
               ▼
┌──────────────────────────────┐
│   Coherent Representation    │
└──────────────────────────────┘

This synergy is particularly evident in incomplete multi-view clustering, where models must group objects observed through different modalities (e.g., matching a person's identity using both video gait and audio voice when one or the other is occasionally missing).

In traditional pipelines, missing views are filled in prior to clustering, which often introduces view-specific noise. However, by unifying cross-view consistency and clustering into a single objective, papers like COMPLETER by Yijie Lin et al. leverage contrastive learning to maximize mutual information across available views while simultaneously predicting missing data.

When the clustering objective is baked directly into the recovery process, the model avoids generating features that are statistically reasonable but cluster-irrelevant. As Yiming Wang et al. show in Incomplete Multiview Clustering via Cross-View Transfer, using a self-supervised clustering loss to guide latent representations ensures that recovered features are explicitly optimized for group separability.

Lishan Feng and Jingya Chang describe this in An adaptive weighted self-representation method for incomplete multi-view clustering as an iterative process: as the model identifies cleaner cluster centroids, it gains a clearer template for what the missing values should look like, which in turn sharpens the cluster assignments.

Implementation Across Data Domains

The philosophy of joint optimization manifests differently depending on the underlying data structure, but the mathematical intuition remains identical.

Graph Neural Networks

In graph-based learning, missing node attributes can cause "feature drift" during neighborhood aggregation. To combat this, the Attribute-Missing Graph Clustering Network (AMGC), developed by Wenxuan Tu et al., treats clustering labels as "prompts" that actively guide feature completion.

For more complex graph topologies, Guanghui Zhu et al. propose AutoAC, which uses a bi-level optimization strategy. Here, the attribute completion parameters and the graph neural network weights are updated simultaneously, ensuring that the neighborhood embeddings remain tightly aligned with the downstream classification goal.

Time-Series Analysis

Time-series data from medical sensors or traffic monitors is notoriously irregular and riddled with gaps. In BRITS, Wei Cao et al. treat missing values as learnable variables within a bidirectional recurrent neural network (RNN). By training the network with a joint loss function that balances imputation error with classification error, the model learns trajectories that are optimized for downstream predictions, such as patient mortality.

This end-to-end paradigm is mirrored in clinical settings by Michael Moor et al. (Early Recognition of Sepsis) and in spatial-temporal forecasting by Chengqing Yu et al. (GinAR), proving that maintaining internal consistency across the pipeline prevents the compounding of errors over time.

Why It Works: Task-Driven Empirical Risk

The mathematical justification for joint optimization traces back to foundational principles of empirical risk minimization. In Learning from incomplete data with infinite imputations, the authors demonstrate that if you treat the distribution of missing values as a free parameter within the learning objective, the optimization naturally concentrates on a finite set of imputations that maximize the model's predictive power.

By framing imputation not as a generative problem (modeling the data for its own sake) but as a functional problem (modeling the data to minimize a specific loss), the model gains several key advantages:

  1. Robustness to Non-Random Missingness: When data is "Not Missing At Random" (NMAR)—such as trauma patients missing specific test results because they were too critical to test—sequential methods fail because the observed data distribution is fundamentally biased. As shown by Alexander Van Esbroeck et al. in Evaluating Trauma Patients, joint optimization bypasses this bias by anchoring the imputation directly to the outcome prediction.
  2. Feature and Label Synergy: In unsupervised feature selection, integrating imputation ensures that reconstructed values align strictly with the most discriminative features. Yanyong Huang et al. leverage this in TIME-FS to filter out view-specific noise. Similarly, Guoxian Yu demonstrates in Weakly-Supervised Multi-view Learning that simultaneously optimizing for cross-view alignment and label replenishment prevents the accumulation of errors that plagues multi-stage pipelines.
  3. Localization of Structure: Rather than forcing a global reconstruction that might not fit all subsets of data, joint optimization allows the model to focus on local structures. For example, En Zhu (Localized Incomplete Multiple Kernel k-means) demonstrates that letting a clustering objective guide kernel element imputation within localized neighborhoods yields far more representative data recovery than global alignment strategies.

Ultimately, joint imputation and task optimization works because it respects a simple truth: data is collected to be used. By breaking down the wall between data cleaning and data analysis, machine learning models can finally see the forest and fill in the trees.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 239 papers
  2. Version 1 · 2026-10-04 · 207 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_joint_imputation_and_clustering_983ea26f,
  title = {joint imputation and clustering},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/joint-imputation-and-clustering/rdr_983ea26f065d40018baa1d0d446d6d09},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_joint_imputation_and_clustering_983ea26f_v2,
  title = {joint imputation and clustering},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/joint-imputation-and-clustering/rdr_983ea26f065d40018baa1d0d446d6d09/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake