Task Aware Imputation for Classification

20 concepts12 papers

When dealing with real-world datasets, missing values are an unavoidable headache. The traditional pipeline treats this as a two-stage problem: first, fill in the missing features using an unsupervised statistical method (like mean imputation, $K$-nearest neighbors, or a generative model), and second, train a classifier on the completed data.

However, this decoupled approach has a fundamental flaw: unsupervised imputation aims to reconstruct the entire underlying data distribution $P(X)$. It wastes model capacity capturing variance that may be completely irrelevant to the downstream classification task, and worse, it can inadvertently smooth over the very decision boundaries we want to preserve. As noted by Solé‐Casals et al., failing to leverage class labels during imputation severely limits a model's effectiveness in supervised learning scenarios.

Task-aware imputation (or discriminant imputation) flips this paradigm. Instead of asking "What is the most statistically probable value for this missing feature?" it asks "What value should this feature take to best support the correct classification decision?" By tightly coupling feature reconstruction with the target task, we can leverage downstream labels as a powerful proxy signal to resolve ambiguities in incomplete data.


Discriminant vs. Generative Imputation

To understand why task-aware imputation works, consider the difference between modeling a full data distribution versus optimizing for a specific prediction error.

In a seminal early framework, Bengio et al. introduced a method using recurrent neural networks with feedback loops to handle missing or asynchronous data. Rather than attempting to model the true density of the missing variables, the network learns to "fill in the blanks" solely to minimize final prediction error. This approach is highly parameter-efficient; as the number of input variables scales, modeling the full joint probability distribution becomes exponentially harder, whereas optimizing a discriminant function remains highly tractable.

Traditional Pipeline:
Incomplete Data ──> [ Unsupervised Imputation (Minimize Reconstruction Error) ] ──> Completed Data ──> [ Classifier ] ──> Prediction

Task-Aware Pipeline:
Incomplete Data ──┐
                  ├──> [ Joint Optimization (Minimize Classification Error) ] ──> Prediction
Target Labels   ──┘

This task-driven guidance becomes crucial when data is Missing Not At Random (MNAR)—meaning the missingness pattern itself correlates with the hidden value or the outcome. For example, in trauma care, certain diagnostic tests might only be skipped if a patient is too critical or clearly healthy. Van Esbroeck et al. demonstrated that in these scenarios, incorporating target labels into a joint optimization framework provides a vital signal that traditional statistical imputation misses, allowing the model to recover functionally relevant feature estimates directly from the outcome context.


Enforcing Supervised Constraints on Reconstruction

Modern task-aware approaches implement this intuition by embedding classification objectives directly into the mathematical formulation of the reconstruction process.

One common strategy is to modify matrix completion or sparse coding objectives. In standard matrix completion, the goal is simply to find a low-rank matrix that minimizes reconstruction error on observed entries. Huang et al. altered this by integrating a supervised loss term directly into the objective function of their framework, AFASMC:

$$\min_{W, H} \mathcal{L}_{\text{reconstruction}}(X, WH) + \gamma \mathcal{L}_{\text{supervised}}(Y, f(WH))$$

By minimizing both errors simultaneously, the class labels act as a geometric constraint, ensuring that the recovered feature values are functionally consistent with known target categories.

Similarly, Caiafa et al. combined neural networks with sparse coding to simultaneously learn a data dictionary, sparse representations, and a classifier. Because the semantic context of the label backpropagates into the sparse reconstruction process, the model resolves ambiguities in incomplete inputs by explicitly preserving class-distinguishing boundaries.


Preserving Class Geometry and Sub-populations

When a dataset suffers from extreme missingness or severe class imbalance, unsupervised imputation tends to wash out minority class characteristics, biasing the filled-in values toward the majority distribution. Task-aware methods prevent this by making the imputation process explicitly class-conditional.

A clear example of this is found in generative architectures. Bennamoun et al. developed a Conditional Generative Adversarial Imputation Network (CGAIN) that feeds class labels into the generator as conditioning variables. This forces the network to synthesize missing values tailored to specific sub-populations, ensuring minority class boundaries are not overwhelmed. In the context of multivariate time series, Miao et al. similarly utilized classification feedback to generate missing steps that align tightly with specific class trajectories.

Even simpler algorithms benefit from this geometric awareness. For instance, Huang demonstrated that restricting the search space of nearest-neighbor imputation to samples within the same predicted class prevents the interpolation of irrelevant features from opposing categories.

Taking this concept to its logical conclusion, Yang et al. proposed a two-stage framework that completely decouples feature reconstruction from the raw data distribution. It first optimizes a similarity (kernel) matrix based entirely on class labels to maximize class separability, and subsequently reconstructs the missing features solely to align with that optimized matrix. This radical shift focuses entirely on the functional utility of the data, achieving high classification accuracy even when more than 60% of the features are missing.


Closing the Loop: Handling Incomplete Labels

A natural chicken-and-egg problem arises in task-aware imputation: if filling in missing features depends on knowing the class labels, what happens if the labels themselves are missing or incomplete during training?

Recent research addresses this by co-optimizing feature imputation and label reconstruction simultaneously:

  • Pseudolabeling for MNAR Bias: To prevent information loss when labels are missing systematically, Du et al. developed a framework called BiasCorr. It assigns pseudolabels and "soft" selection values to unlabeled samples, allowing their incomplete features to safely contribute to the optimization gradient without introducing selection bias.
  • Self-Expression Learning: In multi-dimensional domains like emotion recognition, Xu et al. utilized a weighted self-expression mechanism that treats incomplete labels not as an obstacle, but as a constructive guide to perform feature selection and label replenishment in tandem.
  • Hierarchical Pre-processing: In multi-label biological settings where annotations are sparse due to experimental limitations, Yu et al. showed that leveraging taxonomy similarities and flat co-annotation correlations allows models to pre-estimate and replenish missing labels, establishing a robust foundation before downstream classification begins.

By treating data completion as a holistic, task-oriented optimization problem rather than a statistical data-cleaning chore, these methods ensure that every imputed value directly serves the ultimate goal: making accurate predictions.

Key Foundational Work

Recurrent Neural Networks for Missing or Asynchronous Data (1995) — contributed 3 concept elements to this direction.
Evaluating Trauma Patients: Addressing Missing Covariates with Joint Optimization (2014) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_label_guided_data_imputation_a4c068ef,
  title = {label-guided data imputation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/label-guided-data-imputation/rdr_a4c068ef3a9040848759ab23e832764d},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_label_guided_data_imputation_a4c068ef_v1,
  title = {label-guided data imputation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/label-guided-data-imputation/rdr_a4c068ef3a9040848759ab23e832764d/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}

See something wrong? Report a mistake