Unified Learning for Imputation and Feature Selection

This direction is merged into another direction: joint imputation and clustering. Its last version stays readable and citable here.

Version 1 of 110 papers20 conceptsfrozen 2026-10-04

In traditional machine learning pipelines, dealing with incomplete data is a two-step process: first, you "fill in the blanks" using an imputation method (like mean substitution or matrix completion), and then you perform feature selection on the resulting dataset.

However, this sequential approach suffers from a fundamental "chicken and egg" problem. To find the most important features, you need a complete dataset; but to fill in missing values accurately, you need to know which features actually matter for the underlying structure of the data. Recent research has shifted toward a unified paradigm where imputation and feature selection are performed simultaneously, creating a synergistic feedback loop that outperforms sequential processing.

The Failure of Sequential Pipelines

The primary weakness of the "impute-then-select" strategy is error propagation. If the initial imputation step introduces noise or fails to capture the specific relationships between variables, that error is baked into the data before the feature selection algorithm ever sees it.

As noted in A novel feature selection framework for incomplete data, traditional methods treat imputation as a preprocessing step independent of the downstream task. This is problematic because the goal of imputation in a machine learning context isn't necessarily to recover the "true" missing value, but to recover a value that preserves the discriminative power of the features.

The Synergistic Feedback Loop

The core intuition behind unified frameworks like UNIFIER, developed by Yanyong Huang, Tianrui Li, and colleagues in Unified View Imputation and Feature Selection Learning for Incomplete Multi-view Data, is to treat missing values as variables to be optimized alongside feature weights.

In this joint optimization framework:

  1. Feature weights guide imputation: The model prioritizes reconstructing values for features that appear to be highly informative or relevant to the data's manifold structure.
  2. Imputed values refine selection: As the missing values are filled with more "structure-aware" estimates, the feature selection mechanism gains a clearer picture of which variables are truly redundant.

This creates a mutual reinforcement effect. By alternating between predicting missing values and updating feature weights, the model ensures that the reconstructed data is specifically optimized to enhance the identification of informative features.

Preserving Structure Over Accuracy

A surprising insight from this field is that successful feature selection does not actually require perfect reconstruction of missing entries. Chuyang Ke and Jean Honorio demonstrate in Provable Guarantees for Sparsity Recovery with Deterministic Missing Data Patterns that imputation strategies only need to preserve a property called mutual incoherence.

Mutual incoherence ensures that irrelevant features do not become indistinguishable from relevant ones during the recovery process. This suggests that the "goal" of joint learning isn't pixel-perfect accuracy, but rather the preservation of the underlying geometry of the data. This is why manifold learning is a recurring theme in these papers; by focusing on the local structures of samples and features, researchers like Wanzhu Wang can recover missing labels and features simultaneously by ensuring they both adhere to the same low-dimensional "shape" (Multi-label Feature Selection based on Label-specific features and Manifold Learning).

Scaling to Multi-View and Mixed-Missing Data

Modern datasets often involve "multi-view" data—information coming from different sensors or sources. This introduces the "mixed-missing" problem, where some samples might be missing a few variables, while others are missing entire views.

To address this, researchers have expanded the unified framework to include:

  • Instance Selection: Frameworks like JUICE, proposed by Yuxin Cai 0001 and Dongjie Wang, integrate instance selection into the loop. By identifying the most representative samples, the model can use cross-view neighborhood information to fill in missing data more accurately (Joint Learning of Unsupervised Multi-view Feature and Instance Co-selection with Cross-view Imputation).
  • Tensorial Learning: In TIME-FS, Yanyong Huang and Minghui Lu use tensor-based anchor graphs to unify missing-view imputation and feature selection. This approach treats the multi-view data as a high-dimensional cube (tensor), allowing the model to capture complex correlations across different views that matrix-based methods might miss.

By moving away from isolated preprocessing and toward integrated optimization, these methods ensure that even when data is "messy," the resulting models are built on the most robust and informative foundations possible.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_unified_learning_for_imputation_and_feat_3db46005,
  title = {Unified Learning for Imputation and Feature Selection},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/unified-learning-for-imputation-and-feature-selection/rdr_3db460050792427cbfe1b73715a6d347},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_unified_learning_for_imputation_and_feat_3db46005_v1,
  title = {Unified Learning for Imputation and Feature Selection},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/unified-learning-for-imputation-and-feature-selection/rdr_3db460050792427cbfe1b73715a6d347/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}