RefiDiff: Refinement-Aware Diffusion for Efficient Missing Data Imputation

Md. Atik Ahamed, Qiang Ye, Qiang ChengarXiv 2025 · DOI · arXiv

TL;DR

This paper introduces RefiDiff, a novel imputation framework that unifies predictive and generative approaches for handling missing data in complex, high-dimensional datasets. By combining a progressive refinement strategy with a Mamba-based diffusion model, RefiDiff achieves state-of-the-art performance, particularly in challenging MNAR scenarios, while being significantly more efficient than existing diffusion-based methods.

What Problem Does This Paper Address?

Accurate data imputation is crucial for reliable downstream analysis, but it's a significant challenge, especially with high-dimensional, mixed-type datasets where missing values can occur under various mechanisms, including the difficult "Missing Not At Random" (MNAR) scenario. Existing imputation methods often fall into two camps: predictive methods that are fast but deterministic, and generative methods that offer uncertainty but are computationally expensive. Many struggle to effectively integrate local data characteristics with global patterns, leading to limitations in performance, robustness, and efficiency, particularly for MNAR data. This paper aims to bridge this gap by proposing a unified framework that leverages the strengths of both predictive and generative paradigms.

Methods

RefiDiff employs a four-stage methodology:

  1. Pre-processing: Standardizes numerical features and binary-encodes categorical features to create a unified data representation.
  2. Warm-up Refinement: A novel one-pass predictive stage that uses lightweight machine learning models (like XGBoost) to generate initial imputations for missing values. This "warms up" the data, making it easier for the subsequent diffusion stage.
  3. Diffusion Imputation: A global generative stage that uses a Mamba-based denoising network. Mamba, a selective state-space model, is chosen for its ability to efficiently capture long-range dependencies in data with linear complexity, a significant advantage over traditional Transformers. This stage iteratively refines the imputations using a diffusion process, carefully preserving observed data.
  4. Post-processing and Polishing: De-standardizes and decodes the imputed data back to its original format. An optional polishing step further refines the imputations using the predictive models from the warm-up stage.

The core novelty lies in the progressive refinement strategy (warm-up and polishing) and the use of a Mamba-based denoising network within the diffusion framework.

Datasets

The authors evaluated RefiDiff on nine public real-world datasets:

  • Numerical-only: California, Magic, Bean, Gesture, Letter
  • Mixed-type: Default, News, Adult, Shoppers

These datasets are standard benchmarks in the imputation literature and are available from Kaggle or the UCI Machine Learning Repository. The paper also details the statistics of these datasets in Table 3.

Key Insights and Ideas

  • Unifying Predictive and Generative Paradigms: The paper demonstrates that combining a fast, local predictive "warm-up" with a global generative diffusion process leads to more robust and efficient imputation than relying on either approach alone.
  • Mamba's Efficiency for Long-Range Dependencies: The use of Mamba in the denoising network is a key innovation. Its linear complexity allows it to effectively model relationships across many features and samples, a crucial advantage for high-dimensional data, without the quadratic cost of Transformers.
  • Progressive Refinement for Stability: The warm-up and polishing stages provide initial plausible values and then refine them, leading to more stable and accurate imputations, especially in challenging MNAR scenarios.
  • Tuning-Free Imputation: RefiDiff is designed to be a plug-and-play solution, requiring no architectural or hyperparameter tuning across diverse datasets, making it highly practical.

Key Results

  • State-of-the-Art Performance: RefiDiff outperforms existing methods, including strong baselines like DIFFPUTER, across MCAR, MAR, and MNAR settings, with particularly significant gains in MNAR scenarios.
  • Efficiency Gains: RefiDiff is approximately 4x faster than DIFFPUTER, a comparable diffusion-based method, due to its efficient Mamba-based denoiser and one-pass warm-up strategy.
  • Robustness: The framework shows consistent performance and lower variance across different random masks, indicating its reliability.
  • Parameter Efficiency: The Mamba-based denoiser uses significantly fewer parameters than DIFFPUTER's Transformer-based denoiser while achieving comparable or better performance.

Limitations and Future Work

  • Binary Encoding for Categorical Data: While RefiDiff handles mixed-type data, it uses binary encoding for categorical features. The authors suggest that future work could explore native treatments of categorical variables for potentially improved semantic fidelity.
  • Streaming Data and Adaptive Refinement: The authors plan to extend RefiDiff to handle streaming data and develop adaptive refinement strategies for sparse datasets.
  • Domain-Specific Applications: Future work will also focus on applying RefiDiff to specialized domains like healthcare and finance.

Summary updated

BibTeX
@misc{ahamed2025refidiff,
  author = {Md. Atik Ahamed and Qiang Ye and Qiang Cheng},
  title = {{RefiDiff: Refinement-Aware Diffusion for Efficient Missing Data Imputation}},
  year = {2025},
  eprint = {2505.14451},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2505.14451},
}
Figure