When dealing with real-world datasets, missing values are an unavoidable headache. The traditional workflow is straightforward: fill in the blanks using a "best guess"—whether via simple mean or median replacement, or more sophisticated machine learning models—and then pass the completed dataset to a downstream predictive model.
This approach suffers from a fundamental flaw known as deterministic imputation bias. When a missing entry is replaced by a single point estimate, the downstream model treats this guess as an absolute, indisputable fact. As Mark Mathieson noted in Ordered Classes and Incomplete Examples in Classification, ignoring the underlying distribution of missing values introduces systematic bias and leads to overconfident predictions. By flattening a probability distribution into a single value, we strip away the inherent uncertainty of the missing information, forcing models to make high-stakes decisions based on synthetic data treated as ground truth.
How Deterministic Guesses Sabotage Learning
Why exactly does this cause models to fail? The damage occurs through three primary mechanisms:
1. Error Propagation and Compounding
In deep learning architectures, an incorrect deterministic guess made early in the pipeline propagates through subsequent layers. As Marcin Przewiez Likowski et al. point out in MisConv: Convolutional Neural Networks for Missing Data, standard convolutional networks are designed for complete data grids; forcing a hard imputation early in the pipeline causes errors to compound across layers. Similarly, Hibiki Taguchi et al. demonstrate in Graph Convolutional Networks for Graphs Containing Missing Features that failing to propagate the uncertainty of imputed values through graph convolutional layers significantly degrades representation quality and predictive accuracy.
2. Missingness-Induced Heteroskedasticity
The absence of features naturally changes the variance of the outcome. In Conformal Prediction with Missing Values, Margaux Zaffran et al. explain that missing data patterns introduce heteroskedasticity because losing informative features increases prediction uncertainty. When a pipeline fills in these values before prediction without adjusting its confidence intervals, it produces prediction intervals that are dangerously narrow, creating a false sense of security.
3. Statistical Distortion and False Discoveries
In downstream statistical testing, treating imputed values as actual observations inflates false discovery rates. Haeun Moon et al. show in Augmented Doubly Robust Post-Imputation Inference for Proteomic data that standard imputation methods introduce systematic biases in high-dimensional data by failing to account for the uncertainty of the imputation process. This lack of variance representation leads to what Yi Deng et al. describe in Multiple Imputation for General Missing Data Patterns in the Presence of High-dimensional Data as "improper" imputations, where confidence intervals are artificially compressed because they do not reflect the true variability of the underlying data.
Propagating Distributions Instead of Points
To solve this, a growing body of research focuses on replacing point-estimate imputations with probabilistic frameworks that preserve and propagate uncertainty.
One elegant mechanism is to represent incomplete data points as probability distributions rather than scalar values. For instance, in Regression SVM for Incomplete Data, Łukasz Struski et al. propose representing missing attributes as Gaussian distributions within a Support Vector Regression framework, allowing the model to explicitly weigh the uncertainty of missing features.
In deep learning, instead of filling in a "best guess" pixel or feature, the framework proposed by Marcin Przewiez Likowski et al. analytically calculates the expected value of the convolution operation over the missing data distribution. This prevents the network from locking into an incorrect deterministic choice, allowing uncertainty to flow naturally through the network layers.
Uncertainty Quantification in Matrix Completion
A similar paradigm shift is occurring in matrix and tensor completion. While traditional matrix completion algorithms excel at generating point estimates for collaborative filtering or recommendation systems, they historically lacked "error bars." As Yuxin Chen in Inference and Uncertainty Quantification for Noisy Matrix Completion and Jerome P. Reiter in Introduction to the Special Section on Missing Data emphasize, the absence of uncertainty quantification makes it impossible for practitioners in high-stakes fields to know how much to trust a recovered value.
To address this, frameworks like the Low Rank Gaussian Copula developed by Yuxuan Zhao et al. in Matrix Completion with Quantified Uncertainty through Low Rank Gaussian Copula provide quantified uncertainty intervals alongside imputations across diverse data types. Similarly, Henry Shaowu Yuchi et al. introduce Bayesian frameworks in Bayesian Uncertainty Quantification for Low-Rank Matrix Completion that model underlying row and column subspaces to provide reliable error bounds, ensuring that downstream users can distinguish between highly reliable predictions and highly speculative guesses.
End-to-End and Imputation-Free Architectures
Perhaps the most robust way to avoid deterministic imputation bias is to bypass explicit pre-imputation entirely, or to build architectures that can dynamically "mute" unreliable data.
In Uncertainty-Gated Stochastic Sequential Model for EHR Mortality Prediction, Eunji Jun et al. introduced an "Uncertainty-Gated" recurrent network for electronic health records. The model learns to dynamically down-weight or mute unreliable imputed values, preventing imputation errors from corrupting downstream clinical predictions.
In clustering tasks, where noisy imputations can break the mathematical properties of similarity kernels (as explored by Fangchen Yu et al. in Boosting Spectral Clustering on Incomplete Data via Kernel Correction and Affinity Learning), frameworks like the one proposed by Jiaqi Jin et al. in Imputation-free and Alignment-free: Incomplete Multi-view Clustering Driven by Consensus Semantic Learning eliminate imputation altogether. By learning shared consensus prototypes directly from available views, they avoid introducing the synthetic noise that typically degrades multi-view representation learning.
By shifting the focus from predicting the missing value to modeling the range of what is possible, these approaches ensure that machine learning models remain honest about what they do not know.
Key Foundational Work
Ordered Classes and Incomplete Examples in Classification (1996) — contributed 3 concept elements to this direction.