In standard machine learning, missing data is often treated as a nuisance—a hole to be filled by the mean, a median, or a sophisticated guess. This approach relies on a comforting but often false assumption: that data is Missing At Random (MAR). Under MAR, the reason a value is missing has nothing to do with the value itself.
However, real-world data is frequently Missing Not At Random (MNAR). As Aude Sportisse, Claire Boyer, and Julie Josse explain in Estimation and Imputation in Probabilistic Principal Component Analysis with Missing Not At Random Data, MNAR occurs when the absence of a value is tied to the value itself. Think of a survey where high-income individuals refuse to disclose their salary, or a clinical setting where a doctor only orders a specific test because they suspect a specific disease. In these cases, the "missingness" is not noise; it is a signal.
The Geometry of Missingness
One might assume that if the underlying relationship in a dataset is linear, a simple linear model should handle missing values gracefully. Marine Le Morvan and colleagues challenge this intuition in Linear predictor on linearly-generated data with missing values: non consistency and solutions. They prove that even when a target variable is a perfectly linear function of the input, the presence of missing values makes the optimal predictor non-linear.
To see why, imagine a 2D plane where points are missing along a specific axis. The "best guess" for a missing value depends on the pattern of other missing values in that specific row. A linear model would require an exponential number of parameters—one for every possible combination of missing features—to remain accurate. Instead, researchers have found that non-linear architectures like Multilayer Perceptrons (MLPs) or the ReMasker framework by Tianyu Du can automatically adapt to these patterns by learning missingness-invariant representations.
Missingness as a Predictive Signal
If the reason data is missing is informative, then the "mask" (the binary matrix indicating what is missing) should be treated as a feature, not just a metadata field.
In clinical informatics, this is known as observation bias. Qian Dong and William K. Cheung 0001 developed PATNet (PATNet: Propensity-Adjusted Temporal Network for Joint Imputation and Prediction Using Binary EHRs With Observation Bias) to explicitly model this. By using propensity scores to account for why a clinician chose to record a specific data point, PATNet improves mortality forecasts. Similarly, Yue Li introduced MixEHR (A latent topic model for mining heterogenous non-randomly missing electronic health records data), which treats patients as "documents" and diseases as "topics," recognizing that the absence of a lab result is often as diagnostic as the result itself.
This perspective shift is also critical for fairness. Raymond Feng and Flávio P. Calmon demonstrate in Adapting Fairness Interventions to Missing Values that standard "impute-then-classify" pipelines can amplify bias. If missingness correlates with sensitive attributes (like race or socioeconomic status), smoothing over those gaps with mean imputation erases the structural disparities the model needs to understand to remain fair.
Joint Modeling via Latent Variables
How do we mathematically unify the observed data and the missingness mechanism? The emerging consensus points toward Joint Modeling. Instead of imputing first and predicting second, these models learn the distribution of the data $X$ and the missingness mask $M$ simultaneously.
- Shared Latent Factors: In matrix completion, A Yuanhong and Bingyi Jing suggest that the same hidden factors that determine a value (e.g., a user's preference for a movie) also determine if that value is missing (e.g., the user didn't watch the movie because they knew they wouldn't like it). Their framework (Leveraging Shared Factor Structures for Enhanced Matrix Completion with Nonconvex Penalty Regularization) uses these shared factors to improve recovery accuracy by up to 24%.
- Deep Generative Models: The Variational Selective Autoencoder (VSAE) by Yu Gong and Hossein Hajimirsadeghi uses a selective proposal distribution to handle heterogeneous data types. By modeling the joint distribution $P(X, M)$, the VSAE can generate realistic imputations even when the missingness is highly non-random (Variational Selective Autoencoder: Learning from Partially-Observed Heterogeneous Data).
Robustness Without Imputation
While joint modeling is powerful, it can be computationally expensive. An alternative is to make models inherently robust to missingness. Minh Nguyen and colleagues proposed Knockout (Knockout: A simple way to handle missing inputs), a training strategy that replaces features with placeholder values. This forces the network to learn how to marginalize over missing inputs at inference time, effectively teaching the model to "work around" the gaps without needing an explicit imputation step.
Summary of the Shift
The research direction across these 33 papers represents a move away from "fixing" incomplete data and toward "interpreting" it. By treating the missingness pattern as a structured, informative signal—and using latent variable models to bridge the gap between what we see and what we don't—researchers are building models that are more accurate, more fair, and more reflective of the complex biases inherent in real-world data collection.