Real-world datasets are frequently riddled with missing values. Whether due to sensor failures, partially filled user profiles, or incomplete medical records, data sparsity poses a fundamental challenge to machine learning models. Traditional heuristics—such as discarding incomplete rows or filling gaps with static mean values—often collapse under high missingness because they fail to capture the complex underlying distribution of the data.
To build systems robust to extreme data incompleteness, modern research moves away from static, isolated preprocessing. Instead, it focuses on frameworks that natively incorporate missingness into the representation learning process, treating the absence of data not as a flaw to be patched, but as a geometric or structural constraint to be modeled.
The Imputation-Learning Feedback Loop
Why separate data recovery from downstream tasks? In traditional workflows, an imputation algorithm fills missing values before a predictive model ever sees the data, which can propagate estimation errors and distort downstream gradients. To solve this, early work by Zoubin Ghahramani et al. in Supervised learning from incomplete data via an EM approach introduced a unified framework using the Expectation-Maximization (EM) algorithm. By treating supervised learning as a joint density estimation problem using mixture models, the framework learns parameters and fills in missing values simultaneously, ensuring that the optimization of the predictive task directly guides the data recovery.
This concept of a continuous feedback loop has evolved significantly. For instance, in Leachable Component Clustering, Miao Cheng et al. integrated data imputation and partition learning into a single framework. By utilizing a Bayes alignment strategy to preserve the data's intrinsic statistical distributions, the model continuously refines its cluster partitions and distribution parameters. This iterative feedback loop allows the system to extract underlying patterns and maintain stable performance even when a large majority of data elements are missing.
Multi-View Alignment and Prototype Synchronization
Multi-view learning—where models integrate diverse data modalities like audio, video, or text—faces severe degradation when entire views are unavailable for certain samples. If a model relies on rigid contrastive learning, missing views cause the latent representations of identical concepts to drift apart, destroying the shared semantic space.
To address this, Jiaqi Jin et al. proposed Deep Incomplete Multi-View Clustering with Cross-View Partial Sample and Prototype Alignment. Their framework combines partial sample alignment with global prototype synchronization, ensuring that cluster centers remain stable even as data sparsity increases.
[View 1: Audio] ───► (Partial Sample Alignment) ◄─── [View 2: Video] (Missing for Sample B)
│ │
▼ ▼
[Local Representation] [Local Representation]
│ │
└───────────────► [Global Prototype Bank] ◄──────────┘
(Global Synchronization)
Similarly, Yanwanyu Xi developed a self-supervised framework in LRGR: Self-Supervised Incomplete Multi-View Clustering via Local Refinement and Global Realignment to prevent the "blurry" results typical of graph convolutional networks when dealing with missing views. By balancing local feature distinctiveness with global label consistency, the model ensures accurate grouping under extreme missingness. Furthermore, Pengfei Zhu et al. demonstrated in Latent Heterogeneous Graph Network for Incomplete Multi-View Learning that combining neighborhood constraints with explicit view-existence patterns allows classification accuracy to remain stable, experiencing less than a 3% drop even when half of the data views are missing.
Exploiting Low-Rank Subspaces
A core intuition behind handling high data missingness is that high-dimensional data often resides within a much lower-dimensional, highly structured subspace. Consequently, models do not always need to reconstruct every missing value explicitly to perform downstream tasks accurately; they only need to constrain the optimization to this lower-dimensional manifold.
In Classification with Low Rank and Missing Data, Elad Hazan et al. proved that similarity-based classification can be executed using only the commonly observed attributes. This approach provides robust performance guarantees regardless of whether the missingness pattern is random or adversarial, bypassing explicit reconstruction entirely while matching the performance of an optimal full-data classifier.
When regression or matrix completion is required, enforcing low-rank constraints yields remarkable resilience. For example, Masaaki Takada et al. introduced HMLasso: Lasso with High Missing Rate, which leverages a weighted positive semidefinite matrix approximation to maintain sparse regression accuracy at missingness rates up to 90%. Similarly, in the domain of video processing, Bo Shen et al. developed Smooth Robust Tensor Completion for Background/Foreground Separation with Missing Pixels: Novel Algorithm with Convergence Guarantee. By decomposing corrupted video data into a low-rank background and a smooth foreground, their method achieves superior reconstruction metrics even when 90% of the pixels are absent.
Native Missingness Tracking via Dual Streams
Rather than treating missingness as a hidden corruption to be masked out, another highly effective strategy is to track it explicitly as an informative feature. The presence or absence of data points often follows temporal or structural patterns that can guide neural architectures.
In multivariate time-series forecasting, recurrent networks often suffer from sharp performance drops when faced with large temporal gaps because the hidden state decays unpredictably. To mitigate this, Jing Peng et al. designed S4M: S4 for multivariate time series forecasting with Missing values. This framework uses a dual-stream architecture that processes raw data in parallel with explicit missingness masks, allowing Structured State Space (S4) models to maintain stable accuracy across long gaps by explicitly contextualizing when data was lost.
This native tracking of missingness is equally valuable in distributed settings where data cannot be centralized due to privacy concerns. In Efficient Federated Incomplete Multi-View Clustering, Suyuan Liu et al. integrated indicator matrices directly into the global optimization objective. By natively accounting for missing views without relying on zero-padding or artificial imputations, the framework preserves client privacy and minimizes communication overhead while remaining highly robust to data incompleteness.
Key Foundational Work
Supervised learning from incomplete data via an EM approach (1993) — contributed 1 concept elements to this direction.
Learning Bayesian Networks from Incomplete Data (1997) — contributed 1 concept elements to this direction.