Introduction
Missing data is a pervasive challenge across scientific disciplines, from clinical trials to computer vision. Traditional recovery methods, such as Low-Rank Matrix Completion (LRMC), typically assume that data points reside on a single linear subspace. However, real-world data is often more complex, consisting of multiple categories or clusters that each follow their own local structures. This complexity introduces a "chicken-and-egg" dilemma: accurate data completion requires knowledge of the underlying cluster structure, yet accurate clustering requires complete data.
Deep-Union Completion (DUC) addresses this by embedding data into a non-linear "Union of Subspaces" (UoS). By integrating completion and clustering into a unified deep learning framework, DUC overcomes the biases introduced by naive imputation methods like zero-filling, which often disrupt underlying data geometry.
Methodology
The DUC architecture is composed of three integrated stages designed to learn the data's geometric structure while simultaneously filling in missing values:
- Pseudo-Completion Layer: To avoid the biases of naive zero-filling, this layer uses partially connected neural pathways to provide an initial estimate of missing values. This "warm-up" phase ensures the network maintains active connections even when a high percentage of data is missing.
- Autoencoder: The pseudo-completed data is mapped into a lower-dimensional latent space through a standard autoencoder, facilitating the extraction of essential features.
- Closed-Form Clustering Layer: This layer is the core innovation. Instead of relying on explicit clustering penalties that require extensive hyperparameter tuning, DUC uses a projection operator ($P$). By requiring the decoder to reconstruct data from $ZP$ (where $Z$ is the latent embedding), the model implicitly enforces a Union of Subspaces structure. If the reconstruction is successful, the data must follow a UoS model.
Experimental Results
DUC was evaluated against several state-of-the-art methods on diverse benchmarks, including image datasets (COIL20, Extended Yale B, ORL) and non-image regression datasets (Boston Housing, Heart Disease).
Quantitative Performance
The proposed method demonstrates massive accuracy gains. On real-world datasets like COIL20 and Extended Yale B, DUC outperformed the next best algorithms by over 40% in clustering accuracy across almost all missing-data intervals. Notably, on the COIL20 dataset, the model achieved near-zero completion error even with up to 60% of the data missing.
Qualitative Reconstruction
Visual results highlight DUC's resilience to extreme data loss. Even when observed data is reduced to 20%—appearing as near-random noise to the human eye—DUC successfully reconstructs recognizable faces and objects, preserving the underlying semantic content.
Discussion and Conclusion
DUC represents a significant advancement in data completion by treating the Union of Subspaces as a universal generalizer. It effectively bridges the gap between imputation and structural learning. While the model currently requires an estimate of the number of subspaces ($K$), its architectural approach to implicit clustering eliminates much of the manual tuning required by previous methods. Future work may focus on scaling the closed-form clustering layer for massive datasets through mini-batching techniques to handle big data applications more efficiently.
Summary updated