Abstract
Subspace-Preserving Recovery (SPR) is a fundamental task in high-dimensional data analysis, ensuring that data points are represented using only samples from their own subspace. While well-studied for single-view data under strict equality constraints, multi-view scenarios and noisy real-world data present significant theoretical gaps. This paper introduces the Regularized Multi-view Joint Sparse Representation (RMJSR) model, providing a new theoretical foundation that moves beyond unrealistic equality constraints. We derive Individual and Universal Theoretical Guarantees (ITG and UTG) based on a novel geometric "Multi-view Oracle Point." Numerical experiments confirm that our theoretical bounds accurately predict the success of subspace recovery and classification tasks.
Introduction
Many high-dimensional datasets, such as video trajectories or facial images, can be modeled as a union of low-dimensional subspaces. The goal of Subspace-Preserving Recovery (SPR) is to ensure that a data point's representation is "clean"—it should only involve points from the same class or subspace. In multi-view learning, where data is captured through multiple modalities (e.g., text and images), achieving SPR is more complex. Previous theories often relied on strict equality constraints ($y = Dc$), which fail in the presence of noise or sampling gaps. This work addresses these limitations by analyzing a regularized joint representation model that is more robust for practical engineering applications.
The RMJSR Model and Theoretical Guarantees
The authors propose the Regularized Multi-view Joint Sparse Representation (RMJSR) model, which replaces hard constraints with a regularized objective: $$\min_{C} \|C\|_{1,2} + \frac{\gamma}{2} \sum_{v=1}^{V} \|y^v - D^v C(:, v)\|_2^2$$ The $\|C\|_{1,2}$ norm encourages row-sparsity, ensuring the same representative samples are selected across all $V$ views.
A key contribution is the derivation of two types of guarantees:
- Individual Theoretical Guarantee (ITG): Conditions for the correct recovery of a specific multi-view test sample.
- Universal Theoretical Guarantee (UTG): Conditions ensuring the entire dataset's classification or clustering task succeeds.
Geometric Intuition: The Oracle Point
The success of RMJSR is tied to the geometric properties of the data. The authors introduce the Multi-view Oracle Point, a dual-problem solution that serves as a reference for determining if a representation will be subspace-preserving.
Recovery success depends on:
- Multi-view Coherence: The proximity of samples from other classes to the "Oracle Point" of the current sample.
- Multi-view Circumradius: The spread of samples within the same class. If intra-class samples are too tightly clustered, recovery may fail despite high inter-class separation.
Experimental Validation
The theoretical framework was validated using synthetic multi-view datasets with varying inter-class coherence ($\mu_M$) and sampling density ($\rho$).
The results demonstrate a clear phase transition: as inter-class coherence decreases and sampling density increases, the Multi-view Subspace-Preserving (MSP) error drops to zero. Crucially, the experiments show that when the Universal Multi-view Recovery Condition (UMRC) is met, the classification error reaches 0%, confirming the practical utility of the derived theoretical bounds.
Conclusion
This work provides a robust theoretical bridge between individual sample recovery and universal dataset performance in multi-view subspace learning. By relaxing the equality constraint through regularization and providing geometric conditions for success, the RMJSR model offers a more realistic framework for high-dimensional data analysis. Future work will focus on extending these guarantees to non-linear manifolds and improving computational efficiency for large-scale datasets.
Summary updated