Introduction
In complex domains such as autonomous driving and medical diagnostics, data is often collected from multiple heterogeneous sources or "views"—for instance, combining LiDAR, radar, and camera feeds. Traditional multi-view representation learning, exemplified by Canonical Correlation Analysis (CCA), typically seeks a global structure shared across all views. However, this approach often fails to capture "local" dependencies where only a subset of sensors are correlated. This paper proposes two novel frameworks, MOCCA (Multi-view One-vs-Rest CCA) and RMVAE (Robust Multi-view AutoEncoder), designed to capture these local relationships and improve robustness against missing or noisy data.
Proposed Methods
MOCCA: Multi-view One-vs-Rest CCA
To avoid the exponential complexity of modeling every possible subset of views, MOCCA reformulates the problem into $K$ "one-vs-rest" sub-problems. For each view $k$, the model attempts to predict its representation using a combination of the remaining $K-1$ views.
The core innovation is the application of a group-sparse penalty ($L_\infty$ norm) during the optimization process. This penalty encourages the model to select only the most informative subset of views to reconstruct the target view, effectively performing automatic view selection and uncovering the underlying redundancy structure of the data.
RMVAE: Robust Multi-view AutoEncoder
While MOCCA focuses on linear relationships, RMVAE extends these principles to non-linear representations using neural networks. RMVAE utilizes an AutoEncoder architecture where the latent space is forced to capture cross-view redundancies through view-level dropout. During training, entire views are randomly masked, requiring the encoder to learn a representation that can reconstruct all views even when some inputs are missing. This "stress testing" ensures the model does not become overly reliant on any single sensor.
Experimental Setup
The methods were evaluated using a synthetic four-view dataset ($X_1, X_2, X_3, X_4$) generated from four underlying feature sets ($A, B, C, D$). The views were constructed with specific redundancies (e.g., $X_1$ contains features $B, C, D$ while $X_2$ contains $A, C, D$), such that any two views provide sufficient information to reconstruct the others. This controlled environment allows for direct verification of whether the models can recover the known latent structure.
Results and Discussion
The effectiveness of the group-sparse penalty in MOCCA is clearly demonstrated through the visualization of the redundancy matrix.
As shown in Figure 1, the group-sparse penalty acts as a discovery tool, filtering out irrelevant correlations and highlighting the true local dependencies between views. Without this penalty, the relationship map is noisy and difficult to interpret; with it, the underlying block structure of the synthetic data is recovered.
For RMVAE, quantitative analysis confirmed that the model successfully learns to aggregate information. Reconstruction error decreased monotonically as more views were provided at test time, indicating that the latent representation effectively exploits the redundancies learned during the view-level dropout training phase.
Conclusion and Future Work
This work highlights the importance of modeling local view relationships for robust representation learning. By utilizing group-sparsity in MOCCA and view-level dropout in RMVAE, the authors provide computationally efficient methods for discovering and exploiting data redundancy. Future research directions include scaling MOCCA for high-dimensional datasets via greedy step-wise selection and extending RMVAE with flow-based generative models to handle more complex data distributions.
Summary updated