In the era of big data, information is rarely monolithic. A single object or event is often captured through multiple lenses—different sensors, varying camera angles, or distinct feature types. This "multi-view" nature of data presents a significant challenge: how can we effectively fuse these divergent sources into a single, robust representation? This survey paper, originally presented at IJCAI-18, addresses this by bridging the gap between two traditionally separate fields: Multi-view Learning (MVL) and Domain Adaptation (DA).
The Challenge of View Divergence
Real-world applications, such as autonomous driving or cross-modal retrieval, grapple with "view divergence." For instance, a self-driving car must reconcile high-resolution camera images with sparse radar point clouds. Similarly, a face recognition system might need to match a visible-light photograph against an infrared scan. The statistical distributions of these views are often so different that direct comparison is impossible.
Historically, researchers approached this through two distinct lenses:
- Multi-view Learning (MVL): Focuses on fusing information when corresponding samples exist across views (e.g., an image paired with its textual caption).
- Domain Adaptation (DA): Focuses on transferring knowledge from a labeled "source" domain to an unlabeled "target" domain where samples do not directly correspond.
The authors argue that both fields share a fundamental goal: mitigating view divergence to create a shared, discriminative feature space.
A Unified Mathematical Framework
The core contribution of this work is a unified learning framework that treats MVL and DA as a single optimization problem. This perspective allows for a systematic categorization of existing algorithms and provides a roadmap for future research.
The Unified Perspective Formula
The general objective function proposed is: $$\min \sum A(f_i(X_i), f_j(X_j)) + \lambda \sum R(f_k(X_k))$$
- $A(\cdot)$ (Alignment Term): This acts as the bridge between views. It measures the alignment of different views in the learned shared space. In MVL, this might involve maximizing correlation (e.g., Canonical Correlation Analysis); in DA, it often involves minimizing distribution distances like Maximum Mean Discrepancy (MMD).
- $R(\cdot)$ (Feature Learning Regularizer): This term ensures the learned features are useful for downstream tasks. It may preserve the data's geometric structure or incorporate label information to enhance discriminative power.
To understand the specific operators and variables used in these optimization problems, the authors provide a comprehensive notation table.
Methodological Spectrum
The paper reviews three primary categories of techniques within this unified framework:
- Subspace Learning: These methods seek a low-dimensional "common ground." Classic examples include Canonical Correlation Analysis (CCA) and its variants, which find linear projections that maximize the correlation between views.
- Deep Learning: Deep Neural Networks (DNNs) are employed to learn complex, non-linear mappings. Methods like Deep CCA or Multi-view Deep Networks have become the state-of-the-art for capturing intricate relationships in high-dimensional data.
- Low-Rank Modeling: By applying matrix factorization and nuclear norm constraints, these methods uncover the latent structure shared across views while filtering out view-specific noise. This is particularly effective for creating representations robust to sensor-specific artifacts.
Key Insights and Performance Trends
Several critical insights emerge from this unified view:
- The Correspondence Divide: The primary distinction between MVL and DA lies in the "sample-wise relationship." MVL assumes paired samples (the same object in different views), whereas DA assumes different objects belonging to similar underlying categories.
- View-Invariant Projections: A significant shift in the field has been moving from learning links between specific views to learning a single, view-invariant projection. This is vital for "Unknown Probe View" scenarios, where the system must identify an object regardless of the camera angle or modality used during testing.
- Efficiency vs. Complexity: While deep methods generally outperform shallow ones due to their non-linear capacity, they come with high computational costs. The survey highlights Marginalized Denoising Auto-Encoders as a middle ground—offering high efficiency and comparable results for large-scale applications.
Future Directions and Limitations
Despite significant progress, several hurdles remain:
- Computational Scaling: Many robust methods rely on operations like eigen-decomposition or nuclear norm minimization, which scale cubically ($O(n^3)$) with the number of samples, making them difficult to apply to "big data."
- Extreme Imbalance and Incompleteness: Real-world data is often imbalanced (e.g., many RGB images but few forensic sketches) or incomplete (missing views for certain samples). Developing models that can handle these "incomplete multi-source" scenarios is a high priority.
- End-to-End vs. Hybrid Models: While end-to-end deep learning is popular, traditional feature engineering combined with domain adaptation often remains superior for smaller, specialized datasets where data is scarce.
Summary updated