Abstract
Multi-view learning is essential for processing data from heterogeneous sources, such as medical imaging or multi-modal social media content. However, effectively balancing the "consensus" (shared information) with "complementary" (view-specific) details remains a significant challenge. We introduce the Adversarial Co-atTention Network for multi-view Subspace learning (ANTS), a framework that integrates adversarial training with a co-attention mechanism. ANTS disentangles shared and specific representations while providing interpretability through feature weighting. Experimental results across five diverse datasets demonstrate that ANTS consistently outperforms state-of-the-art methods in both accuracy and robustness.
Introduction
In real-world applications, data often originates from multiple views. For instance, a medical diagnosis may integrate X-rays, blood tests, and patient history. Multi-view learning aims to leverage these diverse sources to improve predictive performance. Traditional approaches like Canonical Correlation Analysis (CCA) focus primarily on finding a common subspace, often discarding unique, view-specific information that could be critical for classification. Furthermore, many deep learning models operate as "black boxes," offering little insight into which features drive their predictions. ANTS addresses these issues by explicitly modeling both shared and specific information and incorporating an interpretable co-attention mechanism.
The ANTS Framework
The ANTS architecture is designed to extract robust, unified representations while preserving the unique nuances of each data source.
The framework consists of several key components:
- Co-attention Encoder: This module computes an affinity matrix between views, allowing the model to weight features based on their cross-view correlation. This acts as a "heat map" of importance, enhancing interpretability and filtering out noise.
- Adversarial Subspace Learning: To ensure a truly unified shared representation, ANTS employs a GAN-style discriminator. While encoders strive to create a shared latent space, the discriminator attempts to identify the source view of a given representation. This adversarial game forces the encoders to produce view-invariant features.
- Cross-Reconstruction and Residuals: A unique "cross-reconstruction" loss is used, where the model attempts to reconstruct View A from View B's representation. The difference between the original input and this reconstruction—the residual—is captured as view-specific information.
- Label Guidance: Unlike purely unsupervised subspace methods, ANTS integrates the classification task directly into the training process, ensuring that the learned features are optimized for the end goal.
Experimental Evaluation
The performance of ANTS was evaluated against several baselines, including Linear CCA, Deep CCA (DCCA), and Deep Information Bottleneck (Deep IB), across datasets such as Noisy MNIST, XRMB, CelebA, and CUB-200-2011 Birds.
As shown in the results, ANTS consistently achieves the highest scores across all metrics. On the Noisy MNIST dataset, it reached an accuracy of 98.05%, surpassing the next best baseline by nearly 2%. Its robustness is particularly evident in the high-dimensional CUB Birds dataset, where it outperformed the second-best algorithm by over 8%. Qualitative analysis of the co-attention maps confirmed that the model focuses on relevant physical features, such as beaks and wings, rather than background noise. On the WebKB dataset, ANTS achieved a leading accuracy of 0.708 and an F1 score of 0.686.
Conclusion
ANTS provides a powerful solution for multi-view learning by effectively balancing consensus and complementarity. Its use of adversarial regularization and co-attention not only improves performance but also offers much-needed interpretability. While the current model shows great promise, future work will focus on improving the scalability of the co-attention mechanism for high-view counts (via "Centroid-ANTS") and reducing the sensitivity to hyperparameter tuning.
Summary updated