OpenViewer: Openness-Aware Multi-View Learning

Shide Du, Zihan Fang, Yanchao Tan, Changwei Wang, Shiping Wang, Wenzhong Guo

Multi-view learning (MVL) has become a cornerstone of modern AI, enabling models to achieve high accuracy by integrating data from diverse sources such as images, text, and sensors. However, traditional MVL models typically operate under a "closed-set" assumption, where all testing data is expected to belong to categories seen during training. In real-world deployments, models inevitably encounter "unknown unknowns," leading to two critical failures: a lack of interpretability in how data is integrated and insufficient generalization, often resulting in overconfident misclassifications of unseen categories.

The Openness Challenge in Multi-view Learning. Traditional models struggle with interpretability (Challenge I) and generalization to unknown categories (Challenge II) when moving from closed to open environments.

The OpenViewer Framework

To bridge the gap between laboratory performance and real-world reliability, the OpenViewer framework introduces an openness-aware approach to multi-view learning. It is designed to be both robust to unknown categories and transparent in its internal processing. The framework is built upon three innovative pillars:

1. Pseudo-Unknown Sample Generation

To prepare for the unknown, OpenViewer simulates an open environment during training. Using a mechanism inspired by "Mixup" techniques, the model blends features from different known classes to create "pseudo-unknown" samples. This allows the model to learn a representation for non-categories, preventing it from assigning high confidence scores to objects it has never seen before.

2. Expression-Enhanced Deep Unfolding Network

Unlike standard "black-box" deep learning architectures, OpenViewer utilizes a Deep Unfolding Network. The architecture is derived from a formal mathematical optimization problem (using the Alternating Direction Method of Multipliers, or ADMM). By "unfolding" the iterations of this algorithm into network layers, each module gains a specific, interpretable function:

  • RF-Module: Eliminates redundant features within views.
  • CD-Module: Learns a "consistency dictionary" to map correlations between different views.
  • DN-Module: Specifically targets and processes noise and outliers.
  • CW-Fusion: Dynamically merges views based on their unique contribution to class discriminability.
The OpenViewer Architecture. The framework integrates pseudo-unknown generation, functional deep unfolding modules (CD, RF, DN), and a perception-augmented training regime to handle open-set data.

3. Perception-Augmented Open-Set Training

The framework employs a specialized training regime and loss function. It is designed to maximize the distance between known categories in the feature space while simultaneously suppressing the confidence scores of pseudo-unknown samples. This dual-action approach ensures the model remains cautious when encountering unfamiliar data.

Experimental Results and Insights

OpenViewer was evaluated across six diverse datasets, including animal attributes (AWA), web objects (NUSWIDE), and face recognition (VGGFace2). The primary metric, the Open-Set Classification Rate (OSCR), demonstrated that OpenViewer consistently outperforms state-of-the-art methods.

  • Superior Generalization: On the ESP-Game dataset, OpenViewer achieved a Correct Classification Rate (CCR) of 83.52% at a 50% False Positive Rate, significantly higher than competing models like TMC and RCML.
  • Enhanced Interpretability: Heatmap and t-SNE visualizations confirm that the model effectively filters noise and creates tight, distinct clusters for known classes while maintaining a separate, low-confidence space for unknown samples.
  • Dynamic Weighting: The framework's ability to dynamically weight views ensures that the most informative data sources (e.g., text descriptions vs. noisy images) are prioritized during the fusion process.

Conclusion

OpenViewer represents a significant advancement in making multi-view learning ready for real-world deployment. By replacing opaque black-box structures with mathematically grounded unfolding networks and proactively training for the unknown, it provides a reliable and interpretable path forward for AI in open-set environments.

Summary updated

Abstract

Multi-view learning methods leverage multiple data sources to enhance perception by mining correlations across views, typically relying on predefined categories. However, deploying these models in real-world scenarios presents two primary openness challenges. 1) Lack of Interpretability: The integration mechanisms of multi-view data in existing black-box models remain poorly explained; 2) Insufficient Generalization: Most models are not adapted to multi-view scenarios involving unknown categories. To address these challenges, we propose OpenViewer, an openness-aware multi-view learning framework with theoretical support. This framework begins with a Pseudo-Unknown Sample Generation Mechanism to efficiently simulate open multi-view environments and previously adapt to potential unknown samples. Subsequently, we introduce an Expression-Enhanced Deep Unfolding Network to intuitively promote interpretability by systematically constructing functional prior-mapping modules and effectively providing a more transparent integration mechanism for multi-view data. Additionally, we establish a Perception-Augmented Open-Set Training Regime to significantly enhance generalization by precisely boosting confidences for known categories and carefully suppressing inappropriate confidences for unknown ones. Experimental results demonstrate that OpenViewer effectively addresses openness challenges while ensuring recognition performance for both known and unknown samples. The code is released at https://github.com/dushide/OpenViewer.
BibTeX
@inproceedings{du2025openviewer,
  author = {Shide Du and Zihan Fang and Yanchao Tan and Changwei Wang and Shiping Wang and Wenzhong Guo},
  title = {{OpenViewer: Openness-Aware Multi-View Learning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  url = {https://openreview.net/forum?id=Ast4VZYvia},
}
Figure