OpenViewer: Openness-Aware Multi-View Learning

Shide Du, Zihan Fang, Yanchao Tan, Changwei Wang, Shiping Wang, Wenzhong Guo

Introduction

Multi-view learning (MVL) leverages correlations across diverse data sources—such as combining image features with text descriptions—to enhance predictive performance. However, traditional MVL models often operate under a "closed-set" assumption, failing when encountering "unknown" classes in real-world deployments. This phenomenon, termed "agnostophobia," leads models to misclassify novel samples into known categories with high confidence. Furthermore, the "black-box" nature of many deep MVL models limits their utility in safety-critical applications where interpretability is paramount.

Conceptual overview of openness challenges in multi-view learning, highlighting the lack of interpretability and insufficient generalization.

Proposed Methodology

OpenViewer addresses these challenges through a three-pronged openness-aware strategy designed to bridge the gap between controlled research environments and the "open" real world.

1. Pseudo-Unknown Sample Generation

To mitigate agnostophobia, the framework simulates unknown samples during training. Inspired by the "Mixup" technique, the authors generate pseudo-unknown samples by interpolating features from different known categories. This allows the model to "practice" encountering data that does not conform to its predefined buckets, improving its ability to reject truly unknown samples during testing.

2. Expression-Enhanced Deep Unfolding Network (EDUN)

The core of OpenViewer's interpretability lies in its deep unfolding architecture. Rather than using standard black-box layers, the network is derived from the iterations of the Alternating Direction Method of Multipliers (ADMM) optimization algorithm. This process "unfolds" the mathematical optimization into four distinct, functional modules:

  • RF-Module: Eliminates redundant features within individual views.
  • CD-Module: Ensures cross-view consistency via dictionary learning.
  • DN-Module: Filters out noise and outliers.
  • CW-Fusion: Dynamically weights views based on their class-separability (complementarity).

3. Perception-Augmented Training Regime

The training loss is designed to optimize the feature space for open-set recognition. It employs a distance margin and center loss to compress known class features into tight, distinct clusters while explicitly penalizing high-confidence scores for pseudo-unknown samples. This effectively manages entropy, ensuring "sharp" predictions for knowns and high uncertainty for unknowns.

The OpenViewer framework architecture, illustrating the flow from pseudo-unknown generation through the EDUN modules to the perception-augmented training regime.

Experimental Evaluation

The framework was evaluated across six diverse datasets, including Animals, AWA, NUSWIDEOBJ, and multi-modal datasets like ESP-Game. Using the Open-Set Classification Rate (OSCR) as the primary metric—which measures the trade-off between identifying knowns and rejecting unknowns—OpenViewer consistently outperformed state-of-the-art competitors such as TMC, RCML, and ORLNet.

Ablation studies confirmed the necessity of the interpretable modules; removing the noise-processing (DN) or dictionary-learning (CD) components resulted in significant performance degradation. Visualizations using t-SNE further demonstrated that OpenViewer creates a clear "no-man's land" for unknown samples, facilitating reliable rejection compared to traditional methods.

Conclusion

OpenViewer represents a significant step toward making multi-view learning robust for real-world applications. By grounding network architecture in mathematical optimization and explicitly accounting for unknown classes, it provides both transparency and superior open-set generalization. Future research will explore handling heterogeneous data structures and incomplete data where certain views may be missing.

Summary updated

Abstract

Multi-view learning methods leverage multiple data sources to enhance perception by mining correlations across views, typically relying on predefined categories. However, deploying these models in real-world scenarios presents two primary openness challenges. 1) Lack of Interpretability: The integration mechanisms of multi-view data in existing black-box models remain poorly explained; 2) Insufficient Generalization: Most models are not adapted to multi-view scenarios involving unknown categories. To address these challenges, we propose OpenViewer, an openness-aware multi-view learning framework with theoretical support. This framework begins with a Pseudo-Unknown Sample Generation Mechanism to efficiently simulate open multi-view environments and previously adapt to potential unknown samples. Subsequently, we introduce an Expression-Enhanced Deep Unfolding Network to intuitively promote interpretability by systematically constructing functional prior-mapping modules and effectively providing a more transparent integration mechanism for multi-view data. Additionally, we establish a Perception-Augmented Open-Set Training Regime to significantly enhance generalization by precisely boosting confidences for known categories and carefully suppressing inappropriate confidences for unknown ones. Experimental results demonstrate that OpenViewer effectively addresses openness challenges while ensuring recognition performance for both known and unknown samples. The code is released at https://github.com/dushide/OpenViewer.
BibTeX
@inproceedings{du2025openviewer,
  author = {Shide Du and Zihan Fang and Yanchao Tan and Changwei Wang and Shiping Wang and Wenzhong Guo},
  title = {{OpenViewer: Openness-Aware Multi-View Learning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  url = {https://openreview.net/forum?id=Ast4VZYvia},
}
Figure