Enhancing Multi-view Open-set Learning via Ambiguity Uncertainty Calibration and View-wise Debiasing

Zihan Fang, Zhiyong Xu, Lan Du, Shide Du, Zhiling Cai, Shiping Wang

TL;DR

Multi-view learning models often fail in real-world, open-set scenarios because they overfit to view-specific biases and assume all classes are known. This paper introduces MOCD, a framework that combines a novel open-set data augmentation strategy calibrated via Dempster-Shafer theory with an independence-enforcing debiasing module. The result is a model that reliably rejects unknown classes with low confidence while maintaining top-tier accuracy on known categories.

What Problem Does This Paper Address?

Most multi-view learning (MVL) systems operate under a "closed-world" assumption: they assume that every object encountered during testing will belong to one of the classes seen during training. When deployed in the wild, these models face two major hurdles:

  1. Unknown Class Recognition: Real-world data contains unseen categories. Instead of flagging these as "unknown," standard models often misclassify them into known categories with dangerously high confidence.
  2. Static View-Induced Bias: Different views (e.g., text descriptions vs. images) can provide conflicting or spurious cues. If a model observes a narrow slice of the view space during training, it might establish a false association between a specific view feature (like a color histogram or texture) and a class label.

As shown in the paper's introductory analysis, directly fusing multi-view data without addressing these biases can actually amplify noise, leading to poor generalization when the model encounters unfamiliar inputs.

Figure 1: The impact of static view-induced bias on the perfor- mance of closed-set and open-set classification on different views of the BBCNews and NUSWIDE-OBJ datasets. For clar- ity, we denote each text view of BBCNews as TS-1–TS-3 (Text Segments), and each visual feature of NUSWIDE-OBJ as CH (Color Histogram), CM (Color Moments), CORR (Color Cor- relation), ED (Edge Distribution), and WT (Wavelet Texture).

Methods

The authors propose the MOCD (Multi-view Open-set learning via ambiguity uncertainty Calibration and view-wise Debiasing) framework, which consists of three main components:

  • Multi-view Semantic Alignment Network (MSAN): This network extracts features from each view independently using Multi-Layer Perceptrons (MLPs) and refines them using neighborhood aggregation to capture local structural relationships. The view-specific features are then averaged into a unified, view-consistent representation space.
  • O-Mix (Open-set Mixup): To simulate unknown and ambiguous samples during training, the authors design a novel data augmentation strategy. Unlike vanilla Mixup, which linearly interpolates labels based purely on a mixing coefficient, O-Mix uses Dempster-Shafer Theory (specifically, Generalized Basic Probability Assignment) to explicitly split the label uncertainty between known class ambiguity (e.g., is this a cat or a dog?) and open-set uncertainty (is this something entirely unknown?).
  • Ambiguity Perception Network (APN) & Contrastive Debiasing: An auxiliary network (APN) is trained specifically on the virtual samples generated by O-Mix to capture atypical patterns. Crucially, the framework applies a Hilbert-Schmidt Independence Criterion (HSIC) loss to enforce statistical independence between the main view-consistent representations (from MSAN) and the biased, ambiguous features (from APN). This prevents the core model from memorizing non-generalizable, view-specific shortcuts.
Figure 2: Three key components of the proposed framework: (1) MSAN extracts view-consistent representations from original inputs across all views; (2) O-Mix simulates ambiguous samples by mixing view-specific instances and generating ambiguity- calibrated soft labels via DS-theory; (3) APN processes synthesized samples to model non-prototypical patterns. Overall, the three modules are jointly optimized to balance semantic alignment, ambiguity modeling, and view-wise debiasing.

Datasets

The framework was evaluated across six diverse multi-view benchmark datasets:

  • BBCNews: 5 classes of text documents split into different text segments.
  • Caltech20: 20 classes of objects with multiple visual feature extractions (e.g., Gabor, GIST).
  • Hdigit: 10 classes of digit images.
  • Iaprtc12: 6 classes of image-text pairs.
  • NUSWIDE-OBJ: 31 classes of object images represented by various visual features (Color Histogram, Edge Distribution, etc.).
  • VGGFace2: 50 classes of face images.

Key Insights and Ideas

  • The Double-Edged Sword of Multi-View Fusion: While combining multiple views generally improves closed-set accuracy, it can severely degrade open-set performance if the model relies on biased, non-generalizable features from a single dominant view.
  • Calibrating "Unknownness": Instead of treating all mixed data points equally, applying Dempster-Shafer theory allows the model to mathematically allocate a portion of its belief space to the "focal element outside the frame of discernment"—effectively teaching the model what an "unknown" class looks like.
  • Debiasing through Independence: Rather than just minimizing classification error, forcing the core semantic features to be statistically independent of the view-specific ambiguous features acts as a powerful regularizer, preventing the network from taking easy, biased shortcuts during training.

Key Results

The authors evaluated performance using the Open-Set Classification Rate (OSCR), measuring the Correct Classification Rate (CCR) against the False Positive Rate (FPR).

MOCD consistently outperformed state-of-the-art multi-view methods (like CoGCN, MMDynamics, and MAMC) and uncertainty-based approaches (like TMC and TUNED). For instance, on the challenging VGGFace2 dataset at a strict 1.0% FPR (meaning the model rarely mistakes an unknown for a known), MOCD achieved a CCR of 27.40%, vastly outperforming the next best method, TUNED, which sat at 15.36%.

Furthermore, t-SNE visualizations demonstrated that MOCD forms much tighter, more compact clusters for known classes while cleanly segregating unknown samples into distinct spaces.

Figure 3: OSCR curves plotting the CCR over the FPR on all test multi-view datasets for all compared methods.

Limitations and Future Work

The authors note a delicate hyperparameter trade-off regarding the debiasing penalty ($\beta$). While enforcing independence helps eliminate static view-induced bias, excessive debiasing can inadvertently suppress highly discriminative, view-specific cues that are actually necessary for resolving difficult closed-set classifications. Future work could explore adaptive or dynamic weighting mechanisms to balance debiasing and feature discrimination automatically.

Summary updated

Abstract

Existing multi-view learning models struggle in open-set scenarios due to their implicit assumption of class completeness. Moreover, static view-induced biases, which arise from spurious view-label associations formed during training, further degrade their ability to recognize unknown categories. In this paper, we propose a multi-view open-set learning framework via ambiguity uncertainty calibration and view-wise debiasing. To simulate ambiguous samples, we design O-Mix, a novel synthesis strategy to generate virtual samples with calibrated open-set ambiguity uncertainty. These samples are further processed by an auxiliary ambiguity perception network that captures atypical patterns for improved open-set adaptation. Furthermore, we incorporate an HSIC-based contrastive debiasing module that enforces independence between view-specific ambiguous and view-consistent representations, encouraging the model to learn generalizable features. Extensive experiments on diverse multi-view benchmarks demonstrate that the proposed framework consistently enhances unknown-class recognition while preserving strong closed-set performance.
BibTeX
@misc{fang2025enhancing,
  author = {Zihan Fang and Zhiyong Xu and Lan Du and Shide Du and Zhiling Cai and Shiping Wang},
  title = {{Enhancing Multi-view Open-set Learning via Ambiguity Uncertainty Calibration and View-wise Debiasing}},
  year = {2025},
  eprint = {2508.01227},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2508.01227},
}
Figure