Robust Fuzzy Multi-view Learning under View Conflict

Siyuan Duan, Yuan Sun, Dezhong Peng, Yingke Chen, Xi Peng, Peng HuarXiv 2026 · arXiv

TL;DR

This paper introduces Robust Fuzzy Multi-View Learning (R-FUML), a novel framework designed to handle "view conflicts" (e.g., when an image and its text description contradict each other due to sensor faults or noise) in multi-view classification. By leveraging Fuzzy Set Theory and a cyclical training strategy, R-FUML accurately quantifies prediction uncertainty and prevents models from overfitting to conflicting data, significantly outperforming existing trusted learning baselines.

What Problem Does This Paper Address?

Multi-view learning combines information from diverse sources—such as RGB images, depth maps, and text descriptions—to make highly accurate predictions. In safety-critical applications like autonomous driving or medical diagnosis, these systems must be "trusted," meaning they need to output not just a classification label, but also a reliable measure of uncertainty.

Existing Trusted Multi-View Classification (TMVC) methods typically assume that all views are perfectly aligned and semantically consistent. However, in the real world, sensors fail, data gets corrupted, and transmission errors occur, leading to View Conflict (VC). For example, an RGB camera and a text description might clearly indicate a "Bathroom," but a malfunctioning depth sensor might provide data that looks like a "Bedroom".

Illustration of view conflict (VC) and the motivation of R-FUML

When faced with such conflicts, standard Evidential Deep Learning (EDL) methods fall short: they underestimate uncertainty, get misled by dominant conflicting views, and overfit to the conflicting data during training, causing performance to plummet.

Methods

To tackle view conflicts during both training and inference, the authors discard traditional evidential fusion strategies in favor of a framework grounded in Fuzzy Set Theory. R-FUML consists of two primary components:

1. Robust Multi-view Fusion (RMF)

Instead of outputting standard softmax probabilities or evidential belief masses, R-FUML models network outputs as fuzzy memberships (possibility measures). To capture how certain a model is that a sample does not belong to other categories, they introduce a metric called category credibility (combining both possibility and necessity measures). During inference, RMF dynamically calculates fusion weights to downweight views that display high uncertainty or high conflict relative to the other views, ensuring clean, aligned views dominate the final decision.

2. Robust Learning Against VC (RLVC)

To prevent the network from overfitting to conflicting data during training, the authors design a four-stage robust learning pipeline that exploits the natural "memorization effect" of deep neural networks (where networks learn clean patterns before memorizing noise):

  • Stage 1: Pre-training: The network is trained on all data until it begins to overfit.
  • Stage 2: Cyclical Training: The learning rate is cyclically alternated to shift the model repeatedly between underfitting and overfitting, and per-sample loss trajectories are recorded.
  • Stage 3: Conflictive View Division: A Gaussian Mixture Model (GMM) is fitted to the loss statistics to cleanly separate clean views from conflicting ones.
  • Stage 4: Robust Learning: The network parameters are re-initialized, and the model is retrained. Instead of discarding conflicting views entirely, the model applies an adaptive penalty based on their conflict probability.

Datasets

The framework was comprehensively evaluated on eight public datasets:

  1. Handwritten (HW)
  2. MSRC-V1 (MSRC)
  3. NUS-WIDE-OBJ (NUSOBJ)
  4. Fashion-MV (Fashion)
  5. Scene15 (Scene)
  6. LandUse
  7. Leaves100 (Leaves)
  8. PIE

To rigorously test robustness, the authors artificially introduced view conflicts by altering information in $V-2$ random views (where $V$ is the total number of views) at conflict rates ranging from 0% to 60%. They also added Gaussian noise to 10% of the test instances.

Key Insights and Ideas

  • Fuzzy Entropy beats Evidential Mass: Traditional evidential deep learning scales uncertainty based strictly on total evidence and category counts, ignoring how that evidence is distributed. Measuring the normalized Shannon entropy of fuzzy category credibility provides a much more realistic, conflict-aware uncertainty estimation.
  • Harnessing Overfitting Cyclically: While deep networks eventually memorize conflicting data (causing their training loss to drop and blend with clean samples), forcing the network through cyclical training loops keeps the loss profiles of clean and conflicting samples distinct enough to be separated by a GMM.
  • Soft Penalization over Hard Discarding: Completely discarding conflicting samples can throw away useful underlying features or borderline supervisory signals. RLVC's approach of softly penalizing views based on posterior probabilities preserves data utility while neutralizing harmful contradictions.

Key Results

Classification Accuracy

As the view conflict rate scales up from 0% to 60%, R-FUML degrades much more gracefully than its competitors. At the highest conflict rate (60%), R-FUML outperformed all 15 baseline methods across all eight datasets. For example, it achieved absolute accuracy improvements of 3.20% on HW, 2.10% on Scene, and 3.33% on LandUse compared to the next-best state-of-the-art methods.

Uncertainty Estimation

The authors used FPR95 (the false positive rate at a 95% true positive rate) to evaluate how effectively the models separate clean in-distribution samples from conflicting out-of-distribution samples. R-FUML achieved dramatically lower scores than alternative methods (e.g., 0.086 on the Fashion dataset compared to SAEML's 0.923 and FUML's 0.410), demonstrating highly reliable uncertainty quantification.

Limitations and Future Work

The authors note a minor trade-off: under conditions with low or zero view conflict (0% to 20% VC), R-FUML occasionally performs slightly worse than its non-robust baseline counterpart (FUML) on specific datasets like NUSOBJ and PIE. This occurs because the unsupervised GMM division stage can occasionally misclassify clean samples as conflicting when no actual conflicts are present in the data.

Future work will likely focus on refining the adaptive thresholding mechanisms to make the conflict division stage more precise under low-noise or perfectly aligned data conditions.

Summary updated

Abstract

Trusted multi-view classification aims to deliver reliable fusion for accurate predictions and has recently attracted substantial attention in both academia and industry. However, existing TMVC methods typically assume strict alignment across different views during both training and testing phases, which is often impractical in real-world scenarios. This limitation motivates us to revisit TMVC and extend it to a more challenging setting: how to mitigate the impact of view conflict (VC) during both training and inference. To tackle this setting, existing TMVC methods suffer from three critical limitations: underestimated uncertainty, misleading decisions, and overfitting to VC. To address these issues, this paper proposes a novel Robust Fuzzy Multi-View Learning (R-FUML) framework grounded in Fuzzy Set Theory. Specifically, R-FUML models network outputs as fuzzy memberships to quantify category credibility and uses an entropy-based method for reliable multi-view fusion. To this end, we present a Robust Multi-view Fusion (RMF) strategy that accounts for both view-specific uncertainty and inter-view conflicts, thereby alleviating the adverse impacts of VC on decision-making. To identify and conquer VC during training, we further design a Robust Learning Against VC (RLVC) framework. RLVC isolates conflicting samples by leveraging neural networks' memory effects and then retrains the model by applying a penalty to these conflicting views. Extensive experiments across eight public datasets demonstrate that R-FUML consistently outperforms 15 state-of-the-art baselines in robustness and uncertainty estimation. The code will be released upon acceptance.
BibTeX
@misc{duan2026robust,
  author = {Siyuan Duan and Yuan Sun and Dezhong Peng and Yingke Chen and Xi Peng and Peng Hu},
  title = {{Robust Fuzzy Multi-view Learning under View Conflict}},
  year = {2026},
  eprint = {2605.24475},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.24475},
}

Loading related content...

Figure