TL;DR
This paper introduces Robust Fuzzy Multi-View Learning (R-FUML), a novel framework designed to handle "view conflicts" (e.g., when an image and its text description contradict each other due to sensor faults or noise) in multi-view classification. By leveraging Fuzzy Set Theory and a cyclical training strategy, R-FUML accurately quantifies prediction uncertainty and prevents models from overfitting to conflicting data, significantly outperforming existing trusted learning baselines.
What Problem Does This Paper Address?
Multi-view learning combines information from diverse sources—such as RGB images, depth maps, and text descriptions—to make highly accurate predictions. In safety-critical applications like autonomous driving or medical diagnosis, these systems must be "trusted," meaning they need to output not just a classification label, but also a reliable measure of uncertainty.
Existing Trusted Multi-View Classification (TMVC) methods typically assume that all views are perfectly aligned and semantically consistent. However, in the real world, sensors fail, data gets corrupted, and transmission errors occur, leading to View Conflict (VC). For example, an RGB camera and a text description might clearly indicate a "Bathroom," but a malfunctioning depth sensor might provide data that looks like a "Bedroom".
When faced with such conflicts, standard Evidential Deep Learning (EDL) methods fall short: they underestimate uncertainty, get misled by dominant conflicting views, and overfit to the conflicting data during training, causing performance to plummet.
Methods
To tackle view conflicts during both training and inference, the authors discard traditional evidential fusion strategies in favor of a framework grounded in Fuzzy Set Theory. R-FUML consists of two primary components:
1. Robust Multi-view Fusion (RMF)
Instead of outputting standard softmax probabilities or evidential belief masses, R-FUML models network outputs as fuzzy memberships (possibility measures). To capture how certain a model is that a sample does not belong to other categories, they introduce a metric called category credibility (combining both possibility and necessity measures). During inference, RMF dynamically calculates fusion weights to downweight views that display high uncertainty or high conflict relative to the other views, ensuring clean, aligned views dominate the final decision.
2. Robust Learning Against VC (RLVC)
To prevent the network from overfitting to conflicting data during training, the authors design a four-stage robust learning pipeline that exploits the natural "memorization effect" of deep neural networks (where networks learn clean patterns before memorizing noise):
- Stage 1: Pre-training: The network is trained on all data until it begins to overfit.
- Stage 2: Cyclical Training: The learning rate is cyclically alternated to shift the model repeatedly between underfitting and overfitting, and per-sample loss trajectories are recorded.
- Stage 3: Conflictive View Division: A Gaussian Mixture Model (GMM) is fitted to the loss statistics to cleanly separate clean views from conflicting ones.
- Stage 4: Robust Learning: The network parameters are re-initialized, and the model is retrained. Instead of discarding conflicting views entirely, the model applies an adaptive penalty based on their conflict probability.
Datasets
The framework was comprehensively evaluated on eight public datasets:
- Handwritten (HW)
- MSRC-V1 (MSRC)
- NUS-WIDE-OBJ (NUSOBJ)
- Fashion-MV (Fashion)
- Scene15 (Scene)
- LandUse
- Leaves100 (Leaves)
- PIE
To rigorously test robustness, the authors artificially introduced view conflicts by altering information in $V-2$ random views (where $V$ is the total number of views) at conflict rates ranging from 0% to 60%. They also added Gaussian noise to 10% of the test instances.
Key Insights and Ideas
- Fuzzy Entropy beats Evidential Mass: Traditional evidential deep learning scales uncertainty based strictly on total evidence and category counts, ignoring how that evidence is distributed. Measuring the normalized Shannon entropy of fuzzy category credibility provides a much more realistic, conflict-aware uncertainty estimation.
- Harnessing Overfitting Cyclically: While deep networks eventually memorize conflicting data (causing their training loss to drop and blend with clean samples), forcing the network through cyclical training loops keeps the loss profiles of clean and conflicting samples distinct enough to be separated by a GMM.
- Soft Penalization over Hard Discarding: Completely discarding conflicting samples can throw away useful underlying features or borderline supervisory signals. RLVC's approach of softly penalizing views based on posterior probabilities preserves data utility while neutralizing harmful contradictions.
Key Results
Classification Accuracy
As the view conflict rate scales up from 0% to 60%, R-FUML degrades much more gracefully than its competitors. At the highest conflict rate (60%), R-FUML outperformed all 15 baseline methods across all eight datasets. For example, it achieved absolute accuracy improvements of 3.20% on HW, 2.10% on Scene, and 3.33% on LandUse compared to the next-best state-of-the-art methods.
Uncertainty Estimation
The authors used FPR95 (the false positive rate at a 95% true positive rate) to evaluate how effectively the models separate clean in-distribution samples from conflicting out-of-distribution samples. R-FUML achieved dramatically lower scores than alternative methods (e.g., 0.086 on the Fashion dataset compared to SAEML's 0.923 and FUML's 0.410), demonstrating highly reliable uncertainty quantification.
Limitations and Future Work
The authors note a minor trade-off: under conditions with low or zero view conflict (0% to 20% VC), R-FUML occasionally performs slightly worse than its non-robust baseline counterpart (FUML) on specific datasets like NUSOBJ and PIE. This occurs because the unsupervised GMM division stage can occasionally misclassify clean samples as conflicting when no actual conflicts are present in the data.
Future work will likely focus on refining the adaptive thresholding mechanisms to make the conflict division stage more precise under low-noise or perfectly aligned data conditions.
Summary updated