Uncertainty-Guided View-Strength-Aware Feature Utilization for Multi-View Classification

Li Lv, Qian Guo, Li Zhang, Liang Du, Bingbing Jiang, Lu Chen, Xinyan LiangAAAI 2026 · OpenReview

TL;DR

This paper introduces the Uncertainty-Guided View-Strength-Aware Feature Utilization (UVF) framework for multi-view classification. UVF tackles the challenge of varying view quality by adaptively selecting features from "weak" views and dynamically weighting all views during fusion based on their estimated uncertainty, leading to improved accuracy and robustness.

What Problem Does This Paper Address?

In multi-view classification (MVC), data is represented by multiple sets of features (views), each offering a unique perspective. The core idea is that combining these views can lead to better classification performance than using any single view alone. However, a significant challenge arises because not all views are created equal. Some views might be noisy, redundant, or simply less informative than others due to factors like sensor quality, data corruption, or inherent data characteristics. Traditional MVC methods often treat all views uniformly, applying the same processing and fusion strategies regardless of their individual quality. This can lead to performance degradation as noisy or misleading information from weaker views can negatively impact the final decision. This paper aims to address this by developing a method that intelligently leverages the varying strengths and uncertainties of different views.

Methods

The proposed UVF framework introduces three key components:

  1. Uncertainty Estimation Module: This module quantifies the "discriminative strength" or reliability of each view. It does this by estimating the uncertainty associated with the predictions made by each individual view. Views that consistently produce confident, skewed probability distributions are considered "strong" (low uncertainty), while those with ambiguous or flat distributions are "weak" (high uncertainty).
  2. Differentiated Feature Selector (DFS): Based on the uncertainty estimates, the DFS adaptively processes each view. For strong views, it largely preserves the original features to retain their rich information. For weak views, it applies a gating mechanism (a learnable network) to selectively filter out less informative or noisy features, effectively "gating" them to suppress detrimental information. This is achieved using an L1-regularized MLP that outputs a gating vector, which is then element-wise multiplied with the feature vector. An MSE constraint is also added to ensure the gated features remain close to the original input.
  3. Uncertainty-Guided Fusion Module: This module integrates the processed features from all views. Instead of using fixed or uniform weights, it assigns dynamic weights to each view's contribution based on its estimated uncertainty. This ensures that more reliable views have a greater influence on the final classification decision, enhancing robustness. The fusion process combines cognitive uncertainty (epistemic uncertainty, approximated using MC Dropout) and prediction uncertainty.

The entire framework is trained end-to-end with a composite loss function that includes cross-entropy loss for classification, L1 regularization for feature sparsity in the gating mechanism, and an MSE loss for preserving feature similarity.

Datasets

The paper evaluates the UVF framework on six challenging multi-view classification datasets:

  • Animals with Attributes (AWA)
  • NUS-WIDE-128 (NUS)
  • Reuters
  • MVoxCeleb
  • YoutubeFace
  • AWA (subset)

These datasets cover diverse modalities like image, text, audio, depth, and video, and vary in the number of categories and view features.

Key Insights and Ideas

  • View Quality Matters: The fundamental insight is that assuming uniform view quality in MVC is a flawed premise that hinders performance. Recognizing and adapting to varying view strengths is crucial.
  • Uncertainty as a Signal: The paper effectively uses uncertainty estimation not just to identify weak views but also to guide the fusion process, allowing more confident views to contribute more.
  • Adaptive Feature Selection: The Differentiated Feature Selector (DFS) offers a novel way to "clean" weak views by adaptively filtering features, rather than simply discarding the entire view or processing it uniformly.
  • Synergy of Components: Ablation studies show that each component (DFS, Strength-Guided Gating, Uncertainty-Guided Fusion) contributes to the overall performance, with DFS providing the most significant improvement, and SG and UG offering complementary gains.

Key Results

The UVF framework consistently outperforms state-of-the-art multi-view classification methods across various datasets and metrics (accuracy, precision, recall, F1-score).

  • On the VoxCeleb and NUS datasets, UVF achieved accuracy improvements of approximately 0.33% and 0.42% over the second-best models.
  • UVF consistently achieved top or near-top performance on datasets like AWA, Reuters5, Reuters3, and YoutubeFace, with accuracies reaching 91.05%, 82.01%, 86.09%, and 86.38%, respectively.
  • The paper also highlights consistent gains in precision, recall, and F1-score, indicating improved robustness and better handling of minority classes and class imbalance.
  • Ablation studies showed that the DFS component alone significantly improved performance (e.g., 89.69% on AWA without DFS vs. 82.27% with DFS), and further gains were observed when combined with SG and UG.

Limitations and Future Work

  • Computational Cost: While not explicitly detailed as a limitation, the introduction of uncertainty estimation and adaptive feature selection likely adds computational overhead compared to simpler fusion methods.
  • Hyperparameter Tuning: The framework involves hyperparameters (e.g., for the L1 and MSE constraints, and fusion weights $\lambda_1, \lambda_2$) that would require careful tuning.
  • Future Work: The authors suggest that their work paves the way for further exploration into uncertainty-aware fusion strategies, implying that more sophisticated uncertainty modeling and fusion techniques could be developed. The paper also mentions the potential for interpretability through strength-aware processing, which could be a direction for future research.

Summary updated

Abstract

In multi-view classification tasks (MVC), each view provides an unique perspective on the data, offering complementary information that can improve classification performance when properly integrated. However, traditional methods typically adopt a uniform processing strategy for all views before fusion, overlooking the fact that different views may require different treatments due to variations in their quality and informativeness. To address this limitation, we propose a novel framework called Uncertainty-Guided View-Strength-Aware Feature Utilization (UVF) for multi-view classification. Our approach introduces a view uncertainty estimation module to quantify the discriminative strength of each view. Based on this estimation, a Differentiated Feature Selector (DFS) adaptively selects features, retaining informative dimensions in weak views while preserving original features in strong views. Furthermore, we employ an uncertainty-guided fusion strategy that assigns dynamic weights to each view's contribution based on its uncertainty score, enhancing the robustness and reliability of the final decision. Experimental results on benchmark datasets demonstrate that our method significantly outperforms conventional approaches, achieving better classification accuracy and interpretability through strength-aware feature processing and fusion.
BibTeX
@inproceedings{lv2026uncertainty,
  author = {Li Lv and Qian Guo and Li Zhang and Liang Du and Bingbing Jiang and Lu Chen and Xinyan Liang},
  title = {{Uncertainty-Guided View-Strength-Aware Feature Utilization for Multi-View Classification}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2026},
  url = {https://openreview.net/forum?id=j82fCnvgdI},
}
Figure