Trusted Unified Feature-Neighborhood Dynamics for Multi-View Classification

Haojian Huang, Chuanyu Qin, Zhe Liu, Kaijing Ma, Jin Chen, Han Fang, Chao Ban, Hao Sun, Zhongjiang He

TL;DR

The TUNED (Trusted Unified Feature-NEighborhood Dynamics) model improves multi-view classification by moving beyond rigid evidence fusion rules. By integrating local neighborhood structures and using a Selective Markov Random Field (S-MRF), it adaptively filters out conflicting or noisy views, ensuring robust decision-making even when data sources disagree.

What Problem Does This Paper Address?

Multi-view classification (MVC) involves integrating data from diverse sources—like different camera angles, sensors, or feature sets—to make a single prediction. While having more data is generally better, it introduces two major headaches: domain gaps and view inconsistencies.

Existing methods often use Evidential Deep Learning (EDL) to quantify uncertainty. However, they typically rely on the Dempster-Shafer Theory (DST) for fusion. DST has a critical flaw: it is notoriously sensitive to conflicting evidence. If one view is highly confident but wrong (e.g., due to a sensor glitch or adversarial attack), it can catastrophically derail the entire model. Furthermore, most models treat data points as isolated instances, ignoring the rich "neighborhood" relationships (how similar samples relate to each other) that provide vital context for resolving conflicts.

Comparison between conventional EDL models and the proposed TUNED approach.

Methods

The authors propose the TUNED framework, which operates in two primary stages to create a more resilient fusion process:

  1. Local Feature-Neighborhood (F-N) Extraction: Instead of just extracting raw features, TUNED uses view-specific DNNs for features and GNNs (Graph Convolutional Networks) to capture the local neighborhood structure within each view. This ensures the model understands not just what a sample is, but how it relates to its "neighbors" in that specific domain.
  2. Global Consensus & Selective Fusion:
    • A shared parameterized evidence extractor learns a global consensus across all views.
  3. The core innovation is the Selective Markov Random Field (S-MRF). Unlike the fixed rules of DST, S-MRF treats views as nodes in a graph. It dynamically calculates weights between views based on their coherence. If a view is identified as "conflicting" or "unreliable," the S-MRF effectively suppresses its influence by reducing its connection weights to zero.
Figure 2: Illustration of the TUNED workflow, which comprises two stages. In the first stage, local view-specific feature- neighborhood (F-N) structures are extracted and fused to obtain evidence from the multi-view dataset. In the second stage, this evidence is integrated with global consensus F-N features through joint learning. The final step involves local-global evidence evaluation via EDL, followed by evidence fusion using the S-MRF module, leading to reliable inference outcomes.

Datasets

The model was benchmarked on eight diverse multi-view datasets:

  • Image/Object: PIE, Caltech101, Animal, ALOI, CUB, NUS-WIDE-OBJECT.
  • Handwritten Digits: HandWritten (6 feature sets).
  • Scene Recognition: Scene15.

To test robustness, the authors created conflictive versions of these datasets by intentionally introducing noise or misalignments into specific views, following established methodologies in the field.

Key Insights and Ideas

  • Neighborhoods as a Buffer: By incorporating neighborhood structures, the model gains a "second opinion" from similar samples. This makes it much harder for a single noisy view to flip a prediction.
  • Order Independence: Traditional DST implementations can be sensitive to the order in which views are fused. TUNED’s graph-based S-MRF approach is inherently order-independent, leading to more stable training and inference.
  • Dynamic Weighting: The S-MRF acts as an automated "trust filter." As shown in the paper's visualizations, when a conflict is introduced in a specific view, the model learns to drop the weights associated with that view almost entirely.
Figure 3: S-MRF weights for the HandWritten dataset. (a) Represents the weights under the normal setting, while (b) illustrates the weights under the conflict setting. And the red node indicates the view where conflict have been introduced.

Key Results

TUNED consistently outperformed state-of-the-art (SOTA) baselines like TMC, ECML, and RMVC, particularly in high-conflict scenarios:

  • Accuracy Gains: On "normal" test sets, TUNED achieved the best results on 6 out of 8 datasets.
  • Conflict Robustness: The performance gap widened significantly on conflictive datasets. For example, on the Animal dataset under conflict, TUNED improved accuracy by a staggering 37.73% over the best baseline. On Scene15 and ALOI, improvements were 18% and 15.4%, respectively.
  • Stability: As shown in Figure 4, TUNED exhibits much lower variance and higher median accuracy compared to competitors as the number of corrupted views increases.
Figure 4: Robustness comparison: Impact of conflict views on model performance across different configurations.

Limitations and Future Work

The authors identify one primary limitation: incomplete multi-view data. The current TUNED model assumes that all views are present for every sample, even if some are noisy. In real-world scenarios, views are often entirely missing (e.g., a sensor is offline). Future work will focus on extending the F-N structure logic to handle these "missing view" cases without degrading the neighborhood dynamics.

Summary updated

Abstract

Multi-view classification (MVC) faces inherent challenges due to domain gaps and inconsistencies across different views, often resulting in uncertainties during the fusion process. While Evidential Deep Learning (EDL) has been effective in addressing view uncertainty, existing methods predominantly rely on the Dempster-Shafer combination rule, which is sensitive to conflicting evidence and often neglects the critical role of neighborhood structures within multi-view data. To address these limitations, we propose a Trusted Unified Feature-NEighborhood Dynamics (TUNED) model for robust MVC. This method effectively integrates local and global feature-neighborhood (F-N) structures for robust decision-making. Specifically, we begin by extracting local F-N structures within each view. To further mitigate potential uncertainties and conflicts in multi-view fusion, we employ a selective Markov random field that adaptively manages cross-view neighborhood dependencies. Additionally, we employ a shared parameterized evidence extractor that learns global consensus conditioned on local F-N structures, thereby enhancing the global integration of multi-view features. Experiments on benchmark datasets show that our method improves accuracy and robustness over existing approaches, particularly in scenarios with high uncertainty and conflicting views. The code will be made available at https://github.com/JethroJames/TUNED.
BibTeX
@inproceedings{huang2025trusted,
  author = {Haojian Huang and Chuanyu Qin and Zhe Liu and Kaijing Ma and Jin Chen and Han Fang and Chao Ban and Hao Sun and Zhongjiang He},
  title = {{Trusted Unified Feature-Neighborhood Dynamics for Multi-View Classification}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i16.33914},
}
Figure