Multi-Level Confidence Learning for Trustworthy Multimodal Classification

Xiao Zheng, Chang Tang, Zhiguo Wan, Chengyu Hu, Wei ZhangAAAI 2023 · DOI

Abstract

Multimodal learning aims to exploit complementary information from diverse sources, yet it often struggles with feature redundancy and "blind fusion" where the informativeness of specific modalities is ignored. This paper introduces the Multi-Level Confidence Learning Network (MLCLNet), a framework designed for trustworthy multimodal classification. MLCLNet incorporates confidence learning at both the feature and label levels, utilizing sample-specific weighting and mathematical orthogonality to enhance model robustness. Evaluated on multi-omics medical datasets, MLCLNet demonstrates superior performance over existing state-of-the-art methods.

Introduction

In safety-critical domains like computer-aided diagnosis, multimodal data (e.g., DNA, mRNA, and miRNA) provides a holistic view of patient health. However, two primary challenges persist: high-dimensional redundancy and the lack of joint perception regarding modality informativeness. Most existing fusion techniques focus on the mechanics of integration rather than the reliability of the underlying features. MLCLNet addresses these issues by explicitly modeling confidence across multiple stages of the learning pipeline, ensuring that the final decision is based on the most informative and certain representations.

Methods

The MLCLNet architecture comprises three integrated modules designed to filter noise and capture high-level correlations.

The MLCLNet architecture, illustrating the flow from raw multi-omics modalities through feature and label confidence learning modules.

Feature Confidence Learning Module (FCLM)

FCLM addresses input-level redundancy by learning a sample-specific weight vector for each modality. By applying $l_1$-norm regularization, the module encourages sparsity, effectively "switching off" noisy feature dimensions that do not contribute to the classification task.

Cross-modal Label Fusion Module (CLFM)

To avoid the complexities of raw feature fusion, CLFM utilizes Graph Convolutional Networks (GCNs) to learn structure-preserved representations. These representations are transformed into initial labels and fused via a "cross-modal label tensor," capturing the intricate correlations between how different modalities perceive class boundaries.

Label Confidence Learning Module (LCLM)

LCLM introduces a "label guided objective loss" based on the principle of mathematical orthogonality. In an ideal scenario, a classification vector should be perfectly certain (orthogonal to other class vectors). LCLM minimizes inter-class ambiguity by penalizing high inner products between different class vectors, leading to more separable and confident predictions.

Results

MLCLNet was evaluated on four benchmark datasets from The Cancer Genome Atlas (TCGA): BRCA, LGG, ROSMAP, and KIPAN. These datasets involve complex tasks such as cancer subtype classification and Alzheimer’s Disease diagnosis.

Performance comparison of MLCLNet against state-of-the-art methods on the LGG and ROSMAP datasets.

As shown in the performance comparison, MLCLNet consistently outperforms 12 state-of-the-art competitors, including MOGONET and TMC. On the LGG dataset, it achieved an accuracy of 83.5%, while on ROSMAP, it reached 84.4%. Notably, on the KIPAN dataset, the model achieved a near-perfect accuracy of 99.9%. Ablation studies further confirmed that the Cross-modal Label Fusion (CLFM) is the most critical component, highlighting the importance of high-level label correlation over low-level feature integration.

Conclusion

MLCLNet provides a robust framework for trustworthy multimodal classification by integrating multi-level confidence learning. While the current implementation excels in multi-omics medical data, future work will address the scalability of the label tensor for tasks with higher class counts and explore the biological interpretability of the learned feature weights.

Summary updated

BibTeX
@inproceedings{zheng2023multi,
  author = {Xiao Zheng and Chang Tang and Zhiguo Wan and Chengyu Hu and Wei Zhang},
  title = {{Multi-Level Confidence Learning for Trustworthy Multimodal Classification}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i9.26346},
}
Figure