Dynamic SRM Curriculum for Trustworthy Multi-modal Classification

Jian Zhu, Cui Yu, Xin Zou, Zhangmin Huang, Chenshu Hu, Jun Sun, Bo Lyu, Lei Liu, Chang Tang, Li-Rong DaiICASSP 2025 · OpenReview

Abstract

Trustworthy multi-modal learning integrates multiple sources of data reliably. However, the current methods still focus on performance improvement by developing deep multi-modal networks. These approaches frequently encounter challenges due to the inherent non-convex nature of deep neural networks and their vulnerability to local minima, ultimately leading to a diminished ability for generalization. To address this problem, we present a novel curriculum termed the Dynamic SRM Curriculum (DSRMC). Within DSRMC, the deep trustworthy multi-modal networks undergo training with data provided sequentially, progressing from simple to complex samples. This training strategy mimics the human learning process, commencing with fundamental concepts and gradually advancing to tackle more complex and abstract ideas. Building upon DSRMC, we propose an innovative Curriculum Trustworthy Multi-modal Learning (CTML) method. CTML makes it easier to place the learned model in a flatter area, which improves its overall ability for generalization. Comprehensive experiments on three public datasets demonstrate that the proposed CTML performs better than state-of-the-art methods, achieving a maximum improvement of 6.7% on macroF1.
BibTeX
@inproceedings{zhu2025dynamic,
  author = {Jian Zhu and Cui Yu and Xin Zou and Zhangmin Huang and Chenshu Hu and Jun Sun and Bo Lyu and Lei Liu and Chang Tang and Li-Rong Dai},
  title = {{Dynamic SRM Curriculum for Trustworthy Multi-modal Classification}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2025},
  url = {https://openreview.net/forum?id=xzioT2Un9x},
}

Loading related content...

Figure