Large multimodal models evaluation: a survey
Lists
Only you see this
Abstract
As large multimodal models (LMMs) advance rapidly across diverse multimodal understanding and generation tasks, the need for systematic and reliable evaluation frameworks becomes increasingly critical. To address this need, this survey provides a structured overview of LMM evaluation, centered around two main axes: multimodal evaluation for understanding and generation. (1) For understanding, a dual-perspective framework is introduced to distinguish benchmarks between general capabilities, which emphasize common tasks, and specialized capabilities, which reflect expert-level competence in domain-specific fields. (2) For generation, evaluation is organized by output modality, including image, video, audio, and 3D content. (3) From a community perspective, this survey further highlights authoritative leaderboards and foundational tools that have been instrumental in establishing a comprehensive evaluation ecosystem for LMMs. By unifying general-specialized understanding and modality-specific generation evaluations, this survey clarifies the current landscape and provides guidance for future research in the LMM evaluation field.
BibTeX
@article{zhang2025large,
author = {Zicheng Zhang and Junying Wang and Farong Wen and Yijin Guo and Xiangyu Zhao and Xinyu Fang and Shengyuan Ding and Ziheng Jia and Jiahao Xiao and Ye Shen and Yushuo Zheng and Xiaorong Zhu and Yalun Wu and Ziheng Jiao and Wei Sun and Zijian Chen and Kaiwei Zhang and Kang Fu and Yuqin Cao and Ming Hu and Yue Zhou and Xuemei Zhou and Juntai Cao and Wei Zhou and Jinyu Cao and Ronghui Li and Donghao Zhou and Yuan Tian and Xiangyang Zhu and Chunyi Li and Haoning Wu and Xiaohong Liu and Junjun He and Yu Zhou and Hui Liu and Lin Zhang and Zesheng Wang and Huiyu Duan and Yingjie Zhou and Xiongkuo Min and Qi Jia and Dongzhan Zhou and Wenlong Zhang and Jiezhang Cao and Xue Yang and Junzhi Yu and Songyang Zhang and Haodong Duan and Guangtao Zhai},
title = {{Large multimodal models evaluation: a survey}},
journal = {Science China Information Sciences},
year = {2025},
doi = {10.1007/s11432-025-4676-4},
}Loading related content...