Lightweight Multi-Model Based Document Image Captioning

Qinqin Ma, Xizai Yang, Yongbo Huo, Bing Pang, Xinrui Zhan, Xiaosheng Cheng, Yu Zhu, Xianglong Yang2025 · DOI

Abstract

Against the backdrop of rapid advancements in multimodal intelligent systems, image captioning—serving as a vital bridge between visual understanding and natural language generation—has attracted considerable attention. However, current mainstream multimodal pre-trained models still face significant challenges in the context of Chinese image captioning, including limited generalization capabilities, unnatural language output, and inadequate comprehension of complex scenes. When processing Chinese natural scene images, these models often struggle to achieve a balance between semantic precision and linguistic fluency. To tackle these issues, this study proposes a joint optimization framework that incorporates Chinese-language data with a lightweight tuning strategy to improve model performance. Specifically, a large-scale Chinese image-text aligned dataset is constructed and paired with a parameter-efficient finetuning approach to facilitate effective adaptation to the Chinese linguistic environment. Experimental results demonstrate that the proposed method consistently outperforms existing approaches across multiple metrics, delivering superior language generation quality and adaptability to diverse scene content. These results validate the effectiveness of our method in enhancing Chinese natural scene image captioning.
BibTeX
@misc{ma2025lightweight,
  author = {Qinqin Ma and Xizai Yang and Yongbo Huo and Bing Pang and Xinrui Zhan and Xiaosheng Cheng and Yu Zhu and Xianglong Yang},
  title = {{Lightweight Multi-Model Based Document Image Captioning}},
  year = {2025},
  doi = {10.1109/icnc-fskd67701.2025.11198127},
}

Loading related content...

Figure