Leopard: A Vision Language Model For Text-Rich Multi-Image Tasks

Mengzhao Jia, Wenhao Yu, Kaixin Ma, Tianqing Fang, Zhihan Zhang, Siru Ouyang, Hongming Zhang, Meng Jiang, Dong YuOpenReview 2025 · Submission to ICLR 2025 · Withdrawn · OpenReview

Its authors withdrew this paper, so there is no full text to read or summarize.

Abstract

Text-rich images, where text serves as the central visual element guiding the overall understanding, are prevalent in real-world applications, such as presentation slides, scanned documents, and webpage snapshots. Tasks involving multiple text-rich images are especially challenging, as they require not only understanding the content of individual images but reasoning about inter-relationships and logical flows across multiple visual inputs. Despite the importance of these scenarios, current multimodal large language models (MLLMs) struggle to handle such tasks due to two key challenges: (1) the scarcity of high-quality instruction tuning datasets for text-rich multi-image scenarios, and (2) the difficulty in balancing image resolution with visual feature sequence length. Low-resolution encoding impairs the recognition of embedded text, while high-resolution encoding quickly exceeds the model’s maximum sequence length under multi-image settings. To address these challenges, we propose Leopard, a MLLM designed specifically for handling vision-language tasks involving multiple text-rich images. First, we curated about one million high-quality multimodal instruction-tuning data, tailored to text-rich, multi-image scenarios. Second, we developed an adaptive high-resolution multi-image encoding module to dynamically optimize the allocation of visual sequence length based on the original aspect ratios and resolutions of the input images. Experiments across a wide range of benchmarks demonstrate our model's superior capabilities in text-rich, multi-image evaluations and competitive performance in general domain evaluations. We are committed to open-source models and will release all collected data, code, and checkpoints to the community.
BibTeX
@misc{jia2025leopard,
  author = {Mengzhao Jia and Wenhao Yu and Kaixin Ma and Tianqing Fang and Zhihan Zhang and Siru Ouyang and Hongming Zhang and Meng Jiang and Dong Yu},
  title = {{Leopard: A Vision Language Model For Text-Rich Multi-Image Tasks}},
  year = {2025},
  url = {https://openreview.net/forum?id=oSJqRF0Tkg},
  howpublished = {OpenReview},
  note = {Submitted to ICLR 2025},
}

Loading related content...

Figure