The Impact of Image Resolution on Biomedical Multimodal Large Language Models

Liangyu Chen, James Burgess, Jeffrey J. Nirschl, Orr Zohar, Serena Yeung-LevyarXiv 2025 · DOI · arXiv

Introduction

Biomedical images, ranging from high-resolution pathology slides to detailed microscopy, contain critical fine-grained details essential for accurate diagnosis. However, standard Multimodal Large Language Models (MLLMs) typically downsample images to small, uniform squares (e.g., 336x336 pixels) to manage computational costs. This research investigates the "resolution gap" in biomedical MLLMs, demonstrating that while native-resolution processing significantly boosts performance, a "resolution mismatch" between training and inference can lead to catastrophic accuracy drops.

Overview of the native-resolution MLLM framework across radiology, pathology, and microscopy modalities.

Methodology: Preserving Native Resolution

The study utilizes state-of-the-art MLLM architectures, such as Qwen2-VL, which support dynamic resolution. Unlike traditional models that force-resize inputs, these architectures employ a native-resolution image encoder. This encoder decomposes arbitrary-sized images into a variable number of patches (tokens), thereby preserving high-frequency diagnostic details that would otherwise be lost to blurring.

The researchers compared four primary training and inference configurations:

  1. Low-resolution only: Baseline training and testing on downsampled images.
  2. Native-resolution only: Training and testing on original, high-resolution images.
  3. Misaligned: Training on native resolution but testing on low resolution (and vice versa).
  4. Mixed-resolution: A proposed strategy training on a 50/50 mix of high and low-resolution samples to enhance robustness.

Experimental Setup and Datasets

The approach was evaluated across seven diverse biomedical datasets covering three major modalities:

  • Microscopy: Subcellular protein localization and chemical perturbations.
  • Pathology: Cervical cytology, Whole-Slide Imaging (WSI), and PathVQA.
  • Radiology: VQA-RAD and SLAKE clinical images.

Input sizes varied significantly, from 1024x1024 radiology scans to pathology slides reaching approximately 3000x4000 pixels.

Key Findings: The Alignment Trap and Mixed-Resolution Solutions

The study revealed that consistency between training and inference resolutions is critical. When a model trained on high-resolution images is tested on low-resolution inputs, performance "craters," sometimes dropping by as much as 48.7%. This "alignment trap" suggests that simply using high-resolution models is insufficient if the input data quality varies.

To address this, the authors propose Mixed-Resolution Training. By exposing the model to a variety of resolutions during the training phase, it becomes resolution-agnostic. This strategy achieved results nearly identical to pure native-resolution models (within ~1%) while maintaining functionality across diverse input types. Additionally, the choice of resampling algorithm proved vital; Lanczos resampling significantly outperformed standard Bilinear or Bicubic methods, recovering 95.7% of native-resolution performance when resizing was necessary.

Results and Performance Gains

Native-resolution training and inference provided accuracy improvements between 0.54% and 6.8% over low-resolution baselines. Even on pre-trained models where training details were unknown, native-resolution inference yielded a 4.0% average improvement on medical VQA benchmarks. This highlights the immediate utility of native-resolution processing for existing biomedical AI tools.

Conclusion

While native-resolution processing increases computational demands due to higher token counts, it is essential for preserving diagnostic signals in biomedical AI. Mixed-resolution training offers a practical path forward, balancing computational efficiency with the robustness required for real-world clinical applications. Future work will explore the application of these principles to 3D medical imaging, such as MRI and CT scans.

Summary updated

Abstract

Imaging technologies are fundamental to biomedical research and modern medicine, requiring analysis of high-resolution images across various modalities. While multimodal large language models (MLLMs) show promise for biomedical image analysis, most are designed for low-resolution images from general-purpose datasets, risking critical information loss. We investigate how image resolution affects MLLM performance in biomedical applications and demonstrate that: (1) native-resolution training and inference significantly improve performance across multiple tasks, (2) misalignment between training and inference resolutions severely degrades performance, and (3) mixed-resolution training effectively mitigates misalignment and balances computational constraints with performance requirements. Based on these findings, we recommend prioritizing native-resolution inference and mixed-resolution datasets to optimize biomedical MLLMs for transformative impact in scientific research and clinical applications.
BibTeX
@misc{chen2025impact,
  author = {Liangyu Chen and James Burgess and Jeffrey J. Nirschl and Orr Zohar and Serena Yeung-Levy},
  title = {{The Impact of Image Resolution on Biomedical Multimodal Large Language Models}},
  year = {2025},
  eprint = {2510.18304},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.18304},
}
Figure