XLSAR-Bench: Towards Extremely Large SAR Imagery Understanding of MLLMs

La Jiang, Lingfeng Chen, Jiaqi Zhu, Fengrong Guo, Jiehua Zhang, Panhe Hu, Li Liu, Zhen LiuarXiv 2026 · arXiv

TL;DR

XLSAR-Bench is a new, large-scale benchmark designed to test how well Multimodal Large Language Models (MLLMs) understand Synthetic Aperture Radar (SAR) images at scale. By using images 27 times larger than previous datasets, the authors reveal that while current models can recognize general scenes, they struggle significantly with fine-grained perception and spatial reasoning in complex, high-resolution environments.

What Problem Does This Paper Address?

Synthetic Aperture Radar (SAR) is a powerhouse for Earth observation because it works day or night and sees through clouds. However, SAR images are notoriously difficult to interpret due to "speckle" noise and complex geometries like layover and shadows.

The real challenge for AI, however, is scale. A typical SAR scene might cover an entire city at high resolution. Existing benchmarks usually chop these into tiny patches (e.g., 256x256 pixels). This creates a "Goldilocks" problem: if you downsample a massive image to fit a model's input, you lose tiny targets like ships or cars; if you crop it into patches, the model loses the global context needed to understand spatial relationships or complex scenes. The authors tackle this by introducing a benchmark that forces models to handle both global context and local detail simultaneously.

Methods

The researchers developed a "label-driven annotation pipeline" to create high-quality vision-language pairs for large-scale imagery.

  1. Image Preparation: They reconstructed or mosaicked images from 11 different SAR sources to ensure they had "extremely large" scenes (minimum 3000 pixels on the long side).
  2. Semi-Automatic Annotation: They used a semi-automatic captioner to generate initial drafts, which were then meticulously refined by human experts to ensure accuracy.
  3. Task Taxonomy: They organized the benchmark into ten tasks across two categories:
  4. Perception: Image captioning, land-use classification (overall and regional), counting, and fine-grained visual grounding.
  5. Reasoning: Complex counting (e.g., "count ships to the right of the bridge") and condition-based grounding.
  6. Baseline: They established XLSAR-Baseline by applying LoRA (Low-Rank Adaptation) to a Qwen3.5-9B model to see how much standard domain adaptation helps.

Datasets

The authors introduced XLSAR-Bench, which is built from 11 public SAR datasets, including:

  • Detection sources: LS-SSDD, HRSID, AIR-SARShip, and MSAR-Bridge.
  • Segmentation sources: WHU-OPT-SAR, AIR-PolSAR-Seg, SpaceNet6, FUSAR-Map, EarthMiss, YESeg-OPT-SAR, and AIR-MDSAR-MAP.

How it differs: XLSAR-Bench features 405 images with an average size of 7153 × 7153 pixels. For context, that is roughly 27 times larger (by long-side average) than existing SAR vision-language datasets. It includes over 14,000 VQA instances and 3,260 visual grounding instances.

Key Insights and Ideas

  • The "Global-Local" Gap: Models that are good at "Scene Classification" (global) aren't necessarily good at "Counting" or "Spatial Relationships" (local). Knowing you are looking at a port doesn't mean the model can actually see the individual ships within it.
  • The Grounding Bottleneck: Visual grounding (locating a specific object based on a text description) is the hardest task for current MLLMs. Even proprietary models like GPT-4o-level equivalents struggle when targets are small and embedded in massive scenes.
  • LoRA is a Partial Fix: Domain adaptation via LoRA significantly boosts VQA performance (improving scores by ~40 percentage points), but it does almost nothing for grounding. This suggests that the "language" part of the model learns the domain, but the "vision" part still can't "see" the small targets in large images.
  • Human vs. Machine: Human experts achieved 70-100% accuracy across tasks, but the best models still lag behind, particularly in counting and complex reasoning, where the gap can exceed 60 percentage points.

Key Results

  • Top Performers: Proprietary models like GLM-5.3-flash and GPT-6.1-sol led the pack, with GLM achieving the highest VQA average (67.45% in English).
  • Grounding Failure: Most open-weight models (like LLaVA or InternVL) achieved near-zero scores on fine-grained visual grounding (mIoU < 1%). Even the best proprietary model only reached an mIoU of ~11%.
  • Captioning Quality: While models can describe the "dominant" scene, they often hallucinate details or miss regional distributions (e.g., describing a generic urban area but missing a specific cluster of 100+ ships).

Limitations and Future Work

  • Input Resolution: Most current MLLMs still rely on downsampling or simple tiling, which is insufficient for "extremely large" imagery.
  • Small Target Perception: The "speckle" nature of SAR makes small targets look like noise to models trained primarily on optical images.
  • Future Directions: The authors suggest the need for new architectures that can preserve local visual details while maintaining global scene understanding—perhaps through more sophisticated "zoom" mechanisms or high-resolution visual encoders specifically tuned for the unique physics of SAR.

Summary updated

Abstract

The rapid development of multimodal large language models (MLLMs) has highlighted the need for comprehensive benchmarks to assess their capabilities and limitations in remote sensing (RS). However, evaluating MLLMs in large-size synthetic aperture radar (SAR) scenarios remains challenging, as understanding complete scenes requires both global semantic interpretation and fine-grained target perception. Existing SAR benchmarks predominantly rely on small image patches, limiting their ability to assess these capabilities jointly. To address these limitations, we present XLSAR-Bench, a comprehensive benchmark for evaluating the perception and reasoning capabilities of MLLMs in large-scene SAR imagery. XLSAR-Bench comprises 405 images with an average size of 7153$\times$7153 pixels, ten tasks spanning visual question answering, visual grounding, and detailed image captioning, with all evaluation samples meticulously annotated manually, assisted by a semi-automatic captioner. Experimental results from general-purpose and domain-specific MLLMs reveal persistent challenges in understanding large SAR scenes and accurately perceiving small targets. We have open-sourced XLSAR-Bench to support further research in developing more powerful MLLMs for complete large SAR scenes.
BibTeX
@misc{jiang2026xlsar,
  author = {La Jiang and Lingfeng Chen and Jiaqi Zhu and Fengrong Guo and Jiehua Zhang and Panhe Hu and Li Liu and Zhen Liu},
  title = {{XLSAR-Bench: Towards Extremely Large SAR Imagery Understanding of MLLMs}},
  year = {2026},
  eprint = {2611.19364},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.19364},
}
Figure