SARVIA: Fast-Adapting the Strong SAR Foundation Models That Are Already in Your MLLMs

Yuxuan Li, Zichen Tian, Qianru SunarXiv 2026 · arXiv

TL;DR

SARVIA challenges the status quo of training Synthetic Aperture Radar (SAR) foundation models from scratch by showing that visual encoders in modern Multimodal Large Language Models (MLLMs) already possess strong, latent SAR capabilities. By using a novel "computable visual question" adaptation recipe, SARVIA specializes these encoders in just 2 GPU-hours, outperforming dedicated SAR models that require thousands of hours of pretraining.

What Problem Does This Paper Address?

Synthetic Aperture Radar (SAR) is essential for all-weather Earth observation, but it is notoriously difficult to process. Unlike standard optical images, SAR data is formed by microwave scattering, resulting in grainy "speckle" noise and complex acquisition geometries that make objects look vastly different depending on the sensor's angle.

To handle this, the research community has focused on building "foundation models" through massive, domain-specific pretraining on unlabeled SAR data. However, this is incredibly expensive—often requiring thousands of GPU-hours. The authors ask a provocative question: Is this heavy lifting actually necessary? They discover that general-purpose visual encoders (like those in the Qwen or InternVL families) are already surprisingly good at SAR tasks, suggesting we should focus on specialization rather than re-invention.

Paradigm shift from heavy pretraining to efficient adaptation.

Methods

The authors introduce SARVIA (SAR Visual Adaptation), a framework that treats an MLLM as a flexible task interface for domain specialization.

  1. Self-Supervised QA Construction: Instead of manual labels, SARVIA generates "computable" questions from unlabeled SAR data. These fall into two categories:
  2. General Image Tasks: Spatial reasoning like patch ordering, crop matching, and identifying flipped images.
  3. SAR-Specific Tasks: Questions about native radar measurements (e.g., "Which quadrant has the highest scattering concentration?") and cross-modal correspondence between paired SAR and optical views.
  4. Joint LoRA Adaptation: The model trains low-rank adapters (LoRA) across the visual encoder, the multimodal merger, and the language model. This allows the language component to "teach" the visual encoder about SAR properties using a shared text-based objective.
  5. Loss-Driven Task Allocation: Not all tasks are equally useful. SARVIA uses a dynamic sampler that tracks learning progress. It prioritizes tasks where the model shows "room to improve" and reduces focus on tasks that are either already mastered or persistently stalled.
  6. Backbone Export: Once adapted, the visual adapters are merged, and the visual encoder is exported as a standalone backbone for downstream tasks like object detection and segmentation.
Figure 3. SARVIA overview. Unlabeled SAR observations yield tasks with deterministic answers. Answer loss jointly trains LoRA in the ViT, merger, and language model, while detached task losses adjust future QA draws through the loss-driven task allocation. We merge the visual adapters and export only the ViT for downstream perception.

Datasets

  • Pretraining/Adaptation: The authors used a mix of TerraMesh (Sentinel-1 observations), SARLO-80 (high-resolution SAR-optical pairs), and KuroSiwo (multitemporal SAR).
  • Downstream Benchmarks: Evaluation was conducted across six diverse benchmarks: RSAR and M4-SAR (oriented detection), SARDet-100K (horizontal detection), and OpenEarthMap-SAR, AIR-PolSAR-Seg-2.0, and WHU-OPT-SAR (segmentation).
  • Development Set: They introduced OEM-SLC-Amplitude, a compact dataset for rapid validation that pairs Umbra SLC amplitudes with OpenEarthMap labels.

Key Insights and Ideas

  • The "Free" Head Start: The most striking finding is that "off-the-shelf" MLLM encoders (like Qwen3.5-2B) already outperform established SAR foundation models like TerraMind and SARMAE before any domain-specific training.
  • Language as a Bridge: By framing domain adaptation as a VQA (Visual Question Answering) problem, the authors can use the MLLM's existing reasoning capabilities to refine its visual representations without needing a single human-annotated label.
  • Co-evolution: This paradigm allows SAR models to "co-evolve" with the broader AI field. As open-source MLLMs improve, the SAR community can simply plug in the latest encoder and run the SARVIA recipe to get a new state-of-the-art SAR backbone.
  • Efficiency is King: SARVIA achieves its results in 600 steps (~2 GPU-hours), which is roughly 1/2,700th of the training cost of TerraMind.

Key Results

SARVIA achieves the best scores across all six tested benchmarks, regardless of whether the task is detection or segmentation.

  • Detection: On SARDet-100K, SARVIA (ViT-L) reached 59.15 mAP, significantly higher than SARMAE (54.11) and TerraMind (50.62).
  • Segmentation: On OpenEarthMap-SAR, it achieved 36.90 mIoU, surpassing the previous best of 35.14.
  • Scaling: The method works across different MLLM families (InternVL, Qwen) and scales (from 0.8B to 4B parameters), showing consistent gains after adaptation.
SARVIA performance vs. baselines across six benchmarks.

Limitations and Future Work

While highly effective for SAR, the authors acknowledge that the framework's success depends on the availability of "computable" domain-specific tasks. Extending this to other specialized fields (like medical imaging or infrared) would require identifying similar native measurements or cross-modal correspondences that can be turned into self-supervised questions.

Additionally, the authors noted that while QA accuracy continues to improve with more training, downstream perception performance eventually plateaus or even slightly declines (as seen in Figure 5c), suggesting a delicate balance between "learning to answer questions" and "learning to see."

Figure 5. How adaptive sampling changes training. (a) Cumulative changes in task exposure, colored by task family. (b) Loss and sampling trajectories for two SARLO tasks. Fast NLL is the fast exponential moving average of answer-token negative log-likelihood. Consumed p/p0 is the ratio of the sampling probability used in training to its initial value. (c) Question-answering (QA) accuracy and OEM-SLC-Amplitude mIoU can diverge.

Summary updated

Abstract

Synthetic aperture radar (SAR) foundation models provide transferable representations for diverse perception tasks in all-weather Earth observation. However, their development often relies on costly large-scale SAR pretraining, raising the question of whether such extensive domain-specific pretraining is necessary. We find that modern multimodal large language models (MLLMs) already provide visual encoders that can outperform dedicated SAR foundation models as downstream initializations. This motivates a new paradigm: reuse general visual capabilities and devote domain training to brief specialization. We therefore introduce SARVIA, an adaptive post-pretraining method that specializes modern MLLMs using self-supervised visual questions with computable answers. These questions are constructed from unlabeled observations through general image tasks that capture spatial relations and SAR-specific tasks that exploit native measurements and SAR--optical correspondence. A shared answer-prediction objective jointly trains low-rank adapters across visual and language components, with loss-driven task sampling. We then export the adapted visual encoder as a standalone backbone. With 600 adaptation steps and 2 GPU-hours, SARVIA outperforms the compared foundation models on all six SAR detection and segmentation benchmarks. Compared with dedicated SAR pretraining, SARVIA reduces domain-training costs by orders of magnitude while improving visual transfer across architectures and scales. This new paradigm shifts the emphasis from repeated, costly pretraining to a reusable specialization process that can benefit from advances in open MLLMs. Dataset and code: https://github.com/zcablii/SARVIA.
BibTeX
@misc{li2026sarvia,
  author = {Yuxuan Li and Zichen Tian and Qianru Sun},
  title = {{SARVIA: Fast-Adapting the Strong SAR Foundation Models That Are Already in Your MLLMs}},
  year = {2026},
  eprint = {2611.22273},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.22273},
}
Figure