RIS-LAD: A Benchmark and Model for Referring Low-Altitude Drone Image Segmentation

Kai Ye, YingShi Luan, Zhudi Chen, Guangyue Meng, Pingyang Dai, Liujuan CaoarXiv 2025 · DOI · arXiv

Abstract

Referring Image Segmentation (RIS) in low-altitude drone (LAD) imagery presents unique challenges due to small object sizes, high density, and varying viewpoints. We introduce RIS-LAD, the first fine-grained benchmark specifically for this domain, and the Semantic-Aware Adaptive Reasoning Network (SAARN). SAARN addresses "category drift" and "object drift" through linguistic disentanglement and scale-aware reasoning, achieving state-of-the-art performance on the new benchmark.

Introduction: The Challenges of Low-Altitude Drone Imagery

Referring Image Segmentation (RIS) aims to segment objects based on natural language descriptions. While effective in standard photography and high-altitude satellite remote sensing, existing models falter in low-altitude drone (LAD) scenarios. Drones typically operate at oblique angles (30°–60°) and encounter diverse lighting conditions, including nighttime.

Overview of the RIS-LAD dataset challenges, comparing drone views with remote sensing.

The primary hurdles in LAD imagery are:

  1. Category Drift: Models confuse the target with visually similar classes (e.g., motorcycle vs. bicycle) due to the tiny scale of objects.
  2. Object Drift: Models correctly identify the class but fail to select the specific instance among a dense group of similar objects.

The RIS-LAD Dataset

To bridge this gap, we present the RIS-LAD dataset, comprising 13,871 image-text-mask triplets derived from real-world drone footage (30m–100m altitude). The dataset includes eight object categories and features nighttime scenes, which are largely absent in existing remote sensing benchmarks. Annotations were generated using a semi-automatic pipeline involving SAM-2 and Qwen2.5-VL, followed by rigorous human verification.

Semantic-Aware Adaptive Reasoning Network (SAARN)

SAARN introduces a "divide and conquer" strategy to handle the complexities of LAD imagery. Unlike standard models that inject text features holistically, SAARN employs:

  • Linguistic Disentanglement: Decomposing expressions into global context, category names, and descriptive attributes.
  • Category-Dominated Linguistic Enhancement (CDLE): By injecting category information early into the visual encoder, the model establishes a "semantic safeguard" against category drift.
  • Adaptive Reasoning Fusion Module (ARFM): This module uses a Scale Reasoning Gate (SRG) to dynamically weight linguistic cues across resolutions, ensuring that spatial descriptions and fine-grained attributes are applied where they are most effective.
Comparison of SAARN against RefSegRS, illustrating how SAARN mitigates category drift.

Results and Discussion

SAARN establishes a new state-of-the-art on the RIS-LAD benchmark, achieving an oIoU of 51.54%. Notably, it shows an 18.0% improvement in [email protected] over previous methods, indicating superior localization accuracy. Qualitative results demonstrate that SAARN effectively isolates single instances in crowded scenes where other models often over-segment or hallucinate masks.

Conclusion

RIS-LAD and SAARN provide a robust foundation for precision segmentation in drone-based applications. Future work will focus on reducing computational overhead for real-time deployment and improving performance in extremely cluttered or ambiguous scenes.

Summary updated

Abstract

Referring Image Segmentation (RIS), which aims to segment specific objects based on natural language descriptions, plays an essential role in vision-language understanding. Despite its progress in remote sensing applications, RIS in Low-Altitude Drone (LAD) scenarios remains underexplored. Existing datasets and methods are typically designed for high-altitude and static-view imagery. They struggle to handle the unique characteristics of LAD views, such as diverse viewpoints and high object density. To fill this gap, we present RIS-LAD, the first fine-grained RIS benchmark tailored for LAD scenarios. This dataset comprises 13,871 carefully annotated image-text-mask triplets collected from realistic drone footage, with a focus on small, cluttered, and multi-viewpoint scenes. It highlights new challenges absent in previous benchmarks, such as category drift caused by tiny objects and object drift under crowded same-class objects. To tackle these issues, we propose the Semantic-Aware Adaptive Reasoning Network (SAARN). Rather than uniformly injecting all linguistic features, SAARN decomposes and routes semantic information to different stages of the network. Specifically, the Category-Dominated Linguistic Enhancement (CDLE) aligns visual features with object categories during early encoding, while the Adaptive Reasoning Fusion Module (ARFM) dynamically selects semantic cues across scales to improve reasoning in complex scenes. The experimental evaluation reveals that RIS-LAD presents substantial challenges to state-of-the-art RIS algorithms, and also demonstrates the effectiveness of our proposed model in addressing these challenges. The dataset and code will be publicly released soon at: https://github.com/AHideoKuzeA/RIS-LAD/.
BibTeX
@misc{ye2025ris,
  author = {Kai Ye and YingShi Luan and Zhudi Chen and Guangyue Meng and Pingyang Dai and Liujuan Cao},
  title = {{RIS-LAD: A Benchmark and Model for Referring Low-Altitude Drone Image Segmentation}},
  year = {2025},
  eprint = {2507.20920},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2507.20920},
}
Figure