Abstract
Referring Image Segmentation (RIS) in low-altitude drone (LAD) imagery presents unique challenges due to small object sizes, high density, and varying viewpoints. We introduce RIS-LAD, the first fine-grained benchmark specifically for this domain, and the Semantic-Aware Adaptive Reasoning Network (SAARN). SAARN addresses "category drift" and "object drift" through linguistic disentanglement and scale-aware reasoning, achieving state-of-the-art performance on the new benchmark.
Introduction: The Challenges of Low-Altitude Drone Imagery
Referring Image Segmentation (RIS) aims to segment objects based on natural language descriptions. While effective in standard photography and high-altitude satellite remote sensing, existing models falter in low-altitude drone (LAD) scenarios. Drones typically operate at oblique angles (30°–60°) and encounter diverse lighting conditions, including nighttime.
The primary hurdles in LAD imagery are:
- Category Drift: Models confuse the target with visually similar classes (e.g., motorcycle vs. bicycle) due to the tiny scale of objects.
- Object Drift: Models correctly identify the class but fail to select the specific instance among a dense group of similar objects.
The RIS-LAD Dataset
To bridge this gap, we present the RIS-LAD dataset, comprising 13,871 image-text-mask triplets derived from real-world drone footage (30m–100m altitude). The dataset includes eight object categories and features nighttime scenes, which are largely absent in existing remote sensing benchmarks. Annotations were generated using a semi-automatic pipeline involving SAM-2 and Qwen2.5-VL, followed by rigorous human verification.
Semantic-Aware Adaptive Reasoning Network (SAARN)
SAARN introduces a "divide and conquer" strategy to handle the complexities of LAD imagery. Unlike standard models that inject text features holistically, SAARN employs:
- Linguistic Disentanglement: Decomposing expressions into global context, category names, and descriptive attributes.
- Category-Dominated Linguistic Enhancement (CDLE): By injecting category information early into the visual encoder, the model establishes a "semantic safeguard" against category drift.
- Adaptive Reasoning Fusion Module (ARFM): This module uses a Scale Reasoning Gate (SRG) to dynamically weight linguistic cues across resolutions, ensuring that spatial descriptions and fine-grained attributes are applied where they are most effective.
Results and Discussion
SAARN establishes a new state-of-the-art on the RIS-LAD benchmark, achieving an oIoU of 51.54%. Notably, it shows an 18.0% improvement in [email protected] over previous methods, indicating superior localization accuracy. Qualitative results demonstrate that SAARN effectively isolates single instances in crowded scenes where other models often over-segment or hallucinate masks.
Conclusion
RIS-LAD and SAARN provide a robust foundation for precision segmentation in drone-based applications. Future work will focus on reducing computational overhead for real-time deployment and improving performance in extremely cluttered or ambiguous scenes.
Summary updated