Introduction
In the field of remote sensing (RS), image segmentation is evolving beyond simple categorical classification. Modern geospatial analysis requires models that can interpret and execute complex human instructions. These tasks generally fall into three categories: referring (e.g., "Find the parking lot near the stadium"), interactive (e.g., segmenting an area based on user clicks), and reasoning (e.g., identifying regions suitable for specific urban development).
Despite the potential, current research remains fragmented. Most existing models are specialized for a single task and fail to leverage the shared semantic knowledge across different instruction types. Furthermore, the RS domain lacks the massive, high-quality instruction-mask datasets common in natural image processing. To address these challenges, we introduce UniGeoSeg, a unified framework designed for open-world geospatial segmentation.
The UniGeoSeg Framework
UniGeoSeg bridges the gap between disparate segmentation tasks through three core architectural innovations:
- Task-Adaptive Text Enhancement (TATE): Rather than treating all instructions uniformly, TATE employs specialized pathways to process different inputs. It uses a coordinate-aware branch for interactive clicks, an attention-based branch for referring expressions, and a multi-query attention branch for complex reasoning. This allows the model to dynamically adjust its focus based on the specific semantic requirements of the task.
- Latent Knowledge Memory (LKM): To foster synergy between tasks, LKM utilizes learnable memory slots that store task-agnostic representations. By distilling knowledge across the entire dataset, the model can apply what it learns about "buildings" in a click-based task to better identify "residential zones" during complex reasoning.
- Progressive Task Scheduling (PTS): This curriculum learning strategy begins training with "easier" interactive tasks to establish basic spatial grounding. As training matures, the focus shifts toward difficult reasoning tasks, preventing overfitting on simple data and enhancing overall generalization.
Data Scaling: GeoSeg-1M and GeoSeg-Bench
A primary contribution of this work is the release of two major resources:
- GeoSeg-1M: The first million-scale dataset for RS instruction-driven segmentation, containing 1.1 million image-mask-instruction triplets across 117 categories. This dataset was constructed by aggregating 20+ existing RS datasets and utilizing an automated pipeline involving GPT-4o for instruction generation and InternVL3 for mask filtering.
- GeoSeg-Bench: A high-quality evaluation suite comprising 6,892 expert-validated samples. This benchmark provides a rigorous test of a model's ability to handle complex geospatial logic and precise spatial grounding.
Experimental Results
UniGeoSeg establishes a new state-of-the-art (SOTA) across multiple benchmarks:
- GeoSeg-Bench Performance: UniGeoSeg significantly outperforms specialized models like Segearth-R1, particularly in Reasoning gIoU (52.96 vs. 51.56) and Interactive gIoU (76.37 vs. 75.00).
- Zero-Shot Generalization: The model demonstrates remarkable robustness on unseen datasets. On the SIOR interactive task, it achieved an average gIoU of 88.77, rivaling or exceeding the performance of general-purpose models like SAM2.
- Established Benchmarks: On EarthReason and RRSIS-D, UniGeoSeg achieved absolute gains of up to 6.65% in cIoU, confirming its superior ability to link visual features with high-level functional roles in diverse geospatial scenes.
Conclusion and Future Work
UniGeoSeg demonstrates that unifying interactive, referring, and reasoning tasks leads to a more powerful and versatile geospatial model. While the current framework relies on optical RGB imagery and high-performance LLMs for data generation, future work will explore the integration of multi-spectral and SAR data to enable "all-weather" geospatial understanding. By providing the GeoSeg-1M dataset and the UniGeoSeg architecture, we aim to provide a foundation for the next generation of intelligent remote sensing systems.
Summary updated