ZODS-RS - Zero-training Oriented Detection & Segmentation for Remote Sensing

Zuan Gu, Tianhan Gao, Langxu ZhaoarXiv 2026 · arXiv · OpenReview

TL;DR

This paper introduces ZODS-RS, a novel zero-training pipeline for detecting and segmenting objects in remote sensing and UAV imagery. It achieves impressive performance across diverse datasets and challenging scenarios like crowded scenes and small objects, without requiring any task-specific fine-tuning.

What Problem Does This Paper Address?

Remote sensing and drone imagery present unique challenges: objects are often oriented (not axis-aligned), vary greatly in scale, and appear in densely packed scenes (like airports or shipyards). Existing models often struggle to generalize to new platforms or viewpoints without extensive retraining. While "training-free" approaches exist, they often fall short on these specific challenges, particularly when trying to unify both object detection (bounding boxes) and instance segmentation (pixel-level masks). This paper aims to bridge this gap by providing a robust, training-free solution that handles these complexities effectively.

Methods

ZODS-RS is built upon a "frozen backbone synergy" using pre-trained DINOv3 for dense features and SAM2 for high-recall proposals. The core innovation lies in its three-stage, closed-form pipeline:

  • PP (Prototype Purification): This stage refines object prototypes using Tyler's scatter estimation and spectral purification, making them more robust to noise and domain shifts. It also incorporates optional anchor-based shrinkage for better alignment.
  • R-SEM (Rotation-Scale Equivariant Matching): This module addresses the challenge of oriented objects and scale variations. It uses rotation-scale equivariant responses and a global Hungarian assignment to match proposals to prototypes, effectively handling geometric variations without training.
  • UAM (Uncertainty-Aware Pixelwise Merging): This stage merges overlapping object predictions by considering pixel-level uncertainty and adaptive priors. It uses a Bayesian approach to stabilize predictions and suppress false positives, especially in crowded regions.

A CWLA (Consistency-Weighted Layer Aggregation) module is also introduced to fuse multi-layer DINOv3 features, acting as a semantic gate to improve feature separability.

Datasets

The paper evaluates ZODS-RS on several datasets:

  • xView: A large-scale dataset for object detection in aerial imagery, known for its diverse categories and varying resolutions.
  • FAIR1M: A benchmark dataset focusing on fine-grained object recognition in high-resolution remote sensing imagery, particularly challenging due to scale and crowding.
  • Custom UAV Dataset: A self-collected dataset to evaluate performance on low-altitude drone imagery.

Notably, the evaluation is performed in a training-free, cross-domain setting, meaning models are evaluated on datasets they were not trained on, often using only a single reference image per class.

Key Insights and Ideas

  • Frozen Backbone Synergy is Key: The paper emphasizes that leveraging powerful, pre-trained models like DINOv3 and SAM2 as fixed components, rather than fine-tuning them, is crucial for effective training-free generalization.
  • Geometric Equivariance Matters: Explicitly handling rotation and scale variations through modules like R-SEM is vital for accurate detection in remote sensing, where these factors are prominent.
  • Uncertainty is Your Friend: UAM's use of uncertainty-aware merging demonstrates that understanding prediction confidence can significantly improve robustness and reduce false positives in complex scenes.
  • No Training is Truly Possible: The success across diverse datasets and challenging scenarios without any fine-tuning highlights the potential of well-designed, training-free pipelines for practical applications.

Key Results

ZODS-RS demonstrates strong performance across all evaluated datasets:

  • On the UAV dataset, it achieves a mask mIoU of 31.10 and significantly improves small-object AP by +30.70 over Grounded-SAM.
  • On xView, it reports an HBB mAP of 16.69.
  • On FAIR1M, it achieves an HBB mAP of 13.06.
  • In cross-domain evaluations (e.g., FAIR1M to xView), ZODS-RS consistently outperforms baselines, showing robust generalization capabilities.
  • The method shows particular strength in high-IoU precision (AP75) and mask quality, indicating accurate localization and stable boundaries.

Limitations and Future Work

The authors acknowledge that for extremely small targets in the FAIR1M and xView datasets, the no-tiling constraint leads to slightly lower recall (APs) compared to some baselines. They suggest that lightweight tiling or zoom-in heuristics could address this.

Future work includes:

  • Extending UAM to output true Oriented Bounding Boxes (OBBs) and shape-aware masks.
  • Improving small-object recall through tiling and denser R-SEM grids.
  • Developing online adaptation of priors and memory for more dynamic scenarios.
  • Leveraging stronger semantic gating with models like Florence/CLIP to handle open-set distractors.
  • Exploring SAM2's streaming capabilities for temporal stability and integrating with tracking.
  • Optimizing deployment for efficiency using ONNX/TensorRT.
  • Broadening evaluation to more diverse remote sensing corpora.

Summary updated

Abstract

Remote-sensing and UAV applications need models that generalize across platforms and viewpoints without task-specific training. Yet training-free pipelines often falter on oriented geometry, scale/rotation variation, and crowded ports or airfields, and rarely unify detection and segmentation. We introduce ZODS-RS, a training-free, closed-form pipeline that outputs horizontal boxes (HBB) and instance masks. Built on DINOv3 dense features and SAM-style proposals, ZODS-RS chains: PP (prototype purification via Tyler covariance), R-SEM (rotation-scale equivariant matching with separable kernels and global Hungarian assignment), and UAM (uncertainty-aware pixelwise merging with adaptive priors and optional negative prototypes). A lightweight CWLA fuses multiple DINOv3 layers. On FAIR1M (HBB) we obtain $\mathrm{mAP}_{0.50:0.95}=\mathbf{13.06}$ and $\mathrm{AP}_S=\mathbf{2.93}$ \emph{(class-averaged over ship/airplane)}; on xView (HBB) we report $\mathrm{mAP}=\mathbf{16.69}$. On our UAV dataset, ZODS-RS achieves mask $\mathrm{mIoU}=\mathbf{31.10}$ and improves small-object AP by $\mathbf{+30.70}$ over Grounded-SAM on a single 5090. This work offers a unified, \emph{no-training} solution for horizontal-box detection plus instance segmentation in aerial imagery; provides explicit closed-form formulations for PP/R-SEM/UAM tightly coupled with DINOv3; and demonstrates \emph{consistent} gains on small and crowded targets and under cross-domain shifts while keeping deployment simple.
BibTeX
@misc{gu2026zods,
  author = {Zuan Gu and Tianhan Gao and Langxu Zhao},
  title = {{ZODS-RS - Zero-training Oriented Detection \& Segmentation for Remote Sensing}},
  year = {2026},
  eprint = {2606.10769},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2606.10769},
}

Loading related content...

Figure