EARL: An Elliptical Distribution aided Adaptive Rotation Label Assignment for Oriented Object Detection in Remote Sensing Images

Jian Guan, Mingjie Xie, Youtian Lin, Guangjun He, Pengming Feng

Abstract

Label assignment is a crucial process in object detection, determining positive or negative samples during training. However, existing strategies often overlook the unique characteristics of remote sensing images (RSIs), such as large variations in scale and aspect ratio. This leads to imbalanced sampling and the inclusion of low-quality samples, limiting detection performance. To address this, we propose Elliptical Distribution aided Adaptive Rotation Label Assignment (EARL). EARL adaptively selects high-quality positive samples using three core strategies: Adaptive Scale Sampling (ADS) for balanced multi-level distribution, Dynamic Elliptical Distribution aided sampling (DED) to fit target shapes and orientations, and Spatial Distance Weighting (SDW) to focus on high-quality samples. Experiments on DOTA, DIOR-R, and HRSC2016 datasets demonstrate that EARL achieves state-of-the-art performance without adding inference overhead.

Introduction

In oriented object detection for remote sensing images (RSIs), "label assignment" defines which pixels or anchors are treated as positive matches for a target during training. Standard detectors often rely on rigid rectangular boxes or center-based rules. However, RSIs captured from aerial perspectives present three major challenges:

  1. Extreme Aspect Ratios: Objects like ships or bridges are highly elongated. Rectangular boxes include significant background noise in the corners.
  2. Scale Variation: Images often contain objects ranging from massive airports to tiny vehicles. Fixed thresholds can cause smaller objects to have insufficient positive samples.
  3. Arbitrary Orientation: Targets are rarely axis-aligned, complicating standard bounding box logic.

EARL addresses these issues by making label assignment flexible and geometry-aware, ensuring that detectors are trained on the most informative parts of an object.

The EARL Framework

EARL is designed as a training-time enhancement for anchor-free detectors. It consists of three main components that work together to refine sample selection:

The EARL framework integrated with the R-FCOS baseline detector, illustrating the ADS, DED, and SDW modules.
  • Adaptive Scale Sampling (ADS): ADS ensures a balanced number of positive samples across different feature map levels. By sampling sequentially from high-level (coarse) to low-level (fine) maps, it prevents large objects from dominating the training process and ensures small objects receive adequate representation.
  • Dynamic Elliptical Distribution aided Sampling (DED): Instead of a static rectangle, DED uses an elliptical mask that adjusts to the object's aspect ratio. This effectively filters out background noise that typically occupies the corners of a rotated bounding box.
  • Spatial Distance Weighting (SDW): SDW refines the loss function by weighting samples based on their distance from the object's center, prioritizing high-quality central samples during the optimization process.

Key Insights: Geometry and Scale

A core innovation of EARL is the shift from rectangular to elliptical sampling regions. In remote sensing, targets are often thin and rotated. A standard box-based approach captures "background noise" in the corners, which can confuse the detector. By using an elliptical distribution, the model focuses strictly on the "foreground samples" that represent the object's actual geometry.

Comparison of rectangular (a) and elliptical (b) sampling regions. The elliptical mask significantly reduces background noise for elongated objects.

Furthermore, the "top-down" priority in ADS is crucial. By selecting the best matches from coarser feature maps first for large objects, the system avoids "crowding out" the finer feature maps needed to detect tiny objects like cars. This mitigates the scale bias inherent in standard datasets.

Experimental Results

EARL was evaluated on three major benchmarks: DOTA, DIOR-R, and HRSC2016. When integrated into the R-FCOS baseline, EARL provided a 1.80% mAP boost on DOTA and a 2.71% mAP boost on DIOR-R.

The most significant gains occurred in difficult categories. For instance, "Soccer Ball Fields" in the DOTA dataset saw an improvement of 4.55%. When applied to the more advanced RTMDet-R detector, EARL achieved a state-of-the-art 80.35% mAP on DOTA while maintaining a high inference speed of 29.3 FPS. Because EARL only modifies the training process, these accuracy gains come with zero additional computational cost during deployment.

Conclusion

EARL provides a robust solution for oriented object detection in RSIs by introducing geometry-aware and scale-adaptive label assignment. While it significantly improves performance for extreme shapes and sizes, future work will focus on improving detection in highly crowded scenes where elliptical masks may still overlap. EARL is an efficient, "plug-and-play" strategy that can be easily integrated into various anchor-free detectors to achieve state-of-the-art results.

Summary updated

Abstract

Label assignment is a crucial process in object detection, which significantly influences the detection performance by determining positive or negative samples during training process. However, existing label assignment strategies barely consider the characteristics of targets in remote sensing images (RSIs) thoroughly, e.g., large variations in scales and aspect ratios, leading to insufficient and imbalanced sampling and introducing more low-quality samples, thereby limiting detection performance. To solve the above problems, an Elliptical Distribution aided Adaptive Rotation Label Assignment (EARL) is proposed to select high-quality positive samples adaptively in anchor-free detectors. Specifically, an adaptive scale sampling (ADS) strategy is presented to select samples adaptively among multi-level feature maps according to the scales of targets, which achieves sufficient sampling with more balanced scale-level sample distribution. In addition, a dynamic elliptical distribution aided sampling (DED) strategy is proposed to make the sample distribution more flexible to fit the shapes and orientations of targets, and filter out low-quality samples. Furthermore, a spatial distance weighting (SDW) module is introduced to integrate the adaptive distance weighting into loss function, which makes the detector more focused on the high-quality samples. Extensive experiments on several popular datasets demonstrate the effectiveness and superiority of our proposed EARL, where without bells and whistles, it can be easily applied to different detectors and achieve state-of-the-art performance. The source code will be available at: https://github.com/Justlovesmile/EARL.
BibTeX
@article{guan2023earl,
  author = {Jian Guan and Mingjie Xie and Youtian Lin and Guangjun He and Pengming Feng},
  title = {{EARL: An Elliptical Distribution aided Adaptive Rotation Label Assignment for Oriented Object Detection in Remote Sensing Images}},
  journal = {IEEE Trans. Geosci. Remote. Sens.},
  year = {2023},
  url = {https://openreview.net/forum?id=rmKiTUWDhn},
}
Figure