EARL: An Elliptical Distribution Aided Adaptive Rotation Label Assignment for Oriented Object Detection in Remote Sensing Images

Jian Guan, Mingjie Xie, Youtian Lin, Guangjun He, Pengming Feng

TL;DR

EARL (Elliptical Distribution aided Adaptive Rotation Label Assignment) is a novel training strategy designed to improve oriented object detection in remote sensing images. By replacing rigid, box-based label assignment with adaptive scale sampling and elliptical distributions, it significantly boosts accuracy for objects with extreme shapes and sizes—all without adding any computational overhead during inference.

What Problem Does This Paper Address?

In the world of computer vision, "label assignment" is the process of deciding which pixels (or anchors) in an image count as a "positive" match for a target object during training. Most standard detectors use simple rules, like checking if a pixel falls within a rectangular bounding box or near the center of an object.

However, remote sensing images (RSIs)—taken from satellites or drones—present unique challenges:

  1. Extreme Aspect Ratios: A ship or a bridge is long and skinny; a standard rectangular box includes a lot of "background noise" in the corners.
  2. Scale Variation: A single image might contain a massive airport and a tiny car. Fixed rules often result in the tiny car having zero positive samples, while the airport has thousands, leading to a massive training imbalance.
  3. Arbitrary Orientation: Objects aren't always aligned with the axes of the image.

Existing strategies often fail to capture enough high-quality samples for these "extreme" cases, leading to detectors that struggle with small or highly elongated rotated objects.

Methods

The authors propose EARL, a three-pronged strategy to make label assignment more flexible and "aware" of object geometry:

  • Adaptive Scale Sampling (ADS): Instead of assigning samples to feature map levels based on a fixed threshold, ADS samples sequentially from high-level (coarse) to low-level (fine) feature maps. It ensures a balanced number of positive samples ($top-k$) for every object, regardless of its size.
  • Dynamic Elliptical Distribution aided Sampling (DED): Rather than using a rectangle, EARL uses an elliptical mask that dynamically adjusts its shape based on the object's aspect ratio. For a square object, it’s a circle; for a long ship, it’s a thin ellipse. This filters out background noise from the corners of the bounding box.
  • Spatial Distance Weighting (SDW): Not all positive samples are created equal. SDW integrates a weighting factor into the loss function, giving more importance to samples near the center of the object and less to those near the boundaries.

Crucially, EARL is a training-time-only modification. Once the model is trained, the "label assignment" logic is discarded, meaning the detector runs at its original speed during deployment.

Datasets

The researchers evaluated EARL on three widely recognized benchmarks for oriented object detection:

  • DOTA: A massive dataset with 2,806 images and 15 categories (e.g., planes, bridges, small vehicles).
  • DIOR-R: Another large-scale dataset featuring 20 common target categories.
  • HRSC2016: A specialized dataset focused specifically on high-resolution ship detection in maritime environments.

Key Insights and Ideas

  • The "Top-Down" Priority: One of the cleverest parts of ADS is the sequential sampling from $P_7$ down to $P_3$. By picking the best matches from higher levels first for large objects, it prevents them from "crowding out" the feature maps needed for smaller objects.
  • Geometry Matters: The shift from rectangles to ellipses is a simple but profound observation. In remote sensing, where objects are often highly elongated (like a harbor or a bus), a rectangle is a poor approximation of the object's actual "informative" area.
  • Mitigating Scale Bias: The paper highlights that standard methods are naturally biased toward the most common scales in a dataset. EARL’s adaptive approach forces the model to pay equal attention to the "tiny car" and the "large airplane."

Key Results

The authors integrated EARL into two baseline detectors: the simple R-FCOS and the more advanced RTMDet-R.

  • Accuracy Gains: On the DOTA dataset, EARL improved the baseline R-FCOS by 1.80% mAP. On DIOR-R, the improvement was even higher at 2.71% mAP.
  • Performance on "Extreme" Objects: The most dramatic improvements were seen in difficult categories. For example, in DOTA, the "Small Vehicle" (SV) category saw a significant boost, and "Soccer Ball Fields" (SBF) improved by 4.55%.
  • State-of-the-Art (SOTA): When applied to RTMDet-R, the system achieved 80.35% mAP on DOTA, outperforming many complex, anchor-based detectors while maintaining a high inference speed of 29.3 FPS.

Limitations and Future Work

The authors acknowledge that while EARL handles scale and rotation well, it still struggles with densely packed targets in "crowd scenes" (e.g., many ships docked side-by-side). In these cases, the elliptical masks can still overlap significantly, leading to ambiguous samples.

Future work may involve combining EARL with specialized loss functions or "decoding" strategies specifically designed for crowded environments to further reduce false negatives in dense clusters.

Summary updated

Abstract

Label assignment is a crucial process in object detection, which significantly influences the detection performance by determining positive or negative samples during training process. However, existing label assignment strategies barely consider the characteristics of targets in remote sensing images (RSIs) thoroughly, e.g., large variations in scales and aspect ratios, leading to insufficient and imbalanced sampling and introducing more low-quality samples, thereby limiting detection performance. To solve the above problems, an elliptical distribution aided adaptive rotation label assignment (EARL) is proposed to select high-quality positive samples adaptively in anchor-free detectors. Specifically, an adaptive scale sampling (ADS) strategy is presented to select samples adaptively among multilevel feature maps according to the scales of targets, which achieves sufficient sampling with more balanced scale-level sample distribution. In addition, a dynamic elliptical distribution (DED) aided sampling strategy is proposed to make the sample distribution more flexible to fit the shapes and orientations of targets and filter out low-quality samples. Furthermore, a spatial distance weighting (SDW) module is introduced to integrate the adaptive distance weighting into loss function, which makes the detector more focused on the high-quality samples. Extensive experiments on several popular datasets demonstrate the effectiveness and superiority of our proposed EARL, where without bells and whistles it can be easily applied to different detectors and achieve state-of-the-art performance. The source code will be available at: https://github.com/Justlovesmile/EARL .
BibTeX
@article{guan2023earl,
  author = {Jian Guan and Mingjie Xie and Youtian Lin and Guangjun He and Pengming Feng},
  title = {{EARL: An Elliptical Distribution Aided Adaptive Rotation Label Assignment for Oriented Object Detection in Remote Sensing Images}},
  journal = {IEEE Trans. Geosci. Remote. Sens.},
  year = {2023},
  url = {https://openreview.net/forum?id=rmKiTUWDhn},
}

Loading related content...

Figure