D2Q-DETR: Decoupling and Dynamic Queries for Oriented Object Detection with Transformers

Qiang Zhou, Chaohui Yu, Zhibin Wang, Fan Wang

Abstract

Despite the promising results, existing oriented object detection methods usually involve heuristically designed rules, e.g., RRoI generation, rotated NMS. In this paper, we propose an end-to-end framework for oriented object detection, which simplifies the model pipeline and obtains superior performance. Our framework is based on DETR, with the box regression head replaced with a points prediction head. The learning of points is more flexible, and the distribution of points can reflect the angle and size of the target rotated box. We further propose to decouple the query features into classification and regression features, which significantly improves the model precision. Aerial images usually contain thousands of instances. To better balance model precision and efficiency, we propose a novel dynamic query design, which reduces the number of object queries in stacked decoder layers without sacrificing model performance. Finally, we rethink the label assignment strategy of existing DETR-like detectors and propose an effective label re-assignment strategy for improved performance. We name our method D2Q-DETR. Experiments on the largest and challenging DOTA-v1.0 and DOTA-v1.5 datasets show that D2Q-DETR outperforms existing NMS-based and NMS-free oriented object detection methods and achieves the new state-of-the-art.

Introduction

In computer vision, standard object detectors typically utilize axis-aligned bounding boxes. However, in specialized domains such as aerial surveillance, satellite imagery, and scene text recognition, objects are frequently tilted or rotated. Oriented Object Detection (OOB) addresses this by predicting rotated rectangles, but it faces significant technical hurdles.

Traditional OOB methods often struggle with the "Angle Problem," where predicting a rotation angle directly leads to boundary discontinuities (e.g., the mathematical gap between 0 and 360 degrees) and unit mismatches between pixels and radians. Furthermore, these methods rely on complex, hand-crafted rules like Rotated Region of Interest (RRoI) generation and Rotated Non-Maximum Suppression (NMS), which complicate the deployment pipeline. Finally, aerial images present a density challenge; they often contain thousands of instances, making the high query count required by Transformer-based detectors computationally prohibitive. D2Q-DETR is designed to overcome these challenges through an end-to-end Transformer architecture.

The D2Q-DETR Framework

D2Q-DETR builds upon the DEtection TRansformer (DETR) architecture, introducing four primary innovations to enhance precision and efficiency in oriented detection.

1. Points Prediction Head

To bypass the mathematical singularities associated with angle regression, D2Q-DETR replaces the standard $(x, y, w, h, \theta)$ regression head with a points prediction head. The model predicts a set of nine representative points for each object. This "angle-free" approach is inherently more flexible, as the spatial distribution of these points naturally captures both the orientation and the scale of the target. During the inference stage, a simple minAreaRect function converts these points back into a standard rotated box, eliminating the need for complex angle-based loss functions.

2. Query Feature Decoupling

In standard DETR models, a single query feature is typically used for both classification and box regression. D2Q-DETR introduces query feature decoupling, splitting the query into two distinct paths within the decoder. This allows the model to learn specialized features: one path focuses on semantic information ("what" the object is), while the other focuses on geometric information ("where" the object is). This separation has been shown to significantly improve model precision, providing a boost of nearly 2% mAP in ablation studies.

3. Dynamic Query Design

Efficiency is a critical concern in aerial imagery due to the sheer number of potential objects. D2Q-DETR utilizes a novel dynamic query strategy to reduce computational overhead. The model begins with a high number of queries in the initial decoder layers to ensure high recall. In subsequent layers, it gradually "prunes" the search space by retaining only the queries with the highest confidence scores. This reduces the computational cost (GFLOPs) from 211.44 to 197.63 without sacrificing detection accuracy.

4. Label Re-assignment Strategy

The authors identified that standard bipartite matching in DETR can sometimes assign low-quality queries to ground-truth boxes, confusing the learning process. D2Q-DETR introduces a label re-assignment strategy that evaluates the Convex-hull Intersection over Union (IoU) of matches. If a query's geometric quality is too low, it is re-assigned as background, forcing the model to prioritize high-quality detections during training.

Experimental Results

D2Q-DETR was evaluated on two of the most challenging benchmarks for aerial imagery: DOTA-v1.0 and DOTA-v1.5.

  • DOTA-v1.0: The model achieved a state-of-the-art 81.24 mAP, outperforming the previous leading NMS-free method, O2DETR, by a substantial 9.09%. It also surpassed top NMS-based methods like ReDet.
  • DOTA-v1.5: On this more difficult dataset, which includes extremely small instances (under 10 pixels), D2Q-DETR achieved 78.85 mAP, demonstrating its robustness in cluttered and high-resolution environments.

Ablation studies confirmed that each component—the points head, feature decoupling, dynamic queries, and label re-assignment—contributed additively to the overall performance gains.

Conclusion

D2Q-DETR represents a significant step forward for end-to-end oriented object detection. By replacing problematic angle regression with a flexible points-based representation and introducing dynamic query pruning, the framework achieves a superior balance of accuracy and efficiency. While future work may focus on making the point-to-box conversion fully differentiable and optimizing for real-time edge deployment, D2Q-DETR currently sets a new benchmark for the field of aerial image analysis.

Summary updated

Abstract

Despite the promising results, existing oriented object detection methods usually involve heuristically designed rules, e.g., RRoI generation, rotated NMS. In this paper, we propose an end-to-end framework for oriented object detection, which simplifies the model pipeline and obtains superior performance. Our framework is based on DETR, with the box regression head replaced with a points prediction head. The learning of points is more flexible, and the distribution of points can reflect the angle and size of the target rotated box. We further propose to decouple the query features into classification and regression features, which significantly improves the model precision. Aerial images usually contain thousands of instances. To better balance model precision and efficiency, we propose a novel dynamic query design, which reduces the number of object queries in stacked decoder layers without sacrificing model performance. Finally, we rethink the label assignment strategy of existing DETR-like detectors and propose an effective label re-assignment strategy for improved performance. We name our method D2Q-DETR. Experiments on the largest and challenging DOTA-v1.0 and DOTA-v1.5 datasets show that D2Q-DETR outperforms existing NMS-based and NMS-free oriented object detection methods and achieves the new state-of-the-art.
BibTeX
@inproceedings{zhou2023d,
  author = {Qiang Zhou and Chaohui Yu and Zhibin Wang and Fan Wang},
  title = {{D2Q-DETR: Decoupling and Dynamic Queries for Oriented Object Detection with Transformers}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2023},
  doi = {10.1109/icassp49357.2023.10095341},
}
Figure