Introduction
Oriented Object Detection (OOD) is a critical task in computer vision, particularly for aerial and satellite imagery where objects like ships, vehicles, and aircraft appear at arbitrary rotations. Traditional detectors typically rely on axis-aligned bounding boxes, but OOD requires predicting an orientation angle, significantly expanding the search space.
Current state-of-the-art detectors generally utilize either CNN-based "anchors" or Transformer-based "queries." However, these static candidates often struggle to capture the complex spatial information and varying aspect ratios inherent in rotated objects. GSDet introduces a novel Gaussian-to-box paradigm, formulating OOD as a 3D Gaussian Splatting problem to overcome these limitations.
Methodology: The Gaussian-to-Box Paradigm
The core innovation of GSDet lies in representing objects as anisotropic 3D Gaussians within a learned feature space. This approach eliminates the need for manually designed anchors or queries.
3D Feature Space and Representation
Since aerial images are 2D, GSDet constructs a synthetic 3D feature space by treating multi-scale feature maps from a backbone network (e.g., ResNet or Swin Transformer) as "supporting planes" stacked along a virtual Z-axis. Objects are then represented as 3D Gaussians defined by their mean (center), anisotropic covariance (shape and rotation), and a confidence score. These Gaussians are initialized randomly, which surprisingly outperforms grid-based initializations.
Splatting and Transformation
Following the principles of Gaussian Splatting, these 3D Gaussians are projected onto the 2D image plane. The resulting 2D Gaussians are mathematically decomposed into oriented bounding boxes $(cx, cy, w, h, \theta)$. This transformation is naturally suited for OOD because the covariance matrix of a Gaussian aligns elegantly with the geometry of a rotated rectangle.
GauAlign and Adaptive Control
To refine these representations, GSDet employs GauAlign, a sampling mechanism that integrates regional features across the Z-axis (feature levels). This allows the model to attend to the most relevant scales for each object, effectively treating "scale" as a spatial dimension. During inference, an adaptive control strategy iteratively refines predictions by keeping high-confidence Gaussians and replacing low-confidence ones with new random candidates, enabling a dynamic "hunting" process.
Experimental Results
GSDet was evaluated on prominent OOD benchmarks, including DOTA-v1.0, DOTA-v1.5, and DIOR-R.
- DOTA-v1.0: GSDet achieved 75.74% $AP_{50}$ with a ResNet50 backbone, outperforming established query-based and anchor-based methods. With a Swin-T backbone and multi-scale training, performance reached 80.04%.
- DIOR-R: The model reached 70.05% $AP_{50}$, marking a 0.7% improvement over the baseline.
- Efficiency: GSDet maintains competitive inference speeds, ranging from 21 to 28 FPS on a ResNet50 backbone, proving faster than many complex two-stage or Transformer-based detectors.
Conclusion and Future Work
GSDet demonstrates that Gaussian Splatting is a powerful and flexible framework for oriented object detection. By treating scale as a spatial dimension and leveraging the mathematical synergy between Gaussians and rotated boxes, it achieves state-of-the-art results without the need for static anchors or queries.
While the current adaptive control strategy is somewhat heuristic, the framework's ability to dynamically refine predictions during inference offers a significant advantage. Future work will focus on refining these update strategies and exploring the framework's application to true 3D data, such as LiDAR point clouds.
Summary updated