Abstract
One-stage object detectors rely on a point feature to predict the detection results. However, the point feature often lacks the information of the whole object, thereby leading to a misalignment between the object and the point feature. Meanwhile, the classification and regression tasks are sensitive to different object regions, but their features are spatially aligned. Both of these two problems hinder the detection performance. In order to solve these two problems, we propose a simple and plug-in operator that can generate aligned and disentangled features for each task, respectively, without breaking the fully convolutional manner. By predicting two task-aware point sets that are located in each sensitive region, the proposed operator can align the point feature with the object and disentangle the two tasks from the spatial dimension. We also reveal an interesting finding of the opposite effect of the long-range skip connection for classification and regression. On the basis of the Object-Aligned and Task-disentangled operator (OAT), we propose OAT-Net, which explicitly exploits point-set features for accurate detection results. Extensive experiments on the MS-COCO dataset show that OAT can consistently boost different state-of-the-art one-stage detectors by ~2 AP. Notably, OAT-Net with Res2Net-101-DCN backbone achieves 53.7 AP on the COCO test-dev.
Introduction
In the world of object detection, one-stage detectors (like FCOS or RetinaNet) are prized for their speed. However, they typically rely on a single point feature to predict both the category of an object and its bounding box. This creates two major hurdles: misalignment and spatial entanglement.
Misalignment occurs because a single point's receptive field often fails to cover the entire object, especially for large or irregularly shaped items. The feature the model uses to "see" the object doesn't actually align with the object's full geometry. Spatial entanglement refers to the fact that classification and regression are fundamentally different tasks. Classification benefits from looking at the most discriminative parts of an object, while regression needs to see the boundaries. Most one-stage detectors force these tasks to share the same features, leading to sub-optimal performance.
Methodology: The OAT Operator
The authors propose the OAT (Object-Aligned and Task-disentangled) operator, a plug-in module designed to generate task-specific features without sacrificing speed. It consists of three main components:
- Regression Feature Alignment (RFA): Instead of a single point, this module predicts a coarse bounding box and samples nine "regression-aware" points. It uses these points to align the features with the object's actual shape via deformable convolutions.
- Spatial Disentanglement (SD) & Classification Feature Alignment (CFA): Recognizing that classification needs to look elsewhere, the SD module learns a set of vectors to shift the regression points to new "classification-aware" locations. This allows the model to sample features from regions most relevant to identifying the object's category.
- Task-Specific Skip Connections: The researchers discovered that long-range skip connections (fusing features from different layers) significantly help classification but actually hurt regression. OAT-Net applies these connections selectively to optimize both tasks.
Theoretical Insights: Invariance vs. Equivariance
A key contribution of this work is identifying the "Invariance vs. Equivariance Conflict." Classification requires translation invariance—a cat is a cat regardless of its position in the frame. Conversely, regression requires translation equivariance—if the object moves, the bounding box must move by the exact same amount.
Because these requirements are fundamentally opposed, forcing both tasks to share the same features and skip connections is inherently sub-optimal. The OAT operator addresses this by disentangling the feature extraction process for each task.
Experimental Results
The researchers evaluated their approach using the MS-COCO dataset, utilizing the train2017 set for training and val2017 for ablation studies. Final benchmarking was performed on the COCO test-dev set.
The OAT operator demonstrated consistent gains across various architectures:
- FCOS: +2.3 AP improvement.
- RepPoints: +1.6 AP improvement.
- State-of-the-Art: OAT-Net, using a Res2Net-101 backbone, reached 53.7 AP on COCO test-dev, outperforming many complex multi-stage detectors.
Ablation studies confirmed that spatial disentanglement alone (moving classification points away from regression points) provided a significant 1.2 AP boost, proving that the spatial location of feature sampling is critical for performance.
Conclusion
The OAT-Net introduces a novel approach to fixing chronic issues in one-stage object detectors. By adaptively predicting two distinct sets of task-aware points, OAT-Net achieves top-tier performance on the COCO benchmark. Future work could explore applying these task-aware point sets to more granular tasks like instance segmentation or keypoint detection. Additionally, the discovery regarding skip connections suggests that future backbone architectures might benefit from task-specific branching earlier in the network.
Summary updated