TL;DR
This paper identifies "slender objects" (those with extreme aspect ratios like skis, bats, or poles) as a significant blind spot for modern object detectors, causing performance to plummet by nearly 19% mAP. The authors demonstrate that the solution isn't better anchors or complex box shapes, but rather a novel "self-adaptation" strategy that allows models to learn optimal feature sampling without being misled by background-heavy object borders.
What Problem Does This Paper Address?
Standard object detection benchmarks like COCO are dominated by "regular" objects with moderate aspect ratios. Consequently, state-of-the-art models are optimized for these shapes and perform poorly on slender objects.
The paper identifies three core challenges:
- Definition Bias: Standard axis-aligned bounding boxes are poor proxies for slenderness, especially when objects are rotated. A diagonal ski might look like a square box to a detector, masking its true "slender" nature.
- Metric Limitations: Standard precision-based metrics (mAP) are ill-defined for slender categories because a single ground truth can match multiple overlapping boxes, making "false positives" ambiguous.
- Data Bias: Existing datasets have very few slender examples, leading to models that effectively ignore them during training.
Methods
The authors propose a unified analytical framework to dissect detectors into four stages: Feature Extraction, Intermediate Prediction, Feature Adaptation, and Detection Formation. Through systematic "diagnosis," they found that traditional fixes like better anchors or 9-point box representations weren't the answer. Instead, they focused on:
- Self-Adaptation of Features: This is the core contribution. Many models try to "supervise" feature sampling to look at object borders. However, for slender objects, borders are mostly empty background. The authors' "self-adaptation" uses Deformable Convolutions (DCN) where offsets are learned adaptively from features themselves, allowing the network to "find" the most informative parts of a thin object without being forced to look at noisy edges.
- Slenderness Prior: They extended the "centerness prior" used in anchor-free detectors. By slowing down the confidence decay along the long axis of an object, they created a prior that better supports training on long, thin shapes.
Datasets
- COCO: Used to establish the initial performance gap.
- COCO+: A new, more balanced benchmark created by the authors. They augmented the COCO validation set with thousands of "extra slender" and "slender" instances from the Objects365 dataset, neutralizing the original data bias and providing a more rigorous testbed.
Key Insights and Ideas
- Borders are Background: A major "aha" moment was realizing that supervising models to look at object borders is counterproductive for slender objects. Because these objects are so thin, their bounding boxes are mostly empty space. Forcing the model to sample features near the box edges just introduces background noise.
- Anchors are for Label Assignment: The paper argues that the primary role of anchors in modern detectors is actually label assignment (deciding which locations are "positives"), rather than providing a geometric starting point for regression.
- Recall over Precision: For evaluating slender objects specifically, the authors advocate for mean Average Recall (mAR) as a more reliable metric than mAP.
Key Results
- Consistent Gains: The self-adaptation strategy improved performance across all tested baselines (RetinaNet, FCOS, RepPoints).
- XS Performance: On the "Extra Slender" (XS) category in COCO+, RetinaNet-ResNet101 saw an mAR jump from 23.6% to 27.0% (+3.4%).
- Overall mAP: Even on the standard COCO dataset, these improvements translated to higher overall scores, such as a +2.2% mAP gain for RetinaNet-ResNet101.
- Slenderness Prior: Adding this prior to FCOS improved XS mAR by 2.1%, proving that accounting for shape during training is a favorable trade-off.
Limitations and Future Work
- Performance Trade-offs: While self-adaptation is generally beneficial, the "slenderness prior" involves a slight trade-off, sacrificing a small amount of accuracy on regular objects to significantly boost slender ones.
- Rotated Boxes: The authors provide a baseline for rotated bounding box evaluation but suggest that more work is needed to fully integrate orientation into the slenderness framework.
- Dataset Scale: While COCO+ is a step forward, the authors note that even larger datasets specifically targeting extreme aspect ratios would be required to fully solve the problem.
Summary updated