TL;DR
This paper introduces AutoAssign, a novel, fully differentiable label assignment strategy for dense object detection. By learning to adaptively assign positive and negative labels based on object appearance and category characteristics, AutoAssign significantly outperforms existing methods and achieves state-of-the-art results on MS COCO.
What Problem Does This Paper Address?
Dense object detection methods, which predict objects at every spatial location in an image, face a crucial challenge: label assignment. This involves deciding which locations are "positive" (likely to contain an object) and which are "negative" (background). Traditional methods rely on human-designed heuristics, such as predefined anchor boxes with specific scales and aspect ratios, or fixed rules for selecting positive locations (e.g., within the center region of an object). While effective, these fixed strategies struggle to adapt to the diverse appearances and distributions of objects across different categories and scenarios. This can lead to suboptimal performance, especially when objects have unusual shapes or when background locations are mistakenly labeled as positive. The paper aims to create a more adaptive and data-driven label assignment mechanism that can automatically learn the best strategy from the data.
Methods
AutoAssign proposes a fully differentiable weighting mechanism that replaces hand-crafted label assignment rules. The core idea is to learn weights for each location, indicating its likelihood of being a positive or negative sample. This is achieved through two key modules:
- Center Weighting: This module addresses the "prior distribution" of objects. It starts with the common "center prior" (objects are more likely to be near their bounding box center) but then learns category-specific Gaussian-shaped weighting functions. This allows the model to adapt the prior distribution to the typical shapes and locations of objects within each category, moving beyond a universal center prior.
- Confidence Weighting: This module adapts the assignment strategy to individual object appearances. It uses a "joint confidence indicator" that combines classification confidence (from the object's class prediction) and localization confidence (derived from regression loss). A novel "Implicit-Objectness" (ImpObj) branch is introduced, which shares supervision with the classification branch and helps to filter out noisy background locations within positive candidates. This module dynamically adjusts the positive and negative weights for each location based on its predicted quality and the object's characteristics.
These two modules work together to generate positive and negative weight maps for all locations within a ground truth bounding box. These weights are then used to modulate the training loss, allowing the entire label assignment process to be optimized end-to-end through backpropagation. Crucially, these weighting modules are only used during training, making AutoAssign inference-cost-free.
Datasets
The primary dataset used for training and evaluation is MS COCO 2017. The paper also reports performance on other datasets to demonstrate generalization:
- PASCAL VOC
- Objects365
- WiderFace
Key Insights and Ideas
- Data-Driven Label Assignment is Key: The paper strongly advocates for moving away from fixed, human-designed heuristics in label assignment towards a fully data-driven approach.
- Unified Spatial and Scale Assignment: AutoAssign tackles both spatial location selection and feature level (scale) selection within a single, unified weighting mechanism, unlike previous methods that treated them separately.
- Category-Specific Priors Improve Performance: Learning category-specific center distributions (Center Weighting) provides a significant performance boost, especially for categories with unique shapes.
- Joint Confidence for Robustness: Combining classification and localization confidence (Confidence Weighting) leads to a more robust assignment strategy, effectively filtering out noisy candidates.
- Implicit-Objectness for Better Separation: The novel ImpObj branch, trained implicitly with classification, significantly improves performance by enhancing the separation between foreground and background.
- Inference Efficiency: The learned weighting modules are only used during training, meaning AutoAssign adds no computational overhead during inference.
Key Results
- AutoAssign achieves state-of-the-art performance on the MS COCO 2017 benchmark, with its best model reaching 52.1% AP, outperforming all existing one-stage detectors.
- The method consistently surpasses other sampling strategies by significant margins across various backbones.
- Ablation studies show that both Center Weighting and Confidence Weighting contribute substantially to the overall performance.
- Experiments on PASCAL VOC, Objects365, and WiderFace demonstrate AutoAssign's broad applicability and generalization ability, achieving superior performance without requiring dataset-specific adjustments.
Limitations and Future Work
The authors acknowledge that the current weighting mechanism, while effective, is not yet simple and could potentially be further simplified. They suggest this as a direction for future work.
Summary updated