Abstract
Modern object detection and segmentation frameworks rely heavily on Feature Pyramid Networks (FPN) to handle multi-scale objects. However, standard FPNs suffer from gradient degradation in the backbone and task interference in the detection heads. This paper introduces the Dually Supervised Feature Pyramid Network (DSFPN), which incorporates auxiliary prediction heads on the bottom-up path to strengthen supervision and a decoupled head to separate classification and regression tasks. Evaluated on MS COCO, DSFPN provides a consistent performance boost across various backbones with negligible impact on inference speed.
Introduction
Feature Pyramid Networks (FPN) have become a cornerstone of object detection, utilizing a bottom-up backbone for feature extraction and a top-down subnet for semantic enrichment. Despite their success, two primary inefficiencies persist:
- Supervision Signal Degradation: Training signals applied only to the top-down path often fade before reaching early backbone layers, leading to sub-optimal feature extraction.
- Task Inconsistency: Sharing a single feature space for classification (semantic) and regression (spatial) tasks creates a conflict that hinders the learning of specialized features.
The DSFPN Framework
To resolve these issues, the authors propose two architectural enhancements: Dual Supervision and the Decoupled Head.
Dual Supervision (DS)
The DS module attaches auxiliary prediction heads to the bottom-up subnet. These heads provide direct "shortcuts" for the loss signal, ensuring that the backbone learns discriminative features from the start. These auxiliary components are used exclusively during training and are discarded during inference, maintaining the model's efficiency.
Decoupled Head (DC)
Standard detection heads use shared fully connected (FC) layers for both classification and regression. DSFPN splits these into two distinct branches, each with its own 1,024-node layers. This separation allows the network to optimize features independently for "what" an object is versus "where" it is located.
Experimental Results
The DSFPN was rigorously tested on the MS COCO dataset using various backbones (ResNet-50, ResNet-101, ResNeXt-101).
- Accuracy Gains: Integrating DSFPN into Faster R-CNN (ResNet-50) yielded a 1.3-point improvement in Average Precision ($AP_{bbox}$). On Cascade Mask R-CNN, it achieved state-of-the-art results with 44.5 $AP_{bbox}$ and 38.1 $AP_{mask}$.
- Training Efficiency: The additional supervision leads to faster convergence, reaching higher accuracy levels in fewer iterations compared to baseline FPNs. Visualization of feature maps reveals that DSFPN produces cleaner activations with less noise, particularly at lower pyramid levels.
- Inference Speed: The computational overhead is virtually zero. For Faster R-CNN, inference time only increased from 0.074s to 0.075s, making it highly practical for real-world deployment.
Conclusion
DSFPN demonstrates that simple yet targeted modifications to the supervision flow and head architecture can significantly enhance detector performance. By optimizing the backbone through auxiliary heads and reducing task conflict via decoupling, the framework provides a robust and efficient solution for multi-scale object detection and segmentation. Future work may extend these dual supervision principles to one-stage detectors and other multi-task vision problems.
Summary updated