Abstract
Standard Convolutional Neural Networks (CNNs) excel at pattern recognition but often lack explicit spatial awareness, treating feature recalibration in a spatially agnostic manner. This paper, presented at IJCAI 2020, introduces the Position-Aware Recalibration Module (PRM). PRM enhances CNN performance by integrating relative position encoding with semantic similarity, allowing the network to consider both what a feature represents and where it is located. The module achieves state-of-the-art results on ImageNet and MS COCO benchmarks with negligible computational overhead.
Introduction
While CNNs process images on a grid, their internal mechanisms for deciding feature importance often ignore geometric relationships. Existing solutions like Squeeze-and-Excitation (SE) blocks provide global context but discard spatial information, while Non-Local networks capture dense pixel relationships at a high computational cost. PRM addresses this "missing middle" by using spatial positions to guide feature importance efficiently, teaching the network to recognize features relative to key objects in the scene.
Methods
The PRM module operates through a dual-stream recalibration process that combines semantic and geometric information.
As shown in the architecture above, the process involves several key steps:
- Representative Query Selection: The module identifies the most discriminative point in a feature map, typically the pixel with the highest average activation across channels.
- Relative Position Encoding: It calculates the distance between every pixel and the representative point, encoding this using a Gaussian distribution. This acts as a geometric prior, mimicking biological receptive fields.
- Dual-Stream Recalibration: The module fuses Semantic Similarity (how a pixel's meaning relates to the query and global context) with Position Influence (the pixel's distance from the query).
- Multi-Head Architecture: Similar to Transformers, the feature channels are split into multiple "heads," allowing the network to attend to different spatial relationships or objects simultaneously.
Key Insights
- Explicit Position Learning: While CNNs can infer some position from zero-padding, explicit position learning via Gaussian priors is significantly more effective and interpretable.
- Stage-Dependent Importance: Ablation studies show that spatial position is critical in early network layers (Stages 1 & 2), whereas later layers (Stage 4) rely more on global semantic context as features become more abstract.
Key Results
The researchers evaluated PRM on ImageNet 2012 for classification and MS COCO for object detection and segmentation.
- ImageNet Performance: Adding PRM to a standard ResNet50 improved top-1 accuracy by 1.75% (from 75.90% to 77.65%). Remarkably, a PRM-enhanced ResNet50 matches the performance of a ResNet101 while using only half the parameters and FLOPs.
- Object Detection: On MS COCO, PRM improved the mAP of detectors like RetinaNet and Cascade R-CNN by 1.5% to 1.9%, outperforming other attention modules like SE and GC-Net.
- Efficiency: The module is extremely lightweight. For a ResNet50, the computational cost increases by less than 0.2% (from 4.122G to 4.128G FLOPs).
As illustrated in the figure above, the multi-head formulation consistently outperforms the baseline across various configurations, with performance peaking around 32 to 64 heads.
Conclusion
The Position-Aware Recalibration Module provides a robust and efficient way to introduce spatial awareness into CNNs. By balancing semantic similarity with geometric priors, it achieves significant accuracy gains across diverse tasks. Future work may explore more sophisticated query selection methods and adaptive position encoding designs to further capture complex scene geometries.
Summary updated