Position-Aware Recalibration Module: Learning From Feature Semantics and Feature Position

Xu Ma, Song Fu

Abstract

Standard Convolutional Neural Networks (CNNs) excel at pattern recognition but often lack explicit spatial awareness, treating feature recalibration in a spatially agnostic manner. This paper, presented at IJCAI 2020, introduces the Position-Aware Recalibration Module (PRM). PRM enhances CNN performance by integrating relative position encoding with semantic similarity, allowing the network to consider both what a feature represents and where it is located. The module achieves state-of-the-art results on ImageNet and MS COCO benchmarks with negligible computational overhead.

Introduction

While CNNs process images on a grid, their internal mechanisms for deciding feature importance often ignore geometric relationships. Existing solutions like Squeeze-and-Excitation (SE) blocks provide global context but discard spatial information, while Non-Local networks capture dense pixel relationships at a high computational cost. PRM addresses this "missing middle" by using spatial positions to guide feature importance efficiently, teaching the network to recognize features relative to key objects in the scene.

Methods

The PRM module operates through a dual-stream recalibration process that combines semantic and geometric information.

Architecture of the Position-Aware Recalibration Module (PRM).

As shown in the architecture above, the process involves several key steps:

  1. Representative Query Selection: The module identifies the most discriminative point in a feature map, typically the pixel with the highest average activation across channels.
  2. Relative Position Encoding: It calculates the distance between every pixel and the representative point, encoding this using a Gaussian distribution. This acts as a geometric prior, mimicking biological receptive fields.
  3. Dual-Stream Recalibration: The module fuses Semantic Similarity (how a pixel's meaning relates to the query and global context) with Position Influence (the pixel's distance from the query).
  4. Multi-Head Architecture: Similar to Transformers, the feature channels are split into multiple "heads," allowing the network to attend to different spatial relationships or objects simultaneously.

Key Insights

  • Explicit Position Learning: While CNNs can infer some position from zero-padding, explicit position learning via Gaussian priors is significantly more effective and interpretable.
  • Stage-Dependent Importance: Ablation studies show that spatial position is critical in early network layers (Stages 1 & 2), whereas later layers (Stage 4) rely more on global semantic context as features become more abstract.

Key Results

The researchers evaluated PRM on ImageNet 2012 for classification and MS COCO for object detection and segmentation.

  • ImageNet Performance: Adding PRM to a standard ResNet50 improved top-1 accuracy by 1.75% (from 75.90% to 77.65%). Remarkably, a PRM-enhanced ResNet50 matches the performance of a ResNet101 while using only half the parameters and FLOPs.
  • Object Detection: On MS COCO, PRM improved the mAP of detectors like RetinaNet and Cascade R-CNN by 1.5% to 1.9%, outperforming other attention modules like SE and GC-Net.
  • Efficiency: The module is extremely lightweight. For a ResNet50, the computational cost increases by less than 0.2% (from 4.122G to 4.128G FLOPs).
Impact of the number of heads on top-1 accuracy compared to the baseline.

As illustrated in the figure above, the multi-head formulation consistently outperforms the baseline across various configurations, with performance peaking around 32 to 64 heads.

Conclusion

The Position-Aware Recalibration Module provides a robust and efficient way to introduce spatial awareness into CNNs. By balancing semantic similarity with geometric priors, it achieves significant accuracy gains across diverse tasks. Future work may explore more sophisticated query selection methods and adaptive position encoding designs to further capture complex scene geometries.

Summary updated

Abstract

We present a new method to improve the representational power of the features in Convolutional Neural Networks (CNNs). By studying traditional image processing methods and recent CNN architectures, we propose to use positional information in CNNs for effective exploration of feature dependencies. Rather than considering feature semantics alone, we incorporate spatial positions as an augmentation for feature semantics in our design. From this vantage, we present a Position-Aware Recalibration Module (PRM in short) which recalibrates features leveraging both feature semantics and position. Furthermore, inspired by multi-head attention, our module is capable of performing multiple recalibrations where results are concatenated as the output. As PRM is efficient and easy to implement, it can be seamlessly integrated into various base networks and applied to many position-aware visual tasks. Compared to original CNNs, our PRM introduces a negligible number of parameters and FLOPs, while yielding better performance. Experimental results on ImageNet and MS COCO benchmarks show that our approach surpasses related methods by a clear margin with less computational overhead. For example, we improve the ResNet50 by absolute 1.75% (77.65% vs. 75.90%) on ImageNet 2012 validation dataset, and 1.5%~1.9% mAP on MS COCO validation dataset with almost no computational overhead. Codes are made publicly available.
BibTeX
@inproceedings{ma2020position,
  author = {Xu Ma and Song Fu},
  title = {{Position-Aware Recalibration Module: Learning From Feature Semantics and Feature Position}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.24963/ijcai.2020/111},
}
Figure