Learning Connected Attentions for Convolutional Neural Networks

Xu Ma, Jingda Guo, Sihai Tang, Zhinan Qiao, Qi Chen, Qing Yang, Song Fu, Paparao Palacharla, Nannan Wang, Xi WangICME 2021 · OpenReview

Abstract

Self-attention modules like Squeeze-and-Excitation (SE) and CBAM have become standard components in modern Convolutional Neural Networks (CNNs). However, these modules are typically treated as isolated units, ignoring potential synergies across layers. This paper introduces DCANet (Deep Connected Attention Network), a framework that interconnects attention modules to allow them to share information and learn jointly. By introducing "Attention Connections" with minimal computational overhead, DCANet improves feature discrimination and boosts performance across various tasks, including image classification and object detection.

Introduction

While self-attention has revolutionized computer vision, current architectures often treat attention blocks as "islands." Each block processes features from its immediate predecessor without knowledge of the attention decisions made in earlier or later stages. This lack of coordination can lead to erratic attention focus and "mushy" attention values that fail to decisively emphasize important features.

Inspired by the parallel processing and interconnected nature of the human visual cortex, the authors propose DCANet. The goal is to move from independent attention learning to a coordinated strategy where deeper layers benefit from the contextual insights of shallower ones.

Methods: The Attention Connection

The core innovation of DCANet is the deconstruction of attention modules into three generic stages: Extraction ($E$), Transformation ($T$), and Fusion ($F$).

  • Extraction ($E$): Captures global context, often via Global Average Pooling.
  • Transformation ($T$): Maps context to an attention weight (e.g., via fully connected layers).
  • Fusion ($F$): Applies weights to the feature maps.

DCANet introduces a "hook" where the output of the Transformation stage from the previous layer is fused with the Extraction output of the current layer. This creates a chain of information flow across the network.

The DCANet architecture showing how attention information is connected across layers through Extraction, Transformation, and Fusion stages.

To ensure compatibility between layers with different dimensions, DCANet utilizes Size Matching modules. Spatial matching is handled by average pooling, while channel matching uses lightweight, learnable fully-connected layers. This design ensures that the "Attention Connection" is both generic and efficient.

Key Results and Insights

The authors evaluated DCANet on ImageNet (classification) and MS COCO (detection). The results consistently showed that connecting attention modules outperforms their isolated counterparts. For instance, DCA-SE-MobileNetV2 improved Top-1 accuracy by 1.19% over standard SE-MobileNetV2. In object detection, a ResNet50 backbone with DCA-SE achieved a +1.5% AP boost over a standard ResNet50.

Visualization and Discrimination

A key finding is that DCANet leads to more "decisive" networks. In vanilla SE-ResNets, attention values often cluster around 0.5. With DCANet, the distribution becomes more spread out, indicating the network is making clearer choices about which features to emphasize.

Comparison of Grad-CAM visualizations and SE value histograms between SE-ResNet50 and DCA-SE-ResNet50, showing more focused attention and decisive value distributions in DCANet.

As shown in the Grad-CAM visualizations, DCANet-enhanced models focus on the target object much more progressively and steadily across layers compared to the erratic focus of vanilla models.

Conclusion

DCANet demonstrates that interconnecting attention modules is a simple yet effective way to improve CNN performance. By allowing attention blocks to "talk" to each other, the network learns a more robust and coordinated feature-weighting strategy. While the benefits are most pronounced in shallower to mid-depth networks, the negligible computational cost makes DCANet a highly practical enhancement for modern computer vision architectures.

Summary updated

Abstract

While self-attention mechanism has shown promising results for many vision tasks, it only considers the current features at a time. We show that such a manner cannot take full advantage of the attention mechanism. In this paper, we present Deep Connected Attention Network (DCANet), a novel design that boosts attention modules in a CNN model without any modification of the internal structure. To achieve this, we interconnect adjacent attention blocks, making information flow among attention blocks possible. With DCANet, all attention blocks in a CNN model are trained jointly, which improves the ability of attention learning. Our DCANet is generic. It is not limited to a specific attention module or base network architecture. Experimental results on ImageNet and MS COCO benchmarks show that DCANet consistently outperforms the state-of-the-art attention modules with a minimal additional computational overhead in all test cases. The code is available at: https://github.com/13952522076/DCANet.
BibTeX
@inproceedings{ma2021learning,
  author = {Xu Ma and Jingda Guo and Sihai Tang and Zhinan Qiao and Qi Chen and Qing Yang and Song Fu and Paparao Palacharla and Nannan Wang and Xi Wang},
  title = {{Learning Connected Attentions for Convolutional Neural Networks}},
  booktitle = {ICME},
  year = {2021},
  url = {https://openreview.net/forum?id=tV1urA1YIs},
}
Figure