BA-Net: Bridge Attention in Deep Neural Networks

Ronghui Zhang, Runzong Zou, Yue Zhao, Zirui Zhang, Junzhou Chen, Yue Cao, Chuan Hu, Houbing Song

Introduction

Modern computer vision architectures frequently employ channel attention mechanisms, such as Squeeze-and-Excitation (SE) blocks, to enhance feature representation by focusing on informative channels. However, these modules typically operate in isolation, calculating attention weights based solely on the current layer's features. This "siloed" approach overlooks the potential of earlier layers, which often contain foundational feature information that could guide the current layer's decisions.

BA-Net introduces Bridge Attention (BA), a novel mechanism that generates attention weights by aggregating information across multiple convolutional layers. By bridging the gap between layers, BA-Net recovers information that might otherwise be degraded as data flows through a deep network.

Methodology: The Bridge Attention Module

The core innovation of BA-Net is the integration of features from preceding layers into the attention calculation process. Unlike standard modules that only look at the local feature map $F_n$, the Bridge Attention module establishes connections with preceding layers $F_1, F_2, \dots, F_{n-1}$.

Comparison of standard attention modules versus the proposed Bridge Attention (BA) module.

The authors propose two iterations of this module:

  1. BAv1 (Original Bridge Operation): This version utilizes Global Average Pooling to "squeeze" features from the current and preceding layers. These features are processed with Batch Normalization and combined via element-wise summation to produce final attention weights.
  2. BAv2 (Adaptive Bridge Operation): Recognizing that different layers contribute varying levels of utility, BAv2 introduces an adaptive selection operator. It concatenates features from multiple layers and applies a linear transformation, allowing the network to learn optimal weights for each layer's contribution. This effectively filters noise and emphasizes the most synergistic cross-layer interactions.

BA-Net is designed as a "plug-and-play" enhancement, compatible with both Convolutional Neural Networks (CNNs) like ResNet and Vision Transformers (ViTs) like Swin and PVT.

Key Insights

  • Historical Context: Attention is significantly more effective when it incorporates the "history" of features from earlier in the network.
  • Feature Influence: Using Central Kernel Alignment (CKA), the researchers found that features from earlier layers often exert more influence on the final attention weights than the current layer's own features.
  • Adaptive Selection: The BAv2 module's ability to dynamically weight layer contributions makes it superior to the fixed summation in BAv1, as it reduces information redundancy and noise.

Experimental Results

The effectiveness of BA-Net was validated across major benchmarks, including ImageNet-1K for classification and MS COCO 2017 for object detection and instance segmentation.

ImageNet Top-1 Accuracy vs. Number of Parameters for various models, highlighting the efficiency of BA-Net.

Performance Highlights:

  • ImageNet Classification: BAv2 achieved 80.49% Top-1 accuracy on ResNet50, a 1.61% improvement over the baseline. It consistently outperformed established mechanisms like SE-Net and FcaNet with negligible increases in computational cost (FLOPs).
  • Vision Transformers: Integrating BAv2 into a Swin-Transformer (Swin-T) resulted in a Top-1 accuracy of 81.8%, a 0.6% gain over the standard model.
  • Object Detection: Using a ResNet50 backbone with Faster R-CNN on the COCO dataset, BAv2 improved mean Average Precision (mAP) by 1.6%, demonstrating that bridge attention benefits complex localization tasks.

Conclusion and Future Work

BA-Net demonstrates that bridging layers within a network stage provides a significant performance boost with minimal overhead. While the current work focuses on intra-stage bridging, future research may explore cross-stage bridging (e.g., linking Stage 1 to Stage 4) and further optimization of hybrid CNN-Transformer architectures. The modular nature of Bridge Attention makes it a promising candidate for a wide range of deep learning applications.

Summary updated

BibTeX
@article{zhang2025ba,
  author = {Ronghui Zhang and Runzong Zou and Yue Zhao and Zirui Zhang and Junzhou Chen and Yue Cao and Chuan Hu and Houbing Song},
  title = {{BA-Net: Bridge Attention in Deep Neural Networks}},
  journal = {Expert Systems with Applications},
  year = {2025},
  doi = {10.1016/j.eswa.2025.128525},
}
Figure