Abstract
EMBANet introduces a highly flexible multi-branch framework that generalizes existing attention mechanisms by providing new "Degrees of Freedom" in network design. By utilizing novel Multi-branch Concat (MBC) and Multi-branch Attention (MBA) modules, the architecture achieves state-of-the-art performance across classification, detection, and segmentation tasks while remaining computationally efficient.
Introduction
In computer vision, capturing "multi-scale" features—the ability to see both the forest and the trees—is essential for high performance. While architectures like Res2Net or Inception have made strides here, they often rely on rigid, hand-crafted structures.
The authors identify a lack of Degrees of Freedom (DoF) in current attention-based networks. Most existing modules are "one-size-fits-all" and do not allow researchers to easily adjust the number of branches or the specific transformation operators used to extract features. This rigidity often leads to a trade-off: achieving high-resolution multi-scale features at a massive computational cost, or settling for a simpler model that misses long-range channel dependencies.
Methodology
The core of the paper is the Multi-branch Concat (MBC) module, which serves as a flexible paradigm for scale transformation. It introduces two primary operators:
- Multiplex: The input tensor is replicated across $S$ branches. This allows the model to learn more abundant positional information, making it ideal for large backbone networks where generalization is key.
- Split: The input channels are divided into $S$ groups. This reduces channel redundancy and is optimized for lightweight, mobile-friendly networks.
These operators feed into a multi-scale pyramid convolution structure. To keep the parameter count in check, the authors use group convolutions where the group size increases alongside the kernel size (e.g., a $3\times3$ kernel might use a group size of 1, while a $9\times9$ kernel uses a group size of 16).
By integrating MBC with standard attention mechanisms (like Squeeze-and-Excitation), the authors created the Multi-branch Attention (MBA) module. This was then plugged into standard ResNet bottleneck blocks to create the EMBA block, which forms the foundation of the EMBANet backbone.
Key Insights and Unified Framework
One of the most elegant realizations in the paper is that many famous architectures are actually "special cases" of the MBA module. For instance, Res2Net can be viewed as an MBA where branches are connected in a cascade, and SKNet is essentially an MBA with exactly two branches.
By using different kernel sizes across branches ($3\times3, 5\times5, 7\times7, 9\times9$), the model extracts features at a much more granular level than standard CNNs. Furthermore, instead of just summing branch outputs, the MBA module uses a Softmax function to adaptively weight the importance of different spatial scales, allowing the network to "choose" which scale is most relevant for a given image.
Experimental Results
The authors validated EMBANet across the "triple crown" of computer vision benchmarks: ImageNet-1K, CIFAR-100, and MS-COCO (for object detection and instance segmentation).
- ImageNet Classification: EMBANet-M(Large)V3 achieved a 78.97% Top-1 accuracy on a ResNet-50 backbone, outperforming the baseline ResNet-50 (76.15%) and even the popular SENet (76.71%) and Res2Net (77.99%).
- Efficiency: The "Small" version, EMBANet-S(Small), achieved better accuracy than SENet-50 while using 41.8% fewer parameters and 41.4% lower FLOPs.
- Object Detection: On COCO, EMBANet-M-50 improved the Mean Average Precision (mAP) by approximately 4% over the standard ResNet-50 across various detectors.
- Lightweight Performance: When applied to MobileNetV2, their lightweight variant (EMBANet-L) hit 73.40% Top-1 accuracy, surpassing the standard MobileNetV2 (72.00%).
Conclusion
While the MBC and MBA modules offer significant flexibility, the specific configurations used in this paper were still manually designed. The authors suggest that Neural Architecture Search (NAS) is the logical next step to automatically find the optimal "Degrees of Freedom" for specific sub-tasks, potentially unlocking even higher efficiency and performance without the need for human intuition in the loop.
Summary updated