Abstract
Standard Convolutional Neural Networks (CNNs) typically utilize square convolution kernels, such as the ubiquitous $3 \times 3$ filter. However, research suggests that the central "skeleton" of these kernels—the horizontal and vertical cross—carries significantly more weight and importance for feature extraction than the corner pixels. ACNet introduces the Asymmetric Convolution Block (ACB), a novel architectural component designed to strengthen these skeletons during training. By employing a multi-branch structure during the learning phase and fusing these branches into a single standard kernel for deployment, ACNet provides a "free lunch": improved model accuracy with zero additional computational cost, memory footprint, or latency during inference.
Introduction
In deep learning, enhancing model performance often necessitates increasing the computational budget through deeper architectures or complex neural architecture searches. ACNet challenges this paradigm by focusing on the internal structure of the convolution kernel itself. The authors observed that the central crisscross pixels of a $3 \times 3$ kernel are more critical to performance than the corners. To exploit this, they proposed explicitly reinforcing this "skeleton" during training without burdening the final deployed model.
The Asymmetric Convolution Block (ACB)
The core of ACNet is the ACB, which replaces standard square convolution layers during the training phase.
Training Phase
Instead of a single $d \times d$ convolution, the ACB utilizes three parallel branches:
- A standard $3 \times 3$ kernel.
- A horizontal $1 \times 3$ kernel.
- A vertical $3 \times 1$ kernel.
The outputs of these three branches are summed and then passed through a Batch Normalization (BN) layer. This configuration allows the model to learn richer representations by specifically focusing on the horizontal and vertical features of the input.
Inference and Deployment
Because convolution is a linear operation, the multi-branch structure used during training can be mathematically collapsed into a single standard kernel for inference. This process, known as fusion, ensures that the deployed model has the exact same architecture and speed as the original baseline, while retaining the performance gains from the asymmetric training.
Mathematical Fusion Mechanism
The fusion process occurs in two primary steps:
- BN Fusion: The parameters of the Batch Normalization layers (mean, variance, gamma, and beta) are "folded" into the weights and biases of their respective convolution kernels.
- Branch Fusion: The resulting $1 \times 3$ and $3 \times 1$ kernels are added to the corresponding central rows and columns of the $3 \times 3$ kernel.
This mathematical equivalence allows the three-branch training block to be replaced by a single $3 \times 3$ convolution layer during deployment, maintaining the original model's efficiency.
Experimental Results
ACNet was evaluated across several benchmarks, including CIFAR-10, CIFAR-100, and ImageNet, using architectures like VGG, ResNet, and DenseNet. The results consistently showed performance improvements with zero inference overhead:
- ImageNet Classification: AlexNet saw a Top-1 accuracy increase of 1.52%, while ResNet-18 and DenseNet-121 improved by 0.78% and 0.67%, respectively.
- Rotational Robustness: ACNet-enhanced models demonstrated superior performance on images with rotational distortions. For example, on ImageNet images rotated by 180°, an ACNet-ResNet-18 outperformed the baseline by nearly 1%.
- Hyper-parameter Free: A significant advantage of ACNet is that it requires no additional hyper-parameter tuning, making it a seamless drop-in replacement for standard convolution layers.
Conclusion and Limitations
ACNet provides a practical method for boosting CNN performance by strengthening kernel skeletons. While it introduces some additional computational overhead during the training phase, the "free" accuracy gains at inference time make it highly attractive for production environments. Future research may further explore the relationship between kernel spatial locations and rotational invariance to design even more efficient and robust architectures.
Summary updated