Introduction
Convolutional Neural Networks (CNNs) have revolutionized computer vision, yet their decision-making processes often rely on "style"—global statistics and textures—as much as "shape." While some style information is crucial for classification, other components act as nuisance factors (e.g., lighting variations or artistic filters) that can degrade model robustness.
Existing channel-wise recalibration techniques, most notably Squeeze-and-Excitation (SE) blocks, attempt to address this by modeling inter-channel dependencies. However, SE blocks typically rely on simple global average pooling and computationally expensive fully connected layers. The Style-based Recalibration Module (SRM) addresses these limitations by explicitly capturing style statistics and employing a highly efficient, channel-independent integration mechanism.
Methodology: The SRM Architecture
The SRM module is designed as a lightweight, plug-and-play unit that can be integrated into standard architectures like ResNet. It operates in two primary stages:
1. Style Pooling
Unlike traditional methods that only utilize global average pooling, SRM extracts both the mean and the standard deviation for each channel. In the context of neural style transfer, these two statistics are recognized as effective representations of an image's "style." By incorporating the standard deviation, SRM captures a richer signal of the feature map's distribution, allowing the network to better distinguish between meaningful textures and background noise.
2. Style Integration
Following pooling, the module estimates a recalibration weight for each channel. SRM introduces a Channel-wise Fully Connected (CFC) layer for this purpose. In contrast to the bottleneck MLPs used in SE blocks—which mix information across all channels—the CFC layer determines each channel's weight based solely on its own style statistics. This design choice makes the operation extremely lightweight and encourages channel decorrelation.
The resulting weights are normalized via a Sigmoid function and used to scale the original feature maps, effectively emphasizing task-relevant features while suppressing irrelevant style noise.
Experimental Evaluation
The authors evaluated SRM across a wide array of benchmarks, including ImageNet-1K, CIFAR-10/100, and specialized datasets like the Describable Texture Dataset (DTD) and Stylized-ImageNet.
Performance and Efficiency
On ImageNet, a ResNet-50 equipped with SRM achieved a top-1 accuracy of 77.13%, surpassing both the baseline (75.89%) and the SE-ResNet-50 (76.80%). Remarkably, this performance boost comes with a negligible parameter overhead. While an SE block adds approximately 2.53M parameters to a ResNet-50, SRM adds only 0.06M—a reduction of nearly two orders of magnitude.
Robustness to Style and Texture
SRM demonstrates significant gains in texture-related tasks. On the DTD dataset, SRM-ResNet-56 reached 50.44% accuracy, a substantial improvement over the baseline's 45.46%. Furthermore, on Stylized-ImageNet—where textures are replaced by random artistic styles—SRM outperformed the baseline by over 6%, indicating a superior ability to focus on underlying object shapes by ignoring irrelevant style "noise."
Conclusion
The Style-based Recalibration Module (SRM) provides an efficient and effective way to enhance CNN performance by leveraging global style statistics. By combining mean and standard deviation pooling with channel-wise integration, SRM achieves state-of-the-art results with minimal computational cost. Future research may explore the integration of SRM into modern architectures like Vision Transformers or its application within generative frameworks such as GANs and diffusion models.
Summary updated