Scaled gated networks
Lists
Only you see this
Abstract
Gating transformation demonstrates great potential in recent deep convolutional neural networks design, enriching the feature representation and alleviating noisy signals by modeling the inter-channel dependencies using learnable parameters. However, the utilization of scaling approaches to reduce the redundancy of the hand-crafted attention mechanism has rarely been investigated. This paper proposes a novel scaled gated convolution that enables attention-enhanced CNNs to overcome the paradox between performance and redundancy. Our scaled gated convolution is a simple and effective alternative compared with both vanilla convolution and attention-enhanced convolutions, which can be easily applied to modern CNNs in a plug-and-play manner. Exhaustive experiments demonstrate that stacking scaled gated convolutions in baselines can significantly improve the performance in a broad range of visual recognition tasks, including image recognition, object detection, instance segmentation, keypoint detection, and panoptic segmentation, while obtaining a better trade-off between performance and attentive redundancy.
BibTeX
@inproceedings{lu2022scaled,
author = {Ruiyuan Lu and Jihua Zhu and Xueming Qian and Zhiqiang Tian and Yi Yue},
title = {{Scaled gated networks}},
booktitle = {World Wide Web},
year = {2022},
url = {https://openreview.net/forum?id=gaoFBE6jia9},
}Loading related content...