COMO: Efficient Deep Neural Networks Expansion With COnvolutional MaxOut
Lists
Only you see this
Abstract
In this paper, we extend the classic MaxOut strategy, originally designed for Multiple Layer Preceptors (MLPs), into CO nvolutional M ax O ut (COMO) — a new strategy making deep convolutional neural networks wider with parameter efficiency. Compared to the existing solutions, such as ResNeXt for ResNet or Inception for VGG-alikes, COMO works well on both linear architectures and the ones with skipped connections and residual blocks. More specifically, COMO adopts a novel split-transform-merge paradigm that extends the layers with spatial resolution reduction into multiple parallel splits. For the layer with COMO, each split passes the input feature maps through a 4D convolution operator with independent batch normalization operators for transformation, then merge into the aggregated output of the original sizes through max-pooling . Such a strategy is expected to tackle the potential classification accuracy degradation due to the spatial resolution reduction, by incorporating the multiple splits and max-pooling-based feature selection. Our experiment using a wide range of deep architectures shows that COMO can significantly improve the classification accuracy of ResNet/VGG-alike networks based on a large number of benchmark datasets. COMO further outperforms the existing solutions, e.g., Inceptions, ResNeXts, SE-ResNet, and Xception, that make networks wider, and it dominates in the comparison of accuracy versus parameter sizes.
BibTeX
@article{zhao2021como,
author = {Baoxin Zhao and Haoyi Xiong and Jiang Bian and Zhishan Guo and Cheng-Zhong Xu and Dejing Dou},
title = {{COMO: Efficient Deep Neural Networks Expansion With COnvolutional MaxOut}},
journal = {IEEE Trans. Multimedia},
year = {2021},
doi = {10.1109/tmm.2020.3002614},
}Loading related content...