COMO: Efficient Deep Neural Networks Expansion With COnvolutional MaxOut

Baoxin Zhao, Haoyi Xiong, Jiang Bian, Zhishan Guo, Cheng-Zhong Xu, Dejing Dou

Abstract

In this paper, we extend the classic MaxOut strategy, originally designed for Multiple Layer Preceptors (MLPs), into CO nvolutional M ax O ut (COMO) — a new strategy making deep convolutional neural networks wider with parameter efficiency. Compared to the existing solutions, such as ResNeXt for ResNet or Inception for VGG-alikes, COMO works well on both linear architectures and the ones with skipped connections and residual blocks. More specifically, COMO adopts a novel split-transform-merge paradigm that extends the layers with spatial resolution reduction into multiple parallel splits. For the layer with COMO, each split passes the input feature maps through a 4D convolution operator with independent batch normalization operators for transformation, then merge into the aggregated output of the original sizes through max-pooling . Such a strategy is expected to tackle the potential classification accuracy degradation due to the spatial resolution reduction, by incorporating the multiple splits and max-pooling-based feature selection. Our experiment using a wide range of deep architectures shows that COMO can significantly improve the classification accuracy of ResNet/VGG-alike networks based on a large number of benchmark datasets. COMO further outperforms the existing solutions, e.g., Inceptions, ResNeXts, SE-ResNet, and Xception, that make networks wider, and it dominates in the comparison of accuracy versus parameter sizes.
BibTeX
@article{zhao2021como,
  author = {Baoxin Zhao and Haoyi Xiong and Jiang Bian and Zhishan Guo and Cheng-Zhong Xu and Dejing Dou},
  title = {{COMO: Efficient Deep Neural Networks Expansion With COnvolutional MaxOut}},
  journal = {IEEE Trans. Multimedia},
  year = {2021},
  doi = {10.1109/tmm.2020.3002614},
}

Loading related content...

Figure