Stochastic Activation Function Layers for Convolutional Neural Networks

Loris Nanni, Alessandra Lumini, Stefano Ghidoni, Gianluca MaguoloPreprints.org 2020 · DOI

Abstract

In recent years, the field of deep learning achieved considerable success in pattern recognition, image segmentation and may other classification fields. There are a lot of studies and practical applications of deep learning on images, video or text classification. In this study, we suggest a method for changing the architecture of the most performing CNN models with the aim of designing new models to be used as stand-alone networks or as a component of an ensemble. We propose to replace each activation layer of a CNN (usually a ReLu layer) by a different activation function stochastically drawn from a set of activation functions: in this way the resulting CNN has a different set of activation function layers.
BibTeX
@misc{nanni2020stochastic,
  author = {Loris Nanni and Alessandra Lumini and Stefano Ghidoni and Gianluca Maguolo},
  title = {{Stochastic Activation Function Layers for Convolutional Neural Networks}},
  year = {2020},
  doi = {10.20944/preprints202002.0231.v1},
  howpublished = {Preprints.org},
}
Figure