Deep Neural Networks with Flexible Complexity While Training Based on Neural Ordinary Differential Equations
Lists
Only you see this
Abstract
Most structures of deep neural networks (DNN) are with a fixed complexity of both computational cost (parameters and FLOPs) and the expressiveness. In this work, we experimentally investigate the effectiveness of using neural ordinary differential equations (NODEs) as a component to provide further depth to relatively shallower networks rather than stacked layers (depth) which achieved improvement with fewer parameters. Moreover, we construct deep neural networks with flexible complexity based on NODEs which enables the system to adjust its complexity while training. The proposed method achieved more parameter-efficient performance than stacking standard DNNs, and it alleviates the defect of the heavy cost required by NODEs.
BibTeX
@inproceedings{luo2021deep,
author = {Zhengbo Luo and Sei-ichiro Kamata and Zitang Sun and Weilian Zhou},
title = {{Deep Neural Networks with Flexible Complexity While Training Based on Neural Ordinary Differential Equations}},
booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
year = {2021},
url = {https://openreview.net/forum?id=CV55rt-nXr},
}Loading related content...