Exploiting potential of deep neural networks by layer-wise fine-grained parallelism

Wenbin Jiang, Yangsong Zhang, Pai Liu, Jing Peng, Laurence T. Yang, Geyan Ye, Hai Jin

Abstract

Highlights•Pushing DNN parallelism into a new fine-grained layer-based level.•Proposing an inter-layer pipeline-based parallelism strategy for neighboring layers.•Presenting a mathematical analysis for the influence of the fragment number.•Proposing an intra-layer parallelism strategy for convolution layers.
BibTeX
@article{jiang2020exploiting,
  author = {Wenbin Jiang and Yangsong Zhang and Pai Liu and Jing Peng and Laurence T. Yang and Geyan Ye and Hai Jin},
  title = {{Exploiting potential of deep neural networks by layer-wise fine-grained parallelism}},
  journal = {Future Generation Computer Systems},
  year = {2020},
  doi = {10.1016/j.future.2019.07.054},
}

Loading related content...

Figure