Towards Ubiquitous Intelligent Computing: Heterogeneous Distributed Deep Neural Networks

Zongpu Zhang, Tao Song, Liwei Lin, Yang Hua, Xufeng He, Zhengui Xue, Ruhui Ma, Haibing GuanIEEE Trans. Big Data 2022 · OpenReview

Abstract

For the pursuit of ubiquitous computing, distributed computing systems containing the cloud, edge devices, and Internet-of-Things devices are highly demanded. However, existing distributed frameworks do not tailor for the fast development of Deep Neural Network (DNN), which is the key technique behind many intelligent applications nowadays. Based on prior exploration on distributed deep neural networks (DDNN), we propose Heterogeneous Distributed Deep Neural Network (HDDNN) over the distributed hierarchy, targeting at ubiquitous intelligent computing. While being able to support basic functionalities of DNNs, our framework is optimized for various types of heterogeneity, including heterogeneous computing nodes, heterogeneous neural networks, and heterogeneous system tasks. Besides, our framework features parallel computing, privacy protection and robustness, with other consideration for the combination of heterogeneous distributed system and DNN. Extensive experiments demonstrate that our framework is capable of utilizing hierarchical distributed system better for DNN and tailoring DNN for real-world distributed system properly, which is with low response time, high performance, and better user experience.
BibTeX
@article{zhang2022towards,
  author = {Zongpu Zhang and Tao Song and Liwei Lin and Yang Hua and Xufeng He and Zhengui Xue and Ruhui Ma and Haibing Guan},
  title = {{Towards Ubiquitous Intelligent Computing: Heterogeneous Distributed Deep Neural Networks}},
  journal = {IEEE Trans. Big Data},
  year = {2022},
  url = {https://openreview.net/forum?id=p3Ldls6nKn},
}

Loading related content...

Figure