CLIO: enabling automatic compilation of deep learning pipelines across IoT and cloud

Jin Huang, Colin Samplawski, Deepak Ganesan, Benjamin M. Marlin, Heesung Kwon

Abstract

Recent years have seen dramatic advances in low-power neural accelerators that aim to bring deep learning analytics to IoT devices; simultaneously, there have been considerable advances in the design of low-power radios to enable efficient compute offload from IoT devices to the cloud. Neither is a panacea --- deep learning models are often too large for low-power accelerators and bandwidth needs are often too high for low-power radios. While there has been considerable work on deep learning for smartphone-class devices, these methods do not work well for small battery-powered IoT devices that are considerably more resource-constrained.In this paper, we bridge this gap by designing a continuously tunable method for leveraging both local and remote resources to optimize performance of a deep learning model. Clio presents a novel approach to split machine learning models between an IoT device and cloud in a progressive manner that adapts to wireless dynamics. We show that this method can be combined with model compression and adaptive model partitioning to create an integrated system for IoT-cloud partitioning. We implement Clio on the GAP8 low-power neural accelerator, provide an exhaustive characterization of the operating regimes where each method performs best and show that Clio can enable graceful performance degradation as resources diminish.
BibTeX
@inproceedings{huang2020clio,
  author = {Jin Huang and Colin Samplawski and Deepak Ganesan and Benjamin M. Marlin and Heesung Kwon},
  title = {{CLIO: enabling automatic compilation of deep learning pipelines across IoT and cloud}},
  booktitle = {MobiCom},
  year = {2020},
  doi = {10.1145/3372224.3419215},
}

Loading related content...

Figure