TL;DR
This paper introduces EPAM, a novel predictive energy model for mobile AI applications. By leveraging Gaussian process regression and extensive experimental data, EPAM accurately forecasts energy consumption across diverse AI models, processing sources, and device configurations, paving the way for more energy-efficient mobile AI development.
What Problem Does This Paper Address?
The proliferation of AI applications on mobile devices (smartphones, wearables, vehicles) presents a significant challenge: these applications are notoriously power-hungry. While smaller, quantized deep neural networks (DNNs) offer improvements in speed and energy efficiency, a comprehensive understanding of how various AI models and processing units (CPUs, GPUs, NPUs) consume energy on mobile platforms remains largely unexplored. Developers currently lack a reliable way to predict this energy consumption without extensive, device-specific testing, hindering the design of truly energy-efficient mobile AI solutions and impacting user experience.
Methods
The core of this work is the development of EPAM (Energy Prediction for AI in Mobile devices), a predictive energy model built upon Gaussian process regression. EPAM takes into account a multitude of influencing factors, including DNN structure, memory usage, and the specific processing source (CPU, GPU, NNAPI). The model is trained offline using a large, experimentally derived dataset. The authors chose an ARD (Automatic Relevance Determination) exponential squared kernel function, which allows the model to automatically assign different weights to various input parameters, reflecting their significance to the overall energy consumption. This approach allows EPAM to predict energy consumption for an entire application cycle, encompassing both data processing and inference, irrespective of the specific device or AI model.
Datasets
The authors conducted extensive experiments to generate their own dataset. They utilized four different Android smartphones with varying specifications and tested seven distinct DNN models across three application types: image classification, natural language processing (NLP), and speech recognition. The experimental setup involved measuring latency, power consumption, and memory usage for each segment of the AI pipeline. For training and validation, they used a dataset of 85,500 data points, with 19,496 for validation and 10,000 for testing.
Key Insights and Ideas
- Quantization Pays Off: Quantized DNN models consistently show reduced inference latency (around 13%) and energy consumption (around 25%) compared to their floating-point counterparts.
- Processing Source Matters (A Lot): The choice of processing unit (CPU, GPU, NNAPI) significantly impacts latency and energy. While GPUs generally offer improvements over CPUs, NNAPI's performance can be highly variable and sometimes detrimental, suggesting a strong dependence on hardware support.
- Non-Vision AI is Different: Non-vision applications like NLP and speech recognition exhibit unique latency and energy consumption patterns, often with unpredictable text processing stages and less straightforward optimization paths compared to vision tasks.
- Initiation Delay is Real: All AI applications experience an "initiation delay" at the start, caused by the activation of various hardware components. This delay varies based on the application, processing source, and device.
- DNN Structure is Key, But Not Linear: While larger and more complex DNN structures generally lead to higher latency and energy consumption, this relationship is not strictly linear. Model size, memory usage, and inference energy are closely correlated.
- EPAM's Predictive Power: The proposed EPAM model demonstrates high accuracy in predicting energy consumption, offering a valuable tool for developers to optimize mobile AI applications without extensive empirical testing.
Key Results
EPAM achieved a highly accurate prediction of energy consumption, with an overall Root Mean Squared Error (RMSE) of 0.075 (3.06%). The model's performance was evaluated across various models and processing sources, showing a strong correlation between observed and predicted energy values. The experimental analysis revealed that quantized models offer significant energy savings, and the choice of processing source can lead to substantial differences in performance. For instance, quantized models reduced energy consumption by an average of 25% compared to float models.
Limitations and Future Work
The authors acknowledge that the accuracy of EPAM is directly tied to the size and diversity of the training dataset. While their dataset is substantial, a larger and more varied collection of data would likely further enhance prediction accuracy. The paper also notes that NNAPI's performance can be inconsistent, suggesting that future work could explore more nuanced NNAPI optimization strategies or investigate its behavior across a wider range of devices and hardware configurations. The authors also highlight the need for further research into the unique characteristics of non-vision AI applications and their energy consumption profiles.
Summary updated