Optimal Policy for Deployment of Machine Learning Models on Energy-Bounded Systems

Seyed-Iman Mirzadeh, Hassan GhasemzadehIJCAI 2020 · DOI

Abstract

Deploying high-performance machine learning models on energy-constrained edge devices often necessitates a trade-off between accuracy and battery life. While traditional approaches focus on individual model compression, this paper introduces a novel strategy: finding an optimal mixture of multiple models to maximize performance within a strict energy budget. By formulating the deployment as a constrained optimization problem—specifically a variation of the Knapsack Problem—we demonstrate that alternating between diverse models can achieve superior energy-accuracy profiles compared to any single "generalist" model. Experimental results on ImageNet show a 20% energy reduction with negligible accuracy loss.

Introduction

Edge devices, such as smartwatches and IoT sensors, operate under severe energy constraints. Running sophisticated deep learning models on these platforms rapidly depletes battery life. Conventional solutions typically involve model compression techniques like pruning or quantization to create a single, efficient model. However, these methods often reach a point of diminishing returns where further efficiency gains cause significant accuracy degradation.

This research shifts the focus from optimizing a single model to optimizing the deployment policy of a pool of models. For long-term operations, the most effective strategy may involve strategically switching between high-accuracy "expert" models and low-energy "efficient" models to hit an energy-accuracy "sweet spot" unattainable by a single static model.

Methods: The Optimal Deployment Problem

The authors frame the selection of models as a mathematical optimization task. The workflow involves selecting from a pool of candidate models based on their known performance and energy characteristics to meet specific objectives.

Overview of the optimization framework for model deployment.

Mathematical Formulation

The goal is to maximize total accuracy over a fixed number of inferences ($K$) while adhering to a total energy budget ($B$). The objective function is defined as follows:

The objective function maximizes expected performance while incorporating a penalty term for model switching.

In this equation, $p_i$ represents the performance of model $i$, and $x_i$ is the number of times it is used. To ensure practical deployment, a penalty term ($\lambda \sum u_i y_i$) is included. This term discourages frequent switching between too many different model types, accounting for the computational and energy overhead associated with loading new models into memory.

Optimization Strategy

The problem is identified as an Exact K-item Knapsack Problem (E-KKP), which is NP-complete. To solve it efficiently, the authors propose:

  1. Dynamic Programming: Used for smaller model sets to find exact solutions.
  2. Linear Programming (LP) Relaxation: Used for larger scales to provide near-optimal, computationally efficient solutions.

Experimental Evaluation

The policy was evaluated across two primary datasets:

  • UCI Human Activity Recognition (UCI-HAR): Utilizing sensor data with a mix of Decision Trees, SVMs, and 1-D CNNs.
  • ImageNet: Utilizing a wide array of architectures, including VGG, ResNet, and MobileNet-V2.

Key Results

On the ImageNet dataset, the "Optimal Policy 6" achieved a 20% reduction in energy consumption compared to a standalone SE-ResNet101, with only a marginal 0.34% drop in accuracy. Furthermore, the policy outperformed the highly optimized MobileNetV2, yielding a 0.92% increase in accuracy while consuming 0.35% less energy.

In the UCI-HAR task, the proposed policy achieved 1.3% higher accuracy and 1.7% lower energy consumption than standard pruned neural networks.

Discussion and Conclusion

The core insight of this work is the "Power of Combinations." By leveraging a diverse pool of models, systems can navigate the Pareto frontier of energy and accuracy more effectively than with any single model. The inclusion of a switching penalty ensures that the resulting policies are engineering-sound and avoid "jittery" behavior that would waste resources on memory overhead.

Future research directions include accounting for non-linear battery discharge behaviors and developing adaptive policies that can adjust the model mix based on real-time input difficulty or changing environmental conditions.

Summary updated

BibTeX
@inproceedings{mirzadeh2020optimal,
  author = {Seyed-Iman Mirzadeh and Hassan Ghasemzadeh},
  title = {{Optimal Policy for Deployment of Machine Learning Models on Energy-Bounded Systems}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.24963/ijcai.2020/473},
}
Figure