Jointly Optimizing Memory, Computing and Offloading for Energy-Efficient DNN Inference

Yunchu Han, Zhaojun Nan, Sheng Zhou, Zhisheng NiuarXiv 2026 · arXiv

TL;DR

This paper introduces a joint optimization framework for edge intelligence that simultaneously tunes memory frequency, computing frequency, and task offloading to minimize energy consumption. By addressing the "memory wall" bottleneck, the authors achieve near-optimal energy efficiency for deep neural network (DNN) inference, reducing device energy usage by up to 16.1% compared to traditional compute-only scaling methods.

What Problem Does This Paper Address?

As Deep Neural Networks (DNNs) grow in complexity, they become increasingly hungry for memory. In mobile and edge computing, we typically use Dynamic Voltage and Frequency Scaling (DVFS) to save energy by slowing down the processor when possible. However, most existing work focuses solely on the CPU or GPU frequency.

The authors argue that we have hit a "memory wall": the processing speed of computing units is often limited by how fast data can be fetched from memory. If you only scale the computing frequency while ignoring the memory frequency, the processor might just end up idling while waiting for data, wasting energy. This paper tackles the challenge of jointly optimizing memory frequency, computing frequency, and offloading decisions (deciding whether to run a task locally or send it to an edge server) under strict real-time latency constraints.

Methods

The researchers developed a multi-step approach to solve this complex optimization problem:

  1. Empirical Modeling: Using real hardware (NVIDIA Jetson boards), they measured how different combinations of memory and computing frequencies affect inference time and power. They used non-linear least squares to fit these measurements into a tractable mathematical model.
  2. Problem Formulation: They formulated the energy minimization as a Mixed-Integer Nonlinear Programming (MINLP) problem. This is notoriously difficult to solve because the binary offloading decisions (0 or 1) are tightly coupled with continuous resource variables like bandwidth and frequency.
  3. Decomposition Strategy: To make the problem solvable, they decomposed it into two subproblems:
  4. Local Subproblem: Optimizes memory and computing frequencies for local tasks. This was transformed into an equivalent Geometric Programming (GP) problem, which can be solved efficiently.
  5. Edge Subproblem: Optimizes uplink bandwidth allocation for offloaded tasks using convex optimization.
  6. Coordinate Descent (CD) Algorithm: They developed an iterative CD algorithm to find the best offloading policy. It flips the offloading decision for one device at a time to find a local optimum for the entire system.
System illustration showing local inference vs. edge offloading with frequency scaling

Datasets

The authors evaluated their framework using several diverse datasets and DNN architectures to ensure broad applicability:

  • CIFAR-100: Used for image recognition with ResNet152, DenseNet121, VGG19, and ViT-B/16.
  • UCI Human Activity Recognition (UCI HAR): Used with LSTM models.
  • ImageNet: Used for large-scale evaluation with Swin-Transformer (Swin-B).
  • Open Graph Benchmark (OGB): Specifically the ogbn-arxiv dataset for Graph Neural Networks (GNN).

Key Insights and Ideas

  • The Memory Frequency Dominance: For some models, increasing memory frequency yields much higher latency reductions than increasing compute frequency. For example, on a Jetson TX1, boosting memory frequency reduced ResNet152 inference time by 83%, far outperforming compute-only adjustments.
  • Diminishing Returns of Compute Scaling: If memory frequency is set too low, increasing the GPU frequency has almost no effect on performance because the system is "memory-bound."
  • Joint Optimization is Essential: The "fitting surfaces" generated by the authors (see Figure 3) show that the relationship between memory/compute frequency and latency is non-linear and coupled. A "memory-only" or "compute-only" approach misses the most energy-efficient "sweet spots."
Fig. 3. Inference time of four representative DNNs under different combinations of memory frequency and computing frequency.

Key Results

  • Energy Savings: The proposed joint optimization reduced energy consumption by up to 16.1% compared to benchmarks that only optimize computing frequency.
  • Near-Optimality: The algorithm's performance gap from the theoretical optimal (found via exhaustive search) was at most 9.3%, but it ran significantly faster.
  • Hardware Sensitivity: The benefits varied by hardware. For instance, the impact of memory frequency was more pronounced on the older Jetson TX1 compared to the newer Jetson Orin series, highlighting the need for hardware-aware optimization.
  • Latency Trade-offs: The algorithm successfully exploited the full available "deadline" to lower frequencies and save power, whereas standard policies often finished tasks too early at the cost of higher power draw.
Fig. 1. Average inference time of DNNs with different memory frequencies and computing frequencies on Jetson TX1 and Jetson Xavier NX.

Limitations and Future Work

The authors identify several areas for further exploration:

  • Channel Uncertainty: The current model assumes stable wireless links. Real-world fading and mobility could introduce uncertainty that requires "robust" or "chance-constrained" optimization.
  • MEC Contention: The paper assumes the edge server (MEC) has constant processing time. In reality, multiple devices sharing a server would face queueing delays and resource contention.
  • DNN Partitioning: Instead of a binary "local vs. edge" choice, future work could look at splitting a single DNN model across the device and the edge, optimizing frequencies for each specific block of the network.

Summary updated

Abstract

In edge intelligence systems, computation offloading and dynamic voltage and frequency scaling (DVFS) are widely used to improve energy efficiency. However, as applications like deep neural network (DNN) inference evolve in complexity, their demands for memory resources have progressively increased, establishing memory as a critical bottleneck that impacts both latency and energy consumption. In this paper, we propose a joint optimization scheme integrating memory, computing and offloading for energy-efficient DNN inference. We formulate the energy minimization problem under deadline constraints as a mixed-integer nonlinear programming (MINLP) problem, where the main challenge arises from the coupling between offloading and resource optimization. To address this issue, we decompose the original problem into two subproblems under a fixed offloading policy. The local subproblem is transformed into an equivalent geometric programming (GP) problem to derive the optimal memory frequency and computing frequency, while the edge subproblem is solved via convex optimization. An efficient coordinate descent (CD) algorithm is further developed to jointly optimize all variables. Simulation results demonstrate that our proposed algorithm achieves near-optimal performance with the gap from the optimal policy being at most 9.3%, and reduces the energy consumption of devices by up to 16.1% compared with benchmarks without memory frequency optimization.
BibTeX
@misc{han2026jointly,
  author = {Yunchu Han and Zhaojun Nan and Sheng Zhou and Zhisheng Niu},
  title = {{Jointly Optimizing Memory, Computing and Offloading for Energy-Efficient DNN Inference}},
  year = {2026},
  eprint = {2611.11106},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.11106},
}
Figure