Abstract
In this paper, we propose ReLeTA: Reinforcement Learning based Task Allocation for temperature minimization. We design a new reward function and use a new state model to facilitate optimization of reinforcement learning algorithm. By means of the new reward function and state model, ReLeTA is able to effectively reduce the system peak temperature without compromising the application performance. We implement and evaluate ReLeTA on a real platform in comparison with the state-of-the-art approaches. Experimental results show ReLeTA can reduce the average peak temperature by 4 °C and the maximum difference is up to 13 °C.
Introduction
As multicore processors continue to scale, thermal management has become a critical challenge. High operating temperatures negatively impact hardware reliability, leakage power, and performance. Traditional approaches like Dynamic Voltage/Frequency Scaling (DVFS) often incur significant performance penalties. While Reinforcement Learning (RL) offers a promising path for dynamic task allocation due to its ability to learn from real-world interactions, existing RL models often struggle with ineffective state representations or reward functions that fail to balance thermal goals with performance constraints. Furthermore, many existing studies rely solely on simulation, which may not accurately reflect the complexities of physical hardware.
The ReLeTA Framework
We propose ReLeTA, an OS-level task allocation framework that leverages Deep Q-Learning to minimize peak temperatures. ReLeTA integrates directly into the operating system to manage task affinity across available cores.
State Model and Reward Function
A key innovation in ReLeTA is its sophisticated state model and novel reward function:
- State Model: Unlike models that rely solely on temperature, ReLeTA incorporates core frequency and utilization. This allows the agent to anticipate thermal trends rather than just reacting to current heat levels.
- Relative Reward Function: We define the reward as the difference between the average system temperature and the temperature of the specific core assigned a task. This relative approach helps the agent identify the most thermally efficient core at any given moment, regardless of the overall chip temperature.
Implementation
ReLeTA uses a three-layer neural network to approximate Q-values, ensuring scalability for multicore systems without excessive computational overhead. The system was implemented on a Linux (Ubuntu 16.04) platform, utilizing cpuaffinity for precise task placement.
Experimental Evaluation
We evaluated ReLeTA using eight benchmarks from the PARSEC suite (e.g., bodytrack, facesim, ferret) on an Intel Core i7-4790 quad-core platform. This setup provides a realistic environment to test the framework's effectiveness against state-of-the-art methods like LTB (Long-Term Benefit) allocation.
Results and Discussion
Experimental results demonstrate that ReLeTA significantly outperforms existing RL-based allocators.
- Thermal Performance: ReLeTA reduced the average peak temperature by 4°C compared to LTB, with peak reductions reaching up to 13°C.
- Performance and Latency: While some thermal management techniques achieve cooling through aggressive throttling, ReLeTA maintains high performance. Latency violations were kept between 0% and 3%, significantly lower than the 41% observed in some baseline methods.
- Overhead: The allocation overhead was measured at approximately 0.54ms, making it suitable for real-time operating system scheduling.
Conclusion
ReLeTA provides an effective, low-overhead solution for thermal-aware task allocation on multicore systems. By utilizing a relative reward function and a comprehensive state model, it achieves significant temperature reductions without sacrificing performance. Future work will focus on extending this framework to heterogeneous architectures, such as ARM's big.LITTLE, to address the diverse thermal and performance profiles of modern mobile and data center chips.
Summary updated