Deep Reinforcement Learning for Tropical Air Free-Cooled Data Center Control

Duc Van Le, Rongrong Wang, Yingbo Liu, Rui Tan, Yew-Wah Wong, Yonggang Wen

Introduction

Data centers (DCs) are critical infrastructure but consume immense amounts of energy, with cooling systems often accounting for nearly 40% of total power usage. While "free cooling"—the use of ambient outside air—is effective in temperate climates, it has traditionally been deemed unfeasible in tropical regions like Singapore due to high temperatures and extreme relative humidity (RH). High humidity, when combined with airborne pollutants, can lead to the formation of corrosive pastes that damage server hardware. This paper presents a novel constrained deep reinforcement learning (cDRL) framework designed to make free cooling viable and safe in tropical environments.

System Architecture and Airflow Management

To overcome the challenges of tropical climates, the proposed system utilizes a sophisticated airflow management strategy. The core idea is to adaptively mix outside air with hot server exhaust. Counter-intuitively, mixing damp tropical air with hot exhaust air can lower the relative humidity below the "deliquescent point," preventing dust from becoming corrosive.

3D and Top views of the data center layout showing the airflow paths through mixing chambers and dampers.

The system controls three primary actuators: variable-speed fans for air volume flow, dampers for the mixing ratio of return and fresh air, and a backup cooling coil for precise temperature adjustments.

Constrained Deep Reinforcement Learning (cDRL)

The cooling control problem is modeled as a Constrained Markov Decision Process (CMDP). The objective is to minimize energy consumption (the reward) while strictly adhering to safety constraints regarding temperature and humidity.

Unlike standard RL, which often uses fixed penalties for constraint violations, this framework employs Lagrangian primal-dual optimization. This method adaptively learns penalty weights during training, allowing the agent to prioritize safety thresholds (e.g., Temp < 32°C, RH < 65%) without compromising energy efficiency. To ensure safety during the learning phase, the authors utilized an offline training pipeline featuring high-fidelity simulators that combine physics-based psychrometric models with Multi-Layer Perceptrons (MLPs).

Experimental Setup and Datasets

The study is validated using 18 months of high-resolution data from a custom-built air free-cooled DC testbed at Nanyang Technological University (NTU), Singapore.

Physical equipment of the NTU testbed, including server racks, IT equipment, and various environmental sensors.

The dataset encompasses over 200 measurement points, including ambient weather conditions, server workloads (CPU, HDD, Memory), and power consumption across eight racks. Performance benchmarks confirmed that tropical temperatures up to 37°C do not necessarily throttle computing performance, provided airflow is managed correctly.

Key Results and Discussion

The cDRL approach demonstrated significant advantages over traditional rule-based (hysteresis) controllers and unconstrained RL:

  • Energy Efficiency: The agent consistently achieved lower cooling power consumption.
  • Safety Compliance: cDRL maintained the lowest variance and fewest violations of temperature and humidity thresholds.
  • Real-time Performance: While Model Predictive Control (MPC) required over 500 seconds to calculate a control action, the DRL agent made decisions in just 0.014 seconds, enabling real-time hardware control.

Conclusion

This research proves that free cooling is viable in tropical climates through the application of constrained DRL. By leveraging the non-linear physics of air mixing, the system achieves high energy efficiency while maintaining strict hardware safety standards. Future work will focus on scaling these deployments and coordinating control across distributed data center networks.

Summary updated

BibTeX
@article{le2021deep,
  author = {Duc Van Le and Rongrong Wang and Yingbo Liu and Rui Tan and Yew-Wah Wong and Yonggang Wen},
  title = {{Deep Reinforcement Learning for Tropical Air Free-Cooled Data Center Control}},
  journal = {ACM Trans. Sens. Networks},
  year = {2021},
  doi = {10.1145/3439332},
}
Figure