Introduction
Data centers (DCs) are critical infrastructure but consume immense amounts of energy, with cooling systems often accounting for nearly 40% of total power usage. While "free cooling"—the use of ambient outside air—is effective in temperate climates, it has traditionally been deemed unfeasible in tropical regions like Singapore due to high temperatures and extreme relative humidity (RH). High humidity, when combined with airborne pollutants, can lead to the formation of corrosive pastes that damage server hardware. This paper presents a novel constrained deep reinforcement learning (cDRL) framework designed to make free cooling viable and safe in tropical environments.
System Architecture and Airflow Management
To overcome the challenges of tropical climates, the proposed system utilizes a sophisticated airflow management strategy. The core idea is to adaptively mix outside air with hot server exhaust. Counter-intuitively, mixing damp tropical air with hot exhaust air can lower the relative humidity below the "deliquescent point," preventing dust from becoming corrosive.
The system controls three primary actuators: variable-speed fans for air volume flow, dampers for the mixing ratio of return and fresh air, and a backup cooling coil for precise temperature adjustments.
Constrained Deep Reinforcement Learning (cDRL)
The cooling control problem is modeled as a Constrained Markov Decision Process (CMDP). The objective is to minimize energy consumption (the reward) while strictly adhering to safety constraints regarding temperature and humidity.
Unlike standard RL, which often uses fixed penalties for constraint violations, this framework employs Lagrangian primal-dual optimization. This method adaptively learns penalty weights during training, allowing the agent to prioritize safety thresholds (e.g., Temp < 32°C, RH < 65%) without compromising energy efficiency. To ensure safety during the learning phase, the authors utilized an offline training pipeline featuring high-fidelity simulators that combine physics-based psychrometric models with Multi-Layer Perceptrons (MLPs).
Experimental Setup and Datasets
The study is validated using 18 months of high-resolution data from a custom-built air free-cooled DC testbed at Nanyang Technological University (NTU), Singapore.
The dataset encompasses over 200 measurement points, including ambient weather conditions, server workloads (CPU, HDD, Memory), and power consumption across eight racks. Performance benchmarks confirmed that tropical temperatures up to 37°C do not necessarily throttle computing performance, provided airflow is managed correctly.
Key Results and Discussion
The cDRL approach demonstrated significant advantages over traditional rule-based (hysteresis) controllers and unconstrained RL:
- Energy Efficiency: The agent consistently achieved lower cooling power consumption.
- Safety Compliance: cDRL maintained the lowest variance and fewest violations of temperature and humidity thresholds.
- Real-time Performance: While Model Predictive Control (MPC) required over 500 seconds to calculate a control action, the DRL agent made decisions in just 0.014 seconds, enabling real-time hardware control.
Conclusion
This research proves that free cooling is viable in tropical climates through the application of constrained DRL. By leveraging the non-linear physics of air mixing, the system achieves high energy efficiency while maintaining strict hardware safety standards. Future work will focus on scaling these deployments and coordinating control across distributed data center networks.
Summary updated