Reinforcement Learning for Hybrid Charging Stations Planning and Operation Considering Fixed and Mobile Chargers

Yanchen Zhu, Honghui Zou, Chufan Liu, Yuyu Luo, Yuankai Wu, Yuxuan Liang

Abstract

As electric vehicle (EV) adoption grows, urban charging infrastructure faces the challenge of "spatial rigidity"—fixed stations cannot adapt to shifting demand patterns. This paper introduces ARL-HCSPO, a hybrid framework that combines fixed charging stations with mobile chargers (MCs). By integrating deep reinforcement learning with spatiotemporal demand forecasting, the system optimizes both strategic planning and operational scheduling. Evaluated on real-world data from Shenzhen, China, the framework achieves a 244% increase in coverage and a 79.8% reduction in user wait times compared to traditional methods.

Introduction: The Spatial Rigidity Problem

Urban EV charging demand is highly dynamic, shifting across the city throughout the day. Fixed charging stations often face a "Goldilocks" problem: they are either congested during peak hours or underutilized during off-peak times. This mismatch arises because fixed infrastructure cannot follow the "heat maps" of demand.

Spatiotemporal demand shifts in Nanshan District at 06:00, 12:00, and 19:00, illustrating the dynamic nature of charging needs.

While mobile chargers (MCs) offer a flexible solution, managing a hybrid network of fixed and mobile assets is computationally complex. Planners must simultaneously decide on permanent station locations (strategic planning) and real-time MC dispatch (operational scheduling) amidst fluctuating traffic and charging patterns.

Methodology: The ARL-HCSPO Framework

The authors propose the Adaptive Reinforcement Learning for Hybrid Charging Station Planning and Operation (ARL-HCSPO) framework to bridge the gap between static planning and dynamic demand.

The ARL-HCSPO framework architecture, featuring multi-step demand prediction and an adaptive RL loop for planning and operation.

The system is built on three main pillars:

  1. Multi-step Demand Prediction: A spatiotemporal model using Graph Embedding and TPA-LSTM provides the "foresight" needed for proactive scheduling by predicting future needs across the road network.
  2. Adaptive RL Framework: A Deep Q-Network (DQN) agent manages the planning phase. It iteratively refines station placement using neighborhood operations like Create by Demand and Relocate.
  3. Heuristic Mobile Scheduling: To address real-time needs, the framework employs two MC strategies: a Support Strategy for overloaded stations and a Flexible Area Strategy for high-demand nodes without stations.

Experimental Evaluation

The framework was tested using real-world data from the Nanshan District of Shenzhen, China, featuring a road network of 1,663 nodes and 2,964 edges. Demand was estimated from historical charging sessions.

Key Results

ARL-HCSPO significantly outperformed baselines that rely solely on fixed stations:

  • Coverage: Achieved up to a 244.4% improvement over existing infrastructure plans.
  • User Experience: Average waiting times were slashed by 79.8%.
  • System Efficiency: The overall cost metric, including travel and charging time, was reduced by 40.5%.

Discussion and Future Work

The study highlights a critical synergy: mobile chargers allow fixed stations to be sized for "base load" demand rather than peak spikes, improving ROI. However, limitations remain. Future research should address the impact of MCs on the local power grid and integrate more complex routing algorithms to optimize the energy MCs consume during transit.

Summary updated

Abstract

The success of vehicle electrification relies on efficient and adaptable charging infrastructure. Fixed-location charging stations often suffer from underutilization or congestion due to fluctuating demand, while mobile chargers offer flexibility by relocating as needed. This paper studies the optimal planning and operation of hybrid charging infrastructures that combine both fixed and mobile chargers within urban road networks. We formulate the Hybrid Charging Station Planning and Operation (HCSPO) problem, jointly optimizing the placement of fixed stations and the scheduling of mobile chargers. A charging demand prediction model based on Model Predictive Control (MPC) supports dynamic decision-making. To solve the HCSPO problem, we propose a deep reinforcement learning approach enhanced with heuristic scheduling. Experiments on real-world urban scenarios show that our method improves infrastructure availability - achieving up to 244.4% increase in coverage - and reduces user inconvenience with up to 79.8% shorter waiting times, compared to existing solutions.
BibTeX
@inproceedings{zhu2025reinforcement,
  author = {Yanchen Zhu and Honghui Zou and Chufan Liu and Yuyu Luo and Yuankai Wu and Yuxuan Liang},
  title = {{Reinforcement Learning for Hybrid Charging Stations Planning and Operation Considering Fixed and Mobile Chargers}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/1113},
}
Figure