Reinforcement learning for hybrid charging stations planning and operation considering fixed and mobile chargers

Yanchen Zhu, Honghui Zou, Chufan Liu, Yuyu Luo, Yuankai Wu, Yuxuan Liang

Abstract

The success of vehicle electrification relies on efficient and adaptable charging infrastructure. Fixed-location charging stations often suffer from underutilization or congestion due to fluctuating demand, while mobile chargers offer flexibility by relocating as needed. This paper studies the optimal planning and operation of hybrid charging infrastructures that combine both fixed and mobile chargers within urban road networks. We formulate the Hybrid Charging Station Planning and Operation (HCSPO) problem, jointly optimizing the placement of fixed stations and the scheduling of mobile chargers. A charging demand prediction model based on Model Predictive Control (MPC) supports dynamic decision-making. To solve the HCSPO problem, we propose a deep reinforcement learning approach enhanced with heuristic scheduling. Experiments on real-world urban scenarios show that our method improves infrastructure availability - achieving up to 244.4% increase in coverage - and reduces user inconvenience with up to 79.8% shorter waiting times, compared to existing solutions.

The Challenge of Spatial Rigidity

Traditional electric vehicle (EV) charging infrastructure suffers from "spatial rigidity." Urban planners face a difficult trade-off: building enough fixed stations to handle peak-hour surges leads to idle capital during off-peak hours, while building for average demand results in massive congestion and "queuing loss," where frustrated drivers abandon long lines.

While mobile chargers (MCs) offer a flexible alternative, coordinating them with a permanent fixed network is mathematically complex. This research addresses the Hybrid Charging Station Planning and Operation (HCSPO) problem, which requires simultaneously deciding where to place permanent stations, how many chargers to install, and how to dynamically route a mobile fleet to address real-time demand spikes.

System Architecture

The proposed system integrates fixed infrastructure with a fleet of mobile chargers managed by a central service provider. This hybrid model allows the network to adapt to dynamic charging demand over time, using mobile units to support fixed stations during peaks or to serve "flexible charging areas" where permanent infrastructure is absent.

System overview of the hybrid charging infrastructure, showing the interaction between fixed stations, mobile chargers, and the central service provider.

Technical Framework: ARL-HCSPO

The authors introduce ARL-HCSPO (Adaptive Reinforcement Learning for HCSPO), a multi-layered framework designed for predictive and adaptive management:

  1. Spatiotemporal Demand Prediction: The system employs Graph Attention Networks (GAT) to model spatial relationships between road junctions and TPA-LSTM to capture temporal trends. This allows the system to anticipate demand surges before they manifest.
  2. Adaptive Reinforcement Learning: A Deep Q-Network (DQN) agent manages high-level planning using a "rolling horizon" framework. It can create new stations, adjust charger counts, or relocate assets based on predicted benefits.
  3. Heuristic Mobile Scheduling: Two strategies govern the mobile fleet: a Support Strategy for overwhelmed fixed stations and a Flexible Area Strategy for high-demand zones without stations.
The ARL-HCSPO architecture, featuring multi-step demand prediction (GAT/LSTM) and an adaptive RL module for infrastructure planning and scheduling.

Real-World Validation and Results

The framework was evaluated using real-world data from the Nanshan District in Shenzhen, China, involving a road network of 1,663 nodes. The ARL-HCSPO method significantly outperformed existing real-world plans and standard RL baselines:

  • Infrastructure Availability: Achieved a 205.1% increase in service coverage (benefit) compared to existing infrastructure.
  • User Experience: Reduced average waiting times by 79.8%.
  • Operational Efficiency: Reduced total system costs by 40.5% while maintaining superior service levels.
  • Throughput: Decreased "queuing loss" by approximately 13%.

Key Insights and Future Directions

A critical finding is the "buffer effect" of mobile chargers. By handling peak loads, MCs allow fixed infrastructure to be more cost-effective and less bloated. Furthermore, ablation studies confirmed that the predictive module is vital; without it, the system's ability to provide coverage drops significantly, proving that reactive management is insufficient for urban EV demand.

Future work will look to incorporate more complex traffic conditions, as peak charging demand often coincides with traffic congestion that can delay mobile chargers. Additionally, the researchers aim to model the localized impact on the electrical grid when multiple chargers congregate in specific urban sectors.

Summary updated

Abstract

The success of vehicle electrification relies on efficient and adaptable charging infrastructure. Fixed-location charging stations often suffer from underutilization or congestion due to fluctuating demand, while mobile chargers offer flexibility by relocating as needed. This paper studies the optimal planning and operation of hybrid charging infrastructures that combine both fixed and mobile chargers within urban road networks. We formulate the Hybrid Charging Station Planning and Operation (HCSPO) problem, jointly optimizing the placement of fixed stations and the scheduling of mobile chargers. A charging demand prediction model based on Model Predictive Control (MPC) supports dynamic decision-making. To solve the HCSPO problem, we propose a deep reinforcement learning approach enhanced with heuristic scheduling. Experiments on real-world urban scenarios show that our method improves infrastructure availability - achieving up to 244.4% increase in coverage - and reduces user inconvenience with up to 79.8% shorter waiting times, compared to existing solutions.
BibTeX
@inproceedings{zhu2025reinforcement,
  author = {Yanchen Zhu and Honghui Zou and Chufan Liu and Yuyu Luo and Yuankai Wu and Yuxuan Liang},
  title = {{Reinforcement learning for hybrid charging stations planning and operation considering fixed and mobile chargers}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/1113},
}
Figure