Two-Phase on-Line Joint Scheduling for Welfare Maximization of Charging Station

Qilong Huang, Qing-Shan Jia, Xiang Wu, Xiaohong Guan

TL;DR

This paper tackles the complex challenge of optimizing electric vehicle (EV) charging station operations by jointly scheduling pricing and charging control. They propose a novel two-phase online learning algorithm that maximizes social welfare by considering EV driver behavior, price sensitivity, and the inherent uncertainties in renewable energy generation and EV arrivals.

What Problem Does This Paper Address?

The increasing adoption of electric vehicles presents a significant opportunity for the power sector, but also introduces operational challenges for charging stations. These stations need to balance maximizing their own profits with providing a good quality of service (QoS) to EV drivers, all while dealing with fluctuating renewable energy sources and unpredictable EV demand. Traditional approaches often tackle pricing or charging control separately, or rely on static pricing, failing to capture the dynamic interplay between these factors and the nuanced behavior of EV drivers. Specifically, drivers are sensitive to price fluctuations, and their charging decisions are influenced by a complex mix of price, convenience, and the station's overall operational state. This paper aims to bridge this gap by developing a holistic approach that optimizes both pricing and charging strategies in real-time to achieve a win-win scenario for both the charging station and its users.

Methods

The core of the proposed solution is a two-phase online policy learning algorithm designed to solve a Markov Decision Process (MDP) formulated to maximize expected social welfare.

  • Phase I: Event-Based Policy Iteration for Pricing: This phase focuses on determining the optimal pricing strategy. Instead of considering every possible system state (which can be prohibitively large), it uses an event-based optimization (EBO) framework. Events are defined by specific transitions in system states that share common properties (e.g., charging station occupancy levels). This significantly reduces the state space. Within this framework, an event-based policy iteration is employed to iteratively refine the pricing policy, aiming to improve the expected social welfare.
  • Phase II: Scenario-Based Model Predictive Control (MPC) for Smart Charging: Once the optimal pricing for the current stage is determined, this phase implements scenario-based MPC for smart charging. This involves generating multiple future scenarios based on predicted EV arrivals and renewable energy generation. The MPC then optimizes the charging of EVs, the use of energy storage, and procurement from the grid to meet the demand dictated by the new pricing scheme, while minimizing costs and maximizing efficiency.

The paper also leverages performance difference theory to theoretically analyze the optimality of the proposed algorithm, providing a strong foundation for its effectiveness.

Datasets

  • Wind speed and solar radiation intensity data: Obtained from NREL Solar Radiation Research Laboratory [27].
  • Electricity price data: Used for the charging station's procurement costs.
  • EV trip distance and electric drive efficiency data: Used to model required charging energy [28].

The paper also uses a dataset of charging station capacities in Nanjing to inform the experimental setup, noting that most stations have fewer than 20 charging piles, which justifies the focus on a 20-pile station for experiments.

Key Insights and Ideas

  • Joint optimization is crucial: The paper strongly demonstrates that optimizing pricing and charging control simultaneously leads to significantly better outcomes than treating them independently.
  • Event-based optimization for scalability: By defining events based on system states rather than individual states, the approach becomes computationally tractable for complex, large-scale charging station operations.
  • Price fluctuation sensitivity matters: Explicitly incorporating the EV drivers' sensitivity to price fluctuations into the objective function is key to improving driver satisfaction and long-term station attractiveness.
  • Two-phase approach balances learning and execution: Separating pricing policy learning from smart charging execution allows for efficient real-time decision-making.
  • Trade-off between price and service: Lower prices attract more EVs but can strain resources and increase costs. Higher prices can increase profit per EV but deter drivers. The proposed method finds an optimal balance.

Key Results

  • The proposed two-phase algorithm achieves the highest social welfare compared to various baseline policies, including constant high/low pricing with greedy/delayed charging, and an event-driven pricing strategy.
  • The optimal pricing strategy significantly reduces the cost of power procurement from the grid by intelligently utilizing renewable energy and storage.
  • The algorithm demonstrates fast convergence in policy learning, with decision-making times deemed acceptable for practical implementation.
  • Sensitivity analysis shows that while increasing charging station capacity and arrival rates initially improve social welfare and profit, there are optimal points beyond which further increases yield diminishing returns or even negative impacts due to QoS costs.

Limitations and Future Work

  • Fixed charging power: The current model assumes a fixed charging power for EVs to prolong battery life. Future work could explore dynamic charging power control.
  • Simplified EV arrival model: While the model accounts for price sensitivity, more complex EV arrival behaviors could be investigated.
  • Single charging station focus: The current work focuses on a single charging station. Extending the framework to a network of interconnected charging stations, considering inter-station competition and coordination, would be a valuable next step.
  • Heuristic control for storage: While the MPC optimizes charging, the paper notes that the control of storage and EV charging can be heuristic. Further optimization of these components could yield additional improvements.

Summary updated

Abstract

The widespread adoption of EVs brings practical interest to the operation optimization of the charging station. This paper considers the joint scheduling of pricing and charging control to enhance the operational capability of the station. The following contributions are made. First, a joint scheduling model of pricing and charging control is developed to maximize the expected social welfare of the charging station considering the quality of service and the price fluctuation sensitivity of EV drivers. It is formulated as a Markov decision process with a variance criterion to capture uncertainties during operation. Second, a two-phase on-line policy learning algorithm is proposed to solve this joint scheduling problem. In the first phase, it implements event-based policy iteration to find the optimal pricing scheme. In the second phase, it implements scenario-based model predictive control for smart charging under the updated pricing scheme. Third, by leveraging the performance difference theory, the optimality of the proposed algorithm is theoretically analyzed. The feasibility of the proposed method and the improved social welfare of the charging station are numerically demonstrated based on a typical charging station with distributed generation and storage. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —The popularization of EVs requires the high-efficiency operation of the charging station. The joint scheduling of pricing and charging control can provide a promising way to achieve the balance between EV drivers’ satisfaction and the profit maximization of the charging station. However, it suffers from EV drivers’ uncertain responses to the pricing scheme and the coupled relationship between pricing and charging control. This multi-stage stochastic programming is non-trivial to solve. In this paper, we propose a two-phase on-line policy learning method for this joint scheduling problem. This algorithm can be implemented in the controller of the charging station. In the first phase, the event-based policy iteration can be implemented to iteratively improve the current best pricing scheme until convergence. In the second phase, the scenario-based model predictive control, which is reformulated as a mixed integer linear programming, can be quickly solved for smart charging. Case studies demonstrate the operation enhancement of the station.
BibTeX
@article{huang2024two,
  author = {Qilong Huang and Qing-Shan Jia and Xiang Wu and Xiaohong Guan},
  title = {{Two-Phase on-Line Joint Scheduling for Welfare Maximization of Charging Station}},
  journal = {IEEE Trans Autom. Sci. Eng.},
  year = {2024},
  doi = {10.1109/tase.2023.3316205},
}
Figure