Abstract
This paper proposes a hierarchical coordination strategy for Vehicle-to-Grid (V2G) systems that leverages Deep Reinforcement Learning (DRL) and a Proof of Stake (PoS) algorithm to balance the competing interests of the power grid, aggregators, and electric vehicle (EV) owners. By integrating detailed battery health metrics—State of Health (SOH), State of Power (SOP), and State of Charge (SOC)—into the decision-making process, the system stabilizes the grid and reduces costs without sacrificing the long-term lifespan of vehicle batteries.
Introduction
Vehicle-to-Grid (V2G) technology offers a promising solution for grid stability by using electric vehicles as mobile energy storage units. However, practical implementation faces a multi-stakeholder challenge: the power grid requires stable loads, aggregators seek to minimize costs, and EV owners demand vehicle readiness while fearing battery degradation. Existing strategies often overlook the non-linear physics of battery health or fail to manage large fleets fairly. This study introduces the Multi-Stakeholder Hierarchical V2G Coordination (MHVCS) framework to address these conflicting objectives.
Methodology
The MHVCS framework operates on two distinct levels to manage complexity and ensure physical safety:
- Upper Level (Aggregator Coordination): A Proximal Policy Optimization (PPO) agent manages the total energy flow between the aggregator and the grid. The agent is trained with a multi-objective reward function that penalizes grid load variance and high operational costs while rewarding renewable energy utilization and battery health preservation.
- Lower Level (Individual Allocation): Inspired by blockchain consensus, a Proof of Stake (PoS) algorithm distributes the aggregator's total power requirement among individual EVs. Here, "stake" is defined by a vehicle's available battery energy, ensuring a fair distribution where vehicles with higher energy "collateral" contribute more to the V2G load.
- Battery Conditioning (SOX Model): The system incorporates a detailed model of State of Charge (SOC), State of Power (SOP), and State of Health (SOH). This series-parallel battery array model ensures that charging commands never violate physical safety limits or accelerate degradation unnecessarily.
Experimental Setup
The framework was evaluated in a simulated microgrid environment featuring 509 EVs with stochastic arrival and departure times. The grid data included 24-hour profiles for baseload, wind, and solar power. The MHVCS model was compared against four baselines: Uncontrolled Charging (BL1), Optimal Charging (BL2), Minimum Grid Fluctuation (BL3), and Minimum Cost (BL4).
Key Results
Grid Stability and Load Management
The MHVCS strategy demonstrated superior performance in flattening the load profile. It achieved a Load Variance (LV) of 71,312.8, a significant improvement over the 508,479.1 recorded for uncontrolled charging (BL1). As shown in the load profile comparison, the MHVCS strategy successfully kept the total load within the transformer constraints during peak hours.
Battery Longevity and Economic Efficiency
After a year of simulated operation, the MHVCS maintained a State of Health (SOH) of 97.13%. While slightly lower than pure charging strategies, it significantly outperformed the cost-minimization strategy (BL4, 96.76%), which tends to overstress batteries for profit. Economically, the system achieved a balanced total cost of $3,127.8, proving more sustainable than standard charging while providing essential grid services.
Conclusion
By combining DRL at the aggregator level with a deterministic PoS algorithm at the vehicle level, the MHVCS framework achieves high performance with low computational overhead. The inclusion of SOP and SOH constraints ensures that V2G participation does not come at the cost of battery safety or longevity, providing a viable path forward for multi-stakeholder V2G integration.
Summary updated