TL;DR
This paper introduces an "Edge-Intelligent" framework that uses Deep Reinforcement Learning (DRL) to help Electric Vehicles (EVs) autonomously choose the best charging station in real-time. By deploying a centrally trained Deep Q-Network (DQN) to the "edge" (the vehicles themselves) via a lightweight MQTT communication backbone, the system improves energy throughput by over 5% in congested urban environments compared to traditional heuristics.
What Problem Does This Paper Address?
As Electric Vehicle adoption accelerates, urban charging infrastructure is struggling to keep up. In cities like Rabat, Morocco, public charging points are still relatively scarce, leading to significant risks of station congestion during peak hours.
Current management solutions generally fall into two camps:
- Centralized Approaches: These can find globally optimal assignments but suffer from high communication latency that doesn't scale well with thousands of IoT-connected vehicles.
- Rule-based Heuristics: These are fast and decentralized (e.g., "go to the nearest station") but are rigid and fail to adapt to complex, shifting traffic patterns.
The authors bridge this gap by proposing a decentralized architecture where vehicles make intelligent, local decisions based on a shared policy, avoiding the "bottleneck" of a central controller while remaining more adaptive than simple rules.
Methods
The core of the framework is a Deep Q-Network (DQN) agent. Unlike traditional Tabular Q-Learning, which struggles with the "curse of dimensionality," the DQN uses a neural network to handle a continuous 13-dimensional state vector. This vector includes:
- Vehicle Battery State of Charge (SoC).
- Road distances to four candidate stations.
- Current occupancy ratios at those stations.
- Current queue lengths.
The architecture is designed for Edge Intelligence. While the policy is trained centrally, the goal is for each EV to run inference locally. Communication is handled by an MQTT broker, a lightweight publish-subscribe protocol ideal for IoT, which carries telemetry data and reward signals.

The authors compared four policies: an Adaptive Heuristic baseline, Tabular Q-Learning, the proposed DQN, and a Double DQN (DDQN) variant. They used the SUMO (Simulation of Urban MObility) engine to model realistic traffic and station behavior, including a Markov failure process to simulate intermittent station outages.
Datasets
The study is grounded in real-world geography, utilizing road networks extracted from OpenStreetMap (OSM) for two Moroccan cities:
- Rabat Agdal–Hay Riad: The primary testbed, featuring 2,176 edges and 648 junctions.
- Tangier: Used to test the geographic robustness of the agents.
The simulation incorporates four real-world, geolocated public charging stations in Rabat with heterogeneous power levels (two 50 kW and two 150 kW units).
Key Insights and Ideas
- Queue-Awareness is Key: The DQN's primary advantage comes from its ability to distinguish between "occupancy" (who is currently charging) and "queue length" (who is waiting). This allows the agent to route vehicles to high-power stations (150 kW) preferentially when they are free, maximizing total energy delivery.
- Throughput vs. Latency: Interestingly, the DRL agents didn't significantly outperform the heuristic in terms of waiting time. Instead, their "aha moment" was in energy throughput. They were better at ensuring more vehicles finished their charges and received more total kWh, even if the individual wait times remained similar.
- The "Fidelity" Advantage: While Tabular Q-Learning can be made to scale by discretizing the state space, DQN offers higher "fidelity." It handles the continuous nature of distances and SoC directly, leading to more nuanced decision-making without the combinatorial explosion of a massive Q-table.
Key Results
The system was stress-tested under "Evening Peak" demand (the highest congestion scenario). Compared to the Adaptive Heuristic baseline:
- Energy Delivered: DQN achieved a 5.4% increase in total kWh delivered.
- Charge Completion Rate: DQN saw a 6.8% relative improvement.
- Vehicles Served: The number of vehicles successfully completing a charge increased by 6.9%.
- Inference Speed: Decisions were made in <0.5 ms, well within the 1-second real-time requirement for IoT systems.
Statistical significance was validated using the Wilcoxon signed-rank test across seven independent seeds, confirming that these gains weren't just "lucky" runs.
Limitations and Future Work
The authors acknowledge several areas for improvement:
- Sequential Curriculum Issues: The agents were trained on Morning, then Midday, then Evening scenarios. This meant the "Evening" training inherited a nearly exhausted exploration budget, which may have hindered its performance.
- Simulation vs. Reality: While the communication backbone (MQTT) was present, the evaluation used a shared policy in a single simulation process. True decentralized execution—where each EV has its own independent instance of the policy—is the next step.
- Noise and Faults: The current model assumes perfect SoC estimates and reliable communication. Future work will investigate how the system handles "noisy" battery data and broker unreachability.
- Scaling: The team plans to extend the campaign to larger synthetic networks with up to 32 stations to further test the limits of the DQN architecture.
Summary updated