A Deep Reinforcement Learning Framework for Real-Time Electric Vehicle Charging Management in IoEV-Enabled Smart Cities

Nour-Eddine Moumni, Rachid Alaoui, Driss KiouacharXiv 2026 · arXiv

TL;DR

This paper introduces an "Edge-Intelligent" framework that uses Deep Reinforcement Learning (DRL) to help Electric Vehicles (EVs) autonomously choose the best charging station in real-time. By deploying a centrally trained Deep Q-Network (DQN) to the "edge" (the vehicles themselves) via a lightweight MQTT communication backbone, the system improves energy throughput by over 5% in congested urban environments compared to traditional heuristics.

What Problem Does This Paper Address?

As Electric Vehicle adoption accelerates, urban charging infrastructure is struggling to keep up. In cities like Rabat, Morocco, public charging points are still relatively scarce, leading to significant risks of station congestion during peak hours.

Current management solutions generally fall into two camps:

  1. Centralized Approaches: These can find globally optimal assignments but suffer from high communication latency that doesn't scale well with thousands of IoT-connected vehicles.
  2. Rule-based Heuristics: These are fast and decentralized (e.g., "go to the nearest station") but are rigid and fail to adapt to complex, shifting traffic patterns.

The authors bridge this gap by proposing a decentralized architecture where vehicles make intelligent, local decisions based on a shared policy, avoiding the "bottleneck" of a central controller while remaining more adaptive than simple rules.

Table 1. Comparison with related work on EV charging assignment. R=Real OSM, S=Synthetic, ✓=present, ×=absent.

Methods

The core of the framework is a Deep Q-Network (DQN) agent. Unlike traditional Tabular Q-Learning, which struggles with the "curse of dimensionality," the DQN uses a neural network to handle a continuous 13-dimensional state vector. This vector includes:

  • Vehicle Battery State of Charge (SoC).
  • Road distances to four candidate stations.
  • Current occupancy ratios at those stations.
  • Current queue lengths.

The architecture is designed for Edge Intelligence. While the policy is trained centrally, the goal is for each EV to run inference locally. Communication is handled by an MQTT broker, a lightweight publish-subscribe protocol ideal for IoT, which carries telemetry data and reward signals.

![Fig.

  1. Edge-Intelligent EV charging framework architecture.](/figures/art_6c3fd6c51ba3425b834ea9d5961a42f9)

The authors compared four policies: an Adaptive Heuristic baseline, Tabular Q-Learning, the proposed DQN, and a Double DQN (DDQN) variant. They used the SUMO (Simulation of Urban MObility) engine to model realistic traffic and station behavior, including a Markov failure process to simulate intermittent station outages.

Datasets

The study is grounded in real-world geography, utilizing road networks extracted from OpenStreetMap (OSM) for two Moroccan cities:

  • Rabat Agdal–Hay Riad: The primary testbed, featuring 2,176 edges and 648 junctions.
  • Tangier: Used to test the geographic robustness of the agents.

The simulation incorporates four real-world, geolocated public charging stations in Rabat with heterogeneous power levels (two 50 kW and two 150 kW units).

Table 2. Real EV charging stations in Rabat Agdal / Hay Riad.

Key Insights and Ideas

  • Queue-Awareness is Key: The DQN's primary advantage comes from its ability to distinguish between "occupancy" (who is currently charging) and "queue length" (who is waiting). This allows the agent to route vehicles to high-power stations (150 kW) preferentially when they are free, maximizing total energy delivery.
  • Throughput vs. Latency: Interestingly, the DRL agents didn't significantly outperform the heuristic in terms of waiting time. Instead, their "aha moment" was in energy throughput. They were better at ensuring more vehicles finished their charges and received more total kWh, even if the individual wait times remained similar.
  • The "Fidelity" Advantage: While Tabular Q-Learning can be made to scale by discretizing the state space, DQN offers higher "fidelity." It handles the continuous nature of distances and SoC directly, leading to more nuanced decision-making without the combinatorial explosion of a massive Q-table.

Key Results

The system was stress-tested under "Evening Peak" demand (the highest congestion scenario). Compared to the Adaptive Heuristic baseline:

  • Energy Delivered: DQN achieved a 5.4% increase in total kWh delivered.
  • Charge Completion Rate: DQN saw a 6.8% relative improvement.
  • Vehicles Served: The number of vehicles successfully completing a charge increased by 6.9%.
  • Inference Speed: Decisions were made in <0.5 ms, well within the 1-second real-time requirement for IoT systems.

Statistical significance was validated using the Wilcoxon signed-rank test across seven independent seeds, confirming that these gains weren't just "lucky" runs.

Limitations and Future Work

The authors acknowledge several areas for improvement:

  • Sequential Curriculum Issues: The agents were trained on Morning, then Midday, then Evening scenarios. This meant the "Evening" training inherited a nearly exhausted exploration budget, which may have hindered its performance.
  • Simulation vs. Reality: While the communication backbone (MQTT) was present, the evaluation used a shared policy in a single simulation process. True decentralized execution—where each EV has its own independent instance of the policy—is the next step.
  • Noise and Faults: The current model assumes perfect SoC estimates and reliable communication. Future work will investigate how the system handles "noisy" battery data and broker unreachability.
  • Scaling: The team plans to extend the campaign to larger synthetic networks with up to 32 stations to further test the limits of the DQN architecture.

Summary updated

Abstract

The rapid growth of electric vehicles (EVs) in urban environments demands intelligent, decentralised charging management strategies that operate under strict real-time constraints. This paper proposes an Edge-Intelligent architecture in which each EV selects a charging station (CS) by querying a shared, centrally trained Deep Q-Network (DQN) policy over a lightweight MQTT publish-subscribe backbone; independent per-EV edge inference is the deployment target, not the configuration evaluated here. The reported simulation evaluates that shared DQN policy applied sequentially to each vehicle's observations within a single SUMO run, using MQTT for KPI telemetry while observations are obtained through TraCI. The system is evaluated on two real-world road networks from OpenStreetMap (Rabat Agdal-Hay Riad and Tangier, Morocco); the primary network carries four geolocated public charging stations with heterogeneous power levels (50 kW and 150 kW). Three traffic demand profiles (morning, midday, evening peak) are simulated, and four decision policies are compared: an Adaptive Heuristic, a Tabular Q-Learning agent, the proposed DQN agent, and a Double DQN variant. A multi-seed statistical protocol (7 seeds, Wilcoxon signed-rank test, Cohen's d) evaluates the results. The DQN agent achieves a 6.8% relative improvement in charge rate and a 5.4% increase in delivered energy over the Adaptive Heuristic under Evening Peak demand ($p = 0.016$, Cohen's $d = 0.99$ and $1.51$), while showing no advantage in waiting time; on the second network the heuristic leads on waiting time, so the ranking is not stable across geographies. A scalability analysis shows linear state-space growth with station count and sub-millisecond inference; all runs include intermittent station outages under a Markov failure process.
BibTeX
@misc{moumni2026deep,
  author = {Nour-Eddine Moumni and Rachid Alaoui and Driss Kiouach},
  title = {{A Deep Reinforcement Learning Framework for Real-Time Electric Vehicle Charging Management in IoEV-Enabled Smart Cities}},
  year = {2026},
  eprint = {2611.01751},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.01751},
}
Figure