Abstract
This study evaluates Reinforcement Learning (RL) policies for autonomous vehicles (AVs) navigating a real-world roundabout in Milan, Italy. Utilizing a "human-in-the-loop" framework, the research combines high-fidelity traffic and driving simulators with a physical cockpit to assess both quantitative traffic metrics and qualitative human perception. Results indicate that increasing AV penetration significantly reduces travel time, fuel consumption, and emissions. Crucially, human drivers reported feeling safer and perceiving traffic as smoother in high-AV scenarios, suggesting that optimized RL policies create positive "ripple effects" for all road users.
Introduction
The deployment of autonomous vehicles (AVs) in mixed-traffic environments presents significant challenges, particularly in complex urban layouts like roundabouts. While Reinforcement Learning (RL) has proven effective for training AV policies in isolated simulations, these "vacuum" environments often fail to account for the unpredictable behavior of human-driven vehicles (HVs) and the subjective comfort of human passengers.
A critical concern in mixed autonomy is the "sim-to-real" gap: will a mathematically optimal policy for an AV be perceived as erratic or dangerous by a human driver in an adjacent lane? This research addresses these questions by conducting a dual-mode evaluation—quantitative analysis of traffic flow and qualitative study of human perception—using a professional-grade driving simulator at the Polytechnic University of Milan.
Methods
The researchers developed a multi-layered co-simulation framework to bridge the gap between AI training and human experience.
Simulation Framework
- Traffic Modeling: SUMO (Simulation of Urban MObility) was used for microscopic traffic modeling, handling the physics and logic of multiple interacting agents.
- Visual Environment: VI-WorldSim provided a high-fidelity graphical environment for the driving experience.
- Physical Hardware: The DrisMi Lab at the Polytechnic University of Milan provided a cable-driven cockpit with realistic motion feedback, allowing human participants to interact with the AI-controlled agents in real-time.
RL Policy and Calibration
The AVs were trained using Proximal Policy Optimization (PPO) with a multi-objective reward function designed to minimize crossing time and polluting emissions. To ensure realism, the simulation was calibrated using field measurements from a specific roundabout in Milan, including queue lengths and flow rates for cars, pedestrians, and bicycles. A key technical achievement was the synchronized co-simulation between SUMO and VI-WorldSim, maintaining a latency of just 5ms to prevent "simulator sickness" or lag for the human driver.
Results and Discussion
The study revealed that AVs do not merely optimize their own performance; they act as "traffic pacers" that improve the environment for human drivers.
Quantitative Performance
As AV penetration increased, significant improvements were observed across all metrics:
- Efficiency: At 100% AV penetration, crossing times for AVs were reduced by 10.72%. Interestingly, HVs also saw an 8.52% reduction in travel time, benefiting from the smoother flow established by the AI.
- Sustainability: The RL policy's focus on smooth acceleration led to a 39% reduction in emissions and a 35% drop in fuel consumption at full penetration. Even at 50% penetration, total emissions decreased by approximately 25-30%.
Qualitative Human Perception
The "human-in-the-loop" study provided unique insights into how people perceive mixed autonomy:
- Safety and Predictability: 80% of participants felt that scenarios with high AV penetration (80%) were safer than those with low penetration (20%).
- Preference: 70% of participants preferred driving in the high-AV environment, with 50% specifically noting that the traffic felt "smoother." This suggests that RL policies optimized for efficiency also tend to produce more predictable and comfortable behaviors for human observers.
Conclusion
This research demonstrates that RL-trained AVs can significantly enhance urban traffic efficiency and sustainability while maintaining—and even improving—human comfort and perceived safety. The findings highlight the importance of "altruistic" AI behavior in mixed-traffic scenarios. Future work will focus on expanding the participant pool for qualitative studies and testing the robustness of these policies across diverse urban layouts and adverse weather conditions.
Summary updated