Learning-based cooperative decision-making and control for multiple autonomous vehicles in unsignalized intersections

Ronghua Zhang, Xin Xu, Yang Lu, Xinglong Zhang, Qingwen MaIntelligence 2025 · DOI

Abstract

Cooperative navigation of multiple autonomous vehicles (MAVs) at unsignalized intersections remains a core challenge in intelligent transportation systems. This paper proposes a learning-based cooperative decision-making and control (LCDMC) method for MAVs, which improves policy learning efficiency and ensures safe and efficient cooperative navigation. In the proposed LCDMC algorithm, the global value function is decomposed into two components: a local utility function and a joint-action utility function among vehicles, which incorporates both the offline policy learning phase and the online deployment phase. During the offline phase, the kernel-based least-squares policy iteration method is employed to learn localized decision-making policies from high-dimensional samples. In the online deployment phase, a coordination graph for MAVs is developed, and a collaborative utility function characterizing joint action performance is designed. To solve optimized decision actions, the local utility function is integrated with a message propagation mechanism, and then the decision actions are converted into velocity commands. Furthermore, a receding-horizon reinforcement learning approach is designed to achieve trajectory tracking control of the autonomous vehicles in MAVs. Finally, to verify the effectiveness of the proposed method, numerical simulations of MAVs are performed, and the results demonstrate that the proposed LCDMC method exhibits superior performance in both traffic efficiency and safety for cooperative navigation of MAVs at unsignalized intersections.

Introduction

Unsignalized intersections represent some of the most high-risk and complex environments for autonomous driving. Unlike structured highway environments, these zones require intricate social negotiation, precise timing, and a deep understanding of neighboring vehicles' intentions. Current solutions generally fall into two categories: rule-based systems, which are often too rigid to handle the infinite variety of real-world traffic scenarios, and Multi-Agent Reinforcement Learning (MARL), which frequently struggles with the "curse of dimensionality" and environmental non-stationarity.

This paper introduces the Learning-based Cooperative Decision-Making and Control (LCDMC) framework. LCDMC aims to bridge the gap between these approaches by creating a scalable, learning-based method that allows vehicles to cooperate explicitly through a decomposed global value function, rather than merely reacting to one another as obstacles.

Methodology

The LCDMC algorithm is structured into two distinct operational phases: offline policy learning and online deployment.

Offline Policy Learning

To ensure computational efficiency, the framework utilizes Kernel-based Least-Squares Policy Iteration (KLSPI) rather than massive deep neural networks. The process involves:

  • Dimensionality Reduction: Using Approximate Linear Dependency (ALD) analysis to construct a "kernel dictionary," which extracts the most critical features from high-dimensional sensor data.
  • Local Utility Development: Each vehicle learns a local utility function that evaluates the desirability of specific actions (e.g., accelerate, decelerate, or maintain speed) based on its individual progress and state.

Online Deployment and Coordination

When vehicles approach an intersection, they transition from individual decision-making to a collaborative "negotiation" phase:

  • Coordination Graphs: The system dynamically constructs a graph where vehicles are nodes and potential path intersections are edges. These edges form only when trajectories are predicted to overlap.
  • Max-Plus Message Passing: Vehicles exchange messages via the coordination graph to identify a joint action that maximizes safety and traffic flow for the entire group. This avoids the exponential complexity of checking every possible combination of actions.
  • Receding-Horizon Reinforcement Learning (RHRL): Once a target velocity is decided, this low-level controller ensures the vehicle's physical actuators follow the trajectory precisely while maintaining passenger comfort.

Experimental Results

The effectiveness of LCDMC was verified through numerical simulations involving 30,000 pre-collected samples per vehicle. The framework was tested against Independent Learning (IL) models in three scenarios: a basic T-junction, a complex T-junction, and a high-density cross-intersection.

Safety Performance

In the T-junction and cross-intersection scenarios, LCDMC achieved a 0% collision rate. In stark contrast, the Independent Learning approach suffered collision rates ranging from 22.8% to 26%, highlighting the necessity of explicit cooperation in high-conflict zones.

Traffic Efficiency

LCDMC significantly reduced the time required to clear intersections. In the T-junction scenario, LCDMC-controlled vehicles cleared the zone in approximately 7.4 seconds, while IL agents required 8.8 seconds. Furthermore, LCDMC demonstrated much lower standard deviation in traversal times, indicating smoother and more predictable traffic flow compared to the erratic behavior of selfish agents.

Conclusion

The LCDMC framework provides a robust solution for the cooperative navigation of MAVs at unsignalized intersections. By decomposing global objectives into local and joint utilities and utilizing dynamic coordination graphs, the system achieves near-perfect safety and superior traffic efficiency. Future research will address the challenges of communication reliability (e.g., packet loss and latency) and the integration of these systems into mixed-traffic environments containing human-driven vehicles.

Summary updated

Abstract

Cooperative navigation of multiple autonomous vehicles (MAVs) at unsignalized intersections remains a core challenge in intelligent transportation systems. This paper proposes a learning-based cooperative decision-making and control (LCDMC) method for MAVs, which improves policy learning efficiency and ensures safe and efficient cooperative navigation. In the proposed LCDMC algorithm, the global value function is decomposed into two components: a local utility function and a joint-action utility function among vehicles, which incorporates both the offline policy learning phase and the online deployment phase. During the offline phase, the kernel-based least-squares policy iteration method is employed to learn localized decision-making policies from high-dimensional samples. In the online deployment phase, a coordination graph for MAVs is developed, and a collaborative utility function characterizing joint action performance is designed. To solve optimized decision actions, the local utility function is integrated with a message propagation mechanism, and then the decision actions are converted into velocity commands. Furthermore, a receding-horizon reinforcement learning approach is designed to achieve trajectory tracking control of the autonomous vehicles in MAVs. Finally, to verify the effectiveness of the proposed method, numerical simulations of MAVs are performed, and the results demonstrate that the proposed LCDMC method exhibits superior performance in both traffic efficiency and safety for cooperative navigation of MAVs at unsignalized intersections.
BibTeX
@inproceedings{zhang2025learning,
  author = {Ronghua Zhang and Xin Xu and Yang Lu and Xin Xu and Xinglong Zhang and Qingwen Ma},
  title = {{Learning-based cooperative decision-making and control for multiple autonomous vehicles in unsignalized intersections}},
  booktitle = {Intelligence},
  year = {2025},
  doi = {10.20517/ir.2025.36},
}
Figure