Unveiling Uncertainty-Aware Autonomous Cooperative Learning Based Planning Strategy

Shiyao Zhang, Shiyao Zhang 0001, Liwei Deng, Shuyu Zhang, Weijie Yuan, Hong ZhangIEEE RA-L 2025 · DOI

TL;DR

This paper introduces a novel Deep Reinforcement Learning (DRL) framework called DRLACP that effectively handles uncertainties in autonomous cooperative planning (ACP) for multi-vehicle systems. By integrating perception and communication uncertainties into a Soft Actor-Critic (SAC) algorithm enhanced with Gated Recurrent Units (GRUs), DRLACP enables robust and safe cooperative motion planning even in dynamic and imperfect environments.

What Problem Does This Paper Address?

Future intelligent transportation systems rely heavily on autonomous cooperative planning (ACP) to enhance efficiency and safety in multi-vehicle interactions, such as platooning. However, existing ACP strategies struggle to adequately address various uncertainties inherent in these systems. These uncertainties include:

  • Perception Uncertainty: Errors in sensors (like LiDAR) due to environmental conditions (e.g., extreme weather) or limitations of perception algorithms can lead to inaccurate understanding of the vehicle's surroundings.
  • Planning Uncertainty: Imperfect state information can arise from the planning process itself.
  • Communication Uncertainty: Even with improved multi-vehicle perception, communication outages or unreliable data transmission between vehicles (V2V) can disrupt information fusion and coordinated decision-making.

These uncertainties can significantly degrade the performance and safety of cooperative driving. While some prior work has addressed individual uncertainties or focused on single-vehicle autonomous driving, there's a need for a comprehensive framework that integrates multiple uncertainties into a unified cooperative planning strategy.

Methods

The paper proposes a Deep Reinforcement Learning-based Autonomous Cooperative Planning (DRLACP) framework. The core of this framework is a Soft Actor-Critic (SAC) algorithm augmented with Gated Recurrent Units (GRUs), termed GRU-SAC.

Here's a breakdown of the key methodological components:

  • Uncertainty Modeling:
    • Perception Uncertainty: Modeled using a stochastic set representing deviations in detected object positions, influenced by LiDAR sensor limitations and confidence scores.
  • Communication Uncertainty: Modeled using a communication outage function based on channel quality and outage probability, particularly considering a downlink RSMA scheme.
  • Collision-Free Model: Incorporates affine constraints for Forward Collision Avoidance Constraint (FCAC) and Rear Collision Avoidance Constraint (RCAC) to ensure safe distances between vehicles.
  • GRU-SAC Algorithm:
    • Actor Network: Uses a GRU to process sequential states and generate a stochastic policy for actions. The GRU's ability to capture temporal dependencies is crucial for handling time-varying driving environments.
  • Critic Networks: Two independent critic networks, also utilizing GRUs, estimate Q-values. This allows for learning from historical observations and improving decision-making.
  • Temporal Modeling: The GRU's recurrent nature allows the agent to learn from past observations and predict future actions, which is vital for dynamic scenarios.
  • Problem Formulation: The cooperative motion planning problem is formulated as a learning-based cooperative model predictive control (MPC) problem, aiming to minimize trajectory tracking error, control effort, and collision risks. The reward function is derived from this optimization objective.

Datasets

The paper does not explicitly mention specific datasets used for training. However, the evaluation is performed using the CARLA simulation platform, which is a well-established open-source simulator for autonomous driving research. The authors also mention using "real driving data" for training an LSTM-based model in a related work, but this is not directly applied to their proposed DRLACP.

Key Insights and Ideas

  • Unified Uncertainty Handling: The most significant insight is the successful integration of both perception and communication uncertainties into a single, coherent DRL framework for cooperative planning. This moves beyond addressing uncertainties in isolation.
  • GRU's Advantage in Temporal Dynamics: The use of GRUs in both actor and critic networks proves effective in capturing temporal dependencies in driving scenarios. This leads to more accurate predictions and robust decision-making compared to standard MLP-based approaches.
  • Robustness to Dynamic and Imperfect Environments: The DRLACP framework demonstrates strong performance not only under typical uncertainties but also when faced with sudden environmental changes, such as unexpected lane closures.
  • Balancing Safety and Efficiency: The proposed method achieves a high success rate and efficient maneuver execution while maintaining safety, indicating a good trade-off between competing objectives.

Key Results

  • 100% Success Rate: The DRLACP framework achieved a 100% success rate in all simulation trials, outperforming baseline methods like TCMPC (90%) and SEMPC (85%).
  • Superior Velocity and Safety: DRLACP attained the highest average velocity, indicating efficient maneuver execution, and maintained stable lane-following behavior without collisions.
  • Reduced Prediction Errors: GRU-SAC consistently yielded lower prediction errors than MLP-SAC across various prediction window lengths, highlighting the benefit of temporal modeling.
  • Computational Efficiency: DRLACP achieved significantly lower computation time compared to optimization-based baselines, making it suitable for real-time applications.
  • Robustness Demonstrated: Visualizations in CARLA showed the framework's ability to handle complex multi-vehicle lane changes and react effectively to unexpected obstacles.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that the current work focuses on a specific communication model (RSMA) and could be extended to other communication protocols.
  • While CARLA is a powerful simulator, real-world deployment will introduce additional complexities not fully captured in simulation.
  • Future Work:
    • The authors suggest extending the framework to longer-horizon temporal modeling to further enhance decision-making capabilities for real-world AV applications.
  • Exploring other communication protocols and more complex uncertainty models is also a potential avenue.

Summary updated

BibTeX
@article{zhang2025unveiling,
  author = {Shiyao Zhang and Shiyao Zhang and Liwei Deng and Shuyu Zhang and Weijie Yuan and Hong Zhang},
  title = {{Unveiling Uncertainty-Aware Autonomous Cooperative Learning Based Planning Strategy}},
  journal = {IEEE RA-L},
  year = {2025},
  doi = {10.1109/lra.2025.3621973},
}
Figure