Deep Structured Reactive Planning

Jerry Liu, Wenyuan Zeng, Raquel Urtasun, Ersin Yumer

Abstract

An intelligent agent operating in the real-world must balance achieving its goal with maintaining the safety and comfort of not only itself, but also other participants within the surrounding scene. This requires jointly reasoning about the behavior of other actors while deciding its own actions as these two processes are inherently intertwined—a vehicle will yield to us if we decide to proceed first at an intersection but will proceed first if we decide to yield. However, this is not captured in most self-driving pipelines, where planning follows prediction. In this paper, we propose a novel data-driven, reactive planning objective which allows a self-driving vehicle (SDV) to jointly reason about its own plans as well as how other actors will react to them. We formulate the problem as an energy-based deep structured model that is learned from observational data and encodes both the planning and prediction problems. Through simulations based on both real-world driving and synthetically generated dense traffic, we demonstrate that our reactive model outperforms a non-reactive variant in successfully completing highly complex maneuvers faster, without trading off collision rate.

Introduction: The "Frozen Robot" Problem

Most current self-driving stacks operate sequentially: first, the car perceives the world; then, it predicts where others will go; finally, it plans its own path. The fatal flaw in this sequential approach is that prediction is often unconditioned on planning. In the real world, driving is a social negotiation. If an SDV waits for a "perfect" gap that never comes because it treats other cars as passive obstacles moving on fixed trajectories, it suffers from the "frozen robot" problem—becoming too timid to merge in heavy traffic.

Comparison of non-reactive vs. reactive planning. In the reactive model, prediction and planning are jointly reasoned, allowing the SDV to "nudge" and induce yielding behavior.

Methodology

The authors propose a Deep Structured Reactive Map, formulated as an energy-based model. Instead of separate modules, they define a joint probability distribution over the future trajectories of all actors (the SDV and surrounding vehicles) based on a set of learnable costs.

Energy Decomposition

The total "energy" (cost) of a scene is the sum of:

  • Actor-specific energy: How much a car likes its own path (e.g., smoothness, lane following).
  • Interaction energy: Costs associated with collisions or safety distance violations.

Inference via Loopy Belief Propagation

To make the complex joint reasoning tractable, the authors discretize possible futures into a set of $K$ trajectories. They use Loopy Belief Propagation (LBP) to efficiently approximate marginal probabilities. This allows the SDV to evaluate how its candidate plans change the probability distribution of what others will do, choosing the path that minimizes the expected joint energy.

Experimental Results

The model was trained and evaluated across three major environments: UrbanCity (real-world snippets), CARLA (high-density synthetic traffic), and nuScenes (standard motion forecasting benchmarks).

In closed-loop simulations, where other actors actually responded to the SDV's movements, the reactive model showed significant gains:

  • Higher Success Rates: In CARLA lane-merge scenarios, the reactive model achieved a 70% success rate, compared to just 46% for the non-reactive version.
  • Faster Completion: The reactive model completed maneuvers significantly faster without increasing the collision rate.
  • Competitive Prediction: Even when used solely for prediction, the model outperformed or matched state-of-the-art methods like DESIRE and R2P2.

Conclusion and Future Work

By treating planning and prediction as two sides of the same coin, the SDV shifts from a passive observer to an active participant capable of "nudging" traffic to achieve its goals safely. Future work aims to incorporate more nuanced social energies, such as "politeness," and to model interactions between multiple other actors to further improve performance in extremely dense urban environments.

Summary updated

Abstract

An intelligent agent operating in the real-world must balance achieving its goal with maintaining the safety and comfort of not only itself, but also other participants within the surrounding scene. This requires jointly reasoning about the behavior of other actors while deciding its own actions as these two processes are inherently intertwined - a vehicle will yield to us if we decide to proceed first at the intersection but will proceed first if we decide to yield. However, this is not captured in most self-driving pipelines, where planning follows prediction. In this paper we propose a novel data-driven, reactive planning objective which allows a self-driving vehicle to jointly reason about its own plans as well as how other actors will react to them. We formulate the problem as an energy-based deep structured model that is learned from observational data and encodes both the planning and prediction problems. Through simulations based on both real-world driving and synthetically generated dense traffic, we demonstrate that our reactive model outperforms a non-reactive variant in successfully completing highly complex maneuvers (lane merges/turns in traffic) faster, without trading off collision rate.
BibTeX
@inproceedings{liu2021deep,
  author = {Jerry Liu and Wenyuan Zeng and Raquel Urtasun and Ersin Yumer},
  title = {{Deep Structured Reactive Planning}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2021},
  url = {https://openreview.net/forum?id=JHWxxxKFnv},
}
Figure