Deep Structured Reactive Planning

Jerry Liu, Wenyuan Zeng, Raquel Urtasun, Ersin Yumer

TL;DR

Researchers from Uber ATG and the University of Toronto have developed a joint prediction and planning framework that allows self-driving vehicles (SDVs) to reason about how other drivers will react to their actions. By using an energy-based deep structured model, the SDV can perform "reactive planning," successfully navigating complex maneuvers like tight lane merges and unprotected turns far more effectively than traditional sequential systems.

What Problem Does This Paper Address?

Most current self-driving stacks operate like a relay race: first, the car perceives the world; then, it predicts where others will go; finally, it plans its own path. The fatal flaw in this sequential approach is that prediction is often unconditioned on planning.

In the real world, driving is a social negotiation. If you start to edge into a lane, the driver behind you might slow down to let you in. If you wait for a "perfect" gap that never comes, you’ll be stuck forever. Traditional "non-reactive" planners treat other cars as passive obstacles moving on fixed trajectories. This leads to the "frozen robot" problem—where the SDV is too timid to merge in heavy traffic—or unintended aggression because the model doesn't understand the causal link between its "nudge" and the other driver's response.

Methods

The authors propose a Deep Structured Reactive Map, formulated as an energy-based model. Instead of separate modules, they define a joint probability distribution over the future trajectories of all actors (the SDV and surrounding vehicles) based on a set of learnable costs.

Key technical components include:

  • Energy Decomposition: The total "energy" (cost) of a scene is the sum of actor-specific energy (how much a car likes its own path) and interaction energy (costs associated with collisions or safety distance violations).
  • Discretized Action Space: To make the complex math tractable, they discretize possible futures into a set of $K$ trajectories (lines, circles, spirals).
  • Loopy Belief Propagation (LBP): They use LBP to efficiently approximate marginal probabilities. This allows the SDV to evaluate how its candidate plans change the probability distribution of what others will do.
  • Reactive Objective: The planning goal is to minimize the expected joint energy. Essentially, the SDV chooses the path that results in the best overall outcome, assuming other actors will try to minimize their own costs in response to the SDV's move.

Datasets

The model was trained and evaluated across three major environments:

  1. UrbanCity: A large-scale, in-house dataset from Uber ATG featuring 6,500+ snippets of real-world driving in North American cities, including LiDAR and HD maps.
  2. CARLA: An open-source simulator used for high-density synthetic traffic scenarios.
  3. nuScenes: A public dataset used specifically to validate the model's performance on standard open-loop motion forecasting benchmarks.

Key Insights and Ideas

  • Planning and Prediction are Two Sides of the Same Coin: The authors argue that both should be derived from the same set of learnable costs. This ensures the SDV's "understanding" of others is consistent with its own "decision-making" logic.
  • The Power of the Nudge: The reactive model realizes that by initiating a maneuver, it can "induce" a yielding behavior in others. This is a fundamental shift from being a passive observer to an active participant in traffic.
  • Efficiency via LBP: By treating the problem as a structured graphical model and using LBP, they can perform these complex calculations fast enough for real-time online planning, which is usually the bottleneck for game-theoretic approaches.

Key Results

The researchers tested the model in "closed-loop" simulations, meaning the other actors actually responded to the SDV's movements.

  • Higher Success Rates: In CARLA lane-merge scenarios, the reactive model achieved a 70% success rate, compared to just 46% for the non-reactive version.
  • Faster Completion: The reactive model completed maneuvers significantly faster (lower Time To Completion) without increasing the collision rate.
  • Competitive Prediction: Even when used just for predicting other cars (without the planning component), the model outperformed or matched state-of-the-art methods like DESIRE and R2P2 on the nuScenes and CARLA benchmarks.

Limitations and Future Work

  • Actor-Actor Interactions: For computational efficiency, the authors excluded the energy terms representing interactions between two other actors (e.g., how car A reacts to car B because of the SDV). They suggest approximating this in the future by only considering neighbors.
  • Discrete Trajectories: The model relies on a predefined set of trajectory samples. While effective, it may not capture the full fluidity of continuous motion in extremely edge-case scenarios.
  • Social Norms: The current model focuses on safety and goal-reaching. Future work could incorporate more nuanced social energies, such as lane-following or "politeness" (avoiding making others brake too hard), which are currently not explicitly encoded.

Summary updated

Abstract

An intelligent agent operating in the real-world must balance achieving its goal with maintaining the safety and comfort of not only itself, but also other participants within the surrounding scene. This requires jointly reasoning about the behavior of other actors while deciding its own actions as these two processes are inherently intertwined – a vehicle will yield to us if we decide to proceed first at the intersection but will proceed first if we decide to yield. However, this is not captured in most self-driving pipelines, where planning follows prediction. In this paper we propose a novel data-driven, reactive planning objective which allows a self-driving vehicle to jointly reason about its own plans as well as how other actors will react to them. We formulate the problem as an energy-based deep structured model that is learned from observational data and encodes both the planning and prediction problems. Through simulations based on both real-world driving and synthetically generated dense traffic, we demonstrate that our reactive model outperforms a non-reactive variant in successfully completing highly complex maneuvers (lane merges/turns in traffic) faster, without trading off collision rate. Please see our supplementary document https://tinyurl.com/3nukpn5b for all additional details.
BibTeX
@inproceedings{liu2021deep,
  author = {Jerry Liu and Wenyuan Zeng and Raquel Urtasun and Ersin Yumer},
  title = {{Deep Structured Reactive Planning}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2021},
  url = {https://openreview.net/forum?id=JHWxxxKFnv},
}

Loading related content...

Figure