Online Vehicle Trajectory Prediction using Policy Anticipation Network and optimization-based Context Reasoning

Wenchao Ding, Shaojie Shen

Introduction

Predicting vehicle trajectories in dense urban environments is a critical challenge for autonomous driving. Two primary factors complicate this task: multimodality and complex environmental context. At intersections, vehicles often face multiple valid paths (e.g., turning left vs. going straight), and their movement is strictly governed by lane geometries, traffic signals, and the presence of other agents. Standard regression models often fail to capture these discrete shifts in intent, sometimes predicting physically impossible "average" paths.

Visualization of the complex urban environment in the CARLA simulator, highlighting various contextual factors like traffic signs, road construction, and moving agents.

Proposed Framework

To address these challenges, this paper introduces a hierarchical "Online Two-Level" framework that mimics human driving logic by separating high-level intent from low-level physical execution.

The proposed two-level framework: a Policy Anticipation Network for high-level intent and Optimization-based Context Reasoning for low-level trajectory refinement.

Level 1: High-Level Policy Anticipation

The first stage utilizes an RNN-based encoder-decoder (LSTM) to process a vehicle's history and predict a discrete driving policy, such as forward, yield, or turn left. By identifying the most likely intention first, the system effectively prunes the search space and avoids the pitfalls of multimodal averaging. This predicted policy is then mapped to the local environment to generate a rough "reference trajectory."

Level 2: Low-Level Context Reasoning

The second stage employs non-linear optimization (via the Ceres solver) to refine the reference trajectory into a smooth, feasible path. This process utilizes a multi-layer cost map that accounts for:

  • Static Constraints: Lane boundaries and physical obstacles.
  • Dynamic Constraints: Predicted positions of other moving objects (MO).
  • Traffic Regulations: Soft constraints for rules like red lights, allowing the model to predict trajectories even for law-breaking vehicles.
  • Nonholonomic Constraints: The physical limits of vehicle acceleration and curvature.

Experimental Setup and Results

The framework was evaluated using the CARLA autonomous driving simulator (v0.7.0). The researchers collected over 21,000 frames of data, including both standard driving and traffic violations, to ensure model robustness. Both observation and prediction horizons were set to 4 seconds.

Key results include:

  • Significant Accuracy Gains: The proposed method achieved a 44% accuracy improvement on curved roads and a 56% improvement at intersections compared to standard RNN baselines.
  • Context Sensitivity: The model successfully predicted "hard braking" for pedestrians, a scenario where baseline models often incorrectly predicted continued forward motion.
  • Real-Time Performance: The entire pipeline—including inference, cost map rendering, and optimization—averages 22.6 ms, making it highly suitable for online deployment.

Conclusion and Future Work

By treating trajectory prediction as an optimization problem rather than a black-box regression, this framework provides a transparent and flexible way to incorporate complex environmental rules. Future work aims to replace manual weight tuning for cost terms with Inverse Reinforcement Learning (IRL) and to further explore complex multi-agent interactions where vehicles' intentions are mutually influential.

Summary updated

BibTeX
@inproceedings{ding2019online,
  author = {Wenchao Ding and Shaojie Shen},
  title = {{Online Vehicle Trajectory Prediction using Policy Anticipation Network and optimization-based Context Reasoning}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2019},
  url = {https://openreview.net/forum?id=gmwYFYu1ytdL},
}
Figure