Instance-Aware Predictive Navigation in Multi-Agent Environments

Jinkun Cao, Xin Wang, Trevor Darrell, Fisher Yu

Introduction

Navigating autonomous vehicles in dense, multi-agent environments remains a significant challenge. While standard Reinforcement Learning (RL) suffers from sparse rewards and Imitation Learning (IL) is often brittle in novel scenarios, predictive models offer a promising alternative by forecasting future states to inform planning. However, existing predictive frameworks often rely solely on scene-level semantic segmentation. In crowded traffic, these blurry segmentation maps frequently fail to capture the precise trajectories and collision risks of individual vehicles. This paper introduces Instance-Aware Predictive Control (IPC), a framework designed to bridge this gap by integrating instance-level tracking and prediction into the navigation pipeline.

Methodology

The IPC framework centers on two primary innovations: Multi-instance Event Prediction (MEP) and Sequential Action Sampling (SAS).

Multi-instance Event Prediction (MEP)

Unlike traditional models that predict a monolithic future image, the MEP module utilizes a RetinaNet-inspired architecture to detect individual agents and forecast their future 2D bounding boxes. For each instance, the model outputs a localization confidence score ($c$) and a collision probability ($\phi$), both conditioned on the ego-vehicle's intended actions. This approach allows the system to maintain multiple hypotheses for agent behavior, such as a vehicle choosing between turning left or right at an intersection.

Figure 6: The MEP module predicts individual agent bounding boxes and collision probabilities overlaid on scene-level semantic data.

Sequential Action Sampling (SAS)

To optimize driving decisions, the agent employs a two-stage Sequential Action Sampling strategy. In the first stage, the system prunes action sequences that lead to "obviously bad" outcomes based on scene-level events, such as driving off-road. In the second stage, the remaining candidates are refined using detailed instance-level predictions. The agent ultimately selects the path that minimizes the risk of collision with specific nearby vehicles. The entire model is trained via self-imitation, learning from its own successful trajectories without requiring expensive human-expert demonstrations.

Experimental Setup and Results

The researchers evaluated IPC in the CARLA simulator (Town01 and Town02) under high-density traffic conditions (32 to 64 vehicles) to simulate adversarial environments. To test generalization, the model was also deployed in GTA V, using a pre-trained Mask R-CNN to provide pseudo-supervision in the absence of ground-truth semantic data.

Figure 4: Instance detection and prediction in high-fidelity simulation environments.

Key Findings

  • Superior Performance: IPC consistently outperformed model-free baselines like DDPG and SAC, as well as previous state-of-the-art predictive models (SPC), achieving higher cumulative rewards and lower collision rates.
  • Robustness in Crowds: While performance typically degrades as traffic density increases, IPC demonstrated significantly more graceful degradation compared to scene-only models, proving the necessity of instance-level awareness for safety.
  • Explainability: By explicitly predicting bounding boxes and collision risks, IPC offers a level of interpretability absent in "black-box" RL policies. Observers can visualize the specific predicted vehicle trajectory that prompted the agent to brake or swerve.

Conclusion and Future Work

Instance-Aware Predictive Control represents a significant step forward in multi-agent autonomous navigation. By combining scene-level context with instance-specific forecasting, the model achieves state-of-the-art results in complex simulations. Future research will focus on extending the prediction horizon beyond the current 10-frame limit, transitioning from 2D bounding boxes to 3D spatial reasoning, and addressing the remaining sim-to-real gap for deployment on physical hardware.

Summary updated

Abstract

In this work, we aim to achieve efficient end-to-end learning of driving policies in dynamic multi-agent environments. Predicting and anticipating future events at the object level are critical for making informed driving decisions. We propose an Instance-Aware Predictive Control (IPC) approach, which forecasts interactions between agents as well as future scene structures. We adopt a novel multi-instance event prediction module to estimate the possible interaction among agents in the ego-centric view, conditioned on the selected action sequence of the ego-vehicle. To decide the action at each step, we seek the action sequence that can lead to safe future states based on the prediction module outputs by repeatedly sampling likely action sequences. We design a sequential action sampling strategy to better leverage predicted states on both scene-level and instance-level. Our method establishes a new state of the art in the challenging CARLA multi-agent driving simulation environments without expert demonstration, giving better explainability and sample efficiency.
BibTeX
@inproceedings{cao2021instance,
  author = {Jinkun Cao and Xin Wang and Trevor Darrell and Fisher Yu},
  title = {{Instance-Aware Predictive Navigation in Multi-Agent Environments}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2021},
  url = {https://openreview.net/forum?id=KWCgDoroZSz},
}
Figure