Abstract
Predicting the future trajectories of multiple interacting agents is a critical task for autonomous driving. Current methods often suffer from two primary limitations: computational complexity that scales linearly with the number of agents ($O(n)$) and a lack of diversity in training data, which is typically dominated by simple, straight-line driving. We present SMART, a framework that addresses these challenges through a novel scene-centric architecture and an automated simulation pipeline for diverse behavior generation. SMART achieves constant-time inference regardless of agent count and significantly improves prediction accuracy for complex maneuvers.
Introduction
Autonomous vehicles must navigate complex environments where the future actions of surrounding drivers are uncertain. To ensure safety, a prediction system must be both fast and capable of representing multiple possible futures (multimodality). However, traditional agent-centric models process each vehicle individually, leading to latency issues in crowded scenes. Furthermore, real-world datasets often lack sufficient examples of rare but critical maneuvers, such as sharp turns or aggressive lane changes. SMART overcomes these hurdles by treating the entire scene as a single grid and augmenting training data with realistic, simulated interactions.
The SMART Architecture
At the core of SMART is a scene-centric approach that utilizes a Bird’s-Eye-View (BEV) grid representation. Instead of modeling agents as isolated entities, the system processes the entire environment as a multi-layered image, allowing it to implicitly learn social interactions through convolutional operations.
Key Innovations:
- ConvLSTM with State Pooling: Grid-based models often struggle with "jitter" as agents move between cells. SMART introduces a state-pooling mechanism that transfers an agent's internal memory state to its new predicted location, ensuring temporal consistency.
- Conditional Variational Autoencoder (CVAE): To capture multimodality, the model uses a CVAE to sample various potential behaviors (e.g., turning vs. going straight) from a latent space.
- Constant-Time Inference: By processing the entire grid in a single forward pass, SMART maintains a fixed inference time (approx. 0.07s), making it highly suitable for real-time applications.
Diverse Behavior Simulation
To address the "straight-line bias" in existing datasets, the authors developed an automated simulation pipeline. This system mines real-world maps and trajectories to extract realistic velocity profiles and then applies advanced vehicle dynamics models to create new, challenging scenarios.
The simulation utilizes the Intelligent Driver Model (IDM) for longitudinal control and MOBIL for lane-changing logic. This allows for the generation of thousands of physically plausible scenarios—such as complex intersection crossings—that are consistent with road geometry and traffic rules. Training on this diverse data enables the model to better generalize to real-world edge cases.
Experimental Results
SMART was evaluated on major benchmarks, including Argoverse and KITTI, as well as augmented versions (P-ArgoT and P-KITTI) designed to test maneuver diversity.
- Efficiency: While baseline models like Social GAN see processing times triple as the number of agents increases, SMART maintains a constant 0.07-second latency for up to 10 agents.
- Accuracy: On the Argoverse dataset, SMART achieved a Final Displacement Error (FDE) of 1.02 meters at 5 seconds, a significant improvement over the 2.54 meters recorded by Social GAN.
- Scene Compliance: Predictions generated by SMART are more likely to respect road boundaries and lane constraints compared to other generative models, which often produce trajectories that drift off-road.
Conclusion
SMART represents a significant step forward in multi-agent trajectory prediction. By shifting from an agent-centric to a scene-centric paradigm and leveraging high-fidelity simulation for data augmentation, the framework provides a scalable and robust solution for autonomous navigation. Future work will focus on integrating 3D perspectives and more explicit traffic rule constraints to further refine prediction reliability in complex urban environments.
Summary updated