Introduction
In autonomous driving, predicting the future trajectories of surrounding vehicles is a critical task for safety and planning. While deep learning (DL) models have achieved state-of-the-art performance on standard benchmarks, they often suffer from a significant "generalization gap" when deployed in new environments. A model trained on data from one city (e.g., San Francisco) may perform poorly in another (e.g., Pittsburgh) due to differences in sensor setups, road layouts, and local driving behaviors. This paper introduces Adaptive Prediction Ensemble (APE), a framework designed to bridge this gap by intelligently combining the strengths of neural networks and robust physics-based models.
The Challenge of Generalization
Surprisingly, in out-of-distribution (OOD) scenarios, complex neural networks are often outperformed by simple Constant Velocity (CV) models. While DL models excel at capturing complex maneuvers in familiar settings, they can overfit to training nuances, leading to erratic predictions when faced with unfamiliar data. In contrast, CV models provide a reliable, albeit simple, baseline that adheres to basic physical constraints.
As shown in the figure above, a significant number of scenarios exist where the simple Constant Velocity model achieves a lower Average Displacement Error (ADE) than a state-of-the-art deep neural network. This observation motivates the need for a system that can dynamically choose the most appropriate model for a given situation.
The APE Framework
APE is a hybrid framework that treats the rule-based model as a "safety net" for anomalous or OOD scenarios. It consists of three primary components: a high-capacity Deep Learning Expert, a robust Rule-Based Expert (Constant Velocity), and a Learned Routing Function.
The routing function acts as the system's decision-maker. It shares a scene-encoding backbone with the DL expert to maintain computational efficiency while gaining a deep understanding of the environment. Unlike traditional ensembles that are combined post-hoc, APE trains the routing function concurrently with the DL model. By observing the successes and failures of the DL expert relative to the CV baseline during training, the router learns to output a ranking score that determines which prediction to trust.
Experimental Results and Insights
The authors evaluated APE using a strict zero-shot protocol, training on the Waymo Open Motion Dataset (WOMD) and testing on Argoverse, and vice versa. The results demonstrate substantial improvements in cross-dataset reliability:
- Performance Gains: When generalizing from WOMD to Argoverse, APE reduced the minADE of a vanilla MotionTransformer (MTR) from 6.85m to 3.04m—a 55% improvement.
- Horizon Robustness: The benefits of APE become even more pronounced over longer prediction horizons, showing a 57.3% gain at an 8-second window.
- Model Agnostic: The framework proved effective across different DL architectures, including MTR and Wayformer.
A key insight from the study is that while generating a perfect trajectory in a new environment is difficult for a DL model, discriminating between existing candidates is a much more generalizable task. By framing the problem as a ranking challenge, APE leverages the discriminative power of neural networks to improve overall system robustness.
Conclusion and Future Work
Adaptive Prediction Ensemble provides a practical path toward more reliable autonomous systems by acknowledging the limitations of pure deep learning approaches. While currently limited by the performance of its best available expert, future iterations could incorporate a wider variety of specialized experts for specific traffic scenarios (e.g., intersections or high-density traffic). By combining the high-capacity learning of neural networks with the robust grounding of rule-based models, APE significantly enhances the zero-shot generalization required for real-world deployment.
Summary updated