Abstract
Predicting diverse yet admissible trajectories that adhere to map constraints is challenging. Graph-based scene encoders have been proven effective for preserving local structures of maps by defining lane-level connections. However, such encoders do not capture more complex patterns emerging from long-range heterogeneous connections between nonadjacent interacting lanes. To this end, we shed new light on learning common driving patterns by introducing meTA ROad paTh (TAROT) to formulate combinations of various relations between lanes on the road topology. Intuitively, this can be viewed as finding feasible routes. Furthermore, we propose MEta-road NeTwORk (MENTOR) that helps trajectory prediction by providing it with TAROT as navigation tips. More specifically, 1) we define TAROT prediction as a novel self-supervised proxy task to identify the complex heterogeneous structure of the map. 2) For typical driving actions, we establish several TAROTs that result in multiple Heterogeneous Structure Learning (HSL) tasks. These tasks are used in MENTOR, which performs meta-learning by simultaneously predicting trajectories along with proxy tasks, identifying an optimal combination of them, and automatically balancing them to improve the primary task. We show that our model achieves state-of-the-art performance on the Argoverse dataset, especially on diversity and admissibility metrics, achieving up to 20% improvements in challenging scenarios.
Introduction
Accurate trajectory prediction in autonomous driving requires a deep understanding of the "rules of the road" encoded in High-Definition (HD) maps. While current state-of-the-art models utilize Graph Neural Networks (GNNs) to capture local lane connectivity, they often fail to grasp long-range heterogeneous relationships—such as those found in complex merging, overtaking, or multi-lane turning maneuvers.
Self-supervised learning (SSL) offers a potential solution by introducing auxiliary "proxy tasks" to help the model learn these structures. However, manually balancing these auxiliary tasks against the primary prediction objective is difficult. A poorly weighted proxy task can either dominate the learning process or be ignored entirely. To address these challenges, we introduce TAROT and MENTOR.
Methodology
TAROT: meTA ROad paTh
We treat the road network as a Heterogeneous Information Network. Instead of focusing solely on immediate lane neighbors, we define TAROT (meTA ROad paTh) to represent sequences of semantic relations (e.g., $Successor \rightarrow Left \rightarrow Successor$). Predicting the existence of these paths between distant lane segments serves as a self-supervised Heterogeneous Structure Learning (HSL) task. These tasks act as "navigation tips," teaching the model the underlying grammar and feasible flow of the road topology.
MENTOR: MEta-road NeTwORk
To effectively integrate these proxy tasks, we propose MENTOR. MENTOR utilizes a meta-learning framework based on bi-level optimization to dynamically weight the various HSL tasks. It learns a weighting function $\mathcal{V}(\cdot; \theta)$ that identifies which TAROT tasks are most beneficial for the primary trajectory prediction task at any given training step. This approach naturally prioritizes "hard samples" where the model struggles with map topology, ensuring that auxiliary learning supports the main goal without causing interference.
Experimental Results
We evaluated our approach primarily on the Argoverse motion forecasting dataset, focusing on challenging scenarios such as turns and stationary vehicles where map geometry is the primary cue for prediction.
Quantitative Performance
MENTOR achieves significant improvements in both admissibility (staying within drivable areas) and diversity (predicting a representative set of feasible paths). As shown in the results table, our model outperforms existing baselines like LaneGCN, MMTransformer, and HiVT across multiple metrics.
Key highlights include:
- Admissibility: A substantial increase in Drivable Area Occupancy (DAO), particularly in turn scenarios, where we see improvements of up to 20% over some baselines.
- Diversity: Improvements in the Ratio of avgFDE to minFDE (RF), indicating a more realistic spread of predicted trajectories.
- Robustness: Strong performance in stationary scenarios, demonstrating that the model effectively leverages map structure when motion cues are absent.
Conclusion
By combining the structural insights of TAROT with the automated task balancing of MENTOR, we have developed a trajectory prediction framework that is significantly more "map-aware." Our results on the Argoverse dataset demonstrate that meta-learning is a powerful tool for integrating self-supervised proxy tasks, leading to more diverse and admissible predictions in complex real-world driving environments.
Summary updated