A Novel Temporal Multi-Gate Mixture-of-Experts Approach for Vehicle Trajectory and Driving Intention Prediction

Renteng Yuan, Mohamed A. Abdel-Aty, Qiaojun Xiang, Zijin Wang, Ou ZhengarXiv 2023 · DOI · OpenReview

TL;DR

This paper introduces the Temporal Multi-Gate Mixture-of-Experts (TMMOE) model, a novel approach that simultaneously predicts vehicle trajectory (longitudinal and lateral position) and driving intention. By integrating Temporal Convolutional Networks (TCN) for feature extraction and Long Short-Term Memory (LSTM) for expert gating, TMMOE significantly outperforms existing methods, particularly LSTM, in accurately forecasting vehicle behavior.

What Problem Does This Paper Address?

Accurately predicting a vehicle's future path and its driver's intentions is crucial for the safe and efficient operation of autonomous vehicles and advanced driver-assistance systems. Current methods often tackle trajectory prediction (predicting future positions) and driving intention recognition as separate tasks. However, there's a strong, often overlooked, correlation between a vehicle's motion and its driver's intentions. This separation can lead to suboptimal predictions, especially in complex scenarios. The paper aims to bridge this gap by developing a unified model that leverages these interdependencies for more robust and accurate predictions.

Methods

The core of this work is the Temporal Multi-Gate Mixture-of-Experts (TMMOE) model. It's built upon a three-layer architecture:

  • Shared Layer: This layer uses Temporal Convolutional Networks (TCN) to extract temporal features from the input data. TCNs are chosen for their ability to capture long-term dependencies in sequential data through dilated causal convolutions, which is essential for understanding vehicle movement over time.
  • Expert Layer: This layer employs a Multi-Gate Mixture-of-Experts (MMoE) structure. Here, Long Short-Term Memory (LSTM) networks are used as the "gating mechanism." This allows the model to dynamically weigh the contributions of different "experts" (specialized sub-networks) for each specific task (driving intention, longitudinal position, lateral position), effectively differentiating and leveraging task-specific information.
  • Fully Connected Layer (FCN): This final layer integrates the outputs from the expert layer to produce the final predictions for each task.

A key innovation is the use of a homoscedastic uncertainty algorithm to construct a multi-task loss function. This approach dynamically adjusts the weights of different tasks, allowing the model to balance learning across tasks with varying scales and uncertainties, leading to improved overall performance.

Datasets

The proposed TMMOE model was trained and evaluated using the CitySim dataset. This dataset comprises vehicle trajectory data collected from drones, offering a high sampling frequency (30 Hz) and detailed information about vehicle positions. The study specifically utilized a sub-dataset called freeway-B, which includes data from a 680-meter basic freeway segment with six lanes. This dataset contains 5623 vehicle trajectories extracted from 60 minutes of drone videos.

Key Insights and Ideas

  • Interdependence is Key: The paper strongly emphasizes that driving intentions and vehicle trajectory are not independent. A unified model that considers these relationships offers a significant advantage over siloed approaches.
  • TCN for Temporal Features, LSTM for Gating: The combination of TCN for robust temporal feature extraction and LSTM for adaptive expert gating in an MMoE framework proves highly effective for complex sequential prediction tasks like vehicle behavior.
  • Uncertainty-Aware Multi-Task Learning: Dynamically weighting tasks based on their inherent uncertainty (using homoscedastic uncertainty) is crucial for optimizing performance in multi-task learning scenarios, especially when tasks have different scales or importance.
  • Traffic Conflict Indicators Matter: Sensitivity analysis revealed that incorporating traffic conflict indicators (like TTC, MTTC, DRAC) and system coupling degree significantly improves the accuracy of trajectory prediction. Removing these variables led to a noticeable drop in performance.

Key Results

The TMMOE model demonstrated superior performance compared to a standard LSTM model across all tested input sequence lengths (3s, 6s, 9s).

  • Classification Accuracy (Driving Intention): The TMMOE model achieved a peak accuracy of 0.9544 with a 6-second input sequence, outperforming the LSTM model's best accuracy of 0.8931.
  • Regression Performance (Trajectory Prediction): For longitudinal position prediction, TMMOE achieved an RMSE of 2.323m and MAE of 1.785m. For lateral position, it achieved an RMSE of 0.2963m and MAE of 0.199m. These figures represent a substantial improvement over the LSTM model, which had significantly higher RMSE values (e.g., 8.280m for longitudinal position).
  • ROC Curves: The ROC curves for TMMOE showed higher Area Under the Curve (AUC) values for all three driving intention classes (LK, RLC, LLC) compared to LSTM, indicating a better ability to distinguish between intentions.

Limitations and Future Work

The authors acknowledge a few limitations:

  • Single Dataset: The study relied solely on the CitySim dataset. Future work should involve validating the model's performance on additional, diverse datasets to assess its portability and generalizability.
  • Input Sequence Length: The evaluation focused on specific input sequence lengths (3s, 6s, 9s). Exploring finer input time series lengths could potentially yield further improvements.

The authors suggest that future studies should focus on these areas to enhance the model's robustness and applicability.

Summary updated

Abstract

Accurate Vehicle Trajectory Prediction is critical for automated vehicles and advanced driver assistance systems. Vehicle trajectory prediction consists of two essential tasks, i.e., longitudinal position prediction and lateral position prediction. There is a significant correlation between driving intentions and vehicle motion. In existing work, the three tasks are often conducted separately without considering the relationships between the longitudinal position, lateral position, and driving intention. In this paper, we propose a novel Temporal Multi-Gate Mixture-of-Experts (TMMOE) model for simultaneously predicting the vehicle trajectory and driving intention. The proposed model consists of three layers: a shared layer, an expert layer, and a fully connected layer. In the model, the shared layer utilizes Temporal Convolutional Networks (TCN) to extract temporal features. Then the expert layer is built to identify different information according to the three tasks. Moreover, the fully connected layer is used to integrate and export prediction results. To achieve better performance, uncertainty algorithm is used to construct the multi-task loss function. Finally, the publicly available CitySim dataset validates the TMMOE model, demonstrating superior performance compared to the LSTM model, achieving the highest classification and regression results. Keywords: Vehicle trajectory prediction, driving intentions Classification, Multi-task
BibTeX
@misc{yuan2023novel,
  author = {Renteng Yuan and Mohamed A. Abdel-Aty and Qiaojun Xiang and Zijin Wang and Ou Zheng},
  title = {{A Novel Temporal Multi-Gate Mixture-of-Experts Approach for Vehicle Trajectory and Driving Intention Prediction}},
  year = {2023},
  eprint = {2308.00533},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2308.00533},
}
Figure