Temporal-Spatial Traffic Flow Prediction Model Based on Prompt Learning

Siteng Cai, Gang Liu, Jing He, Yulun Du, Zhichao Si, Yunhao JiangISPRS Int. J. Geo Inf 2024 · DOI

TL;DR

This paper introduces a novel temporal-spatial traffic flow prediction model that leverages "prompt learning" to improve its ability to adapt to different traffic patterns and datasets. By using a prompt network to guide a graph convolutional network, the model achieves better prediction accuracy and robustness, especially when dealing with variations in traffic data distributions.

What Problem Does This Paper Address?

Traffic congestion and accidents are growing problems worldwide, exacerbated by increasing urbanization and vehicle ownership. Accurate traffic flow prediction is crucial for intelligent transportation systems (ITS) to manage traffic effectively, improve safety, and enhance urban mobility. However, existing deep learning models often struggle with "distribution shift" – meaning they perform poorly when the traffic patterns in new, unseen data differ from the data they were trained on. This limits their generalization capabilities and real-world applicability. The paper aims to address this by developing a model that can better adapt to these varying spatiotemporal contexts.

Methods

The core of the proposed model is a spatiotemporal traffic flow prediction prompt network (TPPN). This network integrates several key components:

  • Graph Convolutional Networks (GCNs): Used to capture the spatial dependencies and relationships between different locations in the road network.
  • Prompt Learning: This is the novel aspect. Instead of traditional fine-tuning, prompt learning uses "soft prompts" (learnable continuous vectors) to guide a pre-trained model. These prompts are generated by the TPPN and help the model adapt to specific downstream tasks (i.e., predicting traffic flow for a particular dataset or scenario) without drastically altering its core parameters. This helps prevent "catastrophic forgetting" of knowledge learned during pre-training.
  • Temporal Embeddings: Captures periodic temporal patterns (like time of day and day of the week) to understand traffic dynamics.
  • Spatial Embeddings: Encodes the road network structure using Laplacian positional encoding to represent spatial context.
  • Time Series Embeddings: Captures the historical traffic flow data.

These components are combined to generate a "prompt" that is then used to fine-tune the model, enhancing its adaptive capability. The model also utilizes an infoNCE loss function for improved robustness and discriminative power.

Datasets

The study uses a combination of publicly available datasets for training and evaluation:

  • PEMS03, PEMS04, PEMS07M: Datasets from California, USA, containing traffic flow records from various streets and cities.
  • Chengdu Didi Dataset: Records traffic flow index for the road network in Chengdu, China.

The datasets were split into training, validation, and testing sets at a 6:2:2 ratio, with a data collection frequency of 5 minutes. The authors employed a pre-training strategy using multiple datasets (PEMS03, PEMS04, PEMS07M, and Chengdu Didi) followed by fine-tuning on the Chengdu Didi dataset to evaluate generalization.

Key Insights and Ideas

  • Prompt Learning for Traffic Prediction is Effective: The paper demonstrates that prompt learning, a technique more common in NLP, can significantly improve the adaptability and generalization of spatiotemporal traffic flow prediction models.
  • Soft Prompts Offer Flexibility: Using learnable "soft prompts" provides a more flexible way to adapt models to new data distributions compared to traditional fine-tuning, which can lead to forgetting previously learned knowledge.
  • Temporal Embeddings are Crucial: Ablation studies revealed that removing temporal embeddings had the most significant negative impact on model performance, highlighting their importance in capturing the dynamic and periodic nature of traffic flow.
  • Addressing Distribution Shift is Key: The model's success in handling diverse datasets underscores the importance of developing methods that can robustly predict traffic flow even when test data distributions differ from training data.

Key Results

The proposed model, equipped with the spatiotemporal prompt learning module, outperformed several mainstream benchmark models (AGCRN, MTGNN, STWA, STGCN, TGCN) across the PEMS03, PEMS04, PEMS07M, and Chengdu Didi datasets.

  • Improved Prediction Accuracy: The model achieved better Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and Mean Absolute Percentage Error (MAPE) scores compared to baseline models. For instance, it showed improvements in MAE over MTGNN by 0.1, 0.17, and 0.05 on specific datasets.
  • Efficient Learning: The model demonstrated efficient learning performance, reaching optimal states in a short period, and showed faster convergence compared to models like MTGNN.
  • Robustness to Distribution Shifts: The model's ability to perform well across different datasets indicates its enhanced generalization capability and reduced sensitivity to distribution shifts.

Limitations and Future Work

The authors acknowledge several limitations:

  • Model Complexity and Training Time: The proposed model has relatively high complexity, leading to longer training times, which might limit its application in real-time, large-scale traffic flow prediction scenarios.
  • Prompt Design: While prompt learning is effective, designing more optimal prompt vectors remains an area for further research.
  • Handling Extreme Conditions: The model's performance in special conditions like extreme weather or unexpected events requires further verification and improvement.

Future work will focus on extending the model to other cities with different traffic patterns and exploring its integration into real-time traffic management systems.

Summary updated

BibTeX
@article{cai2024temporal,
  author = {Siteng Cai and Gang Liu and Jing He and Yulun Du and Zhichao Si and Yunhao Jiang},
  title = {{Temporal-Spatial Traffic Flow Prediction Model Based on Prompt Learning}},
  journal = {ISPRS Int. J. Geo Inf.},
  year = {2024},
  doi = {10.3390/ijgi14010011},
}
Figure