Probabilistic Traffic Forecasting with Dynamic Regression

Vincent Zhihao Zheng, Seongjin Choi, Lijun Sun

TL;DR

This paper introduces a "Dynamic Regression" (DR) framework that enhances existing deep learning models for traffic forecasting by explicitly modeling and correcting for correlated errors. By treating errors as a dynamic process, the method significantly improves probabilistic forecasts and provides interpretable insights into error structures.

What Problem Does This Paper Address?

Traffic forecasting is crucial for intelligent transportation systems, aiming to predict future traffic conditions based on historical data. Deep learning (DL) models have shown great promise in this area, but they often make simplifying assumptions about the errors in their predictions. Specifically, they typically assume errors are temporally independent and follow an isotropic distribution (meaning errors are equally likely in all directions). This is often not true in reality. Factors like unobserved variables (e.g., weather, traffic incidents) can lead to errors that are correlated over time and across different locations. Ignoring these correlations can lead to suboptimal predictions and inaccurate uncertainty estimates, especially for longer-term forecasts. This paper addresses this gap by proposing a method to explicitly model these correlated errors.

Methods

The core of the proposed method is a Dynamic Regression (DR) framework that is designed to be model-agnostic, meaning it can be integrated with various existing deep learning traffic forecasting models. The DR framework works by:

  • Modeling Error Dynamics: Instead of assuming errors are independent, the framework models the error term ($R_t$) of the base DL model as a matrix-variate autoregressive (AR) process. This allows the model to capture temporal dependencies in the errors.
  • Non-Isotropic Error Modeling: The errors are assumed to follow a non-isotropic multivariate Gaussian distribution. This means the errors can have different variances across different time steps and spatial locations, and can be correlated. This is achieved by using a Kronecker product structure for the covariance matrix.
  • Joint Optimization: Crucially, the parameters of the error AR model and the non-isotropic covariance matrix are jointly optimized with the parameters of the base DL model. This allows the entire system to learn a more cohesive and accurate representation of the traffic dynamics and their associated errors.
  • Probabilistic Forecasting: By modeling the error distribution, the DR framework naturally enables probabilistic forecasting, providing not just point predictions but also estimates of uncertainty.

Datasets

The authors evaluated their method on two widely used traffic forecasting datasets:

  • PEMSD7 (M): A highway traffic speed dataset.
  • PEMS08: A highway traffic flow dataset.

These datasets are standard benchmarks in the field and have been used in previous state-of-the-art traffic forecasting research.

Key Insights and Ideas

  • Errors are Not Always Independent: The paper provides strong visual evidence (Figure 1 and 3) that residuals from existing DL models exhibit significant contemporaneous and cross-correlations, directly contradicting the common isotropic assumption.
  • Weekly Patterns Dominate Error Correlations: For traffic flow, the lag-2016 (one-week) cross-correlation in residuals was found to be the strongest, suggesting that weekly traffic patterns are a significant driver of forecasting errors.
  • Model-Agnostic Enhancement: The DR framework is designed as a "plug-and-play" module, meaning it can be added to existing DL models without requiring major architectural changes, making it highly practical.
  • Improved Probabilistic Forecasts: By explicitly modeling error dynamics and non-isotropic covariance, the method leads to more realistic and accurate probabilistic forecasts, especially in capturing increasing uncertainty over the prediction horizon.
  • Interpretability of Error Structure: The learned AR coefficients and covariance matrices offer insights into the nature of forecasting errors, revealing spatial dependencies and temporal error propagation that are hidden in end-to-end models.

Key Results

The proposed DR framework consistently improved performance across various metrics and base models:

  • RRMSE (Deterministic Accuracy): Achieved average improvements of 1.67% on PEMSD7(M) and 3.38% on PEMS08. This metric primarily reflects the effectiveness of the error AR module in improving point predictions.
  • CRPS (Probabilistic Accuracy): Showed average improvements of 6.37% on PEMSD7(M) and 11.72% on PEMS08. This metric assesses the overall quality of the predictive distribution.
  • p-risk (Quantile Accuracy): Demonstrated significant improvements, particularly in the 0.9-risk (9.60% on PEMSD7(M) and 13.97% on PEMS08), indicating better calibration for higher quantiles.
  • Reduction in Error Correlations: Figure 4 visually shows a significant reduction in both contemporaneous and cross-correlations in the residuals after applying the DR method, indicating that the model has effectively learned to account for these dependencies.

Limitations and Future Work

The authors acknowledge two primary limitations:

  • Increased Training Costs: Learning the enriched error covariance structure increases training time. For example, applying the method to Graph WaveNet on PEMS08 increased training time by nearly 20%.
  • Scalability Challenges: For datasets with very large spatial dimensions, scalability might become an issue, despite optimization efforts.

Future work suggested includes:

  • Scaling to Higher-Dimensional Datasets: Developing more efficient methods to handle datasets with a vast number of spatial locations.
  • Incorporating Time-Varying Error Covariance Matrices: Extending the model to capture how error correlations might change over time.

Summary updated

Abstract

This paper proposes a dynamic regression (DR) framework that enhances existing deep spatiotemporal models by incorporating structured learning for the error process in traffic forecasting. The framework relaxes the assumption of time independence by modeling the error series of the base model (i.e., a well-established traffic forecasting model) using a matrix-variate autoregressive (AR) model. The AR model is integrated into training by redesigning the loss function. The newly designed loss function is based on the likelihood of a non-isotropic error term, enabling the model to generate probabilistic forecasts while preserving the original outputs of the base model. Importantly, the additional parameters introduced by the DR framework can be jointly optimized alongside the base model. Evaluation on state-of-the-art (SOTA) traffic forecasting models using speed and flow datasets demonstrates improved performance, with interpretable AR coefficients and spatiotemporal covariance matrices enhancing the understanding of the model.
BibTeX
@article{zheng2025probabilistic,
  author = {Vincent Zhihao Zheng and Seongjin Choi and Lijun Sun},
  title = {{Probabilistic Traffic Forecasting with Dynamic Regression}},
  journal = {Transp. Sci.},
  year = {2025},
  doi = {10.1287/trsc.2024.0560},
}
Figure