Enhancing Deep Traffic Forecasting Models with Dynamic Regression

Vincent Zhihao Zheng, Seongjin Choi, Lijun SunarXiv 2023 · arXiv · OpenReview

Abstract

Deep learning models for traffic forecasting often assume the residual is independent and isotropic across time and space. This assumption simplifies loss functions such as mean absolute error, but real-world residual processes often exhibit significant autocorrelation and structured spatiotemporal correlation. This paper introduces a dynamic regression (DR) framework to enhance existing spatiotemporal traffic forecasting models by incorporating structured learning for the residual process. We assume the residual of the base model follows a matrix-variate seasonal autoregressive (AR) model, which is seamlessly integrated into the training process through the redesign of the loss function. Importantly, the parameters of the DR framework are jointly optimized alongside the base model. We evaluate the effectiveness of the proposed framework on state-of-the-art (SOTA) deep traffic forecasting models using both speed and flow datasets, demonstrating improved performance and providing interpretable AR coefficients and spatiotemporal covariance matrices.

Introduction

Most state-of-the-art (SOTA) traffic forecasting models rely on Mean Absolute Error (MAE) or Mean Squared Error (MSE) as their loss functions. While effective for point estimates, these functions implicitly assume that prediction errors are independent and identically distributed (i.i.d.)—essentially treating them as "white noise."

In reality, traffic data is complex. If a model underestimates traffic at one sensor, it is highly likely to underestimate it at neighboring sensors (spatial correlation) and at subsequent time steps (temporal correlation). Furthermore, uncertainty naturally grows as the prediction horizon extends. By ignoring these structured correlations in the error process, current models leave significant performance on the table and fail to provide realistic uncertainty estimates.

The Dynamic Regression Framework

The proposed Dynamic Regression (DR) framework is a "plug-and-play" wrapper for any existing spatiotemporal deep learning model. It upgrades standard point-predictors into sophisticated probabilistic forecasters by explicitly modeling the residual process.

Comparison between existing point-prediction methods and the proposed DR framework for probabilistic forecasting.

The approach consists of two main components:

  1. Matrix-valued Autoregression: Instead of assuming errors ($R_t$) are random, they are modeled as a first-order bi-linear AR process: $R_t = A R_{t-\Delta} B + E_t$. Here, $\Delta$ represents a seasonal lag (e.g., one week ago). The matrix $A$ captures spatial dependencies between sensors, while $B$ captures correlations across the prediction horizon.
  2. Non-Isotropic Covariance Learning: The remaining noise ($E_t$) follows a multivariate Gaussian distribution. To maintain computational tractability for large networks, the authors employ a Kronecker product structure for the covariance matrix ($\Sigma = \Sigma_Q \otimes \Sigma_N + \sigma^2 I$). This allows the model to learn how uncertainty propagates across space ($N$) and time ($Q$) without the overhead of inverting massive $NQ \times NQ$ matrices.

The entire system is trained jointly using a Negative Log-Likelihood (NLL) loss function, allowing the base model and the error-correction module to optimize simultaneously.

Key Insights and Residual Analysis

A major finding is that model performance can be significantly enhanced without increasing the complexity of the base neural network. By refining how residuals are treated, the framework extracts more information from the data.

Correlation matrices showing (a) contemporaneous and (b) seasonal spatiotemporal structure in residuals.

The authors found that incorporating seasonality—specifically a lag of one week ($\Delta = 2016$ steps)—was particularly effective for traffic flow. This suggests that the "mistakes" a model makes today often mirror those made at the same time in the previous week. Visual analysis of the residuals (Figure 1) confirms that the DR framework successfully "whitens" the errors, removing spatial and temporal structures that base models typically miss.

Furthermore, the learned covariance matrices ($\Sigma_Q$) are physically intuitive. Diagonal elements of the temporal covariance matrix expand as the prediction horizon increases, mathematically capturing the reality that the distant future is inherently harder to predict than the immediate next steps.

Results and Conclusion

The DR framework consistently improved performance across all tested models and metrics. For point forecasts, the authors observed an average improvement in Root Relative Mean Squared Error (RRMSE) of 1.67% on PEMSD7 (M) and 3.38% on PEMS08. The gains in probabilistic forecasting were even more substantial, with the Continuous Ranked Probability Score (CRPS) improving by an average of 11.72% on flow data.

While the method introduces a modest increase in training time (e.g., ~20% for Graph WaveNet), the gains in accuracy and the provision of reliable uncertainty estimates are substantial. Future work will focus on scaling the framework to larger datasets and exploring time-varying covariance matrices to better account for dynamic traffic conditions like rush hour.

Summary updated

Abstract

Deep learning models for traffic forecasting often assume the residual is independent and isotropic across time and space. This assumption simplifies loss functions such as mean absolute error, but real-world residual processes often exhibit significant autocorrelation and structured spatiotemporal correlation. This paper introduces a dynamic regression (DR) framework to enhance existing spatiotemporal traffic forecasting models by incorporating structured learning for the residual process. We assume the residual of the base model (i.e., a well-developed traffic forecasting model) follows a matrix-variate seasonal autoregressive (AR) model, which is seamlessly integrated into the training process through the redesign of the loss function. Importantly, the parameters of the DR framework are jointly optimized alongside the base model. We evaluate the effectiveness of the proposed framework on state-of-the-art (SOTA) deep traffic forecasting models using both speed and flow datasets, demonstrating improved performance and providing interpretable AR coefficients and spatiotemporal covariance matrices.
BibTeX
@misc{zheng2023enhancing,
  author = {Vincent Zhihao Zheng and Seongjin Choi and Lijun Sun},
  title = {{Enhancing Deep Traffic Forecasting Models with Dynamic Regression}},
  year = {2023},
  eprint = {2301.06650},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2301.06650},
}
Figure