Scalable Dynamic Mixture Model with Full Covariance for Probabilistic Traffic Forecasting

Seongjin Choi, Nicolas Saunier, Vincent Zhihao Zheng, Martin Trépanier, Lijun SunTransp. Sci 2025 · DOI

TL;DR

This paper introduces a novel "Dynamic Mixture Model" to improve traffic forecasting by accurately modeling the complex, time-varying correlations in prediction errors. By treating these errors as a dynamic mixture of Gaussian distributions with full covariance, the model offers more reliable probabilistic forecasts and provides interpretable insights into traffic dynamics.

What Problem Does This Paper Address?

Traditional deep learning models for traffic forecasting often rely on simplified assumptions about prediction errors, typically assuming they follow independent and identically distributed (i.i.d.) Gaussian or Laplacian distributions. This is problematic because real-world traffic data exhibits complex spatiotemporal correlations – traffic conditions at one location influence nearby locations, and errors at one time step are related to errors at other time steps. Furthermore, these error distributions are not static; they change over time, especially during transitions like rush hour or congestion. Ignoring these dynamics leads to biased models and less accurate probabilistic forecasts. Existing methods that attempt to model these correlations often struggle with computational complexity or fail to capture the multimodal nature of the error distributions.

Methods

The core contribution is a dynamic mixture of zero-mean Gaussian distributions to model the spatiotemporal error. Here's a breakdown of the key methodological innovations:

  • Dynamic Mixture Model: Instead of a single distribution, the model uses a mixture of K Gaussian components. The key innovation is that the mixture weights (w_k) are dynamic and time-varying, allowing the model to adapt to changing error distributions throughout the day. The covariance matrices within each component are assumed to be time-independent for simplicity.
  • Full Covariance Modeling: Each Gaussian component uses a full covariance matrix (Σ_k) to capture both spatial and temporal correlations within that component.
  • Scalable Covariance Parameterization: Directly modeling the large NQ x NQ spatiotemporal covariance matrix is computationally prohibitive. The authors propose a Kronecker product decomposition (Σ_k = Σ_spatial ⊗ Σ_temporal) to break down the problem into smaller, manageable spatial (N x N) and temporal (Q x Q) covariance matrices. This significantly reduces computational complexity from O((NQ)^3) to O(N^3 + Q^3).
  • Cholesky Factorization for Precision Matrices: To further improve stability and efficiency during training, the model parameterizes the precision matrices (inverse of covariance matrices) using their Cholesky factors. This avoids direct matrix inversion during training and helps prevent numerical issues like gradient explosion.
  • Integrated Loss Function: The model is trained by minimizing a combined loss function: LDynMix = (1 - p)LMSE + p*LNLL. This balances the traditional Mean Squared Error (MSE) for mean prediction with the Negative Log-Likelihood (NLL) of the proposed dynamic mixture model for capturing uncertainty.

Datasets

The paper evaluates the proposed method on several widely-used traffic speed datasets:

  • PEMS-BAY-2017-SPEED: Collected from California Transportation Agencies (CalTrans) Performance Measurement System (PeMS), featuring 325 sensors.
  • METR-LA: Collected from highways in Los Angeles County, with 207 sensors.
  • PEMS-BAY-2022-FLOW: A new dataset for traffic flow.
  • PEMS-BAY-2022-SPEED: A new dataset for traffic speed.

All datasets use a 5-minute resolution. The paper also mentions using these datasets for training and evaluating the baseline models (Graph Wavenet and STGCN) with the proposed loss function.

Key Insights and Ideas

  • Real-world traffic errors are not simple: The assumption of i.i.d. errors is a significant oversimplification that hinders accurate probabilistic forecasting.
  • Time-varying distributions are crucial: Traffic error distributions change predictably throughout the day (e.g., weekday vs. weekend patterns), and capturing this dynamism is key to improving forecasts.
  • Spatiotemporal correlations are separable (often): The Kronecker product decomposition is a practical and effective way to model complex spatiotemporal covariances without prohibitive computational cost.
  • Dynamic mixture weights offer interpretability: The learned mixture weights reveal distinct patterns (e.g., "peak-hour" vs. "off-peak") that correspond to different traffic regimes and error characteristics.
  • Balancing MSE and NLL is beneficial: A hybrid loss function that combines traditional MSE with the proposed NLL loss often yields better results than using either extreme alone, suggesting a trade-off between point prediction accuracy and uncertainty modeling.

Key Results

  • Improved Accuracy: The proposed dynamic mixture loss function consistently improves prediction accuracy (RMSE, MAPE, MAE) across various forecasting horizons (15, 30, 45, 60 minutes) when integrated with baseline models like Graph Wavenet (GWN) and Spatio-Temporal Graph Convolutional Networks (STGCN).
  • Significant MAPE Reduction: The model shows particularly strong improvements in Mean Absolute Percentage Error (MAPE), which is critical for forecasting in congested traffic scenarios where speeds are low and errors can be heavily penalized.
  • Better Performance on Longer Horizons: Improvements are more pronounced for longer forecasting horizons, indicating the model's enhanced ability to capture accumulating spatiotemporal dependencies.
  • Interpretable Patterns: Analysis of the learned covariance matrices and mixture weights reveals distinct weekday and weekend patterns, correlating with different traffic states (peak vs. off-peak hours). For instance, the first component often captures strong spatial and temporal correlations associated with "peak-hour patterns."
  • Scalability: The method is shown to be scalable, with computational complexity of O(K(N^3 + Q^3)), making it feasible for real-world applications.

Limitations and Future Work

  • Gaussian Assumption: The paper acknowledges that using Gaussian distributions as component distributions might not fully capture the complex, non-linear characteristics of real-world traffic data. Future work could explore non-Gaussian distributions or more complex probabilistic models.
  • Kronecker Product Assumption: The assumption of a separable Kronecker product structure for the covariance matrix might be too restrictive for some datasets. Future research could explore more flexible ways to parameterize the covariance matrix.
  • Formal Convergence Guarantee: The paper lacks a formal theoretical analysis of the method's convergence properties. Establishing such a guarantee would increase confidence in its stability and robustness.
  • Generalization to Other Domains: While promising for traffic forecasting, the authors suggest the method could be extended to other domains with complex spatiotemporal correlations, such as wind speed forecasting, climate modeling, or financial markets.
  • Precision Matrix Structure: Exploring alternative structures for the precision matrix, such as Gaussian Markov Random Fields (GMRF) or directly modeling sparse spatiotemporal precision, could offer further improvements.

Summary updated

BibTeX
@article{choi2025scalable,
  author = {Seongjin Choi and Nicolas Saunier and Vincent Zhihao Zheng and Martin Trépanier and Lijun Sun},
  title = {{Scalable Dynamic Mixture Model with Full Covariance for Probabilistic Traffic Forecasting}},
  journal = {Transp. Sci.},
  year = {2025},
  doi = {10.1287/trsc.2024.0547},
}
Figure