Knowledge-data fusion oriented traffic state estimation: A stochastic physics-informed deep learning approach

Ting Wang, Ye Li, Rongjun Cheng, Guojian Zou, Takao Dantsujic, Dong Ngoduy

Introduction

Traffic State Estimation (TSE) is a critical component of intelligent transportation systems, aiming to reconstruct full traffic fields (speed, density, flow) from sparse and noisy sensor observations. Historically, TSE has evolved from purely physics-based models to data-driven deep learning approaches. Recently, Physics-Informed Deep Learning (PIDL) has emerged as a powerful paradigm, combining the generalizability of physical laws with the flexibility of neural networks. However, as shown in the evolution of the field, there is a growing shift toward incorporating stochasticity to better reflect real-world dynamics.

Evolution of Traffic State Estimation (TSE) methods from deterministic to stochastic modeling.

The Problem: Deterministic Limitations

Despite the success of PIDL, most current architectures rely on deterministic physical models, such as a one-to-one speed-density relationship. However, real-world traffic exhibits a significant "scattering effect"—where a single density value can correspond to a range of speeds due to driver heterogeneity and environmental factors. Forcing a neural network to adhere to a rigid, deterministic curve leads to "overly centralized guidance," which can result in unreliable estimations, particularly in sparse data scenarios where the model may ignore natural variance.

Proposed Methodology: Stochastic PIDL (SPIDL)

To address these limitations, this study introduces Stochastic Physics-Informed Deep Learning (SPIDL). Instead of a single curve, SPIDL incorporates stochastic fundamental diagrams (SFD) as prior knowledge. The framework treats physics as a range of probabilities rather than a single fixed line. Two specific models are proposed:

  1. $\alpha$-SPIDL (Percentile-based): This approach utilizes a family of percentile-based fundamental diagrams. By calibrating multiple curves (e.g., representing different traffic behaviors), the model provides "soft constraints" that define a feasible search space for the neural network.
  2. $\mathcal{B}$-SPIDL (Distribution-based): This more advanced framework treats the speed-density relationship as a stochastic process following a Beta distribution. It employs a Variational Autoencoder (VAE) to learn the latent distribution of traffic states, minimizing the KL divergence between the learned distribution and the theoretical physics-based distribution.
The framework of $\mathcal{B}$-SPIDL, integrating a VAE with a stochastic fundamental diagram based on the Beta distribution.

Experimental Results

The models were evaluated using the real-world NGSIM dataset, simulating sparse sensor coverage (as low as 20% coverage).

  • Accuracy and Robustness: Both SPIDL models outperformed traditional PIDL in sparse data environments. $\alpha$-SPIDL showed superior stability with extremely limited data, while $\mathcal{B}$-SPIDL demonstrated higher accuracy as sensor density increased, effectively capturing the underlying data generation mechanism.
  • Capturing Scattering: Unlike deterministic models that smooth out traffic oscillations, SPIDL successfully reproduced the scattering effect observed in field data, particularly in high-density, congested conditions (stop-and-go waves).
  • Uncertainty Quantification: A key advantage of the SPIDL framework is its ability to provide confidence intervals (e.g., 95% intervals), offering traffic operators a quantifiable measure of estimation reliability at any given moment.

Conclusion

SPIDL represents a significant step forward in knowledge-data fusion for traffic engineering. By transitioning from deterministic to stochastic physical guidance, the framework achieves more realistic and robust traffic state estimations. Future research will explore extending this approach to complex urban networks using Physics-Informed Graph Convolutional Networks (PIGCN) and integrating diverse data sources like floating car data (GPS) to create a more holographic view of traffic.

Summary updated

Abstract

Physics-informed deep learning (PIDL)-based models have recently garnered remarkable success in traffic state estimation (TSE). However, the prior knowledge used to guide regularization training in current mainstream architectures is based on deterministic physical models. The drawback is that a solely deterministic model fails to capture the universally observed traffic flow dynamic scattering effect, thereby yielding unreliable outcomes for traffic control. This study, for the first time, proposes stochastic physics-informed deep learning (SPIDL) for traffic state estimation. The idea behind such SPIDL is simple and is based on the fact that a stochastic fundamental diagram provides the entire range of possible speeds for any given density with associated probabilities. Specifically, we select percentile-based fundamental diagram and distribution-based fundamental diagram as stochastic physics knowledge, and design corresponding physics-uninformed neural networks for effective fusion, thereby realizing two specific SPIDL models, namely \text{$α$}-SPIDL and \text{$\cal B$}-SPIDL. The main contribution of SPIDL lies in addressing the "overly centralized guidance" caused by the one-to-one speed-density relationship in deterministic models during neural network training, enabling the network to digest more reliable knowledge-based constraints.Experiments on the real-world dataset indicate that proposed SPIDL models achieve accurate traffic state estimation in sparse data scenarios. More importantly, as expected, SPIDL models reproduce well the scattering effect of field observations, demonstrating the effectiveness of fusing stochastic physics model knowledge with deep learning frameworks.
BibTeX
@misc{wang2024knowledge,
  author = {Ting Wang and Ye Li and Rongjun Cheng and Guojian Zou and Takao Dantsujic and Dong Ngoduy},
  title = {{Knowledge-data fusion oriented traffic state estimation: A stochastic physics-informed deep learning approach}},
  year = {2024},
  eprint = {2409.00644},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2409.00644},
}
Figure