Inverting the fundamental diagram and forecasting boundary conditions: how machine learning can improve macroscopic models for traffic flow

Maya Briani, Emiliano Cristiani, Elia Onofri

TL;DR

This paper introduces a hybrid approach that combines machine learning (specifically LSTMs) with traditional macroscopic traffic flow models. By using ML to infer crucial traffic parameters like density and predict future boundary conditions, the authors significantly improve the accuracy of traffic state estimation and forecasting, especially in complex scenarios like congestion.

What Problem Does This Paper Address?

Traffic flow modeling has traditionally been approached in two ways: model-driven (using physics-based equations) and data-driven (using machine learning on observed data). Both have strengths and weaknesses. Model-driven approaches offer physical interpretability but struggle with noisy data and complex real-world phenomena. Data-driven approaches excel with data but can lack robustness and physical grounding. A key challenge is accurately estimating traffic density and predicting future traffic conditions (like congestion or flow at boundaries) to feed into macroscopic models, especially when sensor data is limited or indirect. Specifically, inferring density from flux data is problematic, and predicting future boundary conditions for simulations is difficult.

Methods

The core of this work is a hybrid approach that leverages Long Short-Term Memory (LSTM) networks to bridge the gap between data and models.

  • Density Estimation (Nowcasting): Instead of directly using sensor flux data, which can be incompatible with traffic models, the authors train an LSTM to detect congestion events in real-time. This LSTM output is then used to invert the fundamental diagram, allowing them to derive a more reliable density estimate from the flux data. This density is then injected into a macroscopic LWR (Lighthill-Whitham-Richards) model.
  • Boundary Condition Forecasting: For predicting traffic flow into the future (forecasting), where sensor data is unavailable, another LSTM is trained. This LSTM predicts the total number of vehicles entering or leaving a road segment over a future time interval. This prediction serves as a boundary condition for the macroscopic model, providing a more informed input than simply assuming zero flow.
  • Macroscopic Model: The paper utilizes a multi-class LWR-like macroscopic differential model that describes the joint dynamics of light and heavy vehicles. This model is solved using numerical approximations.

Datasets

The primary dataset used in this paper is provided by the Italian motorway company Autovie Venete S.p.A. This dataset consists of traffic data collected from fixed sensors along highways A23, A28, and parts of A4 in the North-East of Italy. The data includes:

  • Flux (vehicles per hour)
  • Velocity (km/h)
  • Vehicle classification (light vs. heavy)

The data is aggregated per minute and collected from 45 groups of sensors, providing approximately 301,200 records per day. Some sensors (3T sensors) are equipped with advanced technologies that can directly report congestion events as Boolean flags, which are used for supervised learning.

Key Insights and Ideas

  • Inverting the Fundamental Diagram with ML: The ability of LSTMs to detect congestion patterns allows for a more robust inversion of the fundamental diagram, overcoming the ambiguity and unreliability of directly calculating density from flux. This is crucial for accurate state estimation.
  • Predicting Boundary Conditions: Using LSTMs to predict the total future traffic volume (rather than minute-by-minute fluctuations) provides a stable and effective boundary condition for macroscopic models, significantly improving forecast accuracy.
  • Hybrid Approach Superiority: The paper strongly suggests that combining the physical insights of macroscopic models with the pattern recognition capabilities of ML offers a more powerful solution than either approach alone, especially for complex traffic phenomena.
  • Data-Driven Spatial Information Limitation: While data-driven methods are good at temporal prediction, they struggle to infer spatial traffic distribution between sensors. This highlights the continued necessity of mathematical models for capturing spatial dynamics.

Key Results

  • Congestion Detection: The LSTM-based congestion detection classifier achieved an accuracy of 97.70% (or 99.75% when weighted by positive rate) for high-flux sensors.
  • Short-Term Congestion Prediction: The congestion pre-alarm classifier achieved an average accuracy above 95% with a corresponding average loss value of 0.24 for a 4-minute prediction shift.
  • Mid-Term Traffic Volume Forecast: For traffic volume prediction (30 minutes ahead), the ANN-generated inflow boundary conditions resulted in normalized L¹ errors that stayed bounded below 30%, significantly outperforming the error from assuming null inflow, which rapidly increased to 100%.
  • Model Performance: The density-based approach, which incorporates ML-derived density information, consistently outperformed the flux-based approach in academic and real-world tests, better matching observed traffic dynamics, especially in congested scenarios.

Limitations and Future Work

  • Sensor Data Limitations: The paper notes that large density values are difficult to recover from flux and velocity data alone. Also, the data cannot distinguish between an empty road and a fully congested road when flux is zero.
  • Heuristic Limitations: The heuristics used for building the congestion dataset are not real-time applicable as they rely on future data (regularization).
  • Computational Cost: Training deep learning models can be computationally expensive.
  • Future Work: The authors suggest that training a different LSTM for each group of sensors is highly recommended due to dataset regularity differences. They also imply that further refinement of results could be achieved with more data. The paper sets the stage for using these ML-enriched data inputs within more complex macroscopic models.

Summary updated

Abstract

In this paper, we develop new methods to join machine learning techniques and macroscopic differential models, aimed at estimate and forecast vehicular traffic. This is done to complement respective advantages of data-driven and model-driven approaches. We consider here a dataset with flux and velocity data of vehicles moving on a highway, collected by fixed sensors and classified by lane and by class of vehicle. By means of a machine learning model based on an LSTM recursive neural network, we extrapolate two important pieces of information: (1) if congestion is appearing under the sensor, and (2) the total amount of vehicles which is going to pass under the sensor in the next future (30 min). These pieces of information are then used to improve the accuracy of an LWR-based first-order multi-class model describing the dynamics of traffic flow between sensors. The first piece of information is used to invert the (concave) fundamental diagram, thus recovering the density of vehicles from the flux data, and then inject directly the density datum in the model. This allows one to better approximate the dynamics between sensors, especially if an accident/bottleneck happens in a not monitored stretch of the road. The second piece of information is used instead as boundary conditions for the equations underlying the traffic model, to better predict the total amount of vehicles on the road at any future time. Some examples motivated by real scenarios will be discussed. Real data are provided by the Italian motorway company Autovie Venete S.p.A.
BibTeX
@article{briani2024inverting,
  author = {Maya Briani and Emiliano Cristiani and Elia Onofri},
  title = {{Inverting the fundamental diagram and forecasting boundary conditions: how machine learning can improve macroscopic models for traffic flow}},
  journal = {Adv. Comput. Math.},
  year = {2024},
  doi = {10.1007/s10444-024-10206-8},
}
Figure