A Hybrid Model for Forecasting Traffic Flow: Using Layerwise Structure and Markov Transition Matrix

Shaokun Zhang, Zejian Kang, Zhemin Zhang, Congren Lin, Cheng Wang, Jonathan Li

Introduction

Predicting urban traffic flow is a critical yet challenging task due to the intricate interplay of temporal patterns, spatial correlations, and exogenous shocks such as weather and holidays. Traditional forecasting models often suffer from three primary limitations: a narrow focus on single factors, the neglect of spatial interactions between monitoring sites, and the "image fallacy," where deep learning models treat traffic networks as static pixel grids rather than constrained, flowing networks.

To address these gaps, researchers have developed a hybrid forecasting model that integrates layerwise neural networks with a Markov Transition Matrix (MTM). This approach captures the dynamic movement between sites while incorporating real-world road distances and Points of Interest (POIs) to enhance prediction accuracy.

Methodology: The Hybrid Architecture

The proposed model employs a multi-faceted architecture designed to capture the complexity of urban mobility through two primary components: a layerwise temporal structure and a statistical transition model.

Layerwise Temporal Structure

Rather than relying on a single monolithic network, the model utilizes three parallel components to analyze different temporal scales:

  • Long-term: Identifies weekly trends and seasonal shifts.
  • Medium-term: Captures daily periodicity and recurring driver habits.
  • Short-term: Tracks immediate, non-linear fluctuations in traffic volume.

These components can be implemented using various deep learning architectures, such as Long Short-Term Memory (LSTM) networks or Stacked Autoencoders (SAE).

Markov Transition Matrix (MTM)

The core innovation of this research is the integration of the Markov Transition Matrix. Instead of merely predicting a volume at a specific point, the model predicts a matrix of probabilities representing the likelihood of a vehicle transitioning from one site to another. This allows the system to learn the "attraction" between different urban zones, inherently understanding the spatial structure of the city.

The Markov Transition Matrix structure, representing the probability of transitioning from a current state (site j) to a next state (site i).

Spatial and Contextual Integration

To ground the model in reality, the researchers moved beyond simple Euclidean distance. By utilizing the Baidu Maps API, the model incorporates actual navigation distances, accounting for geographical barriers like rivers and mountains. Furthermore, the model treats Points of Interest (POIs)—such as shopping malls and schools—as "pseudo-monitoring sites" that generate or absorb traffic, providing critical context for flow fluctuations.

Experimental Setup and Dataset

The model was evaluated using the LPRXM dataset, derived from License Plate Recognition (LPR) systems in Xiamen, China. The dataset comprises over 216,000 data points collected from 169 detectors between April and August 2016.

Spatial distribution of the 169 LPR detectors across the road network of Xiamen, China.

In addition to traffic volume, the dataset includes 36 types of weather conditions, temperature data, holiday schedules, and 36 categories of POIs. The first four months of data were used for training, with the final month reserved for testing and validation.

Key Results and Insights

The hybrid model, specifically the Layerwise Stacked Autoencoder combined with MTM (LSAE-MTM), was tested against several baselines, including Support Vector Regression (SVR), Random Forest, and standard LSTMs.

  • Superior Accuracy: For 15-minute flow predictions, the LSAE-MTM achieved a Mean Absolute Error (MAE) of 44.11, significantly outperforming SVR (93.46) and standard LSTM (85.74).
  • The MTM Advantage: The inclusion of the Markov Transition Matrix consistently improved results across all test cases. For instance, a layerwise SAE without MTM recorded an MAE of 62.15, which dropped to 44.11 upon integration of the MTM.
  • POI Impact: Integrating POI data provided a notable boost for short-term forecasts, reducing the MAE of the LSAE-MTM model by approximately 9.32 for 30-minute windows. However, this impact was found to diminish as the forecasting horizon extended to 60 minutes.

Conclusion

This research demonstrates that capturing the dynamic movement between sites is as vital as tracking volume at a site. By combining the temporal depth of layerwise neural networks with the spatial intelligence of Markov Transition Matrices, the LSAE-MTM model provides a robust framework for urban traffic management. Future work will explore non-linear fusion methods for temporal components and the integration of real-time social event data to further refine prediction capabilities.

Summary updated

Abstract

Forecasting the traffic flow is greatly significant for traffic safety, energy conservation, and environmental protection. However, in the face of many external uncertainties, making accurate predictions about traffic volumes is a challenging issue. Many previous types of researche only explore the utility of a single factor in their prediction and rarely conduct the multi-factor research. As for the traffic flow prediction, many past types of researche focus primarily on the temporal distribution of the traffic flow on a single point on the road, ignoring the spatial correlation. In terms of global forecasting, it was logically far-fetched to mechanically view traffic as images. In this paper, considering the effects of many exogenous variables and the interaction between monitor sites, we propose a hybrid model to simultaneously predict the traffic flow in multiple positions by combining the layerwise structure and the Markov transition matrix (MTM). More specifically, we employ the layerwise structure to capture the periodicity, trend, and nonlinearity characteristics of traffic flow and, then, generate the MTM that captures the dynamics embodied in the data and produces the corresponding distributions. Considering the spatial correlation of traffic data, the real road network distance was thus introduced in our model. We apply the methodology on the real-world traffic data from Xiamen, and the experimental results show that the satisfactory predictions can be achieved using our model, which demonstrates the value of the transition matrix in traffic forecasts. In addition, we also introduce the point of interest and analyze its impact on the prediction results.
BibTeX
@article{zhang2019hybrid,
  author = {Shaokun Zhang and Zejian Kang and Zhemin Zhang and Congren Lin and Cheng Wang and Jonathan Li},
  title = {{A Hybrid Model for Forecasting Traffic Flow: Using Layerwise Structure and Markov Transition Matrix}},
  journal = {IEEE Access},
  year = {2019},
  doi = {10.1109/access.2019.2901118},
}
Figure