Spatial Copula Model for Imputing Traffic Flow Data from Remote Microwave Sensors

Xiaolei Ma, Sen Luan, Bowen Du, Bin YuSensors 2017 · OpenReview

TL;DR

This paper introduces a novel spatial copula-based model for imputing missing traffic flow data from remote microwave sensors. By leveraging copula theory to model complex dependencies between traffic flow and its marginal distributions, the proposed method significantly outperforms traditional kriging techniques, especially in scenarios with irregular traffic patterns. This advancement is crucial for improving the reliability of traffic data for urban planning and management.

What Problem Does This Paper Address?

Intelligent transportation systems rely heavily on accurate traffic flow data. However, a significant challenge arises from missing data, often due to equipment failure or processing errors, with some studies reporting up to 50% missing values from microwave sensors. This incompleteness hinders effective traffic analysis, planning, and management. Existing interpolation methods, primarily based on covariance functions, struggle with non-Gaussian data and complex traffic anomalies. The paper aims to address this by developing a more robust imputation method that can handle these complexities and provide accurate traffic flow estimates even with substantial missing data.

Methods

The core of the paper's methodology lies in the application of spatial copula models for traffic flow imputation. This approach departs from traditional methods by explicitly modeling the dependence structure between the marginal distributions of traffic flow and the spatial correlation.

Here's a breakdown of the key steps:

  • Correlation Function Fitting: The study first calculates the Euclidean distance between sensors and fits various correlation functions (Gaussian, Exponential, Spherical, Matern) to describe the spatial dependency of traffic flow. The function that minimizes the mean squared error (MSE) is selected as the optimal correlation function.
  • Marginal Distribution Fitting: Five default probability distribution functions (normal, GEV, gamma, log-normal, Box-Cox) are considered to fit the marginal distribution of traffic flow. The function that maximizes the probability is chosen as the optimal marginal distribution.
  • Parameter Estimation: The Canonical Maximum Likelihood (CML) method is employed to estimate the parameters of both the marginal distributions and the copula function. This method is chosen for its ability to handle uncertainty in the correlation and distribution functions.
  • Spatial Copula Interpolation: The trained copula model is then used to predict missing traffic flow values. This involves calculating weights for neighboring sensors based on their spatial correlation and then constructing a conditional probability density function (PDF) for the missing value, which is integrated to obtain the final imputed value.

The novelty lies in using copulas to link the marginal distributions with the spatial correlation, allowing for more flexible modeling of complex, non-Gaussian dependencies that are common in traffic flow data.

Datasets

The study utilizes data collected from 454 remote microwave sensors deployed on the 3rd and 5th ring expressways in Beijing. The data spans three specific dates in 2015:

  • June 1 (Children's day)
  • June 4 (a typical weekday)
  • June 7 (a typical weekend)

The data includes detector locations, timestamps, traffic flow, speed, and occupancy, with a data updating frequency of 2 minutes. The researchers randomly extracted 50% of the observed data for training and used the remaining 50% for testing. They also specifically incorporated data from the 3rd ring expressway at 8:00 on June 4 and 17:00 on June 7 to test different missing data types (Missing at Random - MR, and Missing Completely at Random - MCR).

Key Insights and Ideas

  • Copulas for Traffic Data: The paper demonstrates that copula theory is a powerful tool for modeling the complex, often non-Gaussian, dependencies present in traffic flow data, which traditional methods struggle with.
  • Spatial Dependency Matters: Explicitly modeling spatial correlation alongside marginal distributions is crucial for accurate imputation, especially in dynamic traffic environments.
  • Irregular Traffic is a Challenge: The 3rd ring expressway, characterized by more irregular traffic patterns, proved to be a more challenging testbed, highlighting the limitations of simpler models and the advantages of the proposed copula approach.
  • Time-Insensitive Performance: The copula-based models showed remarkably consistent performance across different timestamps, suggesting robustness to temporal variations in traffic conditions.
  • MR vs. MCR: The study found that the copula model significantly outperformed kriging for the "Missing at Random" (MR) type of missing data, which is more difficult to impute due to larger contiguous missing regions.

Key Results

The study's primary finding is that copula-based models significantly outperform traditional kriging methods for imputing missing traffic flow data.

  • General Superiority: Across various scenarios, the copula models consistently achieved lower Mean Absolute Percentage Error (MAPE) compared to SK, OK, and UK kriging methods.
  • 3rd Ring Expressway Performance: On the 3rd ring expressway, which exhibits more complex and irregular traffic patterns, the copula models showed a more pronounced advantage. For instance, in one comparison, the copula model achieved a MAPE of 0.1530, while the best kriging result was 0.2249.
  • MR Data Type Advantage: For the MR type of missing data, the copula model showed a substantial improvement (up to 10% increase in accuracy) over kriging methods.
  • Time-Insensitivity: The performance of the copula models was largely unaffected by the time of day or day of the week, indicating robustness.
  • Computational Complexity: While kriging methods were faster (around 0.72 seconds), the copula models' computational burden was manageable, especially with parallel processing, and the accuracy gains justified the increased computation.

Limitations and Future Work

  • Temporal Covariance Ignored: The current model does not explicitly consider temporal covariance, meaning it doesn't leverage information from previous time steps to improve imputation at the current time step.
  • One-Dimensional Road Segments: The model is currently applied to individual road segments rather than entire road networks, which could lead to more comprehensive imputation.
  • Computational Cost: While manageable, the computational complexity of copula models can be higher than simpler methods, especially with a large number of reference points.

The authors suggest several avenues for future research:

  • Spatial-Temporal Models: Incorporating temporal covariance to create a more comprehensive spatial-temporal imputation model.
  • Network-Wide Interpolation: Extending the model to handle entire road networks, including freeways and arterial roads, for a more holistic approach.
  • Sensor Deployment Optimization: Using the interpolation capabilities of copula-based models to identify optimal locations for deploying new traffic sensors.

Summary updated

Abstract

Issues of missing data have become increasingly serious with the rapid increase in usage of traffic sensors. Analyses of the Beijing ring expressway have showed that up to 50% of microwave sensors pose missing values. The imputation of missing traffic data must be urgently solved although a precise solution that cannot be easily achieved due to the significant number of missing portions. In this study, copula-based models are proposed for the spatial interpolation of traffic flow from remote traffic microwave sensors. Most existing interpolation methods only rely on covariance functions to depict spatial correlation and are unsuitable for coping with anomalies due to Gaussian consumption. Copula theory overcomes this issue and provides a connection between the correlation function and the marginal distribution function of traffic flow. To validate copula-based models, a comparison with three kriging methods is conducted. Results indicate that copula-based models outperform kriging methods, especially on roads with irregular traffic patterns. Copula-based models demonstrate significant potential to impute missing data in large-scale transportation networks.
BibTeX
@inproceedings{ma2017spatial,
  author = {Xiaolei Ma and Sen Luan and Bowen Du and Bin Yu},
  title = {{Spatial Copula Model for Imputing Traffic Flow Data from Remote Microwave Sensors}},
  booktitle = {Sensors},
  year = {2017},
  url = {https://openreview.net/forum?id=eyJud6ibaf},
}

Loading related content...

Figure