Abstract
Reliable traffic data is fundamental to Intelligent Transportation Systems (ITS). However, real-world data often suffers from missing values and outliers due to sensor failures or communication issues. While Low-Rank Tensor Completion (LRTC) is a popular recovery technique, existing models typically require manual hyperparameter tuning. This paper introduces a Parameter-free Nonconvex (PFNC) model that utilizes a log-based regularizer to recover spatiotemporal traffic data accurately without manual calibration. We propose two variants: TC-PFNC for missing data recovery and RTC-PFNC for robust recovery in the presence of outliers. Evaluations on four real-world datasets show that our approach outperforms state-of-the-art methods in both accuracy and robustness.
1. Introduction
Intelligent Transportation Systems rely on high-quality spatiotemporal data for traffic forecasting and management. Unfortunately, traffic data is frequently "dirty," characterized by random missing values, non-random blocks of missing data, and anomalous outliers. Traditional recovery methods based on convex nuclear norm minimization often lead to biased results by over-shrinking significant singular values. Conversely, nonconvex methods, while more accurate, often require sensitive hyperparameters like the truncation rank, which vary across different datasets and cities.
2. Problem Formulation and Data Representation
Traffic data is inherently low-rank due to daily periodicities and spatial correlations between adjacent sensors. To exploit this structure, we organize traffic data into a 3D tensor $\mathcal{Y}$ with dimensions representing Space, Time-of-Day, and Day.
As shown in Figure 1, the raw time series data contains various artifacts. By transforming this data into a tensor, we can leverage multi-dimensional correlations to fill gaps (Random and Nonrandom Missing) and identify potential outliers. This tensor representation allows the model to capture the "pulse" of the city across different temporal scales simultaneously.
3. Methodology
The core innovation of this work is a log-based nonconvex regularizer: $$f(\mathbf{X}) = \sum_{i=1}^{n} \log(\sigma_i(\mathbf{X}) + \epsilon)$$ This function serves as a smooth approximation of the rank function, penalizing smaller singular values (noise) more heavily than larger ones (signal). Unlike standard nuclear norm methods, this approach avoids the "over-shrinking" of primary traffic patterns.
3.1 TC-PFNC and RTC-PFNC
We develop two models based on this regularizer:
- TC-PFNC: A tensor completion model optimized for scenarios with missing data.
- RTC-PFNC: A robust version that decomposes the observed tensor $\mathcal{Y}$ into a low-rank component $\mathcal{L}$ and a sparse anomaly component $\mathcal{E}$ (i.e., $\mathcal{Y} = \mathcal{L} + \mathcal{E}$).
Both models are solved using the Alternating Direction Method of Multipliers (ADMM), an iterative framework that ensures efficient convergence without the need for manual parameter tuning, making the solution highly "plug-and-play" for traffic engineers.
4. Experimental Evaluation
We evaluated the models on four diverse real-world datasets:
- PeMS (P): Traffic volume from 228 loop detectors in California.
- Seattle (S): Traffic speed from 323 loop detectors.
- Guangzhou (G): Urban traffic speed from 214 road segments.
- Birmingham (B): Parking occupancy from 30 car parks.
The models were tested against two missing patterns: Random Missing (RM) and Non-random Missing (NM), where entire days or sensor "fibers" are absent.
5. Key Results
The TC-PFNC model consistently achieved superior accuracy compared to state-of-the-art baselines like HaLRTC and LRTC-TNN.
As illustrated in Figure 5, TC-PFNC (red line) closely tracks the ground truth (blue line) across all four datasets, even in regions with significant missing data (indicated by green shaded areas). In the PeMS dataset with 60% missing data, TC-PFNC achieved a Mean Absolute Percentage Error (MAPE) of 3.35%, significantly lower than the 4.77% achieved by the best baseline. Furthermore, RTC-PFNC demonstrated exceptional stability in corrupted environments, maintaining low error rates even when 70% of observations were outliers.
6. Conclusion
The proposed PFNC framework offers a robust, parameter-free solution for traffic data recovery. By utilizing a log-based nonconvex regularizer, the model avoids the bias of convex methods and the parameter sensitivity of existing nonconvex approaches. Future work will focus on incorporating spatial network topology and scaling the ADMM solver for real-time, city-wide applications.
Summary updated