Abstract
Correlated Time Series (CTS) data, such as traffic flow and weather patterns, present unique challenges for representation learning due to complex spatiotemporal dependencies. This paper introduces a self-supervised framework that leverages spatiotemporal bootstrapping to learn robust representations. By predicting the features of neighboring sensors and future time steps from a masked input view, the model avoids the pitfalls of traditional contrastive learning, such as false negatives in periodic data. Experimental results on real-world datasets show that the proposed method significantly outperforms existing state-of-the-art models in forecasting accuracy and excels in cold-start scenarios for newly integrated sensors.
Introduction
Correlated Time Series (CTS) are ubiquitous in modern infrastructure, generated by networks of geographically or logically linked sensors. Effectively modeling these series is critical for applications like urban planning and traffic management. However, current self-supervised learning methods face two primary hurdles:
- Granularity Mismatch: Many methods produce instance-level representations (one vector per window), which are insufficient for fine-grained forecasting tasks requiring point-wise understanding.
- Neglect of Relational Context: Standard models often treat sensors in isolation, failing to exploit spatial correlations (e.g., traffic at adjacent stations) or temporal periodicities (e.g., weekly traffic cycles).
Furthermore, contrastive learning—the dominant paradigm—often suffers from "false negatives" in time series data, where similar periodic patterns are incorrectly treated as distinct, confusing the model during training.
Methodology
The proposed framework adopts a bootstrapping approach inspired by BYOL (Bootstrap Your Own Latent), eliminating the need for negative samples.
Architecture
The system utilizes a dual-encoder setup consisting of an Online Encoder, which actively learns by processing a masked "view" of the input data, and a Target Encoder, which provides stable target representations. The Target Encoder is updated via an exponential moving average (EMA) of the online encoder's weights, ensuring consistent learning signals.
Spatiotemporal Bootstrapping
The core innovation lies in the prediction targets. Given a masked sub-series, the model is trained to predict two distinct targets:
- Spatial Target: The representation of a neighboring sensor at the same timestamp.
- Temporal Target: The representation of the same sensor at a future timestamp.
To capture long-range dependencies without violating causality, the backbone employs Dilated Causal CNNs, which provide a wide receptive field while preventing information leakage from future time steps.
Experimental Evaluation
The model was rigorously tested on three major datasets: METR-LA (highway traffic flow), PEMS-BAY (traffic speed), and a Hong Kong Metro dataset. The latter was specifically used to evaluate "cold-start" performance, testing the model's ability to predict flow for three newly opened stations with minimal historical records.
Results and Discussion
The proposed method demonstrated superior performance across all benchmarks. On the PEMS-BAY dataset, the model achieved a 49% reduction in Mean Absolute Error (MAE) and a 37% reduction in Root Mean Squared Error (RMSE) compared to the previous state-of-the-art, TS2Vec.
Qualitative Analysis
A key strength of the model is its ability to track abrupt changes in data streams. While baseline models often smooth out sharp peaks or valleys, the proposed framework accurately captures these dynamics, as shown in the comparison of traffic speed predictions.
Robustness and Transferability
The model exhibited high resilience to missing data; even with a 60% masking rate, performance remained stable. In the Hong Kong Metro cold-start task, the model improved prediction accuracy by 15-19% over baselines, proving its effectiveness in transferring learned spatiotemporal patterns to new nodes in a network.
Conclusion
By integrating spatial and temporal bootstrapping into a self-supervised framework, this work addresses the fundamental limitations of existing CTS representation learning. The model's reliance on predicting neighbors and future states allows it to capture the inherent structure of sensor networks without the need for negative sampling. Future research will focus on relaxing the dependency on a fixed adjacency matrix and extending the framework to anomaly detection tasks.
Summary updated