Spatiotemporal-Linear: Towards Universal Multivariate Time Series Forecasting

Aiyinsi Zuo, Haixi Zhang, Zirui Li, Ce ZhengarXiv 2023 · DOI · arXiv

Abstract

The SpatioTemporal-Linear (STL) framework addresses a significant limitation in current linear forecasting models: their performance degradation in "data-scarce" environments where historical observations are sparse. By augmenting a linear core with specialized spatial and temporal bypasses, STL achieves state-of-the-art performance across both complex Transformer-based and simple Linear benchmarks, offering a robust solution for diverse multivariate time series tasks.

The Challenge of Data Scarcity and Context

Multivariate Time Series Forecasting (TSF) is critical for applications ranging from financial modeling to autonomous vehicle trajectory prediction. Recently, the field has seen a shift toward simple Linear models (e.g., LTSF-Linear, DLinear) which often outperform complex Transformers by avoiding error accumulation and maintaining computational efficiency. However, these models suffer from two primary drawbacks:

  1. Data Scarcity Sensitivity: Linear models typically require long historical sequences to capture underlying patterns. In domains like disease tracking or real-time robotics, such long histories are often unavailable.
  2. Contextual Blindness: Most linear models treat variables as independent (univariate approach), ignoring the spatial correlations between different sensors and the temporal context (e.g., seasonality, time-of-day).

STL bridges this gap by introducing a "universal" architecture capable of handling both information-rich and data-limited scenarios.

The STL Framework

The STL framework employs a parallel "three-route" architecture to process input sequences and produce predictions. This design ensures that the model remains grounded in linear efficiency while incorporating necessary complexities for refinement.

The STL architecture compared to DLinear, highlighting the Core, Temporal, and Spatial routes.
  • The Core Route: Utilizes a Residual Linear (Res-L) layer. This structure combines a standard linear transformation with a skip connection, allowing the model to preserve the primary signal while learning non-linear refinements via an activation function (SiLU or LeakyReLu).
  • The Temporal Route: Focuses on periodic and calendar-based features. It integrates sinusoidal positional embeddings with date-time embeddings. A dynamic gating mechanism regulates the influence of this route; it is most active when observation length $T$ is short, compensating for the lack of historical data.
  • The Spatial Route: Addresses inter-variable dependencies using a Spatial Attention (SpatialAttn) mechanism. By computing interaction scores between variables, the model can rectify predictions based on how different channels influence one another.

The authors provide a mathematical proof (Appendix C) that univariate linear layers produce "fragmentary" coefficients that fail to capture the full spatiotemporal manifold. STL's bypasses serve to "complete" this information through a coarse-refine strategy.

Experimental Setup and Results

STL was benchmarked against five major datasets, including high-dimensional benchmarks (Electricity, Weather), electric transformer data (ETTh1, ETTm1), and the JAAD dataset for autonomous driving trajectory prediction.

Performance comparison (1/MSE) across varying prediction sequences (left) and observation sequences (right). Higher values indicate better accuracy.

STL consistently outperformed both Transformer-based models (Informer, Autoformer, FEDformer) and Linear baselines (DLinear, NLinear). Key findings include:

  • Superior Accuracy: In standard scenarios, STL achieved up to a 14.3% improvement in MSE over DLinear.
  • Resilience to Short History: In data-scarce settings ($T=48$), STL showed a 34% improvement on standard benchmarks and a massive 55% improvement on the JAAD trajectory dataset. While DLinear's performance dropped by nearly 50% when history was shortened, STL remained remarkably stable.
  • Visual Rectification: Qualitative analysis confirms that STL's auxiliary routes prevent the "drift" common in vanilla linear models, effectively pulling predictions back toward the ground truth during volatile shifts.

Conclusion and Future Work

While STL represents a significant step forward, challenges remain. The temporal route's activation threshold is currently a hyperparameter requiring manual tuning, and the inclusion of attention mechanisms increases parameter counts relative to basic linear models. However, STL demonstrates that by treating spatiotemporal context as a first-class citizen within a linear framework, we can create forecasting models that are both simple and robust enough for the messy, data-limited realities of the real world.

Summary updated

BibTeX
@misc{zuo2023spatiotemporal,
  author = {Aiyinsi Zuo and Haixi Zhang and Zirui Li and Ce Zheng},
  title = {{Spatiotemporal-Linear: Towards Universal Multivariate Time Series Forecasting}},
  year = {2023},
  eprint = {2312.14869},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2312.14869},
}
Figure