Abstract
Spatio-temporal forecasting is crucial in many domains, such as transportation, meteorology, and energy. However, real-world scenarios frequently present challenges such as signal anomalies, noise, and distributional shifts. Existing solutions primarily enhance robustness by modifying network architectures or training procedures. Nevertheless, these approaches are computationally intensive and resource-demanding, especially for large-scale applications. In this paper, we explore a novel test-time computing paradigm, namely learning with calibration, ST-TTC, for spatio-temporal forecasting. Through learning with calibration, we aim to capture periodic structural biases arising from non-stationarity during the testing phase and perform real-time bias correction on predictions to improve accuracy. Specifically, we first introduce a spectral-domain calibrator with phase-amplitude modulation to mitigate periodic shift and then propose a flash updating mechanism with a streaming memory queue for efficient test-time computation. ST-TTC effectively bypasses complex training-stage techniques, offering an efficient and generalizable paradigm. Extensive experiments on real-world datasets demonstrate the effectiveness, universality, flexibility and efficiency of our proposed method.
The Challenge of Non-Stationarity in Spatio-Temporal Forecasting
Spatio-temporal forecasting (STF)—predicting variables like traffic flow, air quality, or energy demand—is notoriously difficult because real-world data is non-stationary. Sensors age, seasons change, and unexpected events like road construction cause significant distribution shifts.
Current solutions generally fall into two categories:
- Out-of-Distribution (OOD) Learning: These methods attempt to build robustness during the training phase. However, they often struggle because it is impossible to anticipate every future scenario.
- Continual Fine-Tuning: This involves updating the entire model as new data arrives. While adaptive, it is computationally expensive and often too slow for real-time applications where predictions are required every few minutes.
There is a clear need for a method that adapts models during inference (test-time) in a way that is fast, memory-efficient, and does not require modifying the complex "backbone" architecture of the model.
Proposed Method: ST-TTC
The authors introduce ST-TTC (Spatio-Temporal Test-Time Computing), a "plug-and-play" framework that calibrates predictions in real-time. Unlike traditional methods that update the entire model, ST-TTC keeps the backbone model frozen and only optimizes a lightweight calibrator.
ST-TTC consists of two synergistic components:
- Spectral Domain Calibrator (SD-Calibrator): Instead of correcting errors in the raw time-series domain, the model converts predictions into the frequency domain using a Fast Fourier Transform (FFT). It then applies "Phase-Amplitude Modulation" to tweak the amplitude and phase of specific frequency components. This approach is highly efficient at capturing periodic shifts—such as a rush hour starting slightly later than usual—which often manifest as linear corrections in the spectral domain.
- Flash Gradient Update with Streaming Memory Queue: To maintain the calibrator's accuracy, the system uses a small FIFO (First-In-First-Out) queue of recent data. When a new ground-truth label is received, it performs a single-sample, single-step gradient descent on only the calibrator parameters. This "flash" update ensures the process is lightning-fast and avoids the "catastrophic forgetting" associated with updating the main backbone.
Experimental Evaluation
The researchers evaluated ST-TTC across 14 real-world datasets spanning traffic (PEMS, METR-LA, LargeST), meteorology (KnowAir), and energy (UrbanEV). These datasets range from small networks to massive state-level grids, testing the framework's scalability.
Key Results
- Universal Improvement: ST-TTC consistently improved the performance of various backbones, including Transformers, Graph Neural Networks (GNNs), and MLPs.
- Efficiency: Compared to existing test-time adaptation methods, ST-TTC was 4.64x faster and reduced memory usage by 37.12%.
- Few-Shot & OOD Performance: The method excelled in data-scarce scenarios. In few-shot tests (using only 10% of training data), ST-TTC provided significant boosts, particularly on datasets like KnowAir where distribution shifts are frequent.
- Scalability: On the LargeST benchmark involving tens of thousands of nodes, the additional inference time was less than 4 minutes, a negligible overhead compared to the 14-hour training time of the base model.
Insights and Future Directions
A key insight of this work is that many complex distribution shifts in spatio-temporal data are actually simple shifts in periodicity or intensity. By operating in the spectral domain, these become linear problems that a tiny module can solve. Furthermore, freezing the backbone prevents "information leakage" and instability, proving that one doesn't need to change the model's "brain" to fix its "output."
Future work includes testing this calibration framework on emerging spatio-temporal foundation models and exploring retrieval-augmented techniques to further optimize the calibrator's learning process by finding similar historical patterns.
Summary updated