Introduction
In multivariate time series forecasting, a critical challenge is the multi-delay issue: different variables often influence a target variable at varying time lags. For instance, in weather forecasting, temperature changes might impact energy demand immediately, whereas humidity changes might have a delayed effect. Existing models, including Transformers and standard State Space Models (SSMs) like Mamba, often fail to account for these non-uniform temporal relationships, processing all variables or time points with a rigid, uniform approach.
The TimePro Architecture
TimePro addresses these limitations by building upon the Mamba architecture, known for its linear computational complexity. The model introduces two primary innovations: a fine-grained embedding strategy and the HyperMamba module.
The core of TimePro is the Hyper-Scan mechanism within the HyperMamba block. Unlike the standard selective scan in Mamba, Hyper-Scan operates in three stages:
- Plain State Acquisition: The model scans across the variable dimension to capture global inter-variable correlations.
- Time-Tune Strategy: TimePro learns specific offsets to adaptively sample the most relevant historical time points for each variable. This allows the model to "look back" dynamically, similar to deformable convolutions in computer vision.
- Hyper-State Construction: These sampled points are used to adjust the initial state, creating a "hyper-state" that is simultaneously variable-aware and time-aware.
To maintain high efficiency, TimePro utilizes a hardware-aware design. Initial state acquisition is performed in the GPU's fast SRAM, while more complex time-tuning operations are handled in High Bandwidth Memory (HBM), ensuring the model retains linear $O(NL)$ complexity.
Experimental Evaluation
The authors evaluated TimePro against eight standard benchmarks, including ETT, Exchange, Weather, and ECL. TimePro consistently achieved state-of-the-art results, securing 12 first-place rankings across 16 metrics. On the Weather dataset, for example, it outperformed iTransformer by reducing MSE by 3.1%.
Beyond accuracy, TimePro demonstrates exceptional efficiency. It requires only 67% of the parameters and 78% of the FLOPs of S-Mamba. In terms of speed, TimePro is 2.7x faster than PatchTST and 14.4x faster than TimesNet during inference. Crucially, its memory usage scales linearly with the number of variables and sequence length, allowing it to handle high-dimensional data where Transformer-based models typically fail due to quadratic scaling.
Conclusion
TimePro effectively solves the multi-delay issue in multivariate forecasting through its adaptive Hyper-Scan mechanism. By combining high predictive accuracy with linear efficiency, it sets a new standard for long-term time series forecasting. The authors envision TimePro as a foundational architecture for future large-scale time series prediction models.
Summary updated