TimePro: Efficient Multivariate Long-term Time Series Forecasting with Variable- and Time-Aware Hyper-state

Xiaowen Ma, Zhen-Liang Ni, Shuai Xiao, Xinghao Chen

Introduction

In multivariate time series forecasting, a critical challenge is the multi-delay issue: different variables often influence a target variable at varying time lags. For instance, in weather forecasting, temperature changes might impact energy demand immediately, whereas humidity changes might have a delayed effect. Existing models, including Transformers and standard State Space Models (SSMs) like Mamba, often fail to account for these non-uniform temporal relationships, processing all variables or time points with a rigid, uniform approach.

The TimePro Architecture

TimePro addresses these limitations by building upon the Mamba architecture, known for its linear computational complexity. The model introduces two primary innovations: a fine-grained embedding strategy and the HyperMamba module.

Overview of the TimePro architecture, illustrating the embedding process, the ProBlock structure, and the detailed HyperMamba module with its Hyper-Scan mechanism.

The core of TimePro is the Hyper-Scan mechanism within the HyperMamba block. Unlike the standard selective scan in Mamba, Hyper-Scan operates in three stages:

  1. Plain State Acquisition: The model scans across the variable dimension to capture global inter-variable correlations.
  2. Time-Tune Strategy: TimePro learns specific offsets to adaptively sample the most relevant historical time points for each variable. This allows the model to "look back" dynamically, similar to deformable convolutions in computer vision.
  3. Hyper-State Construction: These sampled points are used to adjust the initial state, creating a "hyper-state" that is simultaneously variable-aware and time-aware.

To maintain high efficiency, TimePro utilizes a hardware-aware design. Initial state acquisition is performed in the GPU's fast SRAM, while more complex time-tuning operations are handled in High Bandwidth Memory (HBM), ensuring the model retains linear $O(NL)$ complexity.

Experimental Evaluation

The authors evaluated TimePro against eight standard benchmarks, including ETT, Exchange, Weather, and ECL. TimePro consistently achieved state-of-the-art results, securing 12 first-place rankings across 16 metrics. On the Weather dataset, for example, it outperformed iTransformer by reducing MSE by 3.1%.

Performance and efficiency trade-offs: TimePro (bottom-left) achieves lower MAE with significantly fewer parameters, FLOPs, and lower training/inference times compared to other SOTA models.

Beyond accuracy, TimePro demonstrates exceptional efficiency. It requires only 67% of the parameters and 78% of the FLOPs of S-Mamba. In terms of speed, TimePro is 2.7x faster than PatchTST and 14.4x faster than TimesNet during inference. Crucially, its memory usage scales linearly with the number of variables and sequence length, allowing it to handle high-dimensional data where Transformer-based models typically fail due to quadratic scaling.

Conclusion

TimePro effectively solves the multi-delay issue in multivariate forecasting through its adaptive Hyper-Scan mechanism. By combining high predictive accuracy with linear efficiency, it sets a new standard for long-term time series forecasting. The authors envision TimePro as a foundational architecture for future large-scale time series prediction models.

Summary updated

Abstract

In long-term time series forecasting, different variables often influence the target variable over distinct time intervals, a challenge known as the multi-delay issue. Traditional models typically process all variables or time points uniformly, which limits their ability to capture complex variable relationships and obtain non-trivial time representations. To address this issue, we propose TimePro, an innovative Mamba-based model that constructs variate- and time-aware hyper-states. Unlike conventional approaches that merely transfer plain states across variable or time dimensions, TimePro preserves the fine-grained temporal features of each variate token and adaptively selects the focused time points to tune the plain state. The reconstructed hyper-state can perceive both variable relationships and salient temporal information, which helps the model make accurate forecasting. In experiments, TimePro performs competitively on eight real-world long-term forecasting benchmarks with satisfactory linear complexity. Code is available at https://github.com/xwmaxwma/TimePro.
BibTeX
@inproceedings{ma2025timepro,
  author = {Xiaowen Ma and Zhen-Liang Ni and Shuai Xiao and Xinghao Chen},
  title = {{TimePro: Efficient Multivariate Long-term Time Series Forecasting with Variable- and Time-Aware Hyper-state}},
  booktitle = {International Conference on Machine Learning},
  year = {2025},
  url = {https://openreview.net/forum?id=s69Ei2VrIW},
}
Figure