Efficiently Enhancing Long-term Series Forecasting via Ultra-long Lookback Windows

Suxin Tong, Jingling YuanAAAI 2025 · DOI

Introduction

In long-term time series forecasting (LTSF), the "lookback window"—the extent of historical data available to a model—is a critical hyperparameter. Intuitively, longer historical contexts should enable models to better capture long-term trends and seasonal cycles. However, state-of-the-art models such as Transformers and CNNs frequently encounter a performance ceiling or computational barrier when lookback windows extend into "ultra-long" territory (e.g., 3000+ time steps).

The authors identify three primary challenges hindering the use of ultra-long windows:

  1. Overfitting: Extended windows often introduce more noise than signal, leading models to memorize historical anomalies rather than learning generalizable patterns.
  2. Computational Constraints: Many architectures, particularly those based on attention mechanisms, suffer from quadratic memory complexity. This often results in "Out of Memory" (OOM) errors during training.
  3. Information Extraction: Distilling relevant predictive patterns from massive historical datasets remains difficult for standard architectures.

Methodology: The IRPA Framework

To address these issues, the authors propose IRPA (Input Refinement and Prediction Auxiliary), a lightweight, "plug-and-play" framework consisting of only four linear layers. IRPA is designed to augment existing forecasting models by effectively leveraging ultra-long history without the associated computational overhead.

The IRPA framework architecture featuring the Input Refinement and Prediction Auxiliary modules.

The framework operates through two core components:

  • Input Refinement Module (IRM): Rather than processing the entire ultra-long series through a heavy backbone model, the IRM uses the long history to "refine" a shorter, manageable input window. It decomposes the series into seasonal and trend components and identifies the most similar historical patches (using Hellinger distance) to enrich the current input.
  • Prediction Auxiliary Module (PAM): This module provides auxiliary guidance by searching the ultra-long history for patches similar to the current context and observing subsequent historical outcomes. This "future-from-the-past" information serves as a predictive hint for the final forecast.

By relying on linear layers and efficient similarity searches, IRPA maintains a computational complexity of $O(l)$, where $l$ is the lookback length, making it significantly more efficient than attention-based alternatives.

Experimental Evaluation

The framework was evaluated on eight widely recognized public benchmarks, including the ETT (Electricity Transformer Temperature) datasets, Weather, Solar, ECL (Electricity Consumption Load), and Traffic. These datasets vary significantly in feature count and sampling frequency, providing a robust testing ground for the framework's versatility.

Key Results and Insights

IRPA demonstrated significant performance improvements across all tested baseline models. On average, it reduced Mean Squared Error (MSE) by 9.7% for iTransformer, 15.0% for PatchTST, 14.4% for TimesNet, and 25.2% for the DLinear model.

MSE performance across different lookback window lengths, highlighting IRPA's stability and avoidance of OOM errors.

A key finding is that the backbone model does not need to process the entire ultra-long window directly. Instead, the ultra-long window serves as a reference library to augment a smaller input. Furthermore, the research highlights a "Frequency-Window Scaling" principle: data with higher sampling frequencies requires proportionally longer lookback windows to achieve optimal forecasting accuracy.

In terms of efficiency, IRPA-enhanced models successfully avoided OOM errors that plagued standard architectures at high lookback lengths. For instance, on the Weather dataset, while a standard PatchTST model failed at a lookback window of 3600, the IRPA-enhanced version maintained a low memory footprint while achieving superior accuracy.

Conclusion and Future Work

IRPA provides a robust solution for scaling time-series models to ultra-long historical contexts. While the optimal lookback window still varies by dataset, the framework's linear efficiency makes it a viable candidate for integration into large-scale time-series foundation models. Future research may explore automated window selection or the use of lightweight non-linear components to further enhance information extraction.

Summary updated

BibTeX
@inproceedings{tong2025efficiently,
  author = {Suxin Tong and Jingling Yuan},
  title = {{Efficiently Enhancing Long-term Series Forecasting via Ultra-long Lookback Windows}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i20.35386},
}
Figure