Learning to Extrapolate and Adjust: Two-Stage Meta-Learning for Concept Drift in Online Time Series Forecasting

Weiqi Chen, Zhaoyang Zhu, Yifan Zhang, Lefei Shen, Linxiao Yang, Qingsong Wen, Liang Sun

Abstract

Concept drift—the evolution of statistical properties over time—poses a significant challenge to time series forecasting. Traditional models often struggle to adapt to these shifts, either lagging behind long-term trends or failing to capture short-term anomalies. This paper introduces LEAF (Learning to Extrapolate and Adjust for Forecasting), a model-agnostic meta-learning framework. LEAF decouples drift into macro-drift (long-term shifts) and micro-drift (sample-specific fluctuations). By operating in a low-dimensional latent space, LEAF employs an extrapolation module to predict future model parameters and a meta-learned surrogate loss for real-time sample adjustment. Empirical evaluations across diverse benchmarks, including new real-world electric load datasets, demonstrate that LEAF significantly reduces forecasting error and enhances the adaptability of various neural architectures.

Introduction

In dynamic environments like financial markets or energy grids, time series data rarely remains stationary. This phenomenon, known as concept drift, renders static forecasting models obsolete as the underlying data-generating processes change. While existing online learning methods attempt to mitigate drift through frequent retraining or reactive updates, they often suffer from "lag" during steady transitions or fail to account for idiosyncratic, sample-level variations.

The LEAF Framework: Decoupling Drift

The core innovation of the LEAF framework is the conceptual separation of concept drift into two distinct scales:

  1. Macro-Drift: Stable, inter-period changes that reflect long-term trends or seasonal shifts.
  2. Micro-Drift: Erratic, intra-period fluctuations that are specific to individual data samples.
Overview of LEAF: (a) types of drift in realistic datasets and (b) the two-stage learning process in latent space.

To handle these efficiently, LEAF maps model parameters into a low-dimensional latent space. This avoids the computational burden of updating entire high-dimensional parameter sets (like those in Transformers) while maintaining the flexibility to adapt.

Methodology: Extrapolation and Adjustment

LEAF operates through a two-stage meta-learning process:

1. Extrapolation Stage (Macro-Drift)

Instead of simply adopting the most recent model, LEAF "forecasts the forecaster." An extrapolation network (typically an LSTM) analyzes the trajectory of optimal latent embeddings ($H^*$) from previous periods to predict the embedding ($H_p$) for the upcoming period. This proactive approach allows the model to stay ahead of steady macro-drifts.

2. Adjustment Stage (Micro-Drift)

To address sample-specific micro-drifts, LEAF utilizes a meta-learned surrogate loss ($L^{surrogate}$). During inference, the model performs a quick gradient descent step on the predicted latent embedding for each specific input sample. This "nudges" the model parameters to better fit the immediate context before the final prediction is made.

Detailed view of the Meta-Learning Phase, illustrating the inner and outer optimization loops.

The entire system is optimized in an "outer loop" to minimize forecasting error on a meta-test set, ensuring that both the extrapolation and adjustment modules learn to facilitate effective adaptation.

Experimental Evaluation

The authors evaluated LEAF against state-of-the-art (SOTA) methods using standard benchmarks (ETT, ECL) and three new Electric Load datasets characterized by aggressive drift.

Key Results

  • Performance Gains: LEAF achieved an average 14.4% reduction in error (MSE/MAE) compared to SOTA online learning methods like FSNet and SOLID.
  • Model Agnosticism: The framework provided consistent improvements across various architectures, including MLPs, RNNs, and Transformers. Notably, standard LSTMs saw a 35.4% decrease in MSE when wrapped in LEAF.
  • Ablation Studies: Removing either the extrapolation or adjustment module resulted in significant performance degradation, confirming the necessity of addressing both macro and micro scales of drift.

Conclusion

LEAF represents a shift from reactive model updating to proactive model forecasting. By decoupling drift and leveraging latent space meta-learning, it provides a robust and efficient solution for online time series forecasting in non-stationary environments. Future work may explore combining LEAF with continual learning to prevent catastrophic forgetting over extended horizons.

Summary updated

Abstract

The non-stationary nature of time series data in many real-world applications makes accurate time series forecasting challenging. In this paper, we consider concept drift where the underlying distribution or environment of time series changes. We first classify concepts into two categories, macro-drift corresponding to stable and long-term changes and micro-drift referring to sudden or short-term changes. Next, we propose a unified meta-learning framework called LEAF (Learning to Extrapolate and Adjust for Forecasting). Specifically, an extrapolation module is first meta-learnt to track the dynamics of the prediction model in latent space and extrapolate to the future considering macro-drift. Then an adjustment module incorporates meta-learnable surrogate loss to capture sample-specific micro-drift patterns. Through this two-stage framework, different types of concept drifts can be handled. In particular, LEAF is model-agnostic and can be applied to any deep prediction model. To further advance the research of concept drift on time series, we open source three electric load time series datasets collected from real-world scenarios, which exhibit diverse and typical concept drifts and are ideal benchmark datasets for further research. Extensive experiments on multiple datasets demonstrate the effectiveness of LEAF.
BibTeX
@inproceedings{chen2025learning,
  author = {Weiqi Chen and Zhaoyang Zhu and Yifan Zhang and Lefei Shen and Linxiao Yang and Qingsong Wen and Liang Sun},
  title = {{Learning to Extrapolate and Adjust: Two-Stage Meta-Learning for Concept Drift in Online Time Series Forecasting}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/542},
}
Figure