Mitigating Non-Stationary Distribution Shift in Time-Series Forecasting

1,089 concepts822 papers27 proposals

Deep learning models typically operate under the assumption that training and testing data are drawn from the same underlying distribution. In time-series forecasting, this assumption frequently breaks down. Real-world data—such as electricity consumption, traffic flows, or financial indices—is inherently non-stationary: its statistical properties, such as mean, variance, and trend, drift continuously over time.

When a model trained on historical data encounters a new temporal window with a significantly shifted mean or variance, its performance degrades rapidly. The model is forced to generalize to out-of-distribution inputs, leading to large prediction errors.

The Stationarization Dilemma

To combat non-stationarity, a natural first step is stationarization—transforming the data so that its statistical properties remain constant over time. A common way to achieve this is by applying instance normalization to each input sequence, subtracting its mean and dividing by its standard deviation.

However, this introduces a fundamental dilemma:

  1. Without stationarization, the model struggles to handle inputs whose absolute values fall outside the range seen during training.
  2. With stationarization, the model loses all information about the absolute scale and trend of the series. If you normalize a sequence representing a rising stock price, the model only sees the relative fluctuations and cannot know the actual price level to predict the next absolute value.

This tension has driven recent research toward architectures that can stationarize data for internal processing while preserving the context necessary to make accurate, real-world predictions.

Reversible Instance Normalization (RevIN)

An elegant solution to this dilemma was proposed by Taesung Kim et al. in Reversible Instance Normalization for Accurate Time-Series Forecasting against Distribution Shift.

Instead of permanently stripping away the mean and variance, the authors introduce a symmetrical, two-step wrapper around any backbone forecasting model:

$$\hat{x} = \frac{x - \mu_x}{\sigma_x}$$

  • Normalization: Before entering the model, the input sequence $x$ is normalized using its own mean $\mu_x$ and standard deviation $\sigma_x$. This maps the input into a stable, zero-mean, unit-variance distribution that the backbone model can easily process without encountering out-of-distribution scaling issues.
  • Forecasting: The backbone model predicts the future values $\hat{y}$ based on the stationarized input $\hat{x}$.
  • Denormalization: To restore the absolute scale, the model's output is symmetrically denormalized using the original input statistics:

$$y = \hat{y} \cdot \sigma_x + \mu_x$$

By reversing the normalization process at the very end, RevIN allows the model to learn invariant temporal patterns while seamlessly adapting its final predictions to the current distribution shift.

De-stationary Attention

While Reversible Instance Normalization handles the global scale shift, it does not fully solve how internal model mechanisms—specifically self-attention—behave when data is stationarized. As explored by Yong Liu et al. in Non-stationary Transformers: Exploring the Stationarity in Time Series Forecasting, standard stationarization can inadvertently diminish the structural differences between different time windows.

When sequences are normalized independently, their relative variances are flattened. In a standard Transformer, the attention mechanism computes similarities based on these flattened values, which can lead to over-smoothing and a loss of critical temporal context.

To fix this, the authors propose De-stationary Attention. Instead of computing attention directly on the stationarized tokens, the mechanism calculates a correction factor based on the original un-stationarized statistics ($\mu$ and $\sigma$). This correction factor is injected back into the attention matrix, effectively allowing the Transformer to compute attention weights as if it were looking at the un-stationarized data, while still benefiting from the training stability of normalized inputs.

Disentangling Invariant and Variant Patterns

Another prominent strategy for tackling temporal distribution shift is to explicitly separate the data into stable, predictable components and volatile, shifting components.

For instance, Haihua Xu 0005 et al. developed the Decoupled Invariant Attention Network (DIAN), which splits multivariate time-series data into spatial and temporal "invariant" and "variant" patterns. To make the model robust against non-stationarity, the authors explicitly simulate distribution shifts during training by swapping volatile components across different timestamps, forcing the network to rely on features that remain consistent over time.

Similarly, decomposing signals into the frequency domain can isolate stable periodic behaviors from sudden environmental or economic shocks. Models like MSGNet by Wanlin Cai et al. leverage frequency-domain analysis to break down complex signals into underlying periodic components, ensuring that the model's core predictive capacity remains robust even when the global trend undergoes a sudden regime shift.

By combining input-level normalization wrappers, attention-level corrections, and structural pattern decomposition, modern time-series architectures can successfully navigate the moving target of non-stationary data.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 780 papers
  2. Version 1 · 2026-10-04 · 845 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_temporal_distribution_shift_non_stationa_eb10a814,
  title = {temporal distribution shift (non-stationary distribution shift)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/temporal-distribution-shift-non-stationary-distribution-shift/rdr_eb10a814e7c14c0c843218a16ece87e9},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 2, frozen text):

@misc{lacuna_temporal_distribution_shift_non_stationa_eb10a814_v2,
  title = {temporal distribution shift (non-stationary distribution shift)},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/temporal-distribution-shift-non-stationary-distribution-shift/rdr_eb10a814e7c14c0c843218a16ece87e9/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake