FreDF: Learning to Forecast in the Frequency Domain

Hao Wang, Licheng Pan, Zhichao Chen, Degui Yang, Sen Zhang, Yifei Yang, Xinggao Liu, Haoxuan Li, Dacheng Tao

TL;DR

FreDF (Frequency-enhanced Direct Forecast) identifies a fundamental theoretical bias in modern time series models caused by ignoring correlations between future time steps (label autocorrelation). By shifting the learning objective into the frequency domain, FreDF naturally decorrelates these labels, leading to significantly more accurate and sample-efficient forecasts across a wide variety of model architectures.

What Problem Does This Paper Address?

Most state-of-the-art time series models today follow the Direct Forecast (DF) paradigm. Instead of predicting one step at a time (which leads to error accumulation), they use a multi-output head to predict an entire future window simultaneously.

While efficient, the authors point out a major flaw: standard DF training assumes that each predicted time step is independent. In reality, time series data is highly autocorrelated—what happens on Tuesday is strongly dependent on what happened on Monday. The authors theoretically demonstrate that ignoring this "label autocorrelation" introduces a significant estimation bias into the model's learning objective. Essentially, the model is being trained with a "blind spot" regarding the internal structure of the future it is trying to predict.

Figure 1: Visualizing label autocorrelation in time series forecasting. (a) shows the generation process of time series with dependencies depicted as arrows. (b) shows the label correlation in the time domain, where each element ρi,j indicates the partial correlation between Yi and Yj given L. (c-d) shows the label correlation in the frequency domain, where each element ρi,j indicates the partial correlation between Fi and Fj given L, shown with the real (c) and imaginary part (d). Due to the symmetry inherent in FFT, the forecast length in the frequency domain is halved.

Methods

The authors propose FreDF, a model-agnostic training strategy that refines the standard learning objective. Instead of just minimizing error in the time domain (e.g., Mean Squared Error), FreDF introduces a dual-loss approach:

  1. Temporal Loss ($\mathcal{L}^{(tmp)}$): The standard MSE loss calculated in the time domain.
  2. Frequency Loss ($\mathcal{L}^{(feq)}$): A new loss term calculated in the frequency domain.

The core idea is to use the Fast Fourier Transform (FFT) to project both the ground-truth labels and the model's predictions onto a set of orthogonal sine and cosine bases. Because these bases are orthogonal, the complex correlations in the time domain are "unpacked" into nearly independent frequency components.

FreDF uses an $\ell_1$ loss in the frequency domain to ensure stability, as different frequencies can have vastly different magnitudes. A weighting parameter $\alpha$ balances the two losses. Because FFT is differentiable, this entire process can be optimized using standard backpropagation.

Figure 2: The workflow of FreDF. Key oper- ations in the time and frequency domains are highlighted in red and blue, respectively.

Datasets

The researchers evaluated FreDF across a comprehensive suite of standard time series benchmarks:

  • ETT (Electricity Transformer Temperature): Four subsets (ETTh1, ETTh2, ETTm1, ETTm2) with hourly and 15-minute resolutions.
  • Weather: 21 meteorological variables recorded every 10 minutes.
  • ECL (Electricity Consumption Load): Hourly data for 321 clients.
  • Traffic: Hourly road occupancy rates from 862 sensors.
  • PEMS (Public Traffic Network): Two subsets (PEMS03, PEMS08) with 5-minute windows.
  • M4: A large-scale dataset for short-term forecasting.

Key Insights and Ideas

  • The "Aha" Moment on Bias: The authors provide a formal proof (Theorem 3.1) showing that the standard Direct Forecast objective is biased against the true Negative Log-Likelihood (NLL) when label autocorrelation exists.
  • Frequency as a Decorrelator: The most striking insight is that the frequency domain acts as a natural "cleansing" environment for autocorrelation. Theorem 3.3 proves that Fourier coefficients become asymptotically uncorrelated, making the independent-step assumption of the DF paradigm actually valid in that space.
  • Sample Efficiency: FreDF is remarkably efficient. The authors found that learning in the frequency domain with only 30% of the training data achieved performance comparable to standard time-domain learning using the full dataset.
  • Versatility: FreDF isn't a new model; it's a new way to train models. It successfully boosted the performance of Transformers (like iTransformer and Autoformer), MLPs (DLinear), and CNNs (TimesNet).

Key Results

FreDF consistently outperformed state-of-the-art baselines across long-term forecasting, short-term forecasting, and data imputation tasks.

  • Accuracy Gains: When applied to the iTransformer model, FreDF reduced the Mean Squared Error (MSE) on the ETTm1 dataset by approximately 4.6% and on the Weather dataset by 9.6%.
  • Qualitative Improvement: Visualizations show that models trained with FreDF are much better at capturing high-frequency components and sharp transitions, whereas standard models often produce "blurry" or overly smoothed forecasts.
  • Robustness: The method showed consistent improvements regardless of the input sequence length or the specific choice of orthogonal basis (e.g., Legendre polynomials also worked well, though Fourier was generally best).

Limitations and Future Work

  • Fixed Bases: The current work primarily uses predefined Fourier bases. The authors note that these might not perfectly adapt to every specific data property. They suggest exploring learned orthogonal bases or techniques like Independent Component Analysis (ICA) in the future.
  • Beyond Time Series: While focused on time series, the authors hypothesize that label autocorrelation exists in other structured data like speech, images, and 3D point clouds. Applying the FreDF philosophy to these domains is a promising next step.

Summary updated

Abstract

Time series modeling presents unique challenges due to autocorrelation in both historical data and future sequences. While current research predominantly addresses autocorrelation within historical data, the correlations among future labels are often overlooked. Specifically, modern forecasting models primarily adhere to the Direct Forecast (DF) paradigm, generating multi-step forecasts independently and disregarding label autocorrelation over time. In this work, we demonstrate that the learning objective of DF is biased in the presence of label autocorrelation. To address this issue, we propose the Frequency-enhanced Direct Forecast (FreDF), which mitigates label autocorrelation by learning to forecast in the frequency domain, thereby reducing estimation bias. Our experiments show that FreDF significantly outperforms existing state-of-the-art methods and is compatible with a variety of forecast models. Code is available at https://github.com/Master-PLC/FreDF.
BibTeX
@inproceedings{wang2025fredf,
  author = {Hao Wang and Licheng Pan and Zhichao Chen and Degui Yang and Sen Zhang and Yifei Yang and Xinggao Liu and Haoxuan Li and Dacheng Tao},
  title = {{FreDF: Learning to Forecast in the Frequency Domain}},
  booktitle = {International Conference on Learning Representations},
  year = {2025},
  url = {https://openreview.net/forum?id=4A9IdSa1ul},
}
Figure