FreqMoE: Enhancing Time Series Forecasting through Frequency Decomposition Mixture of Experts

Ziqi LiuAISTATS 2025

Abstract

FreqMoE introduces a novel architecture for time series forecasting that leverages a dynamic Mixture-of-Experts (MoE) framework in the frequency domain. By replacing static frequency filters with adaptive gating mechanisms, FreqMoE efficiently weights different frequency bands based on input characteristics. This approach achieves state-of-the-art performance on 51 out of 70 standard benchmarks while maintaining a remarkably small parameter footprint (under 50k), significantly outperforming much larger Transformer-based models in both accuracy and computational efficiency.

1. Introduction

Traditional time series forecasting models often struggle with non-stationary patterns, such as complex seasonality and abrupt trends. While recent advancements have shifted focus toward frequency-domain analysis using the Fast Fourier Transform (FFT), many existing methods rely on rigid, static filtering. Specifically, high-frequency signals are frequently dismissed as noise and discarded via low-pass filters. This "one-size-fits-all" strategy can lead to the loss of critical information. FreqMoE addresses this limitation by enabling the model to dynamically determine the relevance of various frequency bands for each specific data sample.

2. Methodology

The FreqMoE architecture is built upon two core components designed to handle frequency-domain information adaptively.

The FreqMoE architecture, featuring the Frequency Decomposition MoE Block and the Residual-connected Prediction Stack.
  • Frequency Decomposition MoE Block: The input time series is first transformed into the frequency domain via FFT. The resulting spectrum is partitioned into disjoint regions, each processed by a specialized "Expert" consisting of complex-valued linear layers. A Gating Network analyzes the input's frequency magnitudes to dynamically assign weights to these experts, allowing the model to prioritize frequency bands containing the most relevant patterns at any given time.
  • Residual-connected Prediction Stack: Instead of a single-pass prediction, FreqMoE employs a stack of prediction blocks. Each block upsamples the frequency representation to project it into the future. The model iteratively refines its forecast by focusing on the residuals (errors) from preceding blocks. Finally, an inverse FFT (iFFT) converts the data back to the time domain for the final output.

3. Experiments and Results

FreqMoE was evaluated across a diverse set of real-world datasets, including long-term forecasting (ETT, Weather, ECL, Exchange Rate) and short-term forecasting (PEMS traffic data).

  • Performance: FreqMoE achieved top-tier results in 51 out of 70 metrics, demonstrating particular strength in long-range forecasting and low-channel data scenarios.
  • Efficiency: The model is exceptionally lightweight. For instance, it achieves superior results to PatchTST (6.89M parameters) using only 15k to 70k parameters. Compared to FEDformer (16.8M parameters), FreqMoE uses approximately 14.5K parameters and offers significantly faster inference (2.09ms vs. 120.5ms per sample).
  • Robustness: Low variance across different random seeds suggests that the model's performance is a robust consequence of its architectural design.

4. Analysis and Discussion

A key contribution of this work is the visualization of gating coefficients, which reveals how the model shifts focus between low, mid, and high frequencies across different datasets.

Gating heatmaps across various datasets, illustrating the model's dynamic focus on different frequency bands.

The "plugin" capability of the Frequency Decomposition MoE module further highlights its utility; when integrated into simpler models like DLinear, it consistently enhances performance, especially for extended forecasting horizons. Ablation studies indicated that three experts typically provide the best balance, as excessive experts can lead to "frequency range fragmentation," where individual experts lack sufficient information to learn meaningful patterns.

5. Conclusion

FreqMoE demonstrates that high-performance time series forecasting does not necessitate massive parameter counts. By operating intelligently in the frequency domain with a dynamic MoE structure, it achieves state-of-the-art results with extreme efficiency. Future work will focus on scaling the model for high-dimensional datasets and further improving the interpretability of frequency prioritization in complex multi-variable environments.

Summary updated

BibTeX
@inproceedings{liu2025freqmoe,
  author = {Ziqi Liu},
  title = {{FreqMoE: Enhancing Time Series Forecasting through Frequency Decomposition Mixture of Experts}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2025},
}
Figure