Learning to Route in Time and Frequency Domains: A Dual-Domain MoE Transformer for Multi-Horizon Forecasting

Qi Li, J. Wang, Han He, Sheya He, Xiaoyu DaiResearch Square 2026 · DOI

TL;DR

The MoE-Transformer is a novel dual-domain framework that significantly boosts long-term time-series forecasting by processing data in both time and frequency domains simultaneously. By using Reinforcement Learning (RL) to dynamically route data to specialized "experts" and introducing an "Extended DFT" to fix spectral misalignment, the model achieves over 50% reduction in error while being 60% faster than current state-of-the-art methods.

What Problem Does This Paper Address?

Predicting the future of complex systems—like electrical grids or traffic flow—is notoriously difficult. These systems exhibit "multi-scale" patterns: some behaviors repeat every hour, others every week, and some are non-stationary trends that don't repeat at all.

Current Transformer-based models often struggle with two things:

  1. Domain Limitation: Most models operate strictly in the time domain, missing the clear periodic signals visible in the frequency domain. Those that do use frequency often suffer from "spectral misalignment," where the frequency grid of the input doesn't match the frequency grid of the full prediction window.
  2. Static Routing: Modern "Mixture-of-Experts" (MoE) architectures use static gating (like picking the top-k experts). These gates don't adapt well to evolving patterns or different forecasting horizons, often leading to "expert collapse" where only a few sub-networks do all the work.

Methods

The authors introduce the MoE-Transformer, which utilizes three primary innovations:

  • Extended Discrete Fourier Transform (Extended DFT): Standard DFTs are calculated based on the input length ($L$). However, the model needs to predict a future window ($T$). The authors calculate an "Extended" spectrum that aligns the input data with the frequency grid of the entire $L+T$ window. This ensures the "notes" the model hears in the input are on the same scale as the "notes" it needs to predict.
  • Dual-Domain MoE Architecture: The model splits into two parallel paths: a T-MoE (Time-domain) and an F-MoE (Frequency-domain). Instead of standard feed-forward layers, it uses sets of specialized experts. The frequency path uses complex-valued neural networks to handle the magnitude and phase of the signal.
  • RL-Based Dynamic Routing: This is the "brain" of the system. Instead of a simple mathematical gate, the model uses Reinforcement Learning agents (specifically using the ISAC algorithm) to decide which expert should handle which piece of data. The RL reward function doesn't just look at accuracy; it also rewards routing consistency (don't switch experts too erratically) and expert diversity (make sure all experts are learning something useful).

Datasets

The model was evaluated on five gold-standard benchmarks for long-term forecasting:

  • ETTh1 & ETTm1: Electricity Transformer Temperature (hourly and 15-min intervals).
  • Electricity: Hourly consumption data for 321 clients.
  • Traffic: Road occupancy rates from 862 sensors in San Francisco.
  • Weather: 21 meteorological indicators collected every 10 minutes.

Key Insights and Ideas

  • The Power of Alignment: The Extended DFT alone provided a 5.8% performance gain. It proves that in frequency-domain modeling, how you define your "grid" is just as important as the model architecture itself.
  • RL as a Stabilizer: One of the most interesting findings is that RL-based routing outperformed static gating by up to 47.2%. By treating routing as a sequential decision-making process (a Markov Decision Process), the model learns to pick experts that are not just good for the current step, but contribute to a stable, long-term forecasting strategy.
  • Specialization Emerges Naturally: In the frequency domain, the RL agent learned to send dominant, low-frequency signals to specific experts while delegating high-frequency "noise" to others. In the time domain, experts specialized in handling trends versus sudden, irregular fluctuations.

Key Results

The MoE-Transformer sets a new bar for the field:

  • Accuracy: It achieved MSE reductions of 50.9% to 56.9% compared to top-tier baselines like PatchTST and TimesNet.
  • Efficiency: Because it uses "sparse" activation (only turning on the necessary experts), it uses 40% less memory and boasts a 60% reduction in inference latency.
  • Scalability: Performance continued to improve as the number of experts increased (tested up to 40), showing that the RL router effectively manages larger "brain" capacities without getting confused.

Limitations and Future Work

While the results are impressive, the paper notes a few areas for growth:

  • Univariate Focus: The current framework processes different variables (channels) independently. While this is a common and effective strategy, it doesn't explicitly model the cross-dependencies between, say, temperature and electricity usage.
  • Deterministic Output: The model provides a "point forecast." Future iterations could incorporate probabilistic methods to provide a range of uncertainty, which is critical for risk management in power grids.
  • Complexity: Implementing dual RL agents alongside a Transformer is computationally heavier during the training phase, even if inference is faster.

Reference: Li, Q., et al. "Learning to Route in Time and Frequency Domains: A Dual-Domain MoE Transformer for Multi-Horizon Forecasting." Research Square Preprint, 2026.

Summary updated

Abstract

<title>Abstract</title> Accurate long-term electrical load forecasting is essential for reliable smart grid operation, but remains challenging due to multi-scale periodic behaviors and non-stationary temporal variations across different prediction horizons. This paper proposes MoE-Transformer, a dual-domain forecasting framework that learns to route representations in both time and frequency domains through reinforcement learning. To address spectral misalignment in multi-step prediction, an Extended Discrete Fourier Transform (Extended DFT) is introduced to align the input spectrum with the frequency grid of the full forecasting window. The model integrates parallel Mixture-of-Experts modules in the time and frequency domains (T-MoE and F-MoE), where domain-specific experts are responsible for modeling complementary temporal dynamics and spectral structures. Expert routing in each domain is formulated as an independent Markov Decision Process and optimized via reinforcement learning to jointly account for forecasting accuracy, routing consistency, and balanced expert utilization. Extensive experiments on five benchmark datasets, including ETTh1, Electricity, and Traffic, across four forecasting horizons demonstrate that MoE-Transformer consistently outperforms state-of-the-art methods, achieving MSE reductions of 50.9–56.9%. The use of sparse expert activation reduces memory consumption by 40% and inference latency by 60%, indicating suitability for real-time forecasting scenarios. Ablation studies further validate the individual contributions of Extended DFT, dual-domain modeling, and reinforcement-based routing, leading to performance improvements of 5.8%, 4.6%, and up to 47.2%, respectively.
BibTeX
@inproceedings{li2026learning,
  author = {Qi Li and J. Wang and Han He and Sheya He and Xiaoyu Dai},
  title = {{Learning to Route in Time and Frequency Domains: A Dual-Domain MoE Transformer for Multi-Horizon Forecasting}},
  booktitle = {Research Square},
  year = {2026},
  doi = {10.21203/rs.3.rs-8835625/v1},
}
Figure