IMRouter: Imbalance-aware Temporal Pattern Routing for LLM-based Time Series Forecasting

Xin Yang, Cai Xu, Haishun Chen, Ying Liu, Jie Zhao, Wei Zhao, Ziyu GuanarXiv 2026 · arXiv

TL;DR

IMRouter addresses "Temporal Pattern Imbalance" (TPI) in LLM-based time series forecasting—a phenomenon where frequent temporal patterns dominate model training, leading to poor performance on rare but critical events. By employing a specialized Mixture-of-Experts (MoE) architecture guided by gradient conflict and density-aware weighting, IMRouter ensures that rare temporal dynamics are adequately represented and aligned with the LLM's semantic space.

What Problem Does This Paper Address?

Large Language Models (LLMs) have recently shown great promise in time series forecasting (TSF) by bridging the gap between numerical data and linguistic prototypes. However, the authors identify a significant hurdle: Temporal Pattern Imbalance (TPI).

In real-world data, certain dynamics (like steady trends) occur much more frequently than others (like sudden oscillations). Because standard models are trained to minimize Mean Squared Error (MSE) across the entire dataset, the optimization process is naturally biased toward these "head" patterns. As shown in the authors' empirical analysis, this leads to a "rich get richer" scenario where the model excels at frequent patterns but fails miserably on the "tail" patterns that occur less often.

Figure 1: Illustration of temporal pattern diversity in a time series. Frequently recurring peak-shaped patterns coexist with less frequent, smoother local patterns (highlighted by red dashed boxes), demonstrating the heterogeneity of tem- poral dynamics.

Methods

The authors propose IMRouter, a framework designed to balance learning across diverse temporal regimes. It consists of two primary components:

  1. Pattern-Adaptive Temporal Representation Learning (PATR): Instead of a single shared encoder, PATR uses a Mixture-of-Experts (MoE). To prevent the router from simply sending everything to the "frequent pattern" expert, they introduce two novel regularizers:
  2. Gradient Conflict Penalty: If two data patches assigned to the same expert produce opposing gradients, it suggests the expert is being forced to learn incompatible dynamics. PATR penalizes these joint assignments.
  3. Density-Aware Weighting: This increases the relative importance of conflicts involving rare patterns, ensuring the "tail" optimization signals aren't drowned out by the "head."
  4. Routing-Guided Semantic Alignment (RGSA): LLM-based TSF requires aligning temporal features with text prototypes (e.g., mapping a "spike" to a specific embedding). RGSA uses mutual information maximization to distill the pattern-specific knowledge from the MoE router into the alignment module. This ensures that the distinctiveness of rare patterns is preserved when the data is "translated" for the LLM.
Figure 3: Illustration of IMRouter

Datasets

The model was evaluated on several widely-used TSF benchmarks:

  • Long-term Forecasting: ETT (h1, h2, m1, m2), Weather, Electricity (ECL), and Traffic.
  • Short-term Forecasting: The M4 benchmark (100,000 univariate series).
  • Few-shot Learning: Evaluated on ETT datasets using only 10% of training data.

The authors used TS2Vec and HDBSCAN clustering on these datasets to quantify TPI, finding that in the Traffic dataset, for example, the top 20% of clusters cover nearly 95% of the samples.

Key Insights and Ideas

  • The Gradient as a Signal for Diversity: Rather than relying on manual labels for "rare" vs "frequent," IMRouter looks at gradient conflict. If the model struggles to satisfy two samples simultaneously, they likely belong to different temporal patterns.
  • TPI is Ubiquitous: The paper provides a statistical breakdown showing that TPI exists across almost all major TSF benchmarks, measured by Gini coefficients and coverage statistics.
  • Alignment is a Bottleneck for Rare Patterns: The authors found that even if an encoder learns a rare pattern, the "cross-modal alignment" (mapping numbers to text) often washes that information away in favor of frequent prototypes. The RGSA module is a direct fix for this "semantic dilution."

Key Results

IMRouter consistently outperformed state-of-the-art (SOTA) baselines like FSCA and CALF:

  • Overall Performance: Achieved a 1.4% reduction in MSE and 2.0% reduction in MAE compared to the strongest baseline (CALF) across seven long-term benchmarks.
  • Tail Performance: On the most infrequent patterns (Cluster 6 in the Traffic dataset), IMRouter achieved an MSE of ~0.78, while competing methods hovered between 0.85 and 0.93.
  • Few-Shot Efficiency: In 10% data scenarios, IMRouter maintained high accuracy, suggesting that its specialized experts learn more efficiently from limited samples of rare patterns.
Figure 2: (a) Frequency distribution of temporal pattern clusters identified from a representative channel in the Traffic dataset, ranked by cluster size in descending order. (b) Cluster-wise forecasting errors (MSE) of three existing methods on the same clusters, showing generally higher errors for less frequent patterns. (c) Dataset-level TPI statistics across Electricity, Weather, and Traffic. Gini measures the inequality of cluster sizes, while Top-20 % Coverage denotes the proportion of samples covered by the largest 20 % of clusters.

Limitations and Future Work

  • Computational Overhead: The use of MoE and complex gradient-based regularizers during training increases the computational cost compared to simpler linear or single-transformer models.
  • Backbone Dependency: The performance is still somewhat tied to the underlying LLM (GPT-2 was used in experiments). Exploring how larger models (like LLaMA 3) interact with TPI would be a natural next step.
  • Dynamic TPI: The current work assumes a static distribution of patterns. In non-stationary series where patterns evolve over time, the router might need more dynamic adaptation.

Summary updated

Abstract

Accurate time series forecasting (TSF) is essential for Web applications, such as traffic monitoring and online resource management. Recent advances in leveraging pre-trained large language models (LLMs) have shown promising performance in TSF. However, existing LLM-based TSF methods largely optimize the overall forecasting objective while overlooking temporal pattern imbalance (TPI), where different temporal patterns occur at highly unequal frequencies. Our empirical analysis reveals that TPI is prevalent across widely used TSF benchmarks, with substantially higher forecasting errors on less frequent patterns. To address this issue, we propose IMRouter (Imbalance-aware Temporal Pattern Routing), a novel framework that mitigates TPI-induced biases in both temporal representation learning and cross-modal alignment. Specifically, IMRouter introduces Pattern-Adaptive Temporal Representation Learning (PATR), which leverages gradient conflict and local gradient density to guide expert specialization. By penalizing incompatible expert assignments with density-aware weighting, PATR reduces the dominance of frequent optimization signals in router training. Furthermore, Routing-Guided Semantic Alignment (RGSA) transfers the pattern-specific knowledge learned by the router to cross-modal alignment through mutual information maximization, preserving distinctive temporal information across different patterns. Extensive experiments on benchmark datasets demonstrate the effectiveness of IMRouter, particularly in improving forecasting performance on less frequent temporal patterns.
BibTeX
@misc{yang2026imrouter,
  author = {Xin Yang and Cai Xu and Haishun Chen and Ying Liu and Jie Zhao and Wei Zhao and Ziyu Guan},
  title = {{IMRouter: Imbalance-aware Temporal Pattern Routing for LLM-based Time Series Forecasting}},
  year = {2026},
  eprint = {2610.39246},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.39246},
}
Figure