TL;DR
TimeRecipe shifts the time-series forecasting paradigm from evaluating entire end-to-end models to systematically benchmarking individual architectural modules. By running over 10,000 experiments across a unified "Canonical Architecture," the authors uncover how specific data properties dictate module effectiveness. Leveraging these insights, they introduce a training-free toolkit that automatically recommends custom model configurations, frequently outperforming current state-of-the-art (SOTA) models.
What Problem Does This Paper Address?
Deep learning has vastly improved time-series forecasting, but the community remains trapped in endless debates over which high-level architecture is superior (e.g., Transformers vs. MLPs). Existing benchmarks typically evaluate models holistically, offering a case-specific "winner" without explaining why certain internal design choices—like instance normalization, series decomposition, or patch embeddings—succeed or fail under different conditions.
This paper addresses this lack of granularity. By deconstructing popular models into their constituent parts, the authors aim to answer a fundamental research question: Which specific modules and model designs are most effective under specific time-series forecasting scenarios?
Methods
The authors propose the TimeRecipe Framework, which unifies time-series forecasting models into a five-component Canonical Architecture:
- Pre-Processing: Options include Instance Normalization (to handle distribution shifts) and Series Decomposition (to separate trend and seasonal components).
- Embedding: Options include Token, Patch, Invert, and Frequency embeddings, alongside a no-embedding baseline.
- Feed-Forward Modeling: Architectures like MLPs, Transformers, and RNNs, paired with either Temporal Fusion or Feature Fusion strategies.
- Projection: A standard linear layer mapping latent predictions to the output horizon.
- Post-Processing: The inverse operations of the pre-processing stage (e.g., denormalization).
By mixing and matching these modules, TimeRecipe can replicate popular SOTA models or discover entirely new architectural combinations.
To make these insights actionable, the authors also developed a training-free model selection toolkit. It uses a LightGBM regression model trained on the benchmark's results to map a new dataset's characteristics directly to the best-performing module configuration.
Datasets
TimeRecipe evaluates configurations across a comprehensive suite of standard time-series benchmarks:
- LTSF Datasets: Electricity Transformer Temperature (ETTh1/2, ETTm1/2), Influenza-like Illness (ILI), Electricity Consumption Load (ECL), Weather, and Exchange Rate.
- PEMS Datasets: Traffic flow and occupancy data (PEMS03, PEMS04, PEMS07, PEMS08).
- M4 Dataset: A massive collection of 100,000 diverse business, financial, and macroeconomic series.
- Time-MMD: Used specifically as an out-of-distribution (OOD) testbed to evaluate the generalization of the model selection toolkit.
Key Insights and Ideas
- No Single King: No single architecture consistently dominates. Module effectiveness is highly dependent on the intrinsic properties of the data (such as trend strength, seasonality, and stationarity).
- The SOTA is Suboptimal: Standard SOTA models often leave performance on the table. Exhaustively searching the modular design space reveals alternative combinations that frequently beat established end-to-end models.
- Data-to-Module Mapping: The authors establish clear, statistically verified correlations between data properties and module choices. For instance, Instance Normalization is highly favored when dataset distribution shifts are severe but seasonality is low, whereas RNN architectures excel when the horizon-to-lookback ratio is small.
Key Results
- Outperforming the SOTA: In 92 out of 102 evaluated scenarios, TimeRecipe identified a custom modular configuration that outperformed existing SOTA models.
- Error Reduction: On average, the top configurations discovered by TimeRecipe reduced forecasting errors by 5.4% compared to the best existing baseline, outstripping them by an average of 13.66 ranking positions.
- Effective Automated Selection: The training-free LightGBM toolkit successfully identified top-performing architectures for unseen tasks. For example, on the Social_12_S dataset, the recommended configurations significantly outperformed the baseline PatchTST model without requiring an expensive architecture search.
Limitations and Future Work
While comprehensive, TimeRecipe does not cover every time-series design. It currently omits highly specialized architectures (like Crossformer) that use tangled temporal/feature fusions, as well as large language model (LLM)-based foundation models (like Time-LLM) due to their massive computational overhead.
Additionally, the framework's insights are fundamentally empirical. The authors note that a deeper, theoretical mathematical analysis explaining why specific modules perfectly align with certain data characteristics remains an open direction for future research.
Summary updated