Unlocking the Power of Patch: Patch-Based MLP for Long-Term Time Series Forecasting

Peiwang Tang, Weitai ZhangAAAI 2025 · DOI

TL;DR

This paper introduces PatchMLP, a novel and surprisingly effective MLP-based model for long-term time series forecasting. It challenges the dominance of Transformer architectures by demonstrating that a simpler, patch-based approach can achieve state-of-the-art results, highlighting the importance of cross-variable interactions and efficient feature decomposition.

What Problem Does This Paper Address?

Long-Term Time Series Forecasting (LTSF) is crucial for predicting future trends in various domains like finance, electricity, and transportation. While Transformer models have shown promise, this paper questions their true effectiveness, attributing their success to a "patch" mechanism rather than the complex self-attention. The authors argue that existing Transformer-based LTSF models often fail to fully capture temporal information and may misapply cross-variable interactions. They aim to develop a simpler, more efficient, and interpretable model that can achieve superior performance.

Methods

The core of this paper is the PatchMLP model, which is entirely based on Multi-Layer Perceptrons (MLPs). Key components include:

  • Multi-Scale Patch Embedding (MPE): Instead of a single linear layer for embedding, MPE uses multiple patch sizes to capture relationships at different temporal scales, allowing for a richer representation of the input time series.
  • Feature Decomposition: The embedded time series are decomposed into "smooth components" and "noisy residuals" using moving averages. This is done on the embedded vectors, not the raw data, to better distinguish fundamental patterns from noise.
  • MLP Layer: This layer consists of two sub-components:
  • Intra-Variable MLP: Processes temporal patterns within a single variable.
  • Inter-Variable MLP: Models interactions between different variables. This is a crucial aspect, as the authors emphasize the importance of cross-variable communication. They use a dot product mechanism to enhance this interaction.
  • Projection Layer: Maps the processed latent vectors back to the desired output sequence length.

The model emphasizes channel independence for processing noise and channel mixing for semantic information interchange between variables.

Datasets

The paper evaluates PatchMLP on eight widely used LTSF benchmark datasets:

  • Solar Energy
  • Weather
  • Traffic
  • Electricity (ECL)
  • ETTh1
  • ETTh2
  • ETTm1
  • ETTm2

Key Insights and Ideas

  • Patching is Key, Not Just Transformers: The paper strongly suggests that the "patch" mechanism, which segments time series into smaller chunks, is the primary driver of Transformer success in LTSF, not necessarily the self-attention mechanism itself.
  • MLPs Can Outperform Transformers: The surprising finding is that a simple MLP architecture, when augmented with effective patching and feature decomposition, can achieve state-of-the-art results, challenging the notion that complex architectures are always necessary.
  • Cross-Variable Interaction is Crucial: The authors highlight that while many models attempt to model interactions between variables, they often misapply it. PatchMLP's design, particularly the Inter-Variable MLP, emphasizes skillful leverage of these interactions.
  • Decomposition on Embeddings is Better: Decomposing the time series after embedding them into a latent space is more effective than decomposing the raw time series, as it can better handle complex and mixed patterns.
  • Simplicity and Efficiency: The paper advocates for simpler, more interpretable models, suggesting that complexity doesn't always equate to better performance in LTSF.

Key Results

  • PatchMLP achieved 100% state-of-the-art (SOTA) results across 16 benchmarks on the tested datasets, surpassing all Transformer architectures.
  • The model demonstrated consistent improvement with increasing input lengths, outperforming some models that showed diminished performance with longer sequences.
  • The ablation studies confirmed the importance of each component: MPE, Feature Decomposition, and the Inter-Variable MLP's dot product mechanism. Removing these components led to performance degradation.

Limitations and Future Work

  • Hyperparameter Sensitivity: While the paper notes that PatchMLP is not highly sensitive to hyperparameters like learning rate, number of blocks, and hidden dimension, it suggests that excessively large values for blocks and hidden dimensions might lead to overfitting due to increased parameters.
  • Future Work: The authors hope their findings will spur new research directions in LTSF, encouraging the development of more efficient, concise, and interpretable models. They suggest that focusing on effective feature extraction and interaction modeling, rather than solely on complex architectures, is a promising avenue.

Summary updated

BibTeX
@inproceedings{tang2025unlocking,
  author = {Peiwang Tang and Weitai Zhang},
  title = {{Unlocking the Power of Patch: Patch-Based MLP for Long-Term Time Series Forecasting}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i12.33378},
}
Figure