For several years, the research community has leaned heavily on Transformer architectures and Graph Neural Networks (GNNs) to solve complex time series forecasting problems. The intuition was straightforward: self-attention captures long-range dependencies, and graphs model spatial relationships. However, a growing body of evidence suggests that these heavy mechanisms may be overkill for temporal data.
A new direction in the field argues that Multi-Layer Perceptrons (MLPs), when structured correctly, can outperform Transformers in both accuracy and efficiency. By replacing quadratic-complexity attention with linear-complexity "mixing" operations, researchers are achieving state-of-the-art results with a fraction of the computational footprint.
The Intuition of Mixing
The fundamental shift in this research direction is the move from attention to mixing. In a standard Transformer, every time point attends to every other time point, a process that grows quadratically with the sequence length. Researchers like Abhimanyu Das have demonstrated with TiDE that a simple MLP-based encoder-decoder can capture these dependencies more efficiently.
The "Mixer" philosophy, popularized in computer vision and now adapted for time series, decomposes the problem into two distinct operations:
- Temporal Mixing: An MLP that operates across the time dimension to capture how values evolve.
- Channel Mixing: An MLP that operates across different variables (channels) to capture inter-series correlations.
This disentanglement is the core of TSMixer, which shows that explicitly mixing temporal and feature dimensions is often more robust than the global dependencies captured by Transformers.
Multi-Scale Decomposition
One reason Transformers were thought necessary was their ability to handle complex, overlapping patterns like seasonality and trends. Modern MLP approaches solve this through multi-scale decomposition. Instead of feeding a raw signal into a single network, models like TimeMixer by Shiyu Wang and MDMixer by Ziqi Zhao break the signal into multiple granularities.
By processing "microscopic" fluctuations and "macroscopic" trends through independent MLP branches, these models achieve a structured understanding of the data. Zichi Zhang further refined this in WaveletMixer by using wavelet transforms to decompose signals into frequency components before mixing, ensuring that the model doesn't lose weak seasonal signals in the noise.
Replacing Graphs with Linear Layers
In spatial-temporal tasks like traffic forecasting, GNNs have been the standard for modeling sensor networks. However, Zepu Wang and colleagues introduced ST-MLP, which utilizes a "Channel-Independence" strategy. By treating each sensor as an independent channel and using cascaded MLPs to integrate spatial data, they achieved a sixteen-fold increase in training efficiency over GNNs.
The intuition here is that explicit graph structures are often noisy or incomplete. Models like RPMixer by Chin-Chia Michael Yeh use random projections and frequency-domain processing to capture spatial-temporal dependencies without requiring a predefined graph, proving that simple linear mixers can effectively learn the underlying topology of a network.
Efficiency as a First-Class Citizen
The most compelling argument for this MLP-centric shift is the massive reduction in resource requirements.
- Training Speed: Yi Xie showed that LAMEE uses only 12% of the training time of a Transformer.
- Memory Footprint: Yong Liu demonstrated that Koopa, by using Koopman predictors (linear operators), slashes memory usage by 76%.
- Model Size: Vijay Ekambaram introduced Tiny Time Mixers (TTMs), pre-trained models with as few as 1 million parameters that rival the performance of massive "Time Series Foundation Models" like Chronos or TimesFM.
Why It Works: The Patching Effect
A key technical catalyst for this success is patching. Instead of treating each time step as a token, models like PatchAD and PITS segment the series into discrete patches. Seunghan Lee found that embedding these patches independently using simple MLPs actually improves accuracy. Patching provides a local context that makes the global mixing operation much more stable, allowing the model to focus on the "shape" of the data rather than individual noisy points.
By stripping away the complexity of self-attention and graph convolutions, this research direction has revealed that the most effective way to model the future is often the simplest: a well-structured hierarchy of linear projections.