Introduction
Time series forecasting is a critical task in various domains, from energy management to traffic planning. However, real-world data is often "non-stationary," meaning its statistical properties like mean and variance shift over time due to underlying trends and seasonality. Standard deep learning approaches often rely on normalization to stabilize training, but this creates a "Normalization Paradox": the very process that makes training possible often wipes out the vital trend information the model needs to predict.
U-Mixer addresses this by combining a hierarchical U-Net structure with an MLP-Mixer backbone and a novel "Stationarity Correction" module. This architecture allows the model to learn complex temporal dependencies while explicitly restoring the non-stationary signals essential for accurate long-term forecasting.
Architecture and Methodology
The U-Mixer architecture is designed to preserve multi-scale features and handle channel-specific distributions without interference.
1. U-Net + Mixer Structure
Unlike traditional flat stacks of layers, U-Mixer utilizes a hierarchical U-Net encoder-decoder. Encoders progressively extract high-level abstractions from patched time series data, while decoders use skip connections to merge these with low-level details from earlier layers. This ensures that "shallow" signal information is not lost as the data passes through deeper parts of the network.
2. Independent Interaction Blocks
To prevent distribution shifts in one variable from corrupting the patterns of another, U-Mixer processes temporal patterns and channel interactions separately. It uses distinct Multi-Layer Perceptron (MLP) blocks for time and space, ensuring robust feature extraction across multivariate datasets.
3. Stationarity Correction (SC)
The core innovation of U-Mixer is the Stationarity Correction module. After the model generates a normalized prediction, the SC module "re-injects" the lost non-stationary information. By leveraging the Wiener-Khinchin theorem and Fast Fourier Transforms (FFT), it ensures the final output matches the autocorrelation and mean of the original input. This restores trends and seasonality while maintaining the learned temporal dependencies.
Experimental Results
U-Mixer was evaluated on eight large-scale real-world benchmarks, including ETT, Electricity, Traffic, and Weather datasets.
The model demonstrated dominant performance, achieving the best results in 56 out of 64 test cases. On average, U-Mixer outperformed state-of-the-art (SOTA) models like TimesNet and DLinear by 14.5% in Mean Squared Error (MSE) and 7.7% in Mean Absolute Error (MAE). Ablation studies confirmed that both the hierarchical U-Net structure and the Stationarity Correction module are critical to these gains; removing either led to significant performance degradation.
Conclusion
U-Mixer represents a significant step forward in time series forecasting by reconciling the need for stable training with the preservation of non-stationary trends. Its hierarchical merging and mathematical correction framework provide a robust solution for complex, real-world forecasting tasks. Future work will focus on optimizing computational efficiency and exploring the integration of Stationarity Correction into other architectures like Transformers or CNNs.
Reference: Ma, X., et al. (2024). U-Mixer: An Unet-Mixer Architecture with Stationarity Correction for Time Series Forecasting. Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI-24).
Summary updated