FTMixer: Frequency and Time Domain Representations Fusion for Time Series Modeling

Zhengnan Li, Yunxiao Qin, Xilong Cheng, Yuting Tan

Abstract

Time series data can be represented in both the time and frequency domains, with the time domain emphasizing local dependencies and the frequency domain highlighting global dependencies. To harness the strengths of both domains in capturing local and global dependencies, we propose the Frequency and Time Domain Mixer (FTMixer). To exploit the global characteristics of the frequency domain, we introduce the Frequency Channel Convolution (FCC) module, designed to capture global inter-series dependencies. Inspired by the windowing concept in frequency domain transformations, we present the Windowing Frequency Convolution (WFC) module to capture local dependencies. The WFC module first applies frequency transformation within each window, followed by convolution across windows. Furthermore, to better capture these local dependencies, we employ channel-independent scheme to mix the time domain and frequency domain patches. Notably, FTMixer employs the Discrete Cosine Transformation (DCT) with real numbers instead of the complex-number-based Discrete Fourier Transformation (DFT), enabling direct utilization of modern deep learning operators in the frequency domain. Extensive experimental results across seven real-world long-term time series datasets demonstrate the superiority of FTMixer, in terms of both forecasting performance and computational efficiency.

Introduction

Time-series forecasting is a critical task in various domains, from energy management to traffic planning. Effective modeling requires capturing two distinct types of patterns: local dependencies (sudden spikes and irregular shifts in the time domain) and global dependencies (periodic cycles and seasonal rhythms in the frequency domain).

Existing models often struggle to balance these. Transformer-based architectures are powerful but computationally expensive, while frequency-based models frequently rely on the Discrete Fourier Transform (DFT). DFT produces complex numbers that are incompatible with standard deep learning operators like Batch Normalization or ReLU. Moreover, global frequency transforms can "smear" local information, making it difficult to predict non-periodic changes. FTMixer addresses these challenges by fusing real-valued frequency representations with time-domain features.

Methodology

FTMixer is a Temporal Convolutional Network (TCN) variant that bridges the gap between time and frequency domains through three primary innovations:

1. The DCT Advantage

Unlike traditional methods using DFT, FTMixer utilizes the Discrete Cosine Transform (DCT). Since DCT operates purely on real numbers, it allows the model to integrate seamlessly with modern deep learning libraries and operators without the overhead of complex-number arithmetic.

2. Frequency Channel Convolution (FCC)

This module is designed to capture global inter-series dependencies. By embedding the entire sequence in the frequency domain, the FCC module performs convolutions across channels to identify broad, multi-variable patterns that define the global trend of the data.

3. Windowed Frequency-Time Convolution (WFTC)

To preserve local information, the WFTC module segments the series into patches (windowing). It applies DCT within each patch to capture local frequency components and combines them with standard time-domain convolutions. This dual-path approach ensures the model tracks both the "forest" (global trends) and the "trees" (local spikes).

The FTMixer architecture showing the FCC and WFTC modules feeding into a unified predictor.

4. Dual-Domain Loss Function (DDLF)

Training is guided by a hybrid loss function. In addition to minimizing Mean Squared Error (MSE) in the time domain, the model uses Mean Absolute Error (MAE) in the frequency domain. This ensures the predicted spectrum aligns with the ground truth, maintaining the natural rhythms of the signal.

Experimental Results

FTMixer was evaluated on seven real-world datasets, including ETT (Electricity Transformer Temperature), ECL (Electricity Consumption Load), Traffic, and Weather. These datasets cover a wide range of periodicities and noise levels.

Performance and Efficiency

FTMixer consistently outperformed state-of-the-art models such as PatchTST, iTransformer, and ModernTCN. For example, on the ETTh1 dataset, FTMixer achieved an MSE of 0.402, significantly better than PatchTST's 0.428.

Beyond accuracy, FTMixer is exceptionally efficient. On the high-dimensional ECL dataset (321 channels), FTMixer required only 304MB of memory, compared to 3350MB for ModernTCN. This efficiency stems from the real-valued DCT and the inherent sparsity of frequency-domain representations, which allow the model to focus on the most influential periodic components.

Visualization of frequency vs. time domain representations (a) and prediction performance against ground truth (b).

Conclusion

FTMixer demonstrates that fusing time and frequency domain representations using real-valued transforms can yield superior forecasting accuracy with a fraction of the computational cost of Transformers. By capturing both local and global dependencies simultaneously, it provides a robust framework for long-term time series modeling. Future work may explore adaptive windowing for the WFTC module and extending the architecture to tasks like anomaly detection and classification.

Summary updated

Abstract

Time series data can be represented in both the time and frequency domains, with the time domain emphasizing local dependencies and the frequency domain highlighting global dependencies. To harness the strengths of both domains in capturing local and global dependencies, we propose the Frequency and Time Domain Mixer (FTMixer). To exploit the global characteristics of the frequency domain, we introduce the Frequency Channel Convolution (FCC) module, designed to capture global inter-series dependencies. Inspired by the windowing concept in frequency domain transformations, we present the Windowing Frequency Convolution (WFC) module to capture local dependencies. The WFC module first applies frequency transformation within each window, followed by convolution across windows. Furthermore, to better capture these local dependencies, we employ channel-independent scheme to mix the time domain and frequency domain patches. Notably, FTMixer employs the Discrete Cosine Transformation (DCT) with real numbers instead of the complex-number-based Discrete Fourier Transformation (DFT), enabling direct utilization of modern deep learning operators in the frequency domain. Extensive experimental results across seven real-world long-term time series datasets demonstrate the superiority of FTMixer, in terms of both forecasting performance and computational efficiency.
BibTeX
@misc{li2024ftmixer,
  author = {Zhengnan Li and Yunxiao Qin and Xilong Cheng and Yuting Tan},
  title = {{FTMixer: Frequency and Time Domain Representations Fusion for Time Series Modeling}},
  year = {2024},
  eprint = {2405.15256},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2405.15256},
}
Figure