Deep Coupling Network For Multivariate Time Series Forecasting

Kun Yi, Qi Zhang, Hui He, Kaize Shi, Liang Hu, Ning An, Zhendong Niu

Introduction

Multivariate time series (MTS) forecasting is a critical task in modern data science, with applications ranging from urban traffic management and energy consumption planning to financial portfolio optimization. The core challenge in MTS forecasting lies in accurately capturing two distinct types of dependencies: intra-series relationships (temporal patterns within a single variable) and inter-series relationships (spatial or causal dependencies between different variables).

While recent advancements in Deep Learning, such as Transformers and Graph Neural Networks (GNNs), have improved forecasting accuracy, they often struggle to model the complex, multi-order interactions that occur across both time and variables simultaneously. This paper introduces the Deep Coupling Network (DeepCN), a novel architecture designed to explicitly model these "hidden" interactions through a recursive coupling mechanism grounded in mutual information theory.

The Challenge: Beyond Sequential and Spatial Modeling

Current state-of-the-art (SOTA) models typically fall into three categories, each with inherent limitations:

  1. Sequential Models (RNNs/CNNs): These focus heavily on temporal patterns but often fail to capture relationships between geographically or logically distant variables.
  2. Graph Neural Networks (GNNs): While effective at modeling inter-series relationships, GNNs often treat temporal and spatial patterns as separate processing steps. This separation misses "multi-order" interactions—for example, how a specific event in Variable A three hours ago, combined with a change in Variable B one hour ago, might influence Variable C in the next hour.
  3. The Time-Lag Effect: Most existing models do not explicitly account for the delayed nature of interactions between variables, which is a fundamental characteristic of real-world systems like traffic flow or disease spread.

The Deep Coupling Network (DeepCN) Architecture

DeepCN addresses these gaps by treating time and variables as dimensions of a single, unified interaction space. The architecture consists of three primary components:

1. The Coupling Mechanism

Inspired by "Cross Networks" used in recommendation systems, the Coupling Mechanism is the heart of DeepCN. It utilizes recursive multiplication to generate multi-order feature interactions. By taking the input data and producing 1st, 2nd, up to $\ell$-order couplings, the model captures non-linear dependencies across both variables and time lags in a single operation. Crucially, the computational complexity of this mechanism remains linear relative to the number of variables and the look-back window, ensuring scalability.

2. Coupled Variable Representation Module

Recognizing that different variables (e.g., different traffic sensors or electricity clients) exhibit unique behaviors, this module employs a variable embedding matrix. This allows the network to learn specific representations for how each individual variable relates to the broader "coupled" feature set generated by the mechanism above.

3. One-Forward Inference Module

To avoid the error accumulation common in autoregressive models—which predict one step at a time—DeepCN utilizes a feed-forward network to predict the entire future horizon in a single "one-forward" step. This approach provides greater stability, particularly for long-term forecasting.

Theoretical Foundation: Mutual Information

The authors provide a rigorous mathematical justification for DeepCN by revisiting MTS relationships through the lens of mutual information. They demonstrate that the total information shared between variables can be decomposed into a sum of interactions across all possible subsets of variables and time lags. This theoretical "Aha!" moment confirms that multi-order coupling is not just an empirical trick but a mathematically sound way to capture the full spectrum of dependencies in a multivariate system.

Experimental Results

DeepCN was evaluated against 23 baseline models, including Informer, Autoformer, PatchTST, and StemGNN, across seven diverse benchmarks: Solar, Wiki, Traffic, ECG, COVID-19, Electricity, and METR-LA.

  • Overall Performance: DeepCN achieved an average improvement of 6.2% in MAE and 7.0% in RMSE across all datasets.
  • Traffic Dataset Success: The model's most significant breakthrough occurred on the Traffic dataset, where it achieved a 15.4% improvement in MAE and a 26.7% improvement in RMSE over the best-performing baseline. This suggests that DeepCN is particularly potent in environments with high inter-variable dependency.
  • Coupling Order Sensitivity: The research found that the optimal "order" of coupling is dataset-dependent. "Tightly coupled" systems (like traffic) benefit from high-order interactions (3rd or 4th order), while "loosely coupled" systems (like ECG) perform better with lower-order couplings to avoid overfitting.

Conclusion and Future Work

DeepCN represents a significant step forward in MTS forecasting by unifying temporal and spatial interaction modeling. While the model is efficient in terms of FLOPs, the authors note that the memory footprint of embeddings and coupling weights can be high. Future research will focus on model compression techniques to reduce parameter volume and exploring Dynamic Coupling, which would allow the model to adapt to non-stationary relationships where interactions between variables change over time.

Summary updated

Abstract

Multivariate time series (MTS) forecasting is crucial in many real-world applications. To achieve accurate MTS forecasting, it is essential to simultaneously consider both intra- and inter-series relationships among time series data. However, previous work has typically modeled intra- and inter-series relationships separately and has disregarded multi-order interactions present within and between time series data, which can seriously degrade forecasting accuracy. In this article, we reexamine intra- and inter-series relationships from the perspective of mutual information and accordingly construct a comprehensive relationship learning mechanism tailored to simultaneously capture the intricate multi-order intra- and inter-series couplings. Based on the mechanism, we propose a novel deep coupling network for MTS forecasting, named DeepCN, which consists of a coupling mechanism dedicated to explicitly exploring the multi-order intra- and inter-series relationships among time series data concurrently, a coupled variable representation module aimed at encoding diverse variable patterns, and an inference module facilitating predictions through one forward step. Extensive experiments conducted on seven real-world datasets demonstrate that our proposed DeepCN achieves superior performance compared with the state-of-the-art baselines.
BibTeX
@article{yi2024deep,
  author = {Kun Yi and Qi Zhang and Hui He and Kaize Shi and Liang Hu and Ning An and Zhendong Niu},
  title = {{Deep Coupling Network For Multivariate Time Series Forecasting}},
  journal = {ACM Trans. Information Systems},
  year = {2024},
  doi = {10.1145/3653447},
}
Figure