CATN: Cross Attentive Tree-Aware Network for Multivariate Time Series Forecasting

Hui He, Qi Zhang, Simeng Bai, Kun Yi, Zhendong Niu

Abstract

Multivariate time series forecasting is essential for managing complex systems like traffic networks and power grids. Traditional models often fail to capture hierarchical dependencies and inter-series correlations. We introduce the Cross Attentive Tree-Aware Network (CATN), which automatically builds a hierarchical tree structure to represent variable relationships. By combining this structure with a multi-level temporal learning mechanism and a cross-attention component, CATN achieves state-of-the-art results, including a 39% improvement in MAPE on key benchmarks.

Introduction

Predicting the future of multiple related variables—such as traffic flow across hundreds of sensors—is a significant challenge. Most existing models either ignore the underlying structure of the data or rely on rigid, pre-defined graphs that may not capture the true complexity of the system.

Traffic flow patterns across different nodes and their geographic proximity, highlighting the need for capturing inter-series correlations.

Real-world systems are often naturally hierarchical. For example, traffic on a main highway influences its various branches differently. CATN addresses these challenges by moving beyond flat graphs to an ordered tree structure that captures grouped, multi-level interactions.

Methodology

CATN's architecture consists of three primary components that integrate structural and temporal information.

The CATN architecture featuring Tree Construction, Global & Local Learning, and the Cross Attention Mechanism.

Tree Construction & Embedding

Instead of a flat graph, CATN employs hierarchical clustering to build a tree where individual time series are "leaves." The model learns embeddings for these nodes and their connections, enabling the passing of "dynamic messages" that represent group-level features.

Global & Local Learning (GLL)

To capture diverse temporal patterns, CATN uses a hybrid approach:

  • Convolutional Component (CNN): Focuses on short-range, local temporal fluctuations.
  • Recurrent Component (Bi-LSTM): Captures long-range, global trends.

Cross Attention Mechanism (CAM)

The CAM links the tree structure with temporal learning by calculating the dynamic relevance between different groups in the tree at various time steps. This allows the model to focus on the most relevant variable clusters for any given prediction.

Experimental Results

CATN was evaluated on four real-world benchmarks: Traffic (PEMS-SF), Electricity, PeMSD7(M), and METR-LA.

Performance Gains

CATN consistently outperformed eight major baselines, including LSTNet, MTGNN, and the Transformer-based Informer. On the Traffic and Electricity datasets, CATN showed an 11.7% improvement in MAE and a 39.03% improvement in MAPE over previous state-of-the-art models.

Stability and Ablation

The model demonstrated remarkable stability across various prediction horizons (3, 6, 9, and 12-step forecasts). Ablation studies proved the necessity of the hierarchical approach; removing the Tree Embedding (TEC) component led to an 18.03% increase in RMSE.

Key Insights

  • Hierarchical Superiority: An ordered tree structure is often more effective and interpretable than an unconstrained graph for capturing "grouped" correlations.
  • Temporal Synergy: Effective forecasting requires the combination of local feature extraction (CNN) and global context (Bi-LSTM).
  • Interpretability: The tree-aware design allows for the visualization of correlation matrices, providing greater transparency into the model's decision-making process.

Limitations and Future Work

CATN is sensitive to the embedding dimension; dimensions that are too small lack representational power, while those that are too large create an "upward aggregation burden." Additionally, deep interaction layers can lead to "over-smoothing." Future work could apply this tree-aware approach to stock market analysis, solar energy forecasting, and epidemic tracking.

Summary updated

Abstract

Modeling complex hierarchical and grouped feature interaction in the multivariate time series data is indispensable to comprehend the data dynamics and predicting the future condition. The implicit feature interaction and high-dimensional data make multivariate forecasting very challenging. Many existing works did not put more emphasis on exploring explicit correlation among multiple time series data, and complicated models are designed to capture long- and short-range pattern with the aid of attention mechanism. In this work, we think that pre-defined graph or general learning method is difficult due to their irregular structure. Hence, we present CATN, an end-to-end model of Cross Attentive Tree-aware Network to jointly capture the inter-series correlation and intra-series temporal pattern. We first construct a tree structure to learn hierarchical and grouped correlation and design an embedding approach that can pass dynamic message to generalize implicit but interpretable cross features among multiple time series. Next in temporal aspect, we propose a multi-level dependency learning mechanism including global&local learning and cross attention mechanism, which can combine long-range dependencies, short-range dependencies as well as cross dependencies at different time steps. The extensive experiments on different datasets from real world show the effectiveness and robustness of the method we proposed when compared with existing state-of-the-art methods.
BibTeX
@inproceedings{he2022catn,
  author = {Hui He and Qi Zhang and Simeng Bai and Kun Yi and Zhendong Niu},
  title = {{CATN: Cross Attentive Tree-Aware Network for Multivariate Time Series Forecasting}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.1609/aaai.v36i4.20320},
}
Figure