Abstract
Multivariate time series forecasting is essential for managing complex systems like traffic networks and power grids. Traditional models often fail to capture hierarchical dependencies and inter-series correlations. We introduce the Cross Attentive Tree-Aware Network (CATN), which automatically builds a hierarchical tree structure to represent variable relationships. By combining this structure with a multi-level temporal learning mechanism and a cross-attention component, CATN achieves state-of-the-art results, including a 39% improvement in MAPE on key benchmarks.
Introduction
Predicting the future of multiple related variables—such as traffic flow across hundreds of sensors—is a significant challenge. Most existing models either ignore the underlying structure of the data or rely on rigid, pre-defined graphs that may not capture the true complexity of the system.
Real-world systems are often naturally hierarchical. For example, traffic on a main highway influences its various branches differently. CATN addresses these challenges by moving beyond flat graphs to an ordered tree structure that captures grouped, multi-level interactions.
Methodology
CATN's architecture consists of three primary components that integrate structural and temporal information.
Tree Construction & Embedding
Instead of a flat graph, CATN employs hierarchical clustering to build a tree where individual time series are "leaves." The model learns embeddings for these nodes and their connections, enabling the passing of "dynamic messages" that represent group-level features.
Global & Local Learning (GLL)
To capture diverse temporal patterns, CATN uses a hybrid approach:
- Convolutional Component (CNN): Focuses on short-range, local temporal fluctuations.
- Recurrent Component (Bi-LSTM): Captures long-range, global trends.
Cross Attention Mechanism (CAM)
The CAM links the tree structure with temporal learning by calculating the dynamic relevance between different groups in the tree at various time steps. This allows the model to focus on the most relevant variable clusters for any given prediction.
Experimental Results
CATN was evaluated on four real-world benchmarks: Traffic (PEMS-SF), Electricity, PeMSD7(M), and METR-LA.
Performance Gains
CATN consistently outperformed eight major baselines, including LSTNet, MTGNN, and the Transformer-based Informer. On the Traffic and Electricity datasets, CATN showed an 11.7% improvement in MAE and a 39.03% improvement in MAPE over previous state-of-the-art models.
Stability and Ablation
The model demonstrated remarkable stability across various prediction horizons (3, 6, 9, and 12-step forecasts). Ablation studies proved the necessity of the hierarchical approach; removing the Tree Embedding (TEC) component led to an 18.03% increase in RMSE.
Key Insights
- Hierarchical Superiority: An ordered tree structure is often more effective and interpretable than an unconstrained graph for capturing "grouped" correlations.
- Temporal Synergy: Effective forecasting requires the combination of local feature extraction (CNN) and global context (Bi-LSTM).
- Interpretability: The tree-aware design allows for the visualization of correlation matrices, providing greater transparency into the model's decision-making process.
Limitations and Future Work
CATN is sensitive to the embedding dimension; dimensions that are too small lack representational power, while those that are too large create an "upward aggregation burden." Additionally, deep interaction layers can lead to "over-smoothing." Future work could apply this tree-aware approach to stock market analysis, solar energy forecasting, and epidemic tracking.
Summary updated