Abstract
Multivariate time series (MTS) forecasting, which analyzes historical time series to predict future trends, can effectively help decision-making. Complex relations among variables in MTS, including static, dynamic, predictable, and latent relations, have made it possible to mine more features of MTS. Modeling complex relations is not only essential in characterizing latent dependency as well as modeling temporal dependence but also brings great challenges in the MTS forecasting task. However, existing methods mainly focus on modeling certain relations among MTS variables. In this paper, we propose a novel end-to-end deep learning model, termed Multivariate Time Series Forecasting via Heterogeneous Graph Neural Networks (MTHetGNN). To characterize complex relations among variables, a relation embedding module is designed in MTHetGNN, where each variable is regarded as a graph node, and each type of edge represents a specific static or dynamic relationship. Meanwhile, a temporal embedding module is introduced for time series features extraction, involving convolutional neural network (CNN) filters with different perception scales. Finally, a heterogeneous graph embedding module is adopted to handle the complex structural information generated by the two modules. Three benchmark datasets from the real world are used to evaluate the proposed MTHetGNN. The comprehensive experiments show that MTHetGNN achieves state-of-the-art results in the MTS forecasting task.
Introduction
Multivariate time series (MTS) forecasting is a cornerstone of decision-making in domains ranging from finance to traffic management and energy. The core challenge lies in capturing two distinct types of dependencies: temporal dependency (how a single variable relates to its own history) and spatial/inter-variable dependency (how different variables influence one another).
Existing methods often struggle because they treat inter-variable relationships as "homogeneous," assuming only one type of connection exists. In reality, these relationships are complex: some are static (fixed physical locations), some are dynamic (changing with time), and some are causal (one variable driving another). Failing to account for this heterogeneity limits forecasting accuracy. MTHetGNN addresses this by treating MTS forecasting as a heterogeneous graph problem, simultaneously modeling static, dynamic, and causal relationships.
The MTHetGNN Architecture
The MTHetGNN framework consists of three integrated modules designed to capture the multifaceted nature of multivariate data:
- Temporal Embedding Module: Instead of standard RNNs, this module uses a suite of 1D Convolutional Neural Network (CNN) filters with varying kernel sizes (e.g., 1×3, 1×5, 1×7). This "Inception-style" approach allows the model to capture patterns at multiple time scales simultaneously.
- Relation Embedding Module: This module constructs three distinct types of adjacency matrices to represent different "views" of the data: a Correlation Graph for linear relationships, a Causality Graph using Transfer Entropy to model non-linear directional "lead-lag" relationships, and a Dynamic Graph that learns latent, time-varying dependencies.
- Heterogeneous Graph Embedding Module: This module fuses the temporal features with the three relational graphs using an attention mechanism. This allows the model to learn the relative importance of each graph type for the final prediction.
Experimental Evaluation
The model was evaluated on three widely-used MTS benchmarks:
- Exchange-Rate: Daily exchange rates of 8 countries.
- Traffic: Highway occupancy rates from 862 sensors in San Francisco.
- Solar-Energy: Solar power output from 137 plants in Alabama.
Performance and Results
MTHetGNN consistently outperformed nine baseline models, including popular architectures like LSTNet, Graph WaveNet, and MTGNN. As shown in the results table, MTHetGNN achieves the lowest Relative Squared Error (RSE) and Relative Absolute Error (RAE) across nearly all datasets and horizons.
Key findings from the experiments include:
- Causality vs. Correlation: Including a dedicated causality graph via Transfer Entropy allows the model to better anticipate how changes in one variable ripple through the system.
- Robustness: The model showed superior performance across different forecasting horizons (3, 6, 12, and 24 steps ahead), capturing both short-term fluctuations and long-term trends.
- Efficiency: Despite the complexity of handling three graph paths, the static graphs (Correlation and Causality) can be computed offline, keeping training and inference times competitive.
Conclusion
MTHetGNN demonstrates that modeling the inherent heterogeneity of relationships in multivariate time series is critical for high-accuracy forecasting. By combining multi-scale temporal embeddings with a flexible, attention-driven heterogeneous graph fusion, the model sets a new state-of-the-art. Future work may explore swapping the CNN temporal module for Transformers or incorporating additional relational embeddings like knowledge graphs to further enhance predictive power.
Summary updated