Abstract
Modeling dynamic graphs requires capturing both spatial and temporal dependencies simultaneously. Traditional Spatio-Temporal Graph Convolutional Networks (ST-GCNs) typically rely on either 1D-Convolutional Neural Networks (CNNs) or Long Short-Term Memory (LSTM) blocks for temporal modeling. This paper proposes a hybrid architecture that leverages CNNs for initial feature distillation and LSTMs for capturing long-term dependencies. Evaluated across five diverse datasets, the hybrid models outperform standard architectures in four out of seven variations, demonstrating superior predictive accuracy and surprising computational efficiency.
Introduction
Dealing with data that is both spatial and temporal is a significant challenge in machine learning. Applications ranging from traffic flow prediction to financial market analysis require models that understand both the connections between entities and how those entities evolve over time. Standard Graph Representation Learning (GRL) often focuses on static snapshots, but real-world graphs are inherently dynamic.
Spatio-Temporal Graph Convolutional Networks (ST-GCNs) address this by stacking temporal layers with spatial layers (Graph Convolutional Networks or GCNs). Historically, researchers have favored either CNNs for their speed and simplicity or LSTMs for their ability to handle long-term dependencies. This research investigates whether a hybrid approach can capture complex temporal patterns more effectively than these "pure" models.
Methodology
The proposed architecture builds upon the "sandwich" framework, which alternates temporal blocks with spatial GCN blocks. The core innovation lies in the composition of these temporal blocks.
In the hybrid models (e.g., CNN-GCN-LSTM), CNN layers act as a "distiller," convolving raw time-series data into meaningful feature channels. These refined features are then processed by the GCN to share information across the graph using Chebyshev polynomial expansion, which efficiently approximates graph convolutions within a K-step neighborhood. Finally, an LSTM layer captures the remaining sequential dependencies. This sequence aims to reduce noise through CNN pre-processing before applying more complex recurrent modeling.
Experimental Setup
The models were evaluated across five diverse datasets to ensure generalizability:
- Finance: Daily returns of 72 S&P 500 companies, with edges defined by NLP-derived news co-references.
- Hungarian Chickenpox: Weekly regional case counts.
- PedalMe: Logistics data from a London bike delivery service.
- Wikipedia: Traffic and edit data for math-related articles.
- Bus: Passenger counts for the Montevideo city bus system.
Results and Discussion
The hybrid CNN/LSTM architectures achieved the best test scores on the majority of dataset variations. As shown in the performance table below, the hybrid models particularly excelled in the Finance and Wikipedia domains.
Key Insights
- CNNs as Pre-processors: Performance dropped when CNNs were removed (pure GCN-LSTM), suggesting that CNNs are vital for distilling local temporal features and reducing noise before the LSTM stage.
- The Speed Paradox: Despite the complexity of LSTMs, the hybrid models were surprisingly fast to train. The author suggests that because CNNs reduce data dimensionality (e.g., to 32 channels), subsequent layers receive a "cleaner" signal, accelerating convergence.
- Scalability: On the Wikipedia dataset—the largest tested with over 1,000 vertices—the hybrid models significantly outperformed the baseline ST-GCN, indicating strong scaling potential for high-dimensional data.
Conclusion and Future Work
The hybrid ST-GCN architecture successfully combines the feature extraction strengths of CNNs with the sequential memory of LSTMs. While the model showed some overfitting on very small datasets like PedalMe, its performance on complex financial and social media data is promising. Future research will explore the sensitivity of the model to the number of CNN channels and its robustness in real-world financial trading scenarios where noise levels are exceptionally high.
Summary updated