Abstract
The real-world traffic networks undergo expansion through the installation of new sensors, implying that the traffic patterns continually evolve over time. Incrementally training a model on the newly added sensors would make the model forget the past knowledge, i.e., catastrophic forgetting, while retraining the model on the entire network to capture these changes is highly inefficient. To address these challenges, we propose a novel Traffic Forecasting Mixture of Experts (TFMoE) for traffic forecasting under evolving networks. The main idea is to segment the traffic flow into multiple homogeneous groups, and assign an expert model responsible for a specific group. This allows each expert model to concentrate on learning and adapting to a specific set of patterns, while minimizing interference between the experts during training, thereby preventing the dilution or replacement of prior knowledge, which is a major cause of catastrophic forgetting. Through extensive experiments on a real-world long-term streaming network dataset, PEMSD3-Stream, we demonstrate the effectiveness and efficiency of TFMoE. Our results showcase superior performance and resilience in the face of catastrophic forgetting, underscoring the effectiveness of our approach in dealing with continual learning for traffic flow forecasting in long-term streaming networks.
Introduction
Real-world traffic networks are dynamic, expanding as new sensors are installed and patterns shift due to urban development or population migration. This evolution presents a significant challenge for AI models. Incremental training on new sensors often leads to "catastrophic forgetting," where the model loses its ability to accurately predict patterns in the original network. Conversely, retraining a massive model from scratch every time the network expands is computationally prohibitive. Existing "one-model-fits-all" approaches struggle to capture the diverse, localized dynamics of an expanding city. The authors propose TFMoE, a system designed to learn continuously from a stream of new data without losing prior knowledge or requiring total retraining.
Methods
The proposed Traffic Forecasting Mixture of Experts (TFMoE) architecture moves away from a single global model toward specialized expert models.
Pre-training and Clustering
The system segments traffic sensors into $K$ homogeneous groups based on flow patterns using Deep Embedded Clustering. Each group is assigned an Expert, consisting of a Reconstructor (a VAE that learns the "essence" of the group's patterns) and a Predictor (a combination of GNNs and 1D-Convolutions).
Continual Learning Strategies
- Reconstructor-Based Knowledge Consolidation: New sensors are assigned to the expert with the lowest reconstruction error. A specific loss function ensures adaptation without drifting from original expertise.
- Forgetting-Resilient Sampling: Instead of storing raw data, TFMoE uses VAE decoders to generate synthetic traffic data for "replay," acting as a memory buffer.
- Reconstruction-Based Replay: "Outlier" sensors that don't fit existing patterns are prioritized for training to capture new urban dynamics.
Graph Structure Learning
The model uses a Gumbel-softmax trick to learn relationships between sensors dynamically, bypassing the need for fixed geographical distance maps.
Datasets
The study utilizes the PEMSD3-Stream dataset, featuring traffic flow data from California (2011–2017). The network grew from 655 sensors in 2011 to 871 by 2017, providing a realistic testbed for handling new sensors and long-term shifts in traffic behavior.
Key Results
TFMoE was evaluated against baselines like GRU, DCRNN, and specialized frameworks like TrafficStream.
- Accuracy: TFMoE consistently achieved the lowest MAE across 15, 30, and 60-minute horizons. At the 60-minute mark, TFMoE achieved an MAE of 17.20, outperforming TrafficStream's 21.22.
- Resilience: While "Expansible" models saw performance plummet over time due to forgetting, TFMoE's error rates remained stable.
- Efficiency: TFMoE (1%) achieves superior results using only 1% of pre-existing nodes for replay. It trains significantly faster than full retraining (e.g., ~244s vs. ~940s+).
Key Insights
- Patterns over Proximity: Clustering by behavior rather than geography is more effective, as distant highways may share more similar patterns than adjacent local streets.
- Generative Memory: Using VAEs to "dream" old data solves privacy and storage issues associated with raw data retention.
- Consolidation Loss: This is critical to prevent the model from collapsing into a single expert, ensuring balanced utilization.
Limitations and Future Work
The initial training task remains computationally heavy ($O(N^2K)$). Furthermore, there is a scarcity of high-quality, long-term evolving traffic datasets. Future work will focus on scaling to megacities with tens of thousands of sensors using sub-tasking or advanced sampling techniques.
Summary updated