TL;DR
Real-world traffic networks continually expand with new sensors, causing standard incremental forecasting models to suffer from catastrophic forgetting, while retraining from scratch is computationally prohibitive. The authors introduce TFMoE (Traffic Forecasting Mixture of Experts), an architecture that groups sensors into homogeneous pattern clusters and assigns an expert pair (a VAE reconstructor and a GNN predictor) to each. By combining graph structure learning, synthetic latent sampling, and targeted 1% replay, TFMoE beats existing continual learning baselines while dramatically reducing memory and prior-node access.
What Problem Does This Paper Address?
In intelligent transportation systems, traffic networks are rarely static: city growth and infrastructure projects continually add new sensors over years. When a model trained on an existing road network is fine-tuned only on newly added sensors, it forgets past dynamics—a classic case of catastrophic forgetting.
Retraining models on the entire historical dataset every time sensors are added is computationally infeasible. Conversely, prior continual learning approaches for streaming traffic graphs (such as TrafficStream) rely on a monolithic "one-model-fits-all" approach and require extracting localized spatial subgraphs around new and replayed sensors. This forces the model to repeatedly access a large fraction (>20%) of historical nodes and store large feature banks from past years.
Methods
TFMoE divides the network dynamics into specialized niches using a Mixture of Experts (MoE) formulation, where each expert consists of a Reconstructor (Variational Autoencoder, or VAE) and a Predictor (Diffusion Convolutional Layer + 1D Temporal Convolutions).
[ Incoming Sensor Data ]
│
┌───────────────┴───────────────┐
▼ ▼
[ VAE Reconstructors ] [ GNN Predictors ]
(Compute Gating Weights) (Forecast Future Flow)
│ │
└───────────────┬───────────────┘
▼
[ Weighted Final Forecast ]
- Pre-training & Clustering: In the initial time period ($\tau = 1$), an autoencoder embeds one-week traffic patterns, followed by balanced $k$-means clustering to partition sensors into $K$ homogeneous clusters ($SG_k$). An expert is assigned to each cluster.
- Graph Structure Learning: Instead of using fixed, distance-based subgraphs (which force models to pull in neighboring pre-existing nodes), TFMoE learns dynamic graph adjacency via the Gumbel-Softmax trick.
- Reconstructor-Based Gating & Consolidation: When evaluating traffic flow, the gating mechanism weights each expert's forecast using its VAE reconstruction likelihood. During subsequent tasks, an adaptation loss (inspired by Learning without Forgetting) preserves the localized grouping learned by earlier reconstructors.
- Forgetting-Resilient Sampling: The decoders of earlier-trained VAEs generate diverse synthetic traffic profiles. These synthetic nodes are fed alongside current-task nodes into the predictor without needing external memory caches.
- Reconstruction-Based Replay: To capture pre-existing sensors whose patterns have drastically evolved (e.g., due to urban shifts), TFMoE identifies nodes with the lowest reconstruction probability across all experts and replays only a small fraction ($n_r = 1\%$).
Datasets
- PEMSD3-Stream: Extracted from the California Caltrans Performance Measurement System (PeMS) across the North Central Area from 2011 to 2017 (analyzing July 10th to August 9th annually in 5-minute increments). The network expands over time from 655 sensors (3,926 edges) in 2011 to 871 sensors (5,572 edges) in 2017.
Key Insights and Ideas
- Modularity beats a monolithic model: New traffic sensors typically resemble a subset of existing patterns rather than introducing completely new global behaviors. Dedicating an expert to a specific pattern group prevents gradient interference across distinct behaviors.
- Decoupling from physical subgraphs: Traditional spatio-temporal GNNs need physical neighborhood subgraphs, inadvertently pulling in old nodes during incremental updates. Learned graph adjacency allows synthetic and real nodes to interact without historical graph dependencies.
- Generative memory over feature caching: Instead of archiving raw sensor readings from previous years, the VAE decoders generate synthetic traffic samples, drastically shrinking the memory footprint.
- Targeting the "most unfamiliar" nodes for replay: Rather than random replay, sorting pre-existing sensors by joint reconstruction error reliably flags nodes undergoing true distribution shifts.
Key Results
- Forecasting Accuracy: On PEMSD3-Stream, TFMoE significantly outperformed prior continual learning models across 15-, 30-, and 60-minute horizons. For 60-minute forecasts, TFMoE achieved an MAE of 17.65 (RMSE 28.95), outperforming TrafficStream (MAE 21.12) and Expansible-TFMoE without replay/sampling (MAE 21.77).
- Minimal Access Footprint: Under a standardized TrafficStream predictor structure, TFMoE achieved an MAE of 14.42 accessing only 1% of historical nodes, outperforming TrafficStream (16.35) and PECMP (16.02), both of which required 10% + $\gamma\%$ node access (often exceeding 20% total due to subgraph structures).
- Training Efficiency: Online training for TFMoE took 255 seconds, compared to >50,000 seconds required to fully retrain standard GNN architectures (DCRNN, STSGCN).
Limitations and Future Work
- Benchmark Scarcity: Due to the novelty of continual traffic forecasting, evaluations were limited to a single streaming benchmark (PEMSD3-Stream).
- Initial Computational Burden: The initial task graph structure learning scales quadratically with the initial node count ($O(N^2K)$), creating potential memory and compute bottlenecks on massive initial networks.
- Suggested Future Directions: Exploring subtask partitioning or distributed training during the initialization phase, and developing dynamic expert adjustment (pruning underutilized experts or splitting overloaded ones).
Summary updated