TL;DR
TopoST is a spatio-temporal foundation model that enables high-performance zero-shot forecasting in new cities by learning a universal rule to specialize model parameters based on local graph topology. By mapping identity-independent structural features to specific model weights, it bridges the gap between general knowledge sharing and the unique, heterogeneous dynamics of individual urban sensors.
What Problem Does This Paper Address?
Urban spatio-temporal forecasting—predicting things like traffic flow, speed, or taxi demand—is notoriously difficult to generalize. Every city has a unique layout (topology), and even within a single city, two sensors just a block apart might exhibit vastly different behaviors (heterogeneity).
Existing Spatio-Temporal Foundation Models (STFMs) try to learn reusable patterns across cities, but they face a fundamental trade-off:
- Uniformity: Treating all nodes the same way, which ignores local dynamics.
- Identity-Dependence: Learning specific "embeddings" for each node, which makes it impossible to transfer the model to a new city where those node identities don't exist.
The authors of TopoST ask: Can we learn a generalizable rule for constructing node-specific dependency models in unseen urban scenarios?
Methods
The core innovation of TopoST is Topology-Conditioned Node Specialization. Instead of learning what a specific node is, the model learns how a node should behave based on where it sits in the graph.
- Local Spatio-Temporal Units: The model doesn't look at a whole city at once. It uses METIS partitioning to break graphs into small, manageable subgraphs. This allows the model to focus on recurring local patterns (like intersections or corridors) that are common across different cities.
- Structural Descriptors: For every node, the model calculates identity-independent features like degree, clustering coefficients, and neighborhood statistics (Statistical Structural Encoding).
- Topology Encoder: A Graph Isomorphism Network (GIN) processes these structural features into "node codes." Because GIN is permutation-equivariant, it ensures that the model's understanding of a node depends only on its structural context, not its arbitrary ID.
- Parameter Generation: These node codes act as keys to a shared "parameter pool." The model generates unique weights for each node's Temporal Attention (how it relates to its own history) and Spatial GCN (how it interacts with neighbors).
Datasets
The authors utilized a massive collection of data for pretraining and evaluation:
- Pretraining: 20 source datasets spanning traffic flow (e.g., PEMS04, PEMS08), traffic speed (e.g., METR-LA), and taxi demand (NYC-TAXI), totaling over 151 million observations.
- Zero-Shot Evaluation: 6 target datasets excluded from pretraining, covering cross-region transfer (California), cross-city transfer (Shanghai, Chicago), and even cross-category transfer (NYC-BIKE demand).
Key Insights and Ideas
- Local Structure Recurrence: The authors analyzed 20+ datasets and found that while global city graphs are unique, local structures repeat constantly. For example, 84.4% of 1-hop neighborhoods in one dataset have an exact structural match in another.
- Persistent Heterogeneity: Even within these recurring structures, nodes behave differently. Within-subgraph differences account for over 70% of the variation in temporal behavior. This proves that "one-size-fits-all" models are destined to fail.
- Transferable Specialization: The "aha moment" is realizing that node specialization itself can be a transferable rule. If you know how a node in a specific structural context behaves in City A, you can apply that logic to a structurally similar node in City B without ever having seen City B before.
Key Results
TopoST demonstrates state-of-the-art performance in zero-shot settings:
- Superior Accuracy: It outperformed existing foundation models like OpenCity, UrbanFM, and UniST across all 18 dataset-metric combinations tested. On average, it reduced error (MAE) by ~10% compared to the next best foundation model.
- Efficiency: TopoST is remarkably compact, with only 2.16M parameters. It uses ~90% less training memory than UrbanFM.
- Scalability: The model supports a parallel execution mode (TopoST-P) that achieves nearly 50x speedups in inference latency for larger graphs compared to sequential processing.
Limitations and Future Work
- The Zero-Shot Gap: While TopoST is the best foundation model, there is still a performance gap between zero-shot models and "expert" models trained specifically on a single target dataset (full-shot). The authors show that few-shot adaptation (fine-tuning on just 10% of target data) significantly closes this gap.
- Static vs. Dynamic Topology: The current framework assumes the graph structure is fixed. Future iterations could explore how to handle dynamic graphs where connections change over time (e.g., road closures).
- Broader Modalities: While the model handles flow, speed, and demand, extending this topology-conditioned approach to other urban modalities like air quality or energy consumption remains an open direction.
Summary updated