X-Light: Cross-City Traffic Signal Control Using Transformer on Transformer as Meta Multi-Agent Reinforcement Learner

Haoyuan Jiang, Ziyue Li, Hua Wei, Xuantang Xiong

Abstract

Reinforcement Learning (RL) has shown promise in optimizing Traffic Signal Control (TSC), yet existing models often struggle with generalization across diverse urban environments. We present X-Light, a novel framework that employs a hierarchical Transformer on Transformer (TonT) architecture to enable zero-shot transferability. By modeling multi-agent traffic coordination as a sequence modeling problem, X-Light captures both local spatial cooperation and global temporal dynamics. Our approach integrates full Markov Decision Process (MDP) trajectories from neighboring intersections and utilizes a General Preprocess Module to handle varying junction geometries. Extensive evaluations on synthetic and real-world maps demonstrate that X-Light outperforms state-of-the-art methods by an average of 7.9% in zero-shot scenarios.

Introduction: The Generalization Gap in Traffic Control

Deploying RL-based traffic signal control in real-world settings is hindered by the "generalization gap." A model trained on a specific city's grid, such as Manhattan, typically fails when deployed in a different urban layout, like the winding streets of Rome. Achieving "out of the box" (zero-shot) transferability is essential for scalable smart city infrastructure. Existing meta-learning approaches often fail to account for the complex spatial-temporal dependencies between neighboring intersections or rely on architectures that do not generalize well across different city "styles."

Overview of X-Light's multi-scenario hierarchical approach, showing the interaction between target and neighbor intersections across different urban layouts.

The X-Light Framework

X-Light addresses these challenges through a hierarchical encoder known as the Transformer on Transformer (TonT). This architecture is designed to act as a meta-learner across diverse traffic scenarios.

Hierarchical TonT Encoder

The TonT encoder consists of two primary levels:

  1. Lower Transformer (Local Cooperation): This layer processes the full MDP trajectory—observations ($o$), actions ($a$), and rewards ($r$)—from both the target intersection and its neighbors. Unlike standard methods that only consider neighbor states, this approach allows the model to understand how specific timing decisions by neighbors influence local traffic flow.
  2. Upper Transformer (Global Generalization): This layer analyzes historical trajectories across different time steps and scenarios. It functions as a meta-agent, learning latent "city styles" to infer the dynamics of the current environment.

Unified Representation and Training

To handle varying intersection geometries (e.g., 3-way vs. 4-way junctions), X-Light employs a General Preprocess Module (GPI) that maps all structures into a unified 4-leg representation. Furthermore, a Dynamic Predictor pretext task is added, requiring the model to predict the next state, which encourages a deeper understanding of traffic physics.

The complete X-Light framework, illustrating the Actor-Critic architecture, the TonT encoder, and the integration of traffic signal phases.

Experimental Results

The researchers evaluated X-Light using the SUMO (Simulation of Urban MObility) engine across seven diverse scenarios, including synthetic grids and real-world maps from Cologne, Ingolstadt, Fenglin, and Nanshan.

Key Performance Metrics

  • Zero-Shot Superiority: X-Light outperformed all baselines (including MetaGAT and GESA) by an average of 7.91% on unseen cities. In complex 5x5 grid scenarios, improvements reached up to 16.3%.
  • Stability: Training curves indicate that X-Light is significantly more stable than previous multi-agent meta-learners, avoiding the volatile performance drops often seen when encountering dissimilar scenarios.
  • Ablation Study: Removing the Lower Transformer resulted in the most significant performance degradation, highlighting the importance of sophisticated local coordination.

Conclusion and Future Work

X-Light demonstrates that hierarchical sequence modeling is a powerful tool for cross-city traffic signal control. By separating local spatial cooperation from global meta-learning, the framework achieves robust generalization. Future work will focus on handling extremely irregular "mega-intersections" and addressing real-world challenges such as sensor noise and communication delays in smart city infrastructure.

Summary updated

BibTeX
@inproceedings{jiang2024x,
  author = {Haoyuan Jiang and Ziyue Li and Hua Wei and Xuantang Xiong},
  title = {{X-Light: Cross-City Traffic Signal Control Using Transformer on Transformer as Meta Multi-Agent Reinforcement Learner}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
  url = {https://openreview.net/forum?id=bBGCThZxy4},
}
Figure