TL;DR
AttendLight is a "universal" reinforcement learning (RL) model for traffic signal control that can be deployed at any intersection without retraining. By utilizing a dual-attention architecture, it seamlessly handles varying numbers of lanes and signal phases, outperforming both traditional rule-based systems and state-of-the-art specialized RL models.
The Scalability Bottleneck in Traffic Control
Traffic signal control (TSC) is a classic optimization problem, but traditional Reinforcement Learning approaches suffer from a significant scalability bottleneck: lack of generalization.
Most existing RL models are "one-offs"—they are trained for a specific intersection topology, such as a standard 4-way crossroad. If you attempt to deploy that model to a 3-way "T" intersection or a road with an extra turn lane, the model's input and output dimensions break. This requires a complete redesign and retraining from scratch. In a city with thousands of unique intersections, this "train-per-intersection" requirement is computationally expensive and practically unfeasible.
To solve this, the authors propose AttendLight, an end-to-end RL algorithm designed to be a single, universal model capable of controlling any intersection configuration it might encounter.
Methods: A Dual-Attention Architecture
The core innovation of AttendLight is the use of two distinct attention mechanisms that decouple the model's architecture from the physical structure of the intersection.
- State Attention: This layer handles the variable number of approaching lanes. It takes raw traffic characteristics (like the number of moving and waiting vehicles) and embeds them into a higher-dimensional space. The attention mechanism then aggregates these lane-level features into a fixed-length "phase representation," regardless of how many lanes are involved in that phase.
- Action Attention: This layer handles the variable number of possible signal phases (actions). It compares the current "phase representations" against the output of a Long Short-Term Memory (LSTM) network, which tracks the history of previous signal changes. This allows the model to decide which phase to activate next, even if the total number of available phases differs between intersections.
The model is trained using a variance-reduced version of the REINFORCE algorithm. The reward function is based on minimizing "intersection pressure"—the difference between the number of vehicles waiting in entering lanes and those in leaving lanes—which serves as a proven proxy for reducing overall travel time.
Datasets and Topologies
The researchers evaluated AttendLight using a mix of real-world and synthetic data across 112 unique intersection instances:
- Real-World Data: Traffic flow datasets from Hangzhou and Atlanta, which are commonly used as benchmarks in TSC research.
- Synthetic Data: Created to cover scenarios lacking real-world data, specifically intersections with three lanes per road.
- Topologies: The experiments covered 11 different intersection topologies, ranging from 3-way to 4-way structures with varying lane counts and signal phase options (from 2 to 8 phases).
Key Insights and Ideas
- Attention as a Structural Bridge: While attention is famous for its role in Natural Language Processing, this paper demonstrates its power as a structural tool. It allows the model to treat an intersection as a flexible graph rather than a rigid matrix, enabling the "universal" property.
- Knowledge Sharing: By training on a diverse set of intersections (the "multi-env" regime), the model learns fundamental principles of traffic flow. The authors found that a model trained on many intersections can actually perform better on some specific cases than a model trained only on that single case, likely due to "knowledge sharing" of successful control strategies across similar patterns.
- Zero-Shot Generalization: The model demonstrates impressive performance on "unseen" intersections. This suggests that the attention mechanisms successfully extract abstract representations of traffic status that are invariant to the specific geometry of the road.
Key Results
AttendLight was tested in two regimes: single-environment (trained and tested on one intersection) and multi-environment (one model for all).
- Superiority over Baselines: In the single-env regime, AttendLight outperformed classical methods (like MaxPressure and SOTL) and state-of-the-art RL models (like FRAP). It achieved a 46% improvement over Fixed-Time policies and a 9% improvement over FRAP.
- Robust Universality: In the multi-env regime, a single AttendLight model achieved performance very close to specialized models. On average, there was only a 15% performance gap when using one universal model for 112 different intersections compared to training 112 separate models.
- Fast Calibration: For critical intersections where maximum performance is needed, the authors showed that a universal model could be "fine-tuned" (Few-Shot training) to reach near-optimal performance in just 200-1,000 episodes, compared to the 100,000+ episodes usually required to train from scratch.
Limitations and Future Work
- Isolated Intersections: The current study focuses on controlling single intersections in isolation. In the real world, intersections are linked in a network. Extending AttendLight to coordinated multi-intersection control is a vital next step.
- Extreme Topologies: While the model is flexible, it may still struggle with extremely unusual or complex intersection designs that are significantly different from anything in the training set.
- Algorithm Improvements: The authors used the REINFORCE algorithm for its simplicity in demonstrating the architecture's effectiveness, but they suggest that more advanced RL algorithms like Actor-Critic or PPO could further improve numerical results.
Summary updated