TL;DR
LibSignal is the first open-source library designed for fair, cross-simulator benchmarking of reinforcement learning (RL) models in traffic signal control. By unifying environments like SUMO and CityFlow with standardized metrics and datasets, it enables researchers to finally compare "apples to apples" in a historically fragmented field.
What Problem Does This Paper Address?
Traffic signal control (TSC) is a cornerstone of urban mobility, but the research landscape is currently a "Wild West." While reinforcement learning has shown immense promise in optimizing traffic flow, comparing different RL algorithms is notoriously difficult.
The primary issue is fragmentation: researchers use different simulators (like the highly realistic SUMO or the RL-optimized CityFlow), disparate datasets, and inconsistent evaluation metrics. Furthermore, RL is highly sensitive to hyperparameters and the specific "physics" of a simulator (e.g., how a car handles a yellow light). Without a unified framework, it is nearly impossible to tell if a new algorithm represents a genuine breakthrough or just a lucky combination of simulator settings and metrics.
Methods
The authors introduce LibSignal, a systematic pipeline built on PyTorch that standardizes the entire TSC research workflow. Key components include:
- Unified Framework: A modular architecture consisting of a "World" (the simulator), a "Generator" (extracting states and rewards), an "Agent" (the RL model), and a "Metrics" module.
- Cross-Simulator Compatibility: A dedicated
converter.pytool allows researchers to transform road networks and traffic flow files between SUMO’s XML formats and CityFlow’s JSON formats. - Standardized Implementations: The library includes 10 reproduced models (3 traditional, 7 RL-based) originally written in various frameworks (like TensorFlow), now unified in PyTorch.
- Calibration: The authors performed extensive environment calibration—aligning settings like yellow light intervals and routing—to ensure that experiments in one simulator are referential to the other.
Datasets
LibSignal incorporates 9 commonly used datasets curated from a survey of 225 papers, covering roughly 65% of the existing literature. These include real-world and synthetic networks such as:
- Jinan & Hangzhou: Real-world intersection data.
- Manhattan: A large-scale 7x28 grid.
- Cologne: Complex, non-grid traffic structures.
- Grid 4x4: A standard synthetic benchmark.
These datasets are provided as "atomic files," making them instantly runnable across different simulators.
Key Insights and Ideas
- Simulators are not interchangeable: Even after calibration, internal mechanisms differ. For instance, SUMO includes random vehicle deceleration and pedestrian signals that CityFlow lacks. LibSignal highlights that while absolute performance varies, the relative ranking of algorithms usually remains consistent across simulators.
- Metric Choice Matters: The authors found that "approximated delay" (often used in literature) is frequently inaccurate. They argue that Average Travel Time is the most robust and reliable metric for evaluating TSC efficiency.
- Q-Learning vs. Actor-Critic: In the TSC domain, Q-learning-based methods (like IDQN and PressLight) tend to converge significantly faster—sometimes by a factor of ten—compared to Actor-Critic methods like IPPO, which struggle with the high-dimensional state spaces of complex traffic networks.
Key Results
- Validation: The PyTorch re-implementations successfully matched the performance of original TensorFlow versions. For example, IDQN on the CityFlow 1x1 task converged to an average travel time of ~116s, closely matching the original paper's ~127s.
- Efficiency: The study confirmed that CityFlow is roughly 10x faster than SUMO (when using the TraCI interface), making it ideal for the heavy iteration required in RL training, while SUMO remains the gold standard for realistic validation.
- Algorithm Performance: Traditional methods like MaxPressure remain surprisingly competitive, but RL methods like PressLight and FRAP consistently achieve the best results in complex, multi-intersection scenarios.
Limitations and Future Work
While LibSignal is a major step forward, the authors acknowledge several hurdles:
- Simulator Support: Currently, it only supports SUMO and CityFlow. While integration for CBEngine is mentioned, expanding to other emerging simulators is a priority.
- Calibration Labor: Aligning simulators is still a manual, intensive process.
- Agent Complexity: Implementing agents for highly irregular, large-scale networks remains difficult, especially for models like FRAP that require specific definitions of "competitive traffic phases."
Future work will focus on integrating more state-of-the-art RL models and further automating the calibration process to support even more diverse simulation environments.
Summary updated