LibSignal: An Open Library for Traffic Signal Control

Hao Mei, Xiaoliang Lei, Longchao Da, Bin Shi, Hua Wei

TL;DR

LibSignal is the first open-source library designed for fair, cross-simulator benchmarking of reinforcement learning (RL) models in traffic signal control. By unifying environments like SUMO and CityFlow with standardized metrics and datasets, it enables researchers to finally compare "apples to apples" in a historically fragmented field.

What Problem Does This Paper Address?

Traffic signal control (TSC) is a cornerstone of urban mobility, but the research landscape is currently a "Wild West." While reinforcement learning has shown immense promise in optimizing traffic flow, comparing different RL algorithms is notoriously difficult.

The primary issue is fragmentation: researchers use different simulators (like the highly realistic SUMO or the RL-optimized CityFlow), disparate datasets, and inconsistent evaluation metrics. Furthermore, RL is highly sensitive to hyperparameters and the specific "physics" of a simulator (e.g., how a car handles a yellow light). Without a unified framework, it is nearly impossible to tell if a new algorithm represents a genuine breakthrough or just a lucky combination of simulator settings and metrics.

Methods

The authors introduce LibSignal, a systematic pipeline built on PyTorch that standardizes the entire TSC research workflow. Key components include:

  • Unified Framework: A modular architecture consisting of a "World" (the simulator), a "Generator" (extracting states and rewards), an "Agent" (the RL model), and a "Metrics" module.
  • Cross-Simulator Compatibility: A dedicated converter.py tool allows researchers to transform road networks and traffic flow files between SUMO’s XML formats and CityFlow’s JSON formats.
  • Standardized Implementations: The library includes 10 reproduced models (3 traditional, 7 RL-based) originally written in various frameworks (like TensorFlow), now unified in PyTorch.
  • Calibration: The authors performed extensive environment calibration—aligning settings like yellow light intervals and routing—to ensure that experiments in one simulator are referential to the other.
Figure 1: Overall framework of LibSignal

Datasets

LibSignal incorporates 9 commonly used datasets curated from a survey of 225 papers, covering roughly 65% of the existing literature. These include real-world and synthetic networks such as:

  • Jinan & Hangzhou: Real-world intersection data.
  • Manhattan: A large-scale 7x28 grid.
  • Cologne: Complex, non-grid traffic structures.
  • Grid 4x4: A standard synthetic benchmark.

These datasets are provided as "atomic files," making them instantly runnable across different simulators.

Key Insights and Ideas

  • Simulators are not interchangeable: Even after calibration, internal mechanisms differ. For instance, SUMO includes random vehicle deceleration and pedestrian signals that CityFlow lacks. LibSignal highlights that while absolute performance varies, the relative ranking of algorithms usually remains consistent across simulators.
  • Metric Choice Matters: The authors found that "approximated delay" (often used in literature) is frequently inaccurate. They argue that Average Travel Time is the most robust and reliable metric for evaluating TSC efficiency.
  • Q-Learning vs. Actor-Critic: In the TSC domain, Q-learning-based methods (like IDQN and PressLight) tend to converge significantly faster—sometimes by a factor of ten—compared to Actor-Critic methods like IPPO, which struggle with the high-dimensional state spaces of complex traffic networks.
Figure 2: Convergence curve of models implemented in their original form (Tensorflow) and in LibSignal (PyTorch). Y-axis are the testing results w.r.t. average travel time (in seconds). Validation for more models can be found in Appendix.

Key Results

  • Validation: The PyTorch re-implementations successfully matched the performance of original TensorFlow versions. For example, IDQN on the CityFlow 1x1 task converged to an average travel time of ~116s, closely matching the original paper's ~127s.
  • Efficiency: The study confirmed that CityFlow is roughly 10x faster than SUMO (when using the TraCI interface), making it ideal for the heavy iteration required in RL training, while SUMO remains the gold standard for realistic validation.
  • Algorithm Performance: Traditional methods like MaxPressure remain surprisingly competitive, but RL methods like PressLight and FRAP consistently achieve the best results in complex, multi-intersection scenarios.

Limitations and Future Work

While LibSignal is a major step forward, the authors acknowledge several hurdles:

  • Simulator Support: Currently, it only supports SUMO and CityFlow. While integration for CBEngine is mentioned, expanding to other emerging simulators is a priority.
  • Calibration Labor: Aligning simulators is still a manual, intensive process.
  • Agent Complexity: Implementing agents for highly irregular, large-scale networks remains difficult, especially for models like FRAP that require specific definitions of "competitive traffic phases."

Future work will focus on integrating more state-of-the-art RL models and further automating the calibration process to support even more diverse simulation environments.

Summary updated

Abstract

This paper introduces a library for cross-simulator comparison of reinforcement learning models in traffic signal control tasks. This library is developed to implement recent state-of-the-art reinforcement learning models with extensible interfaces and unified cross-simulator evaluation metrics. It supports commonly-used simulators in traffic signal control tasks, including Simulation of Urban MObility(SUMO) and CityFlow, and multiple benchmark datasets for fair comparisons. We conducted experiments to validate our implementation of the models and to calibrate the simulators so that the experiments from one simulator could be referential to the other. Based on the validated models and calibrated environments, this paper compares and reports the performance of current state-of-the-art RL algorithms across different datasets and simulators. This is the first time that these methods have been compared fairly under the same datasets with different simulators.
BibTeX
@article{mei2024libsignal,
  author = {Hao Mei and Xiaoliang Lei and Longchao Da and Bin Shi and Hua Wei},
  title = {{LibSignal: An Open Library for Traffic Signal Control}},
  journal = {Machine Learning},
  year = {2024},
  doi = {10.1007/s10994-023-06412-y},
}
Figure