CoordLight: Learning Decentralized Coordination for Network-Wide Traffic Signal Control

Yifeng Zhang, Harsh Goel, Peizhuo Li, Mehul Damani, Sandeep Chinchali, Guillaume Sartoretti

TL;DR

This paper introduces CoordLight, a novel framework for decentralized traffic signal control that significantly reduces average travel times in urban networks. It achieves this by introducing a new "Queue Dynamic State Encoding" (QDSE) for richer traffic state representation and a "Neighbor-aware Policy Optimization" (NAPO) algorithm that enables more effective coordination between traffic signals.

What Problem Does This Paper Address?

Urbanization leads to increasing traffic congestion, which negatively impacts travel times, pollution, and quality of life. Adaptive Traffic Signal Control (ATSC) aims to alleviate this by dynamically adjusting signal timings. While Multi-Agent Reinforcement Learning (MARL) has shown promise, existing decentralized MARL approaches struggle with two main challenges:

  1. Partial Observability and State Representation: Individual traffic signals (agents) only have local information, making it hard to understand the broader network state and predict future congestion. Current state representations often lack the richness to capture complex traffic dynamics.
  2. Decentralized Coordination: Achieving effective coordination among many decentralized agents is difficult. Agents can become myopic, focusing only on their immediate surroundings, which hinders efficient network-wide optimization. Existing coordination methods often struggle with scalability and training stability.

Methods

CoordLight tackles these challenges with two core innovations:

  • Queue Dynamic State Encoding (QDSE): This novel state representation goes beyond simple vehicle counts or pressure. It incorporates features derived from vehicle queueing dynamics, including:
  • Current queue length.
  • Entering and leaving vehicle counts.
  • Information about moving vehicles approaching the intersection.
  • Distance to the foremost moving vehicle. This richer representation allows agents to better predict upcoming congestion and understand local traffic dynamics.

  • Neighbor-aware Policy Optimization (NAPO): This is a fully decentralized MARL algorithm that builds upon Independent Proximal Policy Optimization (IPPO). Key features include:

  • Attention-based Actor-Critic Network: An augmented actor-critic architecture uses multi-head attention to dynamically weigh the importance of neighboring agents' states and actions. This allows agents to focus on influential neighbors and better understand their impact.
  • Neighbor-aware Advantage Calculation: The advantage function is modified to incorporate the weighted influence of neighbors, leading to more accurate estimates of an agent's contribution to regional objectives and a more stable training process.
  • Auxiliary Prediction Losses: The model also includes auxiliary losses for predicting future queue lengths, further enhancing its predictive capabilities.

Datasets

The experiments were conducted using real-world traffic road networks and traffic demand datasets from the CityFlow simulator. Specifically, the paper used:

  • Jinan, China: A 3x4 intersection network.
  • Hangzhou, China: A 4x4 intersection network.
  • New York, USA: A large-scale 7x28 intersection network (196 intersections).

These datasets were used with different traffic demand levels (e.g., DJN(1), DJN(2), DJN(3) for Jinan) to evaluate the robustness of the proposed method.

Key Insights and Ideas

  • Richer State Representation is Crucial: Simply counting vehicles or measuring pressure isn't enough. Understanding queueing dynamics and predicting future vehicle arrivals significantly improves an agent's ability to make proactive, rather than reactive, decisions.
  • Targeted Coordination Outperforms Broad Cooperation: By using attention mechanisms to identify and weigh the influence of specific neighbors, NAPO can achieve more effective coordination than methods that treat all neighbors equally or rely on more general cooperation signals.
  • Balancing Complexity and Accuracy: QDSE strikes a good balance between providing rich traffic information and maintaining manageable complexity, making it practical for real-world deployment.
  • Robustness to Sensor Noise: The QDSE state representation demonstrated good robustness to simulated sensor noise, suggesting its potential for real-world application where perfect sensor data is rare.

Key Results

CoordLight consistently outperformed existing state-of-the-art methods across diverse traffic networks and demand scenarios.

  • Average Travel Time Reduction: CoordLight achieved the lowest average travel times. For example, on the challenging New York dataset (196 intersections), it significantly reduced travel times compared to leading baselines. On the DJN(1) dataset, it achieved an average travel time of 61.58s, outperforming SocialLight by a notable margin.
  • Improved Coordination: The paper showed that CoordLight consistently achieved the lowest average travel time and variance across intersections, indicating superior coordination performance.
  • Ablation Studies:
    • QDSE significantly improved performance over simpler state representations like Vehicle Counts, General Pressure, Efficient Pressure, and Advanced Traffic State.
  • Removing the spatio-temporal network (STN) led to rapid convergence to sub-optimal solutions, highlighting the importance of capturing spatial and temporal dependencies.
  • The state-action decoder was crucial for capturing dependencies among agents and stabilizing training.

Limitations and Future Work

  • Homogeneous Policy: The current implementation trains a homogeneous policy, meaning all agents share the same policy weights. Future work could explore heterogeneous policies for more diverse intersection types.
  • Simulation-Based: Experiments were conducted in simulators. While robustness to sensor noise was tested, real-world deployment would require further validation.
  • Future Directions: The authors suggest extending the method to:
  • Heterogeneous networks with diverse intersection layouts and lane configurations.
  • More realistic, asynchronous signal control settings.
  • Handling imperfect sensor data, traffic priorities (e.g., emergency vehicles), and scenarios like traffic accidents and road closures.

Summary updated

Abstract

Adaptive traffic signal control (ATSC) is crucial in alleviating congestion, maximizing throughput and promoting sustainable mobility in ever-expanding cities. Multi-Agent Reinforcement Learning (MARL) has recently shown significant potential in addressing complex traffic dynamics, but the intricacies of partial observability and coordination in decentralized environments still remain key challenges in formulating scalable and efficient control strategies. To address these challenges, we present CoordLight, a MARL-based framework designed to improve intra-neighborhood traffic by enhancing decision-making at individual junctions (agents), as well as coordination with neighboring agents, thereby scaling up to network-level traffic optimization. Specifically, we introduce the Queue Dynamic State Encoding (QDSE), a novel state representation based on vehicle queuing models, which strengthens the agents' capability to analyze, predict, and respond to local traffic dynamics. We further propose an advanced MARL algorithm, named Neighbor-aware Policy Optimization (NAPO). It integrates an attention mechanism that discerns the state and action dependencies among adjacent agents, aiming to facilitate more coordinated decision-making, and to improve policy learning updates through robust advantage calculation. This enables agents to identify and prioritize crucial interactions with influential neighbors, thus enhancing the targeted coordination and collaboration among agents. Through comprehensive evaluations against state-of-the-art traffic signal control methods over three real-world traffic datasets composed of up to 196 intersections, we empirically show that CoordLight consistently exhibits superior performance across diverse traffic networks with varying traffic flows. The code is available at https://github.com/marmotlab/CoordLight
BibTeX
@article{zhang2025coordlight,
  author = {Yifeng Zhang and Harsh Goel and Peizhuo Li and Mehul Damani and Sandeep Chinchali and Guillaume Sartoretti},
  title = {{CoordLight: Learning Decentralized Coordination for Network-Wide Traffic Signal Control}},
  journal = {IEEE Trans. Intell. Transp. Syst.},
  year = {2025},
  doi = {10.1109/tits.2025.3546261},
}
Figure