HALight: Heterogeneous Multi-Agent Reinforcement Learning for Traffic Signal Control under Lane-Level Missing Observations

Chenkai Hu, Yingqian Zhang, Ming Xie, Yi Fang, Chengxiang Jin, Chao LiarXiv 2026 · arXiv

TL;DR

HALight is a novel multi-agent reinforcement learning (MARL) framework designed for traffic signal control that breaks away from the "idealized" assumptions of previous models. By supporting heterogeneous intersection modeling and a lightweight neighbor-communication mechanism, it maintains high traffic efficiency even when intersections have diverse layouts and sensors frequently fail to provide complete data.

What Problem Does This Paper Address?

Most existing reinforcement learning (RL) methods for traffic signal control (TSC) operate under two major simplifications that rarely hold true in the real world:

  1. Homogeneity: They assume every intersection is identical in terms of lane configuration and signal phases. In reality, cities are a mix of T-junctions, standard crosses, and complex multi-lane intersections.
  2. Full Observability: They assume sensors (like cameras or induction loops) are perfect. In practice, sensor failures, limited coverage, or "blind spots" in electronic toll collection (ETC) sensing lobes lead to missing lane-level data.

When a standard RL agent trained on "average" intersection data meets a unique T-junction, or when its sensors go dark, its performance degrades rapidly, leading to traffic instability and poor coordination.

Illustration of intersection heterogeneity, showing variations across different types and structural differences within the same type.

Methods

The authors propose HALight, a framework that tackles these issues through three core architectural choices:

  • Heterogeneous Actor Modeling: Instead of a one-size-fits-all policy, HALight allows for intersection-specific observation and action spaces. This means an agent at a T-junction isn't trying to use a policy designed for a 4-way cross.
  • Communication-Enhanced Coordination: To compensate for missing local data, HALight incorporates a lightweight, one-hop communication module. Using a dot-product attention mechanism, agents selectively aggregate "hypotheses" from their neighbors. If an agent's own sensors are missing data for an incoming lane, the information from the upstream neighbor helps fill in the blanks.
  • Sequential Policy Updates: Drawing from Heterogeneous-Agent Proximal Policy Optimization (HAPPO), HALight uses a sequential update scheme. Rather than all agents updating simultaneously—which can be unstable in a multi-agent environment—they update one by one, accounting for the changes made by previously updated neighbors.
  • Temporal Modeling: Each agent uses a Gated Recurrent Unit (GRU) to maintain a memory of past traffic states, helping it infer current conditions even during intermittent sensor outages.
Illustration of how non-uniform sensor coverage (like ETC sensing lobes) leads to unavailable lane-level observations.

Datasets

The model was evaluated using the CityFlow simulation platform on two real-world road networks from Hangzhou, China:

  • Xiasha District: A 3×3 grid featuring 7 crossing intersections and 2 T-junctions with varying configurations.
  • Gudang District: A 4×4 grid of 16 crossing intersections with diverse structural layouts.

The researchers tested these networks under both "moderate" and "high" traffic demands, with lane-level observation missing rates ranging from 0% to 40%.

Key Insights and Ideas

  • Spatial-Temporal Compensation: The combination of GRUs (temporal) and neighbor communication (spatial) acts as a powerful redundancy system. When local "eyes" fail, the agent uses its "memory" and its "neighbors' advice" to make a decision.
  • Avoiding the "Average" Trap: Homogeneous models tend to learn a policy that is a mediocre compromise for all intersections. By embracing heterogeneity, HALight allows agents to specialize, which is particularly effective in structurally diverse networks like Xiasha.
  • Lightweight is Better: Unlike complex Graph Attention Network (GAT) approaches that require global information, HALight’s one-hop communication is computationally efficient and scales linearly with the number of intersections, making it viable for large-scale city deployment.

Key Results

HALight consistently outperformed both homogeneous (e.g., PressLight, DynSTGAT) and heterogeneous (e.g., HAPPO) baselines:

  • Robustness to Data Loss: At a 40% missing observation rate, many baseline models failed to converge or showed massive performance spikes. HALight remained stable, maintaining significantly lower Average Travel Time (ATT) and Average Queue Length (AQL).
  • Efficiency: In the Gudang high-demand scenario with 40% data loss, HALight achieved an ATT of ~314 seconds, whereas the best-performing baseline (Het-MAPPO) lagged behind at ~317 seconds, and others failed entirely.
  • Communication Resilience: The model proved robust to communication imperfections, showing only minor performance drops even with 50% packet loss or significant transmission delays.

Limitations and Future Work

While HALight is a significant step toward realistic TSC, the authors note a few areas for improvement:

  • More Complex Traffic: The current study uses relatively standard phase plans. Future work could explore richer, more adaptive phase designs.
  • Long-Range Interactions: The current model only looks at one-hop neighbors. In very dense or highly congested networks, multi-hop communication might be necessary to manage traffic waves that propagate across several blocks.
  • Adaptive Communication: Exploring ways for agents to decide when and what to communicate more selectively could further optimize bandwidth usage in real-world IoT deployments.

Summary updated

Abstract

Multi-agent reinforcement learning has shown great promise for traffic signal control. However, most existing methods rely on two key assumptions: (i) agents are assumed to be homogeneous with unified observation/action spaces; (ii) full and reliable lane-level traffic observations are available to all agents. In practice, urban road networks exhibit substantial structural heterogeneity (e.g., lane configurations and phase designs), while sensor failures and incomplete detector coverage often result in missing incoming lane measurements. Such issues can severely impair coordination and destabilize learning. To address these challenges, we propose HALight, a communication-enhanced heterogeneous multi-agent reinforcement learning framework for traffic signal control under lane-level missing observations. HALight supports intersection-specific observation and action modeling to accommodate heterogeneous intersections, and incorporates a lightweight one-hop neighborhood communication module to enrich spatial context when lane-level observations are partially unavailable. Moreover, HALight adopts a sequential policy update scheme to improve coordination among heterogeneous agents. Experimental results on two real-world road networks demonstrate that HALight outperforms representative homogeneous and heterogeneous baselines in terms of traffic efficiency and convergence behavior. The code and datasets used in this work are available at https://github.com/Huchenkaii/HALight.
BibTeX
@misc{hu2026halight,
  author = {Chenkai Hu and Yingqian Zhang and Ming Xie and Yi Fang and Chengxiang Jin and Chao Li},
  title = {{HALight: Heterogeneous Multi-Agent Reinforcement Learning for Traffic Signal Control under Lane-Level Missing Observations}},
  year = {2026},
  eprint = {2611.16574},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.16574},
}
Figure