City traffic lights are now best run by numeric reinforcement learning, meaning controllers that learn by trial and error in simulation. On large Manhattan networks such controllers beat language-model controllers on both travel time and runtime (SLight). They still break in predictable ways. An intersection loses power and its neighbours keep sending cars into it (MalLight). Sensors drop out and performance falls (LA-Light). A snowstorm changes vehicle physics, and a policy trained in clear weather transfers badly (PromptGAT). Language-model agents handle these cases better, but so far only when they are very large general models called slowly through long prompts.
We propose to stop asking a language model to pick signal phases. Instead it becomes a slow supervisor sitting above a fast numeric controller. Every few minutes it proposes structural changes the controller cannot represent: isolating a failed node, metering inflow from upstream, reversing a tidal lane, or overriding a corrupted sensor. The supervisor has two roles. A proposer writes an intervention. A critic predicts its network-wide consequences and vetoes the ones that would cause spillback, meaning queues backing up into neighbouring intersections. Both roles run on one compact model (about 1–3B parameters) that has been continually pre-trained on traffic data: incident text interleaved with spatio-temporal traffic tokens. The claim to test is that this domain pre-training, rather than model scale or prompting, lets a small model do this supervision well.
Core Claim
Existing work splits along two axes. On the first axis, some systems use language models for closed-loop control. CoLLMLight separates a slow cooperative-reasoning module from a fast phase-selection module, with a general base model tuned by supervised fine-tuning and PPO (a standard policy-gradient RL method). CoMAL has vehicle agents set parameters for a classical car-following controller. LA-Light prompts GPT-4 Turbo with traffic tools. None of these systems uses domain pre-training. CoMAL also reports that a 7B general model collapses at multi-agent coordination where a 72B model does not.
On the second axis, some systems do pre-train on urban data but never close the control loop. CityGPT continually pre-trains on urban text, map geometry and trajectories, and is used for route planning and question answering. OpenCity pre-trains a spatio-temporal forecaster on 151M observations, but it is open-loop and has no language interface.
Our proposal differs from both axes in three concrete ways. First, the language model never chooses phases. It edits the operating conditions of a numeric controller at minute timescales, which removes the latency objection. Second, it reads traffic state as learned spatio-temporal tokens rather than verbalized queue counts. This is the "textual bottleneck" LA-Light identifies. Third, the critic's veto rests on a learned forecast of the intervention's effect rather than on the model's own opinion. This matters because critic roles fix confirmation bias in code generation (Self-collaboration) and causal explanation (LEGO) only when the critic has independent evidence.
The hypothesis is falsifiable. At a matched parameter count, a model given the same supervised fine-tuning but no continual pre-training should be measurably worse at deciding which interventions help. If it is not, the domain pre-training ingredient is unnecessary, and we will report that.
Research Program
The fast layer is an existing shared-policy controller, SLight, run in CityFlow through the LibSignal pipeline. It stays unchanged except for a small, typed intervention interface. Through this interface the supervisor can do five things:
- mark a node as failed, which applies influence masking in the style of MalLight;
- cap green time on selected approaches to meter upstream inflow;
- flip the direction of a designated reversible lane;
- replace a sensor reading with an imputed value;
- attach a short expiry to any of these edits.
Typed edits keep the language model's action space small and checkable. This follows the tool-partitioning lesson from Direction 2. The metering action is motivated by Understanding Road Usage Patterns, which finds that segment congestion originates in a few source neighbourhoods. Targeting those sources is therefore more efficient than broadcasting advice to everyone. The lane action draws on tidal-flow and lane-reversal optimization work. That work shows large gains under asymmetric demand, but treats lane choice as a hard combinatorial problem that must be solved offline.
The supervisor model starts from a compact open base model. It is continually pre-trained on an interleaved corpus with two parts. The numeric part consists of traffic tensors encoded by an OpenCity-style encoder: temporal patches plus Laplacian spatial fingerprints, which are graph eigenvector embeddings of road topology. These are projected into the language model as soft tokens. The text part consists of incident logs, sensor-fault annotations, weather conditions and intervention outcomes. We generate it by running CityFlow and SUMO on the LibSignal networks with injected blackouts, sensor dropouts, roadblocks, snow physics and tidal demand, and recording what happened. Real-city observations from the OpenCity corpus supply the numeric side without paired text.
The training objective has two parts. The first is ordinary next-token prediction on the text. The second is a forecasting loss: given the traffic tokens and an intervention, predict future queue and flow patches. That forecasting head is the critic. The proposer is then aligned with supervised fine-tuning on interventions that won in simulator search. Proposer and critic share weights and differ only in role prompt and output head. Sharing weights keeps deployment cheap and makes the ablations clean.
Catastrophic forgetting, the loss of general reasoning under narrow pre-training, is a known risk. We track it with a held-out set of general reasoning prompts. The proposer mainly needs to read incident text and follow the intervention grammar, so moderate forgetting is tolerable. We report it rather than hide it.
Cost budget. The supervisor runs once every 5 minutes per district, not once per intersection per signal cycle. Its token budget per simulated day is therefore orders of magnitude below the roughly 10⁶ tokens per agent-day that GATSim reports for generative agents. We will measure and report the actual budget.
Baselines
Naive reference point. The first reference is SLight running alone, with no supervisor, under every disruption. It shows how much the disruptions hurt and fixes the zero point for "helps." On nominal demand we also report MaxPressure and fixed-time control, because they locate the problem's difficulty. They are known to lose to learned controllers on these networks (CoLLMLight, SLight).
Strongest nominal competitor. SLight is the strongest controller in nominal conditions. SLight reports an average travel time (ATT) of 870.41 s on NY1 and 856.94 s on NY2. In the same paper, GPT-4o and Traffic-R1 controllers do worse. We retrain SLight ourselves in our CityFlow build with its published configuration. The supervisor must not degrade ATT by more than one standard deviation across seeds on nominal NY1 and NY2 demand. This is a do-no-harm test, not a claim of improvement.
Strongest malfunction competitor. MalLight is the strongest specialized method for failed signals. On the 16-intersection Hangzhou blackout benchmark, MalLight reports a 13% throughput drop versus 42.8% for isolated DQN, and 11 simulated accidents versus 36. We run MalLight on the same fault schedules with the same training budget as SLight. Beating it means a smaller throughput drop than MalLight on the same blackout schedules, or a statistically equal drop plus correct handling of faults announced only in text, which MalLight cannot consume.
Strongest language-model competitor. For sensor outages we reproduce an LA-Light-style GPT-4-class prompted supervisor over the same intervention interface. LA-Light reports 6.9% degradation under sensor outage versus 16.1% for RL baselines, and a 20.4% reduction in average waiting time relative to RL. Beating it means matching its degradation while costing at least ten times less in inference.
Strongest sim-to-real competitor. For the snow regime, PromptGAT reports ATT falling from 332.48 s (direct transfer) to 310.29 s in scenario V4.
Lane reversal. For lane reversal on these networks, no reported result was found on Lacuna. We therefore implement the convex-relaxation method of Optimizing lane reversals as an offline oracle schedule under known demand. It serves as the ceiling the supervisor tries to approach online.
Key ablation. The ablation that isolates our ingredient uses the same small base model and the same supervised fine-tuning data, with the continual pre-training stage removed. Traffic state is verbalized as text, as in CoLLMLight. A second ablation keeps pre-training but drops the critic, so the proposer's edits are applied unchecked. All language-model variants get identical fine-tuning data, the same number of supervisor calls, and the same hyperparameter search budget.
Why not just…?
Why not just use MalLight, which already handles failed intersections? For the hardware blackouts it was designed for, MalLight may well be as good as our system, and we expect it to be the hardest baseline there. Its limits are that it learns only from fault types seen in training, and it cannot use information that arrives as text. Examples are a police report of a lane blockage, a planned closure, or a weather advisory. Our evaluation therefore includes text-only fault announcements and unseen combinations of faults. If MalLight matches us on those, the supervisor's value reduces to interpretability, and we will say so.
Why not just prompt a large general model as the supervisor? LA-Light shows this works for sensor outages, and it is a real threat on quality. The case against it is cost and grounding. GATSim documents hallucinated congestion on routes an agent never travelled. CoMAL shows that small general models fail at coordination. The prompted GPT-4-class supervisor is run head-to-head. If it wins on quality, the contribution narrows to a cost-quality trade-off curve, which we report directly.
Why not just fine-tune a small general model on intervention traces, as CoLLMLight does, and skip pre-training? This is exactly our key ablation. Direction 3 evidence (MonoCoder, the PEFT comparative study) suggests domain pre-training predicts downstream quality more than fine-tuning choices do. But that evidence comes from code, not control. If the ablated model matches, we drop the pre-training stage from the recommendation.
Why not just use the simulator itself as the critic, rolling each proposal forward in CityFlow? CityFlow runs at about 72 steps per second at city scale (CityFlow), so short rollouts are affordable. Under perfect calibration this critic is essentially an oracle. It is included as an upper-bound critic. Its weakness is that deployment sites lack a calibrated simulator, and PromptGAT shows how far uncalibrated physics can be from reality. We therefore train in CityFlow and evaluate in SUMO under the heavy-load and snow regimes. There we test whether the learned critic's ranking of interventions degrades less than the rollout critic's ranking, measured against SUMO ground truth.
Proposed Experiments
Networks and regimes. Experiments use NY1, NY2, Hangzhou and Jinan in CityFlow through LibSignal, plus the CityFlow-to-SUMO transfer regimes V1–V4 from PromptGAT. We add a tidal-demand variant of NY1 with designated reversible lanes. Fault schedules cover six conditions:
- hardware blackouts;
- sensor dropouts;
- sensor spoofing, meaning plausible but wrong readings;
- roadblocks announced only in text;
- snow physics;
- tidal surges.
Faults held out from pre-training and fine-tuning form the unseen-fault test.
Metrics. We measure ATT, throughput drop relative to the fault-free run, and simulated accidents in MalLight's protocol. We also measure average waiting time, supervisor tokens and wall-clock time per simulated hour, and critic ranking accuracy: the Spearman correlation between predicted and realized ATT change across candidate interventions.
Decisive checks. Three results decide the proposal:
- On unseen and text-announced faults, the pre-trained supervisor over SLight must reduce throughput drop relative to both SLight alone and MalLight, with non-overlapping confidence intervals across at least five seeds.
- The no-pre-training ablation must show clearly lower critic ranking accuracy and a larger throughput drop at equal parameters.
- Under CityFlow-to-SUMO shift, the learned critic's ranking accuracy must degrade less than the simulator-rollout critic's.
Failing the first check means the supervisor adds nothing beyond existing numeric methods. Failing the second means domain pre-training is not the operative ingredient. Failing the third means a simulator in the loop is the simpler answer. Any of these outcomes is a publishable negative on the open question SLight and CoLLMLight leave in tension: whether language-model agents earn a place in city-scale traffic control at all.
References
- Scalable Traffic Signal Control with Shared Policy Framework
- MalLight: Influence-Aware Coordinated Traffic Signal Control for Traffic Signal Malfunctions
- LLM-Assisted Light: Leveraging Large Language Model Capabilities for Human-Mimetic Traffic Signal Control in Complex Urban Environments
- LLM Powered Sim-to-real Transfer for Traffic Signal Control
- CoLLMLight: Cooperative Large Language Model Agents for Network-Wide Traffic Signal Control
- CoMAL: Collaborative Multi-Agent Large Language Models for Mixed-Autonomy Traffic
- Urban Generative Intelligence (UGI): A Foundational Platform for Agents in Embodied City Environment
- Open Spatio-Temporal Foundation Models for Traffic Prediction
- Self-Collaboration Code Generation via ChatGPT
- LEGO: A Multi-agent Collaborative Framework with Role-playing and Iterative Feedback for Causality Explanation Generation
- Libsignal: an open library for traffic signal control
- Understanding Road Usage Patterns in Urban Areas
- Adaptive Approaches for Tidal-Flow Lanes in Urban-Road Networks
- Optimizing lane reversals in transportation networks to reduce traffic congestion: A global optimization approach
- GATSim: Urban Mobility Simulation with Generative Agents
- MonoCoder: Domain-Specific Code Language Model for HPC Codes and Tasks
- A Comparative Study of PEFT Methods for Python Code Generation
- CityFlow: A Multi-Agent Reinforcement Learning Environment for Large Scale City Traffic Scenario