Self-driving cars need a map of which lanes connect to which, and recent systems build it on the fly. A camera-based encoder looks at the scene, and a decoder writes out the lane graph one token at a time, much as a language model writes a sentence. The authors of one such system, LaneGraph2Seq, name large-scale pre-training on map data as the obvious next step. The logic mirrors specialized language models: a decoder that has already read millions of real road graphs should know what roads look like before it sees a single camera frame. This proposal takes that step, but it treats the step as a hallucination problem rather than a pure accuracy gain. A decoder with a strong road prior will do two things. It will stop producing impossible graphs, and it will start producing plausible roads the cameras never saw. That second behaviour is the lane-graph version of a language model confidently inventing a citation, and it is the more dangerous of the two when a truck blocks the view of an intersection.
The claim is testable in two sentences. Pre-training a lane-graph decoder on serialized real-world road networks improves topology recovery exactly where visual evidence is weak, and it does so by filling gaps from the prior. We can detect the edges the prior invented by asking the same model about the same edge in different ways and by checking whether the edge survives when its visual evidence is removed. If both halves hold, map pre-training becomes deployable with calibrated abstention. If the invented edges are undetectable, that is a safety finding worth publishing.
Core Claim
Current sequence-based lane-graph models all train their decoder from scratch on perception datasets. LaneGraph2Seq serializes a directed acyclic lane graph by depth-first traversal into vertex and Bézier-curve tokens and reaches 73.3% centerline F1 and 67.6% connectivity F1 on nuScenes. Topo2Seq uses such a decoder only during training, as a regularizer for a query-based detector, and discards it at inference. SeqGrowGraph drops the acyclic assumption by growing the adjacency matrix one node at a time. None of the three pre-trains the decoder, and none measures structural hallucination. On the language side, Agrawal et al. show that fabricated references are exposed by inconsistency under indirect questioning, even when direct questions return confident answers. Co-opNet shows that the most faithful candidate often has lower likelihood than a fluent fabrication. Neither idea has been carried into graph-structured spatial output.
The proposal separates two kinds of wrong edge, and the distinction drives everything that follows. An invalid edge breaks road grammar: a lane that connects across a barrier, or a direction reversal mid-segment. A fabricated edge is grammatically legal and statistically typical but has no support in the scene. We expect pre-training to cut invalid edges sharply, raise recall under occlusion, and raise the fabrication rate in occluded regions. The contribution is to measure this trade-off and to detect fabrications with two signals. Order consistency asks whether the edge appears when the graph is decoded from different depth-first roots, which is the graph analogue of indirect queries. Evidence sensitivity asks how much the edge's probability drops when the camera features under its region are masked.
Research Program
We first build a road-graph corpus. Lane-level HD map graphs from Argoverse 2 and nuScenes map tiles are serialized with the growth-based grammar of SeqGrowGraph, so loops and bidirectional lanes are representable. Following that paper's finding that arbitrary segmentation injects noise, centerlines are re-segmented at forks and merges. Coordinates are quantized in a local ego-centred frame. Two things are held out of pre-training entirely: every map tile overlapping OpenLane-V2 and nuScenes validation scenes, and, in a stricter arm, whole evaluation cities. A second, larger, coarser corpus comes from OpenStreetMap road-level graphs converted into the same grammar. It tests whether road-level topology transfers to lane-level decoding.
A decoder-only transformer is pre-trained with next-token prediction on this corpus. It is then attached by cross-attention to a standard bird's-eye-view encoder, meaning an encoder that lifts multi-camera images into a top-down feature map. The whole system is fine-tuned on OpenLane-V2 subset_A and nuScenes. Grammar validity is enforced for every system by a cheap rule filter at decoding time. This deliberately neutralizes the easy win of pre-training, so that any remaining gain has to come from real road statistics.
Hallucination analysis runs on every predicted edge. Each edge is labelled correct, invalid, fabricated (legal but absent from ground truth), or missed. Labels are stratified by a visibility score computed from lidar-projected occlusion of the lane region. The two detection signals are combined into one score, a flagged edge is emitted as "uncertain" rather than dropped, and we measure how well the score separates fabricated from correct edges.
Baselines
The naive reference point is the same encoder and decoder trained from scratch with the same grammar, filter, and fine-tuning budget. This is LaneGraph2Seq's recipe on the growth grammar. Every claimed gain from pre-training is measured against it.
The strongest existing method on OpenLane-V2 is TopoMLP. It pairs strong Bézier lane and traffic-element detectors with small link-prediction heads and reports 41.2% OLS on subset_A with a ResNet-50 backbone, against 35.6% for TopoNet. It is the strongest competitor because its ablation argues that topology reasoning has almost no headroom once detection is fixed, which directly challenges the premise that a better graph prior helps. We will run it from its published configuration on the same backbone and training schedule. We will also report the lane-to-lane topology metric separately, because an OLS gain driven by traffic-element detection says nothing about our question. Beating it means a higher lane-to-lane topology score at matched backbone, and a higher score in the low-visibility stratum in particular.
The strongest sequence method is Topo2Seq, which reports 37.7% lane segment mAP and 29.9% topology score on subset_A at 14.7 FPS on an A100. We will add our pre-trained decoder to its training-only supervisor slot. This tests whether pre-training also helps when the decoder is thrown away at inference, which would remove latency concerns. On nuScenes the reference is LaneGraph2Seq's 73.3% centerline F1 and 67.6% connectivity F1. The file gives no numerical Argoverse 2 or nuScenes result for SeqGrowGraph; no reported result was found on Lacuna for it, so we will reproduce it.
Three ablations isolate the key ingredient. The first gives equal compute to pre-training on the task's own training graphs only, which separates "more steps" from "more roads." The second pre-trains on randomly generated grammar-valid graphs, which separates knowing the syntax from knowing real road statistics. The third pre-trains on OpenStreetMap only, which tests coarse-to-fine transfer. For the hallucination detector, the baselines are raw token probability of the edge and multi-sample self-consistency under temperature sampling. A useful detector must beat both on area under the ROC curve for flagging fabricated edges.
Why not just…?
Why not just use a better detector with simple heads, as TopoMLP does? This is the most serious threat. The TopoMLP ablation suggests that ground-truth topology on top of predicted detections barely moves the score. We will repeat that ablation on our split, stratified by visibility. If the headroom is also negligible in occluded regions, the accuracy half of our claim fails and only the hallucination analysis remains. Topo2Seq's mAP gain from sequence supervision, 36.4% to 37.7%, is some evidence that sequence priors improve geometry itself, and the Topo2Seq-slot experiment tests whether pre-training amplifies that effect.
Why not just filter invalid edges with hand-written rules, as Chameleon's fast system does? Chameleon uses model-written geometric rules to prune non-connected pairs. Rules plausibly remove invalid edges as well as pre-training does. That is why every arm in our design already carries the filter. Rules cannot say which legal edge exists behind an occluding bus, and that residual is the only place we claim a benefit.
Why not just prompt a general vision-language model or call map tools? TopoAware-Bench finds GPT-4o at 67.8% on binary vector alignment and InternVL3-8B at 2.9% recall on lane connection. The tool-orchestration system of Zhang et al. avoids hallucinated coordinates only by never letting the model produce them, and its image tools rely on corner detection that fails in dense scenes. Neither approach can run onboard at camera frame rate, and neither yields per-edge uncertainty.
Why not just fine-tune an existing driving model longer? STSBench reports driving-tuned models below a general one (DriveMM 39.51% and OmniDrive 29.33%, against 46.07% for InternVL 2.5 8B). That result warns against narrow tuning but does not show pre-training works. Our equal-compute in-domain arm is the direct test.
Proposed Experiments
The main experiment compares from-scratch, in-domain-pre-trained, random-grammar-pre-trained, OpenStreetMap-pre-trained, and HD-map-pre-trained decoders on OpenLane-V2 subset_A and nuScenes. We report lane-to-lane topology score, connectivity F1, and the four-way edge breakdown by visibility decile. Our prediction is specific and falsifiable: pre-training lowers invalid and missed edges most in the lowest-visibility deciles, and it raises fabricated edges there. If fabrication does not rise, the prior is benign and the detector matters less. If recall does not rise, the TopoMLP view wins.
The second experiment evaluates the fabrication detector. Order consistency uses five depth-first roots, and evidence sensitivity uses region-masked bird's-eye-view features. We report the area under the ROC curve for separating fabricated from correct edges, compared against token probability and sampling consistency. We also report selective topology score as the uncertainty threshold varies.
The third experiment is the geographic holdout. We repeat the main comparison with evaluation cities removed from pre-training, to rule out the model having memorized the test roads. If gains vanish under the city holdout, the "prior" was memorization and the result is reported as such.
A final, cheaper probe converts TopoAware-Bench connection questions into edge queries on the model's bird's-eye-view coordinates. This asks whether a small pre-trained graph decoder exceeds the frontier models' recall on the one subtask where they collapse.
References
- LaneGraph2Seq: Lane Topology Extraction with Language Model via Vertex-Edge Encoding and Connectivity Enhancement
- Topo2Seq: Enhanced Topology Reasoning via Topology Sequence Learning
- SeqGrowGraph: Learning Lane Topology as a Chain of Graph Expansions
- Do Language Models Know When They're Hallucinating References?
- Cooperative Generator-Discriminator Networks for Abstractive Summarization with Narrative Flow
- TopoMLP: A Simple yet Strong Pipeline for Driving Topology Reasoning
- Chameleon: Fast-Slow Neuro-Symbolic Lane Topology Extraction
- Are VLMs Ready for Lane Topology Awareness in Autonomous Driving?
- Multimodal Road Network Generation Based on Large Language Model
- STSBench: A Spatio-temporal Scenario Benchmark for Multi-modal Large Language Models in Autonomous Driving