Vision-language models (models that answer questions about images in text) do surprisingly badly on questions that need several spatial steps over real maps. An example is "Which of the districts shaded as high flood risk on map 2 lie within the protected zone on map 4, and how far is the nearest one from the capital?" On FRIEDA, a benchmark of such questions built from government reports in 32 countries, the best models answer about 38% correctly, while humans answer about 85%. Two diagnoses fit that gap. The first says the models cannot compute: containment, intersection, and distance are hard for a text generator. The second says the models cannot read: they misread the legend, the scale bar, or which colored region is which, and every later step inherits that error. This note proposes to separate the two. A vision-language model would read each map into an explicit geometric description: legend entries, labeled regions as pixel polygons, scale, and north direction. A deterministic geometry engine would then execute every topological, metric, and directional step. With that split we can measure, hop by hop, where the error and its regional disparity actually come from.
The point matters beyond maps. Multi-hop systems fail when an early step goes wrong and later steps build on the error. Geographic bias in multimodal models is well documented but rarely located inside a reasoning chain. Our central bet is that the cross-regional gap in multi-step spatial reasoning sits almost entirely in the first hop, reading unfamiliar cartographic conventions, and not in the reasoning that follows. If that is true, the fix is to train map reading, with rewards we can check automatically, on synthetic maps rendered in many regional styles. Larger reasoning models and more retrieval would not close the gap.
Core Claim
The claim has two parts, one diagnostic and one constructive. The diagnostic part is testable on existing data. Suppose a FRIEDA question is answered by an engine fed a hand-checked oracle parse of the maps. Accuracy should then rise close to human level, and the spread of accuracy across country groups should largely disappear. Suppose instead the engine is fed the model's own parse. The regional spread should then return, and it should track parse errors on legends and scales.
The constructive part follows if the diagnostic part holds. A 7B model trained only to produce correct map parses, with no training on final answers, should beat frontier models that reason end to end. It should also hold up better on map styles from regions it never saw in training.
How This Differs From Existing Work
Three papers motivate the proposal. FRIEDA benchmarks multi-step cartographic reasoning and reports that legend misinterpretation is the largest single error source (25.6%). It also finds that distance and containment questions sit at 20–35% accuracy. However, it proposes no method, it does not separate reading from reasoning, and it is limited to Latin-script documents. MapQA builds question–answer pairs from OpenStreetMap and shows that text-to-SQL models fail on intersection and distance questions, mostly through malformed SQL. Its inputs are already structured geometry, so it never faces the reading problem at all. It also covers only two US states. Charting New Territories documents sharp drops in performance on African and South American inputs. Examples include a 61% fall in flag accuracy and map-localization errors over 1000 km. That work is diagnostic only.
Two papers supply the method. Disentangling Extraction and Reasoning in Multi-hop Spatial Reasoning shows that splitting spatial-role extraction from symbolic reasoning improves out-of-distribution transfer on text benchmarks. It also delivers a warning that constrains our design: its Prolog pipeline is near perfect on synthetic SPARTQA (96–97%) but weaker on the more realistic RESQ benchmark, where language is ambiguous. We carry its split from text into pixels. We address its warning by keeping the extractor neural and trained on visually diverse renders, and by making only the geometry symbolic. SpatialThinker trains a 7B model with Group Relative Policy Optimization (GRPO, a reinforcement-learning method that scores several sampled outputs against each other). Its rewards are layered so that format must be correct before any accuracy reward counts, and it scores box overlap with CIoU (complete intersection-over-union). We reuse that reward recipe, but we apply it to map parses and not to 3D boxes.
The closest agentic work treats a hop differently. MMhops-R1 treats each hop as a semantic lookup, going from image to landmark to fact. Geo-R treats each hop as an administrative step, going from country to state to city. In both, every hop stays inside the model. In our design, the model performs only the reading hop, and every geometric hop runs outside it.
Research Program
The system has three components.
The first is a structured map parse that each map is converted into. It contains legend entries, each a symbol or color paired with a text label. It contains named or symbol-coded regions as polygons in pixel coordinates, point features, the scale-bar ratio, the north angle, and links between maps that cover the same area. A small question planner turns the question into a program over these parses, written in a fixed vocabulary of operations: select by legend class, within, intersect, border, distance, direction. The planner is the same vision-language model, prompted with the operation vocabulary. Because the program runs deterministically, a hop can never introduce a hallucinated geometric fact.
The second component is training data with free ground truth. We render maps from OpenStreetMap vector data, the same source MapQA uses, and randomize the cartographic style. The randomized properties include color schemes, hatching and symbol conventions, legend layout, scale-bar styles, the presence and rotation of north arrows, and label scripts (Latin, Arabic, Devanagari, CJK, Cyrillic). Rendering is held out by world region: training uses OpenStreetMap areas and style families from some regions, and testing uses others. Every render comes with an exact parse, so extraction rewards can be verified automatically.
The third component is extraction-only reinforcement learning. Starting from Qwen2.5-VL-7B, we train with GRPO using layered rewards. The parse must be valid before anything else is scored. Legend-label matching accuracy is scored next, then polygon overlap and scale and north-angle error. The final answer is never rewarded. This choice is the key ingredient. It forces any gain to come from better reading, and it keeps the reasoning side identical across all compared systems.
For analysis, we annotate a stratified subset of FRIEDA with oracle parses. The subset is about 150 questions balanced over FRIEDA's countries and its three relation types. Running the engine on oracle parses, on model parses, and on mixtures in which only one element is corrupted yields a hop-level attribution of error. The corrupted element can be the legend, the scale, or the polygons. We report this attribution separately for each country group. That turns "geographic bias" into a statement about which kind of reading error drives it.
Baselines
The naive reference point is the base Qwen2.5-VL-7B answering FRIEDA directly with chain-of-thought prompting. It shows what the backbone does with no reading training and no engine.
The strongest existing systems are frontier models evaluated end to end on FRIEDA. The FRIEDA paper reports 38.20% accuracy for Gemini-2.5-Pro and 37.20% for GPT-5-Think. For an open model of similar scale it reports 25.80% for Ovis2.5-9B-Think, against a human baseline of 84.87%. These are the numbers to beat. FRIEDA reports no per-country or per-region accuracies, so no reported result was found on Lacuna for regional breakdowns. We will recompute all baselines on our region groups using FRIEDA's own protocol and answer-matching rules, in both its Direct and Contextual settings.
The tool-use competitor is the same 7B backbone given a free-form Python interpreter with a geometry library. It is trained with GRPO on final-answer reward, with the same number of updates, samples per prompt, and rendered data as our system. It answers the question of whether explicit parse structure and extraction-only reward matter, or whether giving a model a calculator is enough.
An end-to-end reinforcement-learning baseline uses SpatialThinker-style rewards on answers alone, with no parse and no engine, and the same compute.
Two ablations isolate the proposal's ingredients. The first is our pipeline with an untrained extractor, which isolates the contribution of reading training. The second is our pipeline trained only on Latin-script, single-style renders, which isolates the contribution of style diversity.
We count our system as beating these baselines only if two conditions hold. First, it exceeds 38.20% overall on FRIEDA. Second, its gap between the best and worst region groups is smaller than the gaps of the frontier baselines and the tool-use baseline, with bootstrap confidence intervals that do not overlap.
Why not just…?
Why not just give the model a code interpreter and let it compute? This is the real threat. It differs from our system in only two respects: no prescribed parse format and no extraction-specific reward. MapQA's malformed-SQL failures suggest that free-form code generation over geometry is brittle. However, MapQA starts from structured data, so it says nothing about reading maps. The tool-use baseline above settles the question directly. If it matches our system both overall and on the regional gap, then the parse interface adds nothing, and the honest conclusion is that tools, not structure, are what matter.
Why not just put all the maps into a long-context model? FRIEDA already tested a close variant. Its Contextual setting supplies the surrounding maps from the report, and top models stay near 37–38%. Finding the right map is therefore not the bottleneck. We still run both settings, but we do not expect context length to explain the gap.
Why not just reinforce on final answers end to end, as SpatialThinker and Geo-R do? Answer-level reward cannot tell a misread legend apart from a wrong containment step, so it gives the model no direct pressure to learn unfamiliar conventions. The end-to-end baseline tests whether this matters in practice. If that baseline closes the regional gap as well as extraction-only training does, our attribution story weakens and we will report it that way.
Why not just wait for scale? Vision Language Models See What You Want but not What You See reports that larger models improve on social inference but barely improve on spatial perspective-taking. On ViewSpatial-Bench, GPT-4o scores 34.98% while a targeted 3B model reaches 82.09%. Scale has not delivered on the spatial side.
Decisive Checks
The first check decides the diagnostic claim. If the engine on oracle parses does not substantially exceed 38% on FRIEDA, then computation remains hard even with perfect reading, and the premise is wrong. The second check decides whether bias lives in reading. If oracle parses do not shrink the variance of accuracy across country groups, then regional bias is not a reading problem. The third check is the transfer test. Train on renders from held-in regions and Latin scripts, then test on held-out regional styles and on a small non-Latin set we construct with FRIEDA's question templates. Parse accuracy and answer accuracy should degrade less than in the Latin-only ablation. A secondary check uses MapQA's Southern California-to-Illinois split. There, inputs are already structured, so the engine alone should succeed on the intersection and distance types, confirming that computation is not where the remaining errors lie.
References
- FRIEDA: Benchmarking Multi-Step Cartographic Reasoning in Vision-Language Models
- MapQA: Open-domain Geospatial Question Answering on Map Data
- Charting New Territories: Exploring the Geographic and Geospatial Capabilities of Multimodal LLMs
- Disentangling Extraction and Reasoning in Multi-hop Spatial Reasoning
- SpatialThinker: Reinforcing 3D Reasoning in Multimodal LLMs via Spatial Rewards
- MMhops-R1: Multimodal Multi-hop Reasoning
- Vision-Language Reasoning for Geolocalization: A Reinforcement Learning Approach
- Vision Language Models See What You Want but not What You See
- ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models