Vision-language models, meaning systems that feed image features into a large language model so it can answer questions about pictures, still fail at elementary spatial questions. They confuse left and right from another viewpoint, and they cannot say what lies behind what. The main recent remedy is reinforcement learning with verifiable rewards: the model is rewarded only when a program can check that its answer is correct. So far this remedy has been trained almost entirely on indoor rooms and synthetic block piles, because those are the places where ground truth is easy to compute. Landscape planning offers a different and much larger source of checkable spatial truth. Given an elevation map, standard hydrology algorithms determine exactly where water flows, where it pools, which slopes fall inside a 1–3% drainage band, and what a person standing at a point can see. None of these answers requires human labels, and they can be generated without limit.
We propose to build a procedural terrain environment whose questions are answered by these physical computations, train a vision-language model on it with verifiable rewards, and test two claims. The first is that a physics-checked reward makes spatial reinforcement learning work at the kilometer scale of multi-functional landscapes, where no current model has been evaluated. The second, which is the main scientific bet, is that terrain training transfers back to the standard indoor perspective-taking benchmarks at least as well as indoor training does under the same compute. The reason to expect this is that terrain questions force the model to reason in the frame of a map or of a standing observer rather than in the frame of the camera.
Core Claim
Spatial reinforcement learning so far has two limits. Its reward sources are confined to indoor scenes, as with the scene graphs of SpatialThinker, the synthetic block piles of 3ViewSense, and the ScanNet-derived labels of ViewSpatial-Bench. And the cross-domain transfer literature treats verifiability as available only for math, code and puzzles, as in Reasoning Gym and 1-shot RLVR.
We claim that terrain physics is a third kind of verifier. It is continuous, multi-scale and view-dependent, and it can be computed exactly from a digital elevation model, which is a grid of ground heights. We further claim that a model trained against this verifier gains allocentric spatial skill that carries over to indoor benchmarks. Allocentric here means reasoning from a frame other than the camera's. What would be new is the verifier itself, which is the environment and the reward, together with the transfer test. We do not add another indoor spatial reinforcement learning recipe.
Why the pieces fit
The spatial-deficiency literature tells us where the failure lives. Kamath et al. show that spatial prepositions appear in under 0.2% of web captions, so the problem is partly a data problem. Gao et al. show that Level-2 perspective taking, meaning judging left and right from another observer's viewpoint, stays at or below chance (about 29%) as models scale, because of egocentric bias. 3ViewSense complicates the data story. Its probes find that frozen visual features already hold spatial information (55.8% probe accuracy), and it locates the bottleneck in the missing interface between views. These papers motivate our work, and they point to a specific design: training data rich in perspective changes, plus an explicit top-down scratchpad.
Terrain supplies both almost for free. A map is a canonical orthographic top view. The question "from the bench at A, is the pond on your left?" forces a conversion from the map frame to an observer's frame.
The landscape literature supplies the questions that matter in practice. Li shows that naturalized drainage in low-lying San Francisco depends on fine geometric specifics: channels wider than 3 m with slopes strictly between 1% and 3%. It also shows that the same swales and wetlands yield cooling and air-quality benefits. That is the multi-functional coupling our questions encode. Van Renterghem and colleagues (cited in the file only by author page) report that stormwater depressions attenuate traffic noise by up to 7 dB, which motivates line-of-sight and shielding questions. Pelcner et al. cast land-use allocation as a multi-agent game on a 40×40 grid of UKCEH land-cover parcels, but their agents never see imagery. We use their setting as a perception-grounded planning evaluation, not as a method to compete with.
The method components are reused from prior work. We take GRPO with lexicographic reward gating from SpatialThinker. GRPO is Group Relative Policy Optimization, which scores each sampled answer against the others drawn for the same prompt. Lexicographic gating grants the spatial reward only after format and correctness are satisfied. We also take exponential-decay numeric matching from M2-Reasoning to give partial credit for continuous quantities such as slope or distance.
Research Program
The environment. We generate terrains procedurally from fractal noise, carve drainage with erosion passes, and overlay land cover: buildings, roads, woodland, lawns, water, and permeable versus sealed surfaces. We also import real elevation tiles, including the San Francisco DEM and catch-basin geometry used by Li, and held out entirely for testing.
Each scene is rendered two ways. One is a top-down hillshaded map. The other is one or more oblique ground-level or drone perspective views from known camera poses.
The questions. Each question is answered by a deterministic algorithm, and the families span four kinds of reasoning. For flow routing, a standard steepest-descent flow-direction and accumulation computation answers "which of B or C receives runoff from A" and "where will water pool." For slope-band compliance, the model says whether a proposed swale segment satisfies the 1–3% band. Viewshed computations answer "can the person at A see the road," which serves as a proxy for acoustic and visual shielding. Allocentric direction questions ask "facing the pond from the bench, is the wetland on your left" and are answered by pose geometry.
Two further families are multi-functional. Parcel questions ask which candidate parcel, converted to a rain garden, intercepts the most upstream area while being visible from a footpath. That combines the hydrological and social functions that Jin et al.-style green-view work cares about. Negated and disjunctive assertions, such as "water from A does not reach the road," follow the compound-logic stress cases of 3DSAV.
Difficulty is controlled by terrain roughness, path length, and the size of the viewpoint change.
Training. We start from Qwen2.5-VL at 3B and 7B, the family behind MVSM and SpatialThinker, so comparisons share a base. The model emits a structured trace: first a top-down "map sketch" in JSON listing key cells, heights and features, then its reasoning, then an answer. This transplants the orthographic interface of 3ViewSense to terrain. Rewards cover format, exact answers for discrete questions, numeric matching for continuous ones, and a map-sketch reward that checks the sketched heights and positions against ground truth, all gated lexicographically.
Analysis. Following Zbeeb et al., we compute a "terrain vector": the weight difference between the terrain-trained model and its supervised-only counterpart. We add this vector to an indoor-trained checkpoint and ask whether the gains stack. This probes whether terrain skill and indoor skill live in separable directions of weight space.
Baselines
The do-nothing reference is the untrained base model evaluated with the same prompts, both with and without the map-sketch format. On ViewSpatial-Bench this base sits at 35.85% for Qwen2.5-VL-3B, as derived from ViewSpatial-Bench. On our terrain suite no reported result was found on Lacuna for any vision-language model, so we will establish the base numbers ourselves, alongside GPT-4o-class proprietary models queried zero-shot.
The strongest indoor competitors are three published systems, and each is strongest in a different way. SpatialThinker uses the same RL recipe on indoor data and reports a +12.1% gain over GPT-4o on 3DSRBench, with 76.3% on RoboSpatial-Home and 65.9% on MM-Star. MVSM is the strongest perspective-taking model in the file at 82.09% on ViewSpatial-Bench. 3ViewSense is the orthographic-interface method, reporting 72.9% on ViewSpatial and 54.2% on SPBench-SI. M2-Reasoning-7B serves as the multi-task reference, with 82.3 on CV-Bench and 42.3 on VSI-Bench.
For fairness, the central comparison uses our own reimplementations under matched conditions. We will train one indoor-RL model on STVQA-7K and one terrain-RL model on terrain questions. Both start from the same base, use the same number of GRPO updates and rollouts per prompt, run the same hyperparameter sweep of five learning-rate and KL settings selected on a held-out split of their own domain, and carry the same token budget. Published numbers are cited as context only.
On the terrain suite, beating the indoor models means exceeding the matched indoor-RL model by a margin larger than the seed-to-seed spread, measured over three seeds. On indoor benchmarks, the transfer claim holds only if the terrain-RL model matches or exceeds the matched indoor-RL model on ViewSpatial-Bench and PerspectBench. It does not need to beat MVSM's 82.09%, since MVSM was trained on 43K in-domain samples.
Three ablations isolate the key ingredients. The first removes the map-sketch reward while keeping answer rewards, which tests whether the physics-checked intermediate state matters. The second removes all oblique views so that questions are asked only on top-down maps, which tests whether the viewpoint conversion drives indoor transfer. The third replaces reinforcement learning with supervised fine-tuning on the verifier's answers and traces at equal sample count.
A text-only reasoning baseline trains the same backbone with RLVR on Reasoning Gym's geometry and graph environments, which Reasoning Gym reports improves out-of-domain reasoning. This checks whether any generic reasoning RL would produce the same transfer.
Why not just…?
Why not just hand the model a GIS tool and let it call a flow-routing function? For the pure hydrology questions, a tool-using model will beat ours when it is given the elevation grid. We will report this as an upper bound and not pretend otherwise. The tool, however, needs the elevation grid as input. In our perspective-view and imagery-only conditions the model must infer terrain from pixels, and the tool path requires a separate elevation-reconstruction model. Tools also say nothing about the second claim, transfer to indoor perspective taking, which concerns what the model learns. The tool-use condition with image-only input settles how much of the terrain gap perception accounts for.
Why not just train SpatialThinker-style RL on more indoor data? This is the real threat to the transfer claim. Kamath et al. and ViewSpatial-Bench both suggest that more in-domain spatial data simply helps. Two pieces of evidence argue for terrain. VSR shows that models tie spatial predicates to object identities and fail on held-out concepts. Terrain scenes contain almost no familiar objects, so predicate learning cannot lean on them. And the egocentric lock-in documented by Gao et al. suggests that camera-frame indoor data may reinforce the bias. The matched-compute comparison decides the question. If indoor RL wins on indoor benchmarks, we will report terrain training as a domain extension and not as a source of general spatial skill. A mixed-data condition tests whether the two are complementary.
Why not just fine-tune on the verifier's answers with supervised learning, since the labels are exact anyway? Tahmasbi et al. found that supervised pretraining on single-step transformations was necessary before RL could learn multi-step spatial planning. End-to-end RL collapsed to a −0.216 reward in the static setting, compared with 1.717 for their two-stage pipeline. Supervised learning may therefore be a needed first stage. The open question is whether it is sufficient. The supervised-only ablation at equal samples answers this, and we expect RL's advantage to appear mainly on held-out terrains and on indoor transfer, not on in-distribution accuracy.
Why not just add a math reasoning vector? The gains reported by Zbeeb et al. were confined to text benchmarks: +4.3% on HumanEval and +12.3% on BBH. Nothing in the file suggests that spatial perception improves from math reasoning. The Reasoning Gym geometry baseline and a math-vector-added checkpoint will test this directly on ViewSpatial-Bench and PerspectBench.
Proposed Experiments
The terrain evaluation has three tiers. The first is held-out procedural terrains with unseen roughness and path lengths. The second is real elevation tiles from the San Francisco data of Li, which never appear in training. The third is a perception-grounded planning test on the UKCEH Southwest-UK grid from Pelcner et al.. In that test we render land-cover maps and ask which parcel conversions best serve woodland, water and housing objectives, scoring answers against an exhaustive search. Pelcner's grid-level score of 0.85 is not comparable, because their agent sees indices and not images, and we will not claim to compare against it. As an optional check of real-imagery perception, we will run zero-shot canopy counting and adjacency questions on SelvaBox. No reported vision-language result was found on Lacuna for that dataset.
The indoor transfer evaluation uses ViewSpatial-Bench, PerspectBench, 3DSRBench, CV-Bench, VSI-Bench and 3DSAV-style negated assertions, with MM-Star and MathVista as guards against collateral damage to general ability.
Three results decide the proposal.
First, does terrain RL beat indoor RL on terrain? We expect a large gap, and a small one would mean terrain adds nothing new. We will also run the reverse test: the SpatialThinker-style indoor model evaluated zero-shot on terrain. The file's open problems predict it fails.
Second, at matched compute, does terrain RL equal or beat indoor RL on ViewSpatial-Bench and PerspectBench? PerspectBench is the sharpest test, since no model in Gao et al. exceeded chance. Moving it above 29% would be the headline result. Failing to match indoor RL falsifies the transfer claim.
Third, does the transfer vanish when oblique views are removed? If so, viewpoint conversion is the active ingredient. If transfer survives without them, the map-sketch reward is the likelier cause. The sketch-reward ablation and the terrain-vector addition then show whether the skill is separable from indoor skill.
Across all three, we will report accuracy by question family and difficulty, three seeds per condition, and rollouts consumed. The compute accounting matters because the rollout demands of 1-shot RLVR make cost an honest part of any claim about reinforcement learning transfer.
References
- SpatialThinker: Reinforcing 3D Reasoning in Multimodal LLMs via Spatial Rewards
- 3ViewSense: Spatial and Mental Perspective Reasoning from Orthographic Views in Vision-Language Models
- ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models
- Reasoning Gym: Reasoning Environments for Reinforcement Learning with Verifiable Rewards
- Reinforcement Learning for Reasoning in Large Language Models with One Training Example
- What's "up" with vision-language models? Investigating their struggle with spatial reasoning
- Vision Language Models See What You Want but not What You See
- Urban spatial sustainability landscape planning and design: A study on solving flood disasters in low-lying urban areas based on simulated natural drainage system
- Multi Objective Quantile Based Reinforcement Learning for Modern Urban Planning
- M2-Reasoning: Empowering MLLMs with Unified General and Spatial Reasoning
- Collaborative Enhancements of Community Walking Environments for Low-Carbon Development and Age-Friendly Objectives: A Systematic Review
- 3D Scene Assertion Verification
- Reasoning Vectors: Transferring Chain-of-Thought Capabilities via Task Arithmetic
- Visual Spatial Reasoning
- From Building Blocks to Planning: Multi-Step Spatial Reasoning in LLMs with Reinforcement Learning