Round-Trip Rendering as a Perception Reward for Compact Geometry Reasoners

Vision-language models can describe a photograph in fine detail yet fail simple geometry problems. Reinforcement learning on final answers helps a small model reason, but it quietly damages its ability to see: it starts misreading which lines are parallel or where points lie. We propose a cheap, fully automatic check on what the model saw. Before solving, the model writes a short drawing program that reconstructs the diagram. We run that program, render the picture, and score how closely it matches the input. That match score becomes a second reward, trained jointly with the answer reward. No human annotator and no proprietary judge model is involved. The reward rises only when the model's description of the diagram is faithful, and it can be computed on any unlabeled diagram.

The idea borrows from a lesson in code modeling. Code models become more robust when trained to translate back and forth between source code and a compiler's intermediate representation, a lower-level form that states what the program actually does. Here the diagram plays the role of source and the drawing program plays the role of the intermediate representation. The renderer, unlike a compiler, accepts any well-formed program, so we can close the loop in both directions. The setting is domain specialization. We target a 7B model on plane geometry, and on charts as a secondary domain, where compact specialized models already beat frontier generalists.

Core Claim

A pixel-and-structure consistency reward from re-rendering the model's own diagram transcription will prevent the perception loss that answer-only reinforcement learning causes in compact vision-language models. It will do so in a single training stage, without the separate perception stage and GPT-4o-mini judge that current methods need. It will also match or improve their reasoning accuracy. If this holds, perception can be preserved during specialization by a verifiable, self-supervised signal rather than by curated perception labels.

Where This Sits in Prior Work

Three findings motivate the proposal. MATH-Vision attributes 31.9% of GPT-4V errors to visual recognition and 42.2% to reasoning. It also finds that natural-language chain-of-thought sometimes hurts, because errors pile up between seeing and reasoning. GeoPQA shows the key tension: reasoning-only reinforcement learning degrades visual grounding by up to 15.1%. Its reasoning-only model scores 53.1% on the GeoPQA perception test, against 83.2% for its two-stage model. Specializing Smaller Language Models towards Multi-Step Reasoning documents the text-only version of the same problem: specialization erodes abilities that are not rewarded.

Three works supply methods we reuse and differ from. Learning Where Outcomes Change already has an 8B model transcribe diagrams into line-numbered Matplotlib code before reasoning, which it calls Code-CoT. It then assigns credit to specific intermediate steps with a method called CE-GRPO. That paper supplies our representation, but it treats the transcription as faithful and never executes it against the input image. We make the transcription itself the object of a verifiable reward. GeoX aligns diagrams to a formal geometry language. Its alignment stage rests on only 6,232 expert-annotated caption pairs, and it is trained by supervised imitation alone. Our reward needs no paired captions. IRCoder supplies the conceptual template: bidirectional translation between source and an executable intermediate form improves robustness in 1B–7B models. IRCoder also notes that compiler representations are verbose and require compilable inputs. A geometry construction program is short, and a renderer runs on anything syntactically valid, so both limits are relaxed here.

One work constrains the design. Synthesize Step-by-Step routes charts through a table extractor, and ChartQA shows that such pipelines fail on questions about color or position. Because of this, our transcription must encode visual attributes, not just data values. The model also keeps the raw image in context, which the Code-CoT ablations found necessary.

Research Program

The policy is Qwen2.5-VL-7B, chosen so we can compare directly with GeoPQA. A secondary replication uses Qwen3-VL-8B for comparison with CE-GRPO. Each rollout emits two parts. The first is a construction program in a restricted Matplotlib-like language: points with coordinates, segments, circles, arcs, text labels, and colors. The second is a reasoning trace and a final answer. The reward is

$$r = r_{\text{ans}} + \lambda \, s\big(\mathcal{R}(p), x\big),$$

where $r_{\text{ans}}$ is 1 for a correct final answer and 0 otherwise, $p$ is the emitted program, $\mathcal{R}$ is a fixed deterministic renderer, $x$ is the input diagram, $s$ is a similarity in $[0,1]$, and $\lambda$ sets the weight of the perception term.

The similarity has to be robust to style, because textbook diagrams differ in line width and font from our renderer. We therefore compute $s$ from three parts. The first is a symmetric chamfer distance between edge maps after normalizing for scale and translation. The second is an OCR-based check that each label in $x$ lies near the corresponding named point in $\mathcal{R}(p)$. The third is a color-histogram term for charts. Programs that fail to render receive $s=0$.

The central risk is reward hacking. The model could learn to paint pixels, for example with dense tiny segments, without committing to geometric objects. We address this in two ways. The program language caps the number of primitives. We also report the fidelity of the recovered geometry, not just pixel match, on synthetic diagrams whose ground-truth programs are known.

Training data consists of images and answers from the Geometry3K, PGPS9K, and GeoQA training splits, plus the chart training data used by Synthesize Step-by-Step. Most importantly, $s$ can also be computed on diagrams with no answer at all. This lets us study whether extra unlabeled diagrams help.

The analysis centers on an erosion curve. At fixed checkpoints throughout training, we measure GeoPQA perception accuracy and final-answer accuracy for each method. We also run a validity study of the reward itself. It asks whether $s$ on held-out diagrams correlates with GeoPQA perception correctness and with exact program recovery on synthetic diagrams. If that correlation is weak, the project's premise fails early and cheaply.

Baselines

The naive reference point is the untuned Qwen2.5-VL-7B-Instruct with and without Code-CoT prompting. It tells us how much of any gain comes from the output format rather than from training.

The second reference is vanilla answer-only GRPO (Group Relative Policy Optimization, a reinforcement learning method that compares groups of sampled answers) using the same Code-CoT format. This is the condition GeoPQA shows erodes perception. We expect it to reproduce a perception drop of the reported size, and a failure to reproduce that drop would itself be informative.

The strongest competitor on the perception-preservation question is GeoPQA's two-stage pipeline. It first applies GRPO on perception questions judged by GPT-4o-mini, then applies reasoning GRPO. It reports 79.8% on MathVista geometry problem solving, 76.2% on MathVista geometric reasoning, and 83.2% on the GeoPQA perception test with Qwen2.5-VL-7B (GeoPQA). This baseline is the one that directly attacks the same failure. We rerun it on the same backbone with the same total number of policy updates, rollouts per prompt, and hyperparameter search budget as our method. Its judge calls are logged as an extra cost.

The strongest competitor on reasoning is CE-GRPO on Qwen3-VL-8B. It reports 76.04% average accuracy across nine geometry benchmarks, 3.43 points above trajectory-level GRPO (Learning Where Outcomes Change). Our render reward is independent of how credit is assigned, so we also run CE-GRPO plus the render reward. This tests whether the two methods combine.

For absolute context on GeoQA, GeoX reports 54.9% (GeoX). For the ChartQA human-written split, MatCha trained on the LAMENDA data reports 53.9% (Synthesize Step-by-Step).

Two ablations isolate the key ingredient. The first keeps the program output but drops the render term ($\lambda=0$). The second replaces $s$ with a KL penalty (a measure of divergence from the base model) on the transcription tokens only. The render reward counts as a win if it satisfies two conditions at equal compute. It must lose less than 3 points of perception accuracy relative to the base model, while vanilla GRPO loses roughly 15. It must also match GeoPQA's two-stage accuracy on MathVista geometry problem solving within noise, without any judge calls. Beating GeoPQA outright is a bonus, not the bar.

Why not just…?

Why not just use GeoPQA's perception-first curriculum? This is the most serious threat, because it already recovers perception to 83.2%. Its limits are practical but real. It needs curated perception questions and a proprietary judge, and the file reports roughly 8% annotation noise in its dataset. It also protects perception only through ordering. Nothing in stage two stops perception from drifting again, and the erosion curve will show whether it does. If our erosion curves and final accuracies merely tie GeoPQA's, the contribution narrows to "the same result without labels or an API judge." We would report it as that.

Why not just add a KL penalty to the base model, or mix perception data into reinforcement learning? A KL penalty keeps the policy close to the base model everywhere, which also caps reasoning gains. It does not reward perception becoming better than the base, only staying the same. The KL-on-transcription ablation directly tests whether anchoring to the base model is enough. Mixing in perception data brings back the labeling cost we are trying to remove.

Why not just rely on the answer reward, since correct answers require correct perception? GeoPQA's 53.1% perception score after reasoning-only training is direct evidence that answer reward does not enforce faithful seeing. Models find shortcuts. The $\lambda=0$ ablation reruns this in our exact format.

Why not just train by supervised fine-tuning on synthetic program–diagram pairs? Pairs are free to generate by rendering random programs, and this may well work on synthetic-looking figures. It likely transfers poorly to real textbook diagrams, which is exactly where the render reward still applies without labels. We include a supervised baseline on synthetic pairs, with the same number of tokens seen as the reinforcement learning run, and report it separately on real versus synthetic test images.

Proposed Experiments and Decisive Checks

The main evaluation covers MathVista geometry problem solving and geometric reasoning, the GeoPQA perception test, GeoQA, the nine-benchmark suite from the CE-GRPO paper (including VisOnlyQA as a pure perception check), and the geometry subsets of MATH-Vision. The chart track reports ChartQA human-written and machine-generated splits separately, since the human split contains the color and position questions that table pipelines miss. All numbers are means over three seeds with bootstrap intervals.

Three results would decide the project. First, the render score must correlate with held-out perception correctness. Without that correlation the reward has no basis. Second, the erosion curve must stay flat under the render reward while vanilla GRPO's curve falls, at matched reasoning accuracy. Third, adding unlabeled diagrams, scored only by $s$, must raise perception and answer accuracy. That last result would show that the benefit comes from verifiable self-supervision rather than from the answer labels.

A final check guards against a trivial explanation. On synthetic diagrams, we measure exact recovery of point incidences and parallel and perpendicular relations from the emitted programs. This confirms that a high render score reflects correct geometry rather than pixel mimicry.