When a vision-language model gets a picture-based problem wrong, it is usually impossible to say whether it misread the picture or reasoned badly from a correct reading. Reinforcement learning with verifiable rewards trains a model using an automatic checker, such as a unit test or a solver, in place of human or neural judges. Applied to these models, it inherits that ambiguity, because a wrong final answer is blamed on every token. Current fixes estimate the blame with another neural model, which can itself be wrong. We propose a family of tasks in which the blame can be computed exactly. The model looks at a rendered image of a small randomized program, a random walk, or a Markov chain. It writes out what it sees and then produces a termination certificate: a function of program state that a solver can check mechanically, proving that the program halts or bounding how likely it is to halt. Checking that certificate against the model's own transcription and against the true program splits every failure cleanly. A certificate that is valid for what the model wrote but invalid for the truth is a seeing error. A certificate that is invalid for what the model wrote is a thinking error.
This matters for two reasons. First, the setup turns an argued-about question into a measurable one: how much does routing blame between perception and reasoning actually help, and how accurate must the blame signal be before it stops helping? Proxy-based methods cannot answer this, because their proxies' error rates are unknown. Second, certificates are hard to find and cheap to check, which is the property that makes a domain good for verifiable-reward training. A checked certificate cannot be guessed the way a multiple-choice answer can, so the reward resists the post-hoc rationalization seen under answer-only rewards.
Core Claim
Exact, solver-computed separation of perception errors from reasoning errors gives a cleaner training signal than either final-answer reward or neural-proxy credit routing. The size of that advantage should fall predictably as we inject controlled noise into the blame signal. The resulting curve tells practitioners how good a perception verifier such as a text-only proxy or a visual-fact extractor needs to be. A secondary claim is that perception skills learned on these diagrams, such as reading edge labels, coefficients, and guards, transfer to graph and chart reading. This claim is what keeps the domain from being a toy.
Background and Positioning
Three works motivate the proposal by showing that blame assignment is the bottleneck. Bad Seeing or Bad Thinking? introduces MoCA. MoCA splits responses into <recognition> and <think> blocks and asks a frozen text-only model (Qwen2.5-14B) to solve the problem from the recognition text alone. When the proxy succeeds, the perception tokens are protected from negative updates. This removes the "seesaw" in which improving perception damages reasoning and vice versa. Its blame signal is a neural guess with unknown false-negative rate, and that unknown rate is exactly the quantity we make controllable. Grounding the Score finds that 74% of step errors in VisualProcessBench are visual misreadings. It gates process rewards with a learned visual-fact extractor, but it uses that extractor only for test-time reranking and never in the training loop. From Seeing to Thinking shows that training perception first, with verifiable rewards, shortens later reasoning traces by 20.8%. Its separation comes from a schedule rather than from per-sample blame. We reuse MoCA's block structure and its group-relative policy optimization (GRPO, a policy-gradient method that scores each sampled answer against others for the same prompt). We replace its proxy with an exact checker.
On the reinforcement-learning side, Outcome Rewards Do Not Guarantee Verifiable or Causally Important Reasoning constrains the design. Under answer-only rewards, causal importance of reasoning fell on 19 of 40 ReasoningGym tasks. That paper measures this by truncating the reasoning trace and checking whether the answer changes. We adopt its two metrics, causal importance and sufficiency of reasoning, as diagnostics. Our design aims to make a certificate the answer itself, so reasoning cannot be decorative. Work on solver-grounded rewards, such as Solver-Informed RL and Models That Prove Their Own Correctness, supplies the template of a formal engine as the reward. None of it involves images.
Quantitative termination analysis supplies the checkers. A ranking supermartingale is a nonnegative function of program state that decreases in expectation by a fixed amount each step. Finding one proves the program halts with finite expected running time. Stochastic Invariants for Probabilistic Termination pairs such functions with repulsing supermartingales, which certify that the process is pushed away from bad regions. The pair yields lower bounds on halting probability, and checking linear instances reduces to linear programming in seconds. Quantitative Supermartingale Certificates generalizes this to temporal properties using polynomial templates checked by Z3. On Gambler's Ruin, its lower bound reaches 0.380 at degree 2 and 0.621 at degree 5, against a true value of about 0.671. It also notes solver timeouts as the degree grows. Learning Probabilistic Termination Proofs has a neural network propose candidates that Z3 then checks. These works synthesize certificates from given symbolic programs. We use their checkers as rewards for a model that must first recover the program from an image. We do not claim formal guarantees about the vision model itself, which the file gives no basis for.
Research Program
We would build a procedural generator of small randomized systems. The pool includes probabilistic loops with one to four variables and linear or low-degree polynomial updates, seeded from the WTC-style programs in Learning Probabilistic Termination Proofs. It also includes biased and trapped random walks of the kind studied in Stochastic Invariants for Probabilistic Termination, and finite Markov chains with absorbing states. Each system is rendered in several visual forms, with randomized fonts, layouts, and distractors:
- a code screenshot;
- a flowchart;
- a labeled state-transition graph;
- a multi-panel figure in which guards and update distributions sit in separate subplots. This form is deliberate, because CharXiv reports 30–50% drops for open models on figures with six or more subplots.
Some systems halt almost surely. Others do not, and for those the target is a certified lower bound on halting probability.
The model writes a <recognition> block containing a structured transcription of the system. It then writes a <think> block and a certificate: a ranking function, or a stochastic invariant with a repulsing function, plus the claimed bound. The checker evaluates the certificate twice, once against the transcription and once against the ground-truth system, using linear programming for linear certificates and Z3 for polynomial ones. The reward is validity on the true system, multiplied by the tightness of the claimed bound relative to the best bound a reference solver finds. Trivial bounds therefore earn nothing.
Blame routing follows from the two checks. If the certificate is valid for the transcription but not for the truth, only perception tokens receive negative advantage. If it is invalid for the transcription and the transcription matches the truth, perception tokens are shielded and reasoning tokens are penalized. If both the transcription and the certificate are wrong, both blocks are penalized.
The scientific instrument is a noise knob. With probability $\rho$, the router's perception verdict is flipped, which emulates a proxy with false-negative and false-positive rate $\rho$. Sweeping $\rho$ from 0 to 0.5 traces how much routed blame is worth as a function of verifier quality. At $\rho = 0.5$, routing becomes uninformative and the method collapses toward answer-only training. We then estimate where MoCA's real proxy falls on this curve. We do this by measuring the proxy's disagreement with our exact verdict on the same rollouts.
The backbone is Qwen2.5-VL-7B, so that results are comparable with MoCA's VL-Scaler-7B. Training uses an open GRPO implementation such as DIVA-GRPO, with InternVL as a second backbone to check generality. Following Outcome Rewards Do Not Guarantee Verifiable or Causally Important Reasoning, every reinforcement-learning arm starts from the same small supervised warm-up of 64 expert traces. That paper found this amount prevented reasoning collapse.
Baselines
The naive reference is the untrained backbone, prompted to transcribe and then produce a certificate. It establishes how often certificates are valid by chance and how often transcriptions are already correct. No reported result was found on Lacuna for any model on this new task, so every number here will be measured by us.
The first trained baseline is answer-only reinforcement learning. It uses the same data, the same GRPO settings, and the same number of rollouts. The reward is 1 if the final certificate is valid on the true system and 0 otherwise, so it has the same reward as our method but no routing. This is the cleanest test of whether exact blame adds anything beyond an exact outcome. Beating it means higher held-out certificate validity together with a separate gain in perception accuracy, rather than a trade between them.
The strongest existing method is MoCA from Bad Seeing or Bad Thinking?. We run it with its blindfolded Qwen2.5-14B proxy deciding whether the transcription suffices, on identical data, compute, and tuning budget. For scale, its reported results are a V* score of 86.6 for VL-Scaler-7B against 45.0 for GPT-4o, and a drop of nearly 7 points on perception-heavy tasks when its perception verification is removed. Those numbers come from a different benchmark. On ours, MoCA's numbers will be measured. Beating it means our exact router reaches higher held-out validity at matched compute, with the gap matching what the noise curve predicts at MoCA's measured proxy error rate.
The fourth baseline is the staged schedule of From Seeing to Thinking. It trains transcription accuracy first, then certificate synthesis from gold programs, then the joint task, with total rollouts equal to ours.
The fifth is a non-learned pipeline. The backbone transcribes, and a symbolic template solver in the style of Quantitative Supermartingale Certificates searches for the certificate. Its own reported Gambler's Ruin bound of 0.621 at degree 5 serves as the tightness reference on that instance.
The key ablation is our method with $\rho$ swept upward. It isolates the routing ingredient while holding the reward fixed.
Why not just…?
Why not just transcribe the image and hand the program to an existing solver? For certificate quality on easy, low-dimensional instances, this pipeline may well match or beat a 7B model, and we expect it to. The proposal does not depend on beating it there. Our question is whether exact blame improves the model's perception and reasoning. The pipeline cannot answer that, because it does no learning, and its accuracy is capped by the backbone's transcription errors. We will still report it honestly. If the trained model beats it, the likely reason is that the solver times out at higher degrees and more variables, a limitation Quantitative Supermartingale Certificates itself reports. We will stratify by dimension and degree to show where each approach wins.
Why not just use MoCA's text-only proxy, which already works? It is the main threat. If MoCA matches our exact router, the noise curve will show it, and the conclusion would be that proxy-quality blame is already good enough. That would itself be a useful finding, since no current method can measure its proxy's error rate. The file notes that the proxy produces false negatives when it fails to solve a problem from correct premises. Certificate synthesis is hard enough that we expect such failures to be frequent here. The disagreement measurement will settle whether that happens.
Why not just reward the final answer with an exact checker, since the checker is already exact? That is our answer-only baseline. It removes guessing, but it still blames perception tokens for reasoning failures. Bad Seeing or Bad Thinking? attributes the seesaw to exactly this. We will track perception accuracy and reasoning-given-gold-transcription accuracy separately across training. A seesaw in this baseline and none in ours would be the direct evidence.
Why not just fine-tune on gold transcriptions and gold certificates? Wu et al. report that verifiable-reward training beats supervised fine-tuning for perception in From Seeing to Thinking. We include a fine-tuning arm with matched gradient steps, mainly to test generalization to held-out rendering styles and larger systems.
Proposed Experiments
The first decisive check is in-distribution and out-of-distribution certificate validity and tightness. Out-of-distribution means unseen visual styles, more variables, and more subplots. We compare all arms and also report transcription exact-match separately.
The second is the noise curve: validity against $\rho$. The prediction is a monotone decline that reaches the answer-only baseline near $\rho = 0.5$, with MoCA's measured proxy error marking a point on it. A flat curve would refute the core claim.
The third measures causal importance and sufficiency of reasoning, as defined in Outcome Rewards Do Not Guarantee Verifiable or Causally Important Reasoning, on the <think> blocks. We expect them to stay high in every certificate-rewarded arm and to differ in how stable perception is.
The fourth tests transfer. We evaluate on VisionGraph, for which no reported result was found on Lacuna, on the descriptive split of CharXiv, and on MathVista. For CharXiv, GPT-4o's reported reasoning accuracy is 47.1% and InternVL Chat V1.5's is 29.2%. For MathVista, GPT-4V's reported overall accuracy is 49.9%. These are reference points only; the comparison that matters is our arms against the untrained backbone. Transfer is the claim most likely to fail. Graph-like diagrams should carry over to VisionGraph more readily than to natural charts, and a null result on CharXiv would narrow the contribution to a measurement tool rather than a training recipe.
References
- Bad Seeing or Bad Thinking? Rewarding Perception for Multimodal Reasoning
- Grounding the Score: Explicit Visual Premise Verification for Reliable Vision-Language Process Reward Models
- From Seeing to Thinking: Decoupling Perception and Reasoning Improves Post-Training of Vision-Language Models
- Outcome Rewards Do Not Guarantee Verifiable or Causally Important Reasoning
- Solver-Informed RL: Grounding Large Language Models for Authentic Optimization Modeling
- Models That Prove Their Own Correctness
- Stochastic invariants for probabilistic termination
- Quantitative Supermartingale Certificates
- Learning Probabilistic Termination Proofs
- CharXiv: Charting Gaps in Realistic Chart Understanding in Multimodal LLMs
- VisionGraph: Leveraging Large Multimodal Models for Graph Theory Problems in Visual Context
- MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts