Vision-language models solve a clean, digitally rendered geometry problem far more often than the same problem photographed by a student on a phone. MathReal built a benchmark of 2,000 handheld K-12 math photos and found sharp drops on identical problems; one model lost 13.3 points going from the digital to the photographed version. The usual fixes do not reach the problem. Training on artificially corrupted images teaches a model the signature of the fake noise rather than invariance to real noise, which is the synthetic-to-real gap well documented in image restoration. Asking the model to describe the corruption in words before answering adds reasoning text, and that text can pull attention away from the fine diagram details that were already hard to see.
We propose a quieter fix. During training, the model sees two versions of each problem, a clean render and a corrupted copy. We then require its step-by-step reasoning distribution on the corrupted copy to match, token by token, its own reasoning on the clean copy. The clean copy acts as a teacher, and its gradient is blocked. The model writes no extra words about the corruption; it only learns to reason as if the picture were clean. We argue this matters most in a native or single-stream multimodal transformer, meaning a model whose image patches and text tokens pass through the same layers from the first layer on, with no separate frozen image encoder. In that design, a reasoning-level training signal can flow back into the layers that turn raw pixels into features. We test that claim directly by freezing those early layers. A second question runs through the whole study: does matching reasoning on synthetic corruptions carry over to real phone photos better than matching answers does?
Core Claim
The main claim is this. Matching the reasoning distribution between clean and corrupted inputs, without any verbal diagnosis, closes more of the real-photo gap on math reasoning than three alternatives: corruption-augmented fine-tuning on the same data, Robust-R1-style verbal diagnosis, or a restore-then-reason pipeline. The benefit should be larger when the alignment gradient can reach the pixel-processing layers of a native transformer.
Concretely, let $x^{c}$ be a clean render of a problem and $x^{d}$ a corrupted version of it. Let $y^{c}$ be the reasoning trace and answer the model produces on $x^{c}$ (filtered for correctness), and let $p_\theta(\cdot \mid x, y_{<t})$ be the model's next-token distribution. We train
$$ \mathcal{L}(\theta) = -\log p_\theta(y^{c} \mid x^{d}) + \lambda \sum_{t} \mathrm{KL}\big(\,\mathrm{sg}[p_\theta(\cdot \mid x^{c}, y^{c}_{<t})] \;\big\|\; p_\theta(\cdot \mid x^{d}, y^{c}_{<t})\big), $$
where $\mathrm{sg}$ blocks gradients through the clean branch and $\lambda$ weights the per-step matching term. The first term alone is ordinary augmented fine-tuning. The second term is the new ingredient: it transfers the full shape of the clean-input distribution at every reasoning step, not just the single sampled token. That gives a denser signal about which steps the corruption disturbs.
Where This Sits in Prior Work
Four papers motivate the problem. MathReal measures the collapse on real photos and attributes 40% of errors to reasoning and 25–33% to figure perception. It proposes no fix, and we use it as our main test set. MATH-Vision shows that on clean competition problems reasoning errors (42.2%) outweigh recognition errors (31.9%). MathReal suggests perception errors re-emerge under real capture. Our method has to handle both, which is why we align reasoning rather than pixels.
Awakening Visual Reasoning supplies the method. Its CoRe approach aligns a model's reasoning on text rendered as an image to its reasoning on the original text, using step-wise KL with a stop-gradient on the text branch. With reinforcement learning on top, this raised MATH-500 accuracy on rendered text from 15.42% to 42.97%. CoRe anchors one modality to another, and only on pristine renders, in a modular Qwen3-VL model. We instead anchor a clean image to a degraded image of the same problem, study real camera shifts, and ask how the alignment interacts with native architecture.
Robust-R1 is the closest competitor. It trains a modular Qwen2.5-VL-3B to write out the corruption type, its effect, and a "pristine" reasoning chain, with longer chains for heavier corruption. It reports an R-Bench overall score of 0.5017 versus 0.4845 for the base model. Our method shares its goal but takes the opposite position on verbalization. The reason is BabyVision, which found that long chains of thought shift attention away from image tokens: suppressing them raised attention on visual tokens 5.28×. This constrains our design. If extra words hurt fine visual grounding, the invariance should be learned silently.
The architecture comes from NEO. NEO is a native vision-language model whose first "pre-buffer" layers learn visual features from scratch, while later layers inherit language-model weights. It reaches 54.6 on MMMU and 82.1 on MMBench. NEO and the native pretraining study Beyond Language Modeling were evaluated only on clean data. Single Stream Sequence Fusion argues that shared self-attention lets text steer visual features layer by layer. We turn that argument into a falsifiable prediction about where a reasoning-level gradient does its work.
Restoration research sets the synthetic-to-real constraint. Random Is All You Need and Modeling sRGB Camera Noise with Normalizing Flows explain why models trained on simple synthetic noise fail on real camera-processed noise. TTT-MIM shows that self-supervised adaptation on the test image itself can recover over half the gap on real smartphone noise (28.98 to 33.58 dB on SIDD). This warns us that synthetic corruptions alone may not transfer. It is why we include a small set of physically real pairs and measure their marginal value instead of assuming it.
Research Program
Data. We build paired training data from rendered math and science problems. The sources are scientific diagrams from VeriSciQA and chart-reasoning problems generated with the tool-and-template approach of Synthesize Step-by-Step. We keep MathReal, MATH-Vision and MathVista strictly for evaluation.
Each clean render gets corrupted copies at three levels of realism:
- Simple synthetic corruptions: Gaussian noise, blur and compression.
- Camera-pipeline corruptions: sensor-style noise passed through a simulated image processor, plus perspective warps, page bends, shadows, and pasted handwriting. These match MathReal's three categories of quality loss, perspective change and irrelevant content.
- Physically real pairs: a few thousand clean renders printed and photographed with several phones under varied lighting.
Comparing training on the first two levels with training on all three, at equal pair counts, measures how much real capture contributes beyond realistic simulation.
Models. The primary model is a native transformer in the NEO family. The comparison model is a modular VLM of similar size, Qwen2.5-VL-3B, which lets us compare directly with Robust-R1.
Experiments. The study rests on three experiments. The first fixes the data and compute budget and varies the loss: answer-only augmented fine-tuning, our KL alignment, and Robust-R1-style verbal diagnosis. The second freezes NEO's pre-buffer layers so that the alignment can only reshape the inherited language layers, which approximates a modular design inside one model. The third probes attention mass on image tokens during reasoning, following BabyVision's protocol, for each training regime. This tests whether silent alignment preserves visual attention while verbal diagnosis erodes it.
Baselines
Untrained base model (naive reference). This is the do-nothing reference: each base model evaluated as released. It is run on MathReal's photographed problems and, where available, the clean digital versions of the same problems. If MathReal's clean counterparts are not released, we re-render them from the problem text. The real-minus-clean difference defines the gap we try to close. Our headline metric is the fraction of that gap recovered.
Strongest reported results on MathReal. These set the bar for absolute numbers. MathReal reports 53.9% loose accuracy for Doubao-1.5-thinking-vision-pro, 41.0% strict accuracy for the top closed model, and 17.1% strict accuracy for the best open model, ERNIE-4.5-Turbo-VL. We found no reported result on Lacuna for NEO or Robust-R1 on MathReal, so we will produce those numbers ourselves. We do not expect a 3B–8B model to beat the closed systems in absolute terms. Beating the open-source figure of 17.1% strict accuracy, while recovering a larger fraction of the clean-to-real gap than any other arm, would count as success at our scale.
Corruption-augmented fine-tuning. This is the first term of the loss alone: identical pairs, identical traces, identical steps and hyperparameter search. It is the ablation that isolates the matching term. If alignment does not beat it by a margin larger than the seed-to-seed spread across three seeds, the core claim fails.
Robust-R1-style verbal diagnosis. This is the strongest existing method. We reimplement its supervised stage, with the corruption type and influence tags and depth scaling, on our pairs and the same backbones. That makes the comparison about training signal rather than data. We also report the released Robust-R1 checkpoint on R-Bench against its published 0.5017, as a sanity check on our evaluation code. Beating it means higher MathReal accuracy at equal or shorter output length.
Restore-then-reason pipeline. This pipeline puts Perceive-IR restoration in front of the base model. Perceive-IR is an all-in-one restorer guided by quality and semantic features; it reported a +0.57 dB average gain over PromptIR. We add an off-the-shelf document dewarping step before it, because perspective is a large part of MathReal and a denoiser alone would be a weak strawman.
Frozen-pre-buffer ablation. This tests the native-architecture claim described in the research program.
Why not just…?
Why not just fine-tune on corrupted images with the normal answer loss? This is the most serious threat, because our method contains this loss as its first term. The case against it relies on two findings. CoRe found that ordinary visual fine-tuning transfers reasoning across conditions poorly, and that step-wise distribution matching recovered far more of the gap. The restoration literature adds that models fit the signature of synthetic noise. Neither finding was measured on phone photos of math. Our matched-budget ablation settles it directly. If augmented fine-tuning matches alignment on MathReal, we will report that the denser signal adds nothing for real capture, and the paper becomes a negative result about process-level alignment.
Why not just clean the image first and then reason? Restoration models remove noise partly by smoothing. In diagrams, smoothing can erase decimal points, tick marks and exponents. Masked Image Training notes this high-frequency loss even in methods built for generalization. Denoisers also cannot remove handwriting or bleed-through without knowing what the document says. Dewarping, however, may genuinely fix much of the perspective category. So we break MathReal results down by its 14 distortion subtypes, and we expect the pipeline to stay competitive on perspective while losing on noise, blur and clutter. If it wins overall, a pipeline is the practical answer and we will say so.
Why not just have the model describe the corruption, as Robust-R1 does? Robust-R1 was evaluated on general visual question answering, not symbolic math. MATH-Vision and BabyVision both found that longer chains of thought often hurt on diagrams. BabyVision's attention measurements point to a specific cause. Our same-data comparison, together with the attention probe, tests whether verbal diagnosis reduces attention on image tokens on degraded diagrams and whether that predicts its errors. A combined arm, verbal diagnosis plus alignment, checks whether the two are complementary rather than competing.
Why not adapt to each test photo with masked self-supervision, as TTT-MIM does for denoising? Test-time training recovered more than half of a real-noise gap. That makes it a plausible complement, but its objective rebuilds pixels, not reasoning, and its fixed eight steps add latency to every query. We include a test-time masked-patch arm on top of each trained model as a secondary experiment, to see whether pixel-level adaptation adds to reasoning-level alignment or duplicates it.
Proposed Experiments and Decisive Checks
Evaluation sets. MathReal test (1,520 problems) is the main real-photo benchmark, using its strict and loose accuracy. We report gap recovery against clean versions and error shares by perception versus reasoning. Clean MATH-Vision and MathVista check that alignment does not hurt clean reasoning; MATH-Vision reports GPT-4o at 30.39% and MathVista reports GPT-4V at 49.9%, which give context only. R-Bench checks general robustness against Robust-R1's 0.5017.
Decisive checks. Three outcomes would decide the project:
- Native-architecture claim. It holds only if alignment's advantage over augmented fine-tuning shrinks substantially when NEO's pre-buffer is frozen, and is smaller in the modular model. If freezing changes nothing, the benefit lives in the language layers, and native design is irrelevant to this problem.
- Synthetic-to-real claim. It holds if alignment trained only on simulated corruptions beats augmented fine-tuning on real MathReal photos by more than it does on held-out synthetic corruptions. That pattern would mean the reasoning-level target transfers better than the noise signature. The physical-pairs arm then shows how much real capture still buys.
- Silent-versus-verbal question. If alignment wins on MathReal's noise and blur subtypes and keeps higher image-token attention than verbal diagnosis, the question is answered in favor of silent invariance. If verbal diagnosis wins there, BabyVision's distraction effect does not carry over to degraded math, and we will report that.
References
- MathReal: We Keep It Real! A Real Scene Benchmark for Evaluating Math Reasoning in Multimodal Large Language Models
- Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset
- Awakening Visual Reasoning: Mitigating Post-Training Failure in Vision-Text Compression
- Robust-R1: Degradation-Aware Reasoning for Robust Visual Understanding
- BabyVision: Visual Reasoning Beyond Language
- From Pixels to Words -- Towards Native Vision-Language Primitives at Scale
- Beyond Language Modeling: An Exploration of Multimodal Pretraining
- Unifying Vision-Language Representation Space with Single-Tower Transformer
- Random Is All You Need: Random Noise Injection on Feature Statistics for Generalizable Deep Image Denoising
- Modeling sRGB Camera Noise with Normalizing Flows
- TTT-MIM: Test-Time Training with Masked Image Modeling for Denoising Distribution Shifts
- Synthesize Step-by-Step: Tools, Templates and LLMs as Data Generators for Reasoning-Based Chart VQA
- Perceive-IR: Learning to Perceive Degradation Better for All-in-One Image Restoration
- Masked Image Training for Generalizable Deep Image Denoising