Closing the Visual Reasoning-Access Gap Without Paired Problems

Multimodal language models often solve a math problem when it is typed but fail on the same problem shown as an image, even when they read the image correctly. Recent work traced this to the language model inside the system. Visual inputs fail to switch on the internal "reasoning mode" that text inputs switch on. The current fix requires pairs: take a text problem, render it as an image, find cases where the model solves the text but fails the picture, and train on those. That works for text pasted into a picture. It cannot work for the problems that matter most: geometry figures, function plots and spatial puzzles, which have no text twin to render from.

We propose to remove the pairing requirement. The model would be trained with reinforcement learning on real diagram problems that have checkable answers. A second training term would push the population of the model's internal states on visual problems toward the population of its internal states on unrelated text math problems, measured only along the directions that carry reasoning. No individual image is ever matched to an individual text. The bet is that the access failure is a property of the whole visual distribution, not of individual pairs. If so, matching distributions should be enough to fix it, and it should apply wherever answers can be verified, including figures nobody can render from text.

Core Claim

TIGER motivates this work. It shows the reasoning-access gap exists and, using activation patching, finds that visual tokens under-activate a reasoning subspace of the language backbone. It closes the gap with policy optimization on mined text/image counterfactuals, and reports about a 15% gain on MathVerse for Qwen3-VL-8B. Its pairs come from rendering text problems (OpenR1-Math, MATH-500-I, BBH-I). The file notes that TIGER leaves open whether this extends to diagrams where perception and spatial structure are intertwined.

Our claim is that the paired counterfactual is doing two jobs, and both can be done without pairs:

  • Choosing which examples to train on. This can be done by using the reasoning-subspace activation score itself to flag "access failures" among real visual problems.
  • Supplying a target for what the visual internal states should look like. This can be supplied by a distribution-level alignment loss against text problems that are topic-matched but unpaired.

If this holds, reasoning-access repair moves out of the narrow regime of synthetic text-rendered-as-image and into real diagram benchmarks.

A useful simplification separates this from most unpaired-alignment work. Both modalities pass through the same language backbone, so their internal states share coordinates. The general unpaired toolbox has to bridge different metric spaces, and it suffers from the instabilities documented in that literature: GAN training in vec2vec and the uncorrelated-batch collapse of discrete solvers in Neural Gromov-Wasserstein Optimal Transport. We need neither Gromov-Wasserstein relational matching nor adversarial translation.

We can instead reuse the simpler recipe of Visual Transfer via Wasserstein Domain Confusion. That work aligned simulator and real-world feature distributions with a Wasserstein penalty so a control policy transferred without paired states. We apply the same idea to hidden states inside a reasoning model rather than to a perception encoder for control.

Research Program

The objective, per training batch, is

$$J(\theta) = \mathbb{E}_{x \sim D_{\text{vis}}}\big[r(x, y_\theta)\big] \;-\; \lambda \, SW_2\big(U^\top h_\theta(D_{\text{vis}}),\; U^\top h_\theta(D_{\text{txt}})\big).$$

The symbols are as follows:

  • $D_{\text{vis}}$ is a pool of real image-question problems with verifiable final answers.
  • $r$ is the binary correctness reward used in group-relative policy optimization (GRPO, the reinforcement learning recipe TIGER uses, which compares sampled answers within a group).
  • $y_\theta$ is the sampled solution.
  • $h_\theta(\cdot)$ collects mid-layer backbone hidden states at the first reasoning tokens.
  • $D_{\text{txt}}$ is a separate, unpaired pool of text math problems.
  • $U$ is a low-rank basis for the reasoning subspace.
  • $SW_2$ is the sliced Wasserstein distance. It averages one-dimensional optimal transport costs over random projections, which makes it cheap and stable at mini-batch scale.
  • $\lambda$ is a weight.

The basis $U$ is estimated once from the model's own text rollouts. We follow TIGER's reasoning-subspace analysis and take the leading directions separating hidden states on correctly solved reasoning problems from states on non-reasoning text. $U$ is re-estimated periodically.

The text pool comes from LLM-based data augmentation, used in a deliberately weak role. An LLM writes text problems whose topic and difficulty histogram matches the visual pool (geometry, functions, counting, charts), with answers checked by a solver. Each generated problem is never tied to any particular image. This sidesteps the circularity risk that synthetic reasoning traces carry hallucinated visual grounding. The generator only shapes a marginal distribution and never writes a rationale for an image.

The second ingredient is pairing-free mining. Among visual problems the model currently fails, we rank by how weakly the visual input activates $U$ compared with the text-pool average. Low-activation failures stand in for TIGER's "succeeds on text, fails on image" counterfactuals. They get upweighted in the rollout pool, matching TIGER's finding that a small model-specific subset of 1.5k to 4k examples suffices.

The visual pool is drawn from MMFineReason, which provides curated image problems with verified answers, using only problems disjoint from all evaluation sets. The cross-domain question comes from the reasoning-transfer literature: one-example RL and CPL show that narrow verifiable training transfers broadly in text. We test whether access repair learned on geometry and function plots transfers to charts and spatial puzzles that were held out of training.

Baselines

The naive reference point is the untuned backbone, Qwen3-VL-8B, chosen to match TIGER's reported model. It is evaluated with the same decoding settings. Every gain is reported relative to it.

The strongest existing method is TIGER itself: paired counterfactual mining from rendered OpenR1-Math problems followed by GRPO. Its reported figure is about a 15% gain on MathVerse for Qwen3-VL-8B. The file gives no TIGER numbers on MATH-Vision, MATH-500-I or BBH-I ("no reported result found on Lacuna" for those). We will rerun TIGER with the same number of GRPO steps, rollouts per problem, learning-rate sweep and training-set size as our method. If our rerun falls short of the reported MathVerse gain, we will report both figures. Beating TIGER means matching it within noise on MathVerse and MATH-500-I, where TIGER's renderable regime applies, and exceeding it on non-textual diagram subsets, where TIGER's pairs cannot be built.

The reinforcement-only baseline is GRPO on the identical real visual pool with no alignment term and no activation-based mining. This is the critical ablation. It isolates whether our ingredient adds anything beyond "do RL on real diagrams," and TIGER's own patching result suggests that GRPO alone already raises reasoning-subspace activation.

The supervised baseline is fine-tuning on MMFineReason-123K distilled traces, which that paper reports matches its full 1.8M set. It gets the same token budget.

Two further ablations isolate the mechanism:

  • Shuffled-subspace control: the alignment target is text hidden states from non-reasoning prose, projected on random directions. This tests whether reasoning-specific matching matters or just "looking like text."
  • No-mining control: the alignment term is kept but activation-based mining is removed.

Why not just…?

Why not just run GRPO on real diagram problems and skip alignment? This is the most serious threat. TIGER found that GRPO by itself pulls visual tokens into the reasoning subspace, so the alignment term could be redundant. We do not argue around this; the reinforcement-only baseline settles it. If it matches our method on accuracy and on activation measures, the conclusion is that pairing was never needed and RL suffices. That would itself be a useful correction to TIGER's framing. We expect the explicit term to help mainly when rewards are sparse, so we also compare the two methods at small training pools and early checkpoints.

Why not just create pairs by having a model caption each diagram into text, then apply TIGER? This restores pairing cheaply and is a real alternative, so we run it. The evidence against it is indirect but consistent:

  • MMFineReason found that added perceptual captions gave marginal or negative gains on STEM reasoning.
  • GeoSQA shows that even human text versions of diagrams leave failures.
  • ChartQA shows that text extraction loses links to visual marks.

Captions therefore make noisy pairs, and the mined "counterfactual" may reflect caption errors rather than an access failure. We measure this directly by auditing the fraction of caption-pair counterfactuals whose captions are wrong.

Why not just fine-tune on large distilled reasoning traces? TIGER reports that GRPO beat both supervised fine-tuning and preference optimization on its counterfactuals. MATH-Vision notes that long chains of thought in vision models drift from the visual evidence. The supervised baseline on MMFineReason-123K tests this on our data.

Why not just raise image resolution? MMFineReason reports little benefit from scaling to 2048×2048 on STEM diagrams. MATH-Vision attributes 42.2% of GPT-4V errors to reasoning rather than recognition (31.9%). We therefore do not treat resolution as a competitor, but we keep resolution fixed across all arms.

Proposed Experiments

All methods train the same backbone under a matched compute budget. Evaluation covers MathVerse, MATH-Vision, MathVista, ChartQA's human-written split, and TIGER's MATH-500-I and BBH-I. Reference points come from the file:

  • MATH-Vision: GPT-4o reaches 30.39% versus 68.82% for humans.
  • MathVista: GPT-4V reaches 49.9% versus 60.3% for humans.
  • ChartQA: StructChart with GPT-4 reaches 78.5%.

These frame the absolute numbers, but our claims rest on within-backbone comparisons.

Following the evaluation hierarchy in the perception-alignment-reasoning survey, we score final answers and also sample solution processes for step validity, so that lucky guesses do not inflate gains.

The decisive checks are four.

The first is a regime split. We partition MATH-Vision and MathVerse items into those whose content is mostly renderable text and those that are genuinely diagrammatic, such as MATH-Vision's transformation geometry, where GPT-4o scores roughly 18%. The prediction is that TIGER and our method tie on the first group and our method leads on the second.

The second is mechanism. Reasoning-subspace activation on visual inputs should rise under our method. Activation patching should show that restoring text-like states in $U$ recovers failures, and the shuffled-subspace control should not reproduce the gain.

The third is transfer. A model trained only on geometry and function plots should improve on ChartQA and puzzle categories it never saw, compared against the reinforcement-only baseline trained on the same split.

The fourth is perception cost. Pulling visual states toward text statistics could erase visual detail. We therefore track elementary chart-reading queries of the kind Plot Twist shows models already handle poorly. A drop there would mean the alignment buys reasoning by sacrificing perception, and $\lambda$ would need to be bounded accordingly.