TL;DR
Solving visual geometry problems remains a major blind spot for Large Multimodal Models (LMMs) due to the lack of diverse, high-fidelity training data. To bridge this gap, the authors introduce R-CoT (Reverse Chain-of-Thought), a two-stage pipeline that renders realistic geometric figures with detailed structural descriptions, computes valid multi-step deductions first, and then synthesizes questions backward from the solution. Fine-tuning models on the resulting GeoMM dataset sets new state-of-the-art benchmarks at 2B, 7B, and 8B scales, with R-CoT-8B outperforming GPT-4o by an average of 13% across MathVista and GeoQA.
What Problem Does This Paper Address?
While modern vision-language models excel at general visual question answering (VQA), they routinely stumble on elementary geometry. Existing geometric datasets suffer from critical trade-offs:
- Rewording existing data: Using LLMs to paraphrase human-annotated datasets (e.g., G-LLaVA) expands text volume but leaves image and concept diversity stagnant.
- Template-based data engines: Programmatic engines (e.g., GeomVerse) produce geometrically accurate figures, but the images lack the visual complexity/fidelity of real exam problems, and the rigid question templates lack linguistic variety.
- Direct LMM generation: Prompting advanced vision models to inspect images and draft Q&A pairs introduces severe visual hallucinations and arithmetic mistakes.
The authors address the root cause: synthetic data generation needs the geometric precision of a programmatic engine paired with the linguistic diversity of LLMs—without letting the model hallucinate unanswerable questions.
Methods
The authors propose R-CoT, a two-stage data generation framework:
[Substrate Pool + Line Operations]
│
▼
GeoChain Engine ───► High-Fidelity Geometry Image
│
▼ Detailed Structural Descriptions (Ground Truth)
┌────────────────────────────────────────────────────────┐
│ Reverse A&Q (Text-only LLM reasoning via ERNIE Bot 4.0)│
│ 1. Description Patch Reasoning (single-step facts) │
│ 2. Chain-of-Thought Fusion (multi-step deductions) │
│ 3. Question Generation (work backwards to question) │
└────────────────────────────────────────────────────────┘
│
▼
Solvable, Multi-Step Geometric Q&A Pair
- GeoChain (Image & Description Engine): Samples from a pool of 20 geometric substrates (circles, regular polygons, triangles) and combines them iteratively. Unlike past engines, it introduces realistic auxiliary lines (midlines, chords, radii, secants) and angle annotations, while outputting an exact ground-truth textual description capturing spatial relationships (e.g., collinearity, perpendicularity, tangency).
- Reverse A&Q (Reverse Question Formulation): Rather than asking an LLM to pose a question and solve it, Reverse A&Q feeds the text-only structural description into an LLM through three structured sub-steps:
- Description Patch Reasoning: Breaks the description into small fragments and generates single-step deductions using in-context learning.
- Chain-of-Thought Fusion: Progressively chains single-step reasoning outputs together into coherent multi-step deductions.
- Question Generation: Extracts the final deduction target and works in reverse to draft a clear, solvable question corresponding precisely to the derived multi-step solution.
Because answers and deductive steps are established before formulating the question, the pipeline circumvents hallucinations and insolvable problems.
Datasets
- GeoMM (Introduced): A synthetic dataset containing 87k high-fidelity geometric image–Q&A pairs spanning 20 shape types, 4 complexity levels, and 4 question targets (lengths, angles, areas, and element relationships).
- Training Baselines: Models were fine-tuned using Geo170K combined with GeoMM (or competing synthetic sets: GeomVerse and the synthetic portion of MAVIS).
- Evaluation Benchmarks: Evaluated on the
testminisplit of MathVista and the test set of GeoQA using standard Top-1 accuracy.
Key Insights and Ideas
- Answer-First Inversion Prevents Hallucination: LLMs often fail to estimate problem difficulty when generating geometric questions from scratch, yielding under-constrained or unanswerable prompts. Generating the deductive chain first guarantees that every synthesized question has a strictly verified, step-by-step solution.
- Relational Awareness Matters More than Raw Polygons: Real geometry problems depend heavily on relationships formed by auxiliary lines (e.g., midpoints, tangents). Explicitly modeling element relationships in GeoChain directly translates to better fine-grained visual perception in downstream models, preventing errors like misidentifying alternate interior angles.
- Training Stability and Data Quality: Training on existing synthetic datasets (GeomVerse, MAVIS) degraded MathVista accuracy beyond 50k samples due to limited diversity. In contrast, GeoMM scaled smoothly up to 87k with significantly lower population variance across multiple training runs.
- Invariance to Shape Distribution: Varying the proportion of circles, triangles, or quadrilaterals by up to ~15% shifted downstream test accuracy by less than 0.5%, indicating that performance gains stem from learning underlying deductive geometry reasoning rather than memorizing shape statistics.
Key Results
Fine-tuning multiple open-source LMM architectures with GeoMM consistently boosted performance across the board without adding extra model parameters:
| Model | MathVista (Top-1 Acc %) | GeoQA (Top-1 Acc %) |
|---|---|---|
| Closed-Source Reference | ||
| GPT-4o | 60.6 | 61.4 |
| Open-Source Mathematical SOTA (Previous) | ||
| Math-LLaVA-13B | 56.5 | 47.8 |
| MAVIS-7B | — | 66.7 |
| R-CoT Models (Ours) | ||
| R-CoT-2B (Mini-Monkey-2B) | 57.7 (+2.4 over base) | 62.6 (+0.8 over base) |
| R-CoT-7B (InternLM-XC2-7B) | 62.0 (+3.8 over base) | 67.8 (+4.0 over base) |
| R-CoT-8B (InternVL-2.0-8B) | 73.1 (+2.0 over base) | 75.9 (+1.7 over base) |
- R-CoT-8B set new open-source state-of-the-art marks: outperforming the previous open-source mathematical SOTA by +16.6% on MathVista and +9.2% on GeoQA.
- It outperformed GPT-4o by +12.5% on MathVista and +14.5% on GeoQA.
Limitations and Future Work
- Linguistic Redundancy / Non-Unique Mappings: Because the pipeline relies on an LLM to generate reasoning chains from structural text, identical descriptions can occasionally generate ambiguous or non-unique Q&A pairs.
- Scope Limited to Planar Geometry: GeoChain currently focuses on 2D Euclidean geometry problems and does not yet cover solid (3D) geometry or coordinate-function graphs.
- Future Directions: The authors plan to adapt the Reverse CoT formulation to other diagram-heavy STEM tasks (e.g., physics diagrams, function plots) while further investigating methods to curb visual hallucination in multimodal backbones.
Summary updated