R-CoT: Reverse Chain-of-Thought Problem Generation for Geometric Reasoning in Large Multimodal Models

Linger Deng, Yuliang Liu, Bohan Li, Dongliang Luo, Liang Wu, Chengquan Zhang, Pengyuan Lyu, Ziyang Zhang, Gang Zhang, Errui Ding, Yingying Zhu, Xiang Bai

Abstract

Large Multimodal Models (LMMs) often struggle with geometric reasoning due to a lack of high-quality, logically consistent training data. We present TR-CoT (Theorem-Validated Reverse Chain-of-Thought), a novel framework that synthesizes theorem-grounded geometric diagrams and employs a reverse-reasoning process to generate verifiable question-answer pairs. By leveraging a deterministic graphics engine to validate LLM-generated logic, TR-CoT produces datasets that significantly enhance the geometric reasoning capabilities of LMMs, enabling them to outperform state-of-the-art models like GPT-4o on key benchmarks.

The Geometric Reasoning Gap

While LMMs excel at interpreting natural scenes, their performance drops sharply when solving high school geometry. This "geometric reasoning gap" stems from the scarcity of rigorous mathematical diagrams in training sets. Existing synthetic data generation methods often suffer from three primary traps: low diversity in shapes, logical hallucinations where LLMs misapply theorems, and inconsistencies between textual descriptions and the actual geometric properties of the accompanying images.

Comparison of existing CoT generation methods versus the proposed TR-CoT framework, highlighting common failure modes like logical conflicts and limited theorem coverage.

Methodology: The TR-CoT Framework

The TR-CoT framework addresses these issues through a two-stage process: the TR-Engine for diagram synthesis and the TR-Reasoner for logical data generation.

1. TR-Engine: The Architect

The TR-Engine utilizes a library of 110 fundamental geometric theorems to construct diagrams. Unlike template-based approaches, it dynamically injects geometric elements (e.g., altitudes, medians, or parallel lines) required by specific theorems into a base shape. It calculates exact vertex coordinates using a Python-based graphics engine, ensuring that all geometric properties—lengths, angles, and areas—are mathematically precise and "ground-truth" verified.

2. TR-Reasoner: The Logic Checker

The TR-Reasoner employs a "reverse" approach to generate high-quality Chain-of-Thought (CoT) data:

  • Patch Reasoning Fusion: It decomposes image descriptions into logical segments to generate single-step inferences.
  • Reverse Question Generation: Questions are derived from verified reasoning steps to ensure solvability and avoid "answerability drift," where the generated question cannot be answered by the provided logic.
  • Error Filtering: The system cross-validates LLM-generated reasoning against the deterministic properties from the TR-Engine. If an LLM claims an angle is 90° but the engine knows it is 85°, the data point is discarded. This "deterministic guardrail" ensures the final dataset is free from logical hallucinations.
The detailed TR-CoT workflow, illustrating the transition from theorem-grounded image generation to multi-step reasoning chains and reverse question generation.

Datasets and Evaluation

The researchers introduced two primary datasets to train and evaluate their models:

  • TR-GeoMM: A completely synthetic dataset of 15,000 images and 45,000 high-quality Q&A pairs covering a broad spectrum of theorems.
  • TR-GeoSup: An augmented version of the existing GeoQA dataset, where implicit steps are decomposed into explicit, fine-grained reasoning chains using the TR-Reasoner.

Evaluation on standard benchmarks like MathVista and GeoQA demonstrated substantial improvements. The TR-CoT-InternVL-2.5-8B model achieved a 10.1% gain on MathVista, surpassing GPT-4o and setting new records for open-source models in the 8B parameter class.

Key Insights

  • Reverse Generation Stability: Deriving questions from verified reasoning steps is significantly more reliable than simultaneous Q&A generation, which often leads to logical gaps.
  • Fine-Grained CoT Matters: Breaking problems into tiny, theorem-aware steps improves logical consistency by 24.5% and makes model outputs more human-readable and reliable.
  • Efficiency: Post-training, models produced more concise and logical solutions, using fewer tokens to reach the correct answer compared to baseline models.

Conclusion

TR-CoT provides a robust solution to the data scarcity and logical inconsistency problems in geometric reasoning for LMMs. By combining deterministic geometric engines with LLM-based reasoning, the framework creates a scalable path toward more reliable and mathematically sound multimodal AI. While currently limited by generation speed and initial LLM error rates, the "reverse-validation" framework holds potential for expansion into other complex mathematical and physical reasoning domains.

Summary updated

Abstract

Existing Large Multimodal Models (LMMs) struggle with mathematical geometric reasoning due to a lack of high-quality image-text paired data. Current geometric data generation approaches, which apply preset templates to generate geometric data or use Large Language Models (LLMs) to rephrase questions and answers (Q&A), unavoidably limit data accuracy and diversity. To synthesize higher-quality data, we propose a two-stage Reverse Chain-of-Thought (R-CoT) geometry problem generation pipeline. First, we introduce GeoChain to produce high-fidelity geometric images and corresponding descriptions highlighting relations among geometric elements. We then design a Reverse A&Q method that reasons step-by-step based on the descriptions and generates questions in reverse from the reasoning results. Experiments demonstrate that the proposed method brings significant and consistent improvements on multiple LMM baselines, achieving new performance records in the 2B, 7B, and 8B settings. Notably, R-CoT-8B significantly outperforms previous state-of-the-art open-source mathematical models by 16.6% on MathVista and 9.2% on GeoQA, while also surpassing the closed-source model GPT-4o by an average of 13% across both datasets.
BibTeX
@misc{deng2024r,
  author = {Linger Deng and Yuliang Liu and Bohan Li and Dongliang Luo and Liang Wu and Chengquan Zhang and Pengyuan Lyu and Ziyang Zhang and Gang Zhang and Errui Ding and Yingying Zhu and Xiang Bai},
  title = {{R-CoT: Reverse Chain-of-Thought Problem Generation for Geometric Reasoning in Large Multimodal Models}},
  year = {2024},
  eprint = {2410.17885},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2410.17885},
}
Figure