Abstract
Large Multimodal Models (LMMs) often struggle with geometric reasoning due to a lack of high-quality, logically consistent training data. We present TR-CoT (Theorem-Validated Reverse Chain-of-Thought), a novel framework that synthesizes theorem-grounded geometric diagrams and employs a reverse-reasoning process to generate verifiable question-answer pairs. By leveraging a deterministic graphics engine to validate LLM-generated logic, TR-CoT produces datasets that significantly enhance the geometric reasoning capabilities of LMMs, enabling them to outperform state-of-the-art models like GPT-4o on key benchmarks.
The Geometric Reasoning Gap
While LMMs excel at interpreting natural scenes, their performance drops sharply when solving high school geometry. This "geometric reasoning gap" stems from the scarcity of rigorous mathematical diagrams in training sets. Existing synthetic data generation methods often suffer from three primary traps: low diversity in shapes, logical hallucinations where LLMs misapply theorems, and inconsistencies between textual descriptions and the actual geometric properties of the accompanying images.
Methodology: The TR-CoT Framework
The TR-CoT framework addresses these issues through a two-stage process: the TR-Engine for diagram synthesis and the TR-Reasoner for logical data generation.
1. TR-Engine: The Architect
The TR-Engine utilizes a library of 110 fundamental geometric theorems to construct diagrams. Unlike template-based approaches, it dynamically injects geometric elements (e.g., altitudes, medians, or parallel lines) required by specific theorems into a base shape. It calculates exact vertex coordinates using a Python-based graphics engine, ensuring that all geometric properties—lengths, angles, and areas—are mathematically precise and "ground-truth" verified.
2. TR-Reasoner: The Logic Checker
The TR-Reasoner employs a "reverse" approach to generate high-quality Chain-of-Thought (CoT) data:
- Patch Reasoning Fusion: It decomposes image descriptions into logical segments to generate single-step inferences.
- Reverse Question Generation: Questions are derived from verified reasoning steps to ensure solvability and avoid "answerability drift," where the generated question cannot be answered by the provided logic.
- Error Filtering: The system cross-validates LLM-generated reasoning against the deterministic properties from the TR-Engine. If an LLM claims an angle is 90° but the engine knows it is 85°, the data point is discarded. This "deterministic guardrail" ensures the final dataset is free from logical hallucinations.
Datasets and Evaluation
The researchers introduced two primary datasets to train and evaluate their models:
- TR-GeoMM: A completely synthetic dataset of 15,000 images and 45,000 high-quality Q&A pairs covering a broad spectrum of theorems.
- TR-GeoSup: An augmented version of the existing GeoQA dataset, where implicit steps are decomposed into explicit, fine-grained reasoning chains using the TR-Reasoner.
Evaluation on standard benchmarks like MathVista and GeoQA demonstrated substantial improvements. The TR-CoT-InternVL-2.5-8B model achieved a 10.1% gain on MathVista, surpassing GPT-4o and setting new records for open-source models in the 8B parameter class.
Key Insights
- Reverse Generation Stability: Deriving questions from verified reasoning steps is significantly more reliable than simultaneous Q&A generation, which often leads to logical gaps.
- Fine-Grained CoT Matters: Breaking problems into tiny, theorem-aware steps improves logical consistency by 24.5% and makes model outputs more human-readable and reliable.
- Efficiency: Post-training, models produced more concise and logical solutions, using fewer tokens to reach the correct answer compared to baseline models.
Conclusion
TR-CoT provides a robust solution to the data scarcity and logical inconsistency problems in geometric reasoning for LMMs. By combining deterministic geometric engines with LLM-based reasoning, the framework creates a scalable path toward more reliable and mathematically sound multimodal AI. While currently limited by generation speed and initial LLM error rates, the "reverse-validation" framework holds potential for expansion into other complex mathematical and physical reasoning domains.
Summary updated