Abstract
Large Multimodal Models (LMMs) excel at general visual tasks but consistently struggle with the rigorous spatial logic required for geometric reasoning. This "geometry gap" is primarily due to the lack of high-quality, theorem-grounded training data. We introduce TR-CoT (Theorem-Validated Reverse Chain-of-Thought), a novel framework that synthesizes logically rigorous geometric data. By integrating a theorem-driven image engine with a reverse reasoning validator, TR-CoT produces datasets that significantly enhance the mathematical capabilities of LMMs. Our results show that models trained with TR-CoT can outperform closed-source models like GPT-4o on major geometry benchmarks.
Introduction: The Geometry Gap
While Large Multimodal Models (LMMs) have shown remarkable capabilities in general visual understanding, they consistently underperform in geometric reasoning. This "geometry gap" stems from the nature of existing training data, which primarily consists of natural scenes lacking the rigorous spatial logic and theorem-based dependencies required for mathematical problem-solving.
Previous attempts to address this data scarcity have encountered significant limitations. Rephrasing existing problems lacks diversity, while rigid template-based generation produces overly simple shapes. Most critically, direct LLM generation often leads to "hallucinated" logic, where the model generates a solution that appears plausible but is mathematically inconsistent with the provided diagram. For instance, an LLM might assume a right angle exists where the geometry dictates otherwise, leading to invalid reasoning chains.
Methodology: The TR-CoT Framework
To bridge this gap, we propose the Theorem-Validated Reverse Chain-of-Thought (TR-CoT) framework. This two-stage system ensures that every generated problem is both mathematically sound and logically solvable.
Stage 1: TR-Engine (The Builder)
The TR-Engine constructs geometric diagrams from the ground up using a comprehensive library of 110 fundamental theorems. The process begins with selecting a substrate shape (e.g., a circle or triangle). The engine then performs "dynamic injection," adding elements like tangent lines, points, or arcs based on specific theorem requirements.
Unlike simple drawing tools, the TR-Engine uses analytical geometry to compute the exact coordinates, lengths, and angles of every element. This creates a "ground truth" property set that serves as the deterministic anchor for the subsequent reasoning stage.
Stage 2: TR-Reasoner (The Logic Teacher)
The TR-Reasoner employs a "reverse" reasoning approach to generate Chain-of-Thought (CoT) data. Instead of asking a question and hoping for a correct answer, it derives questions from known geometric properties. This "Reverse CoT" ensures that a valid solution path always exists.
The framework incorporates a stepwise validation process. Every reasoning step produced by the LLM is cross-referenced against the ground-truth properties calculated by the TR-Engine. If the LLM claims an angle is 90° but the engine calculates it as 85°, the entire data pair is discarded. This rigorous filtering ensures that the final dataset is free from the logical hallucinations common in other synthetic methods.
Datasets and Evaluation
We developed two primary datasets to evaluate the TR-CoT framework:
- TR-GeoMM: A fully synthetic dataset containing 15,000 figures and 45,000 high-quality Q&A pairs. It covers a broader range of theorem types than previous synthetic efforts.
- TR-GeoSup: An augmented version of the GeoQA dataset, where implicit reasoning steps were made explicit and validated through our theorem engine.
Models were evaluated on the MathVista and GeoQA benchmarks to measure their geometric reasoning and logical consistency.
Results and Discussion
The implementation of TR-CoT led to significant performance improvements across various model scales (2B to 8B parameters).
- Benchmark Performance: Our top-performing model, TR-CoT-InternVL-2.5-8B, achieved a score of 77.9% on MathVista, notably surpassing GPT-4o, which scored 60.6%.
- Logical Consistency: The inclusion of fine-grained, validated CoT data improved the models' logical consistency by 24.5%. This indicates that the models are learning to follow correct mathematical procedures rather than relying on statistical heuristics to guess answers.
- Error Reduction: Our analysis showed that approximately 16% of LLM-generated geometry data is initially flawed. The TR-CoT validation step successfully reduced this error rate to 5%, highlighting the necessity of grounding LLMs in deterministic mathematical truths.
Conclusion
The TR-CoT framework provides a robust solution to the data scarcity and quality issues in geometric reasoning. By combining theorem-driven generation with reverse reasoning validation, we can produce high-quality synthetic data that significantly boosts LMM performance. Future research will explore extending these principles to 3D spatial reasoning and more complex multimodal domains involving physical constraints.
Summary updated