Learning to Solve Geometry Problems via Simulating Human Dual-Reasoning Process

Tong Xiao, Jiayu Liu, Zhenya Huang, Jinze Wu, Jing Sha, Shijin Wang, Enhong Chen

Abstract: Geometry Problem Solving (GPS) is a classic and challenging math problem that requires a solver to comprehensively understand both text and diagram, master essential geometry knowledge, and appropriately apply it in reasoning. Existing works often follow a neural machine translation paradigm, focusing on enhancing encoders but neglecting the essential characteristics of human geometry reasoning. This paper proposes DualGeoSolver, a framework inspired by dual-process theory to simulate the human dual-reasoning process. DualGeoSolver consists of a Knowledge System (implicit reasoning) and an Inference System (explicit reasoning). The Knowledge System provides diagram information and geometry knowledge based on step-wise goals from the Inference System, which specifies goals and applies knowledge to generate solution tokens. Extensive experiments on GeoQA and GeoQA+ datasets demonstrate DualGeoSolver's superiority in accuracy and robustness.

Introduction

Geometry Problem Solving (GPS) represents a significant hurdle in artificial intelligence, requiring the integration of multi-modal understanding, abstract mathematical knowledge, and complex logical reasoning. To solve a geometry problem, one must parse textual descriptions, interpret geometric diagrams, and recall specific theorems to bridge the gap between given information and the desired solution.

Historically, solvers have been divided into symbolic and neural approaches. Symbolic solvers rely on handcrafted rules, making them interpretable but brittle. Conversely, modern neural solvers often treat GPS as a Sequence-to-Sequence (Seq2Seq) translation task. While these models generalize better, they often lack a structured reasoning backbone, leading to logical inconsistencies or "hallucinations" because they do not explicitly model the application of geometric theorems.

The Dual-Reasoning Framework

Humans do not solve geometry problems by translating text to math in a single pass. Instead, we follow a structured, goal-oriented process: identifying a sub-goal, recalling a relevant theorem, and performing a calculation. This cognitive approach is described by Dual-Process Theory, which suggests human cognition consists of two systems: an intuitive, fast System 1 and an analytical, slow System 2.

Figure 2: An example of the human-like step-wise reasoning process for a geometry problem.

The proposed DualGeoSolver mimics this by splitting the task into a Knowledge System (System 1) and an Inference System (System 2). As shown in the example above, the model breaks down the problem into reasoning goals, such as finding $\angle CAB$ first to eventually determine $\angle AEC$.

Methods: Architecture of DualGeoSolver

DualGeoSolver operates through the iterative interaction of its two primary components.

Figure 4: The overall architecture of DualGeoSolver, featuring the interaction between the Knowledge System and the Inference System.

1. The Knowledge System (Implicit Reasoning)

The Knowledge System serves as the model's "intuitive" retriever. When provided with a reasoning goal from the Inference System, it performs two critical functions:

  • Visual Spotlight Module: This module focuses on specific diagrammatic features relevant to the current goal. For instance, if the goal involves parallel lines, the module highlights those lines in the visual representation ($H_D$).
  • Knowledge Selection Module: It retrieves relevant theorems from a curated Geometry Knowledge Base ($K_G$). This knowledge base contains definitions and explanations (e.g., "If two lines are parallel, then consecutive interior angles are supplementary") that provide the logical basis for the next step.

2. The Inference System (Explicit Reasoning)

The Inference System acts as the analytical engine, managing the high-level logic:

  • Goal Generation Module (GGM): Instead of attempting to solve the entire problem at once, the GGM identifies the next logical sub-goal ($g_t$) based on the current state of knowledge.
  • Token Prediction Module (TPM): Utilizing the retrieved knowledge ($H_t^K$) and visual features ($h_t^{vis}$), the TPM generates the actual mathematical operations (program tokens) required to resolve the current sub-goal.

The two systems work iteratively: System 2 sets a goal $\rightarrow$ System 1 provides the "how-to" (theorems) and "where-to-look" (visuals) $\rightarrow$ System 2 executes the step and sets the next goal.

Experimental Evaluation

The researchers evaluated DualGeoSolver on two benchmark datasets: GeoQA (5,000 problems) and GeoQA+ (an expanded version with more complex reasoning steps and area-based problems). They also introduced a structured collection of geometry knowledge-explanation pairs, verified by human annotators, to serve as the model's "mental library."

Key Results and Insights

DualGeoSolver achieved state-of-the-art (SOTA) performance on both benchmarks:

  • GeoQA: Reached 65.2% accuracy, surpassing previous models like SCA-GPS.
  • GeoQA+: Reached 65.1% accuracy, demonstrating a significant lead over the previous SOTA of 61.8%.

A critical finding was the "geometry gap" in Large Language Models (LLMs). Even powerful models like GPT-4V struggled, scoring around 30%. This is likely due to the disparity between the natural images LLMs are pre-trained on and the abstract, symbolic nature of geometry diagrams. However, when the Knowledge Selection Module's theorems were provided to GPT-4V via prompting, its performance improved significantly, confirming that the bottleneck is often the retrieval and application of domain-specific knowledge.

Conclusion

DualGeoSolver demonstrates that explicitly modeling the human reasoning process—specifically through goal-setting and knowledge injection—is more effective for complex tasks like GPS than black-box neural translation. While the model currently relies on a manually curated knowledge base and a Domain Specific Language (DSL), future work may involve automated knowledge extraction and integration with the broad reasoning capabilities of LLMs to create even more robust mathematical solvers.

Summary updated

Abstract

Geometry Problem Solving (GPS), which is a classic and challenging math problem, has attracted much attention in recent years. It requires a solver to comprehensively understand both text and diagram, master essential geometry knowledge, and appropriately apply it in reasoning. However, existing works follow a paradigm of neural machine translation and only focus on enhancing the capability of encoders, which neglects the essential characteristics of human geometry reasoning. In this paper, inspired by dual-process theory, we propose a Dual-Reasoning Geometry Solver (DualGeoSolver) to simulate the dual-reasoning process of humans for GPS. Specifically, we construct two systems in DualGeoSolver, namely Knowledge System and Inference System. Knowledge System controls an implicit reasoning process, which is responsible for providing diagram information and geometry knowledge according to a step-wise reasoning goal generated by Inference System. Inference System conducts an explicit reasoning process, which specifies the goal in each reasoning step and applies the knowledge to generate program tokens for resolving it. The two systems carry out the above process iteratively, which behaves more in line with human cognition. We conduct extensive experiments on two benchmark datasets, GeoQA and GeoQA+. The results demonstrate the superiority of DualGeoSolver in both solving accuracy and robustness from explicitly modeling human reasoning process and knowledge application.
BibTeX
@inproceedings{xiao2024learning,
  author = {Tong Xiao and Jiayu Liu and Zhenya Huang and Jinze Wu and Jing Sha and Shijin Wang and Enhong Chen},
  title = {{Learning to Solve Geometry Problems via Simulating Human Dual-Reasoning Process}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
  url = {https://openreview.net/forum?id=NBmVr6VJ7j},
}
Figure