Learning by Applying: A General Framework for Mathematical Reasoning via Enhancing Explicit Knowledge Learning

Jiayu Liu, Zhenya Huang, ChengXiang Zhai, Qi Liu

Introduction

Solving Math Word Problems (MWP) is a hallmark of human intelligence, requiring both linguistic comprehension and logical reasoning. Traditional deep learning models often treat this as a direct translation task—mapping problem text to mathematical expressions. However, this "black-box" approach lacks transparency and fails to capture the explicit reasoning steps humans use. This paper introduces Learning by Applying (LeAp), a framework that shifts the paradigm from problem-expression to problem-knowledge-expression, simulating the human cognitive process of acquiring knowledge through its application.

The Problem: Implicit vs. Explicit Reasoning

The authors identify two major flaws in current MWP solvers:

  1. Lack of Transparency: The "knowledge" these models possess is buried in millions of numerical weights, making it impossible for humans to see what the model actually knows.
  2. Implicit Reasoning: Unlike humans, who explicitly learn that "total" often implies addition or that "apples" belong to the category of "fruits," standard models lack a mechanism to learn and apply such explicit rules.

The LeAp Framework

The core of LeAp is a Variational AutoEncoder (VAE) architecture designed to learn an explicit Knowledge Graph (KG) while solving problems. It consists of three main components:

1. Knowledge Encoder ($P_\phi(Z|X)$)

This module reads the problem text and predicts the existence of edges in a mathematical knowledge graph. These edges represent two types of relations:

  • Word-Word: (e.g., "apples" $\leftrightarrow$ "fruits") to improve semantic understanding.
  • Word-Operator: (e.g., "times" $\leftrightarrow$ "$\times$") to guide symbol reasoning.

2. Knowledge Decoder ($P_\theta(Y|Z, X)$)

This component takes the learned knowledge ($Z$) and the problem text ($X$) to generate the math expression ($Y$). It utilizes two novel modules:

  • Semantics-Enhanced Module: Employs a Graph Convolutional Network (GCN) to refine word representations based on word-word relations.
  • Reasoning-Enhanced Module: Uses an attention mechanism to focus on specific words that trigger certain mathematical operators.

3. Autonomous Learning Mechanism

The model is trained to maximize the Evidence Lower Bound (ELBO). Crucially, the authors provide a theoretical proof showing that learning knowledge while solving problems is more accurate than traditional link prediction (learning knowledge in isolation). This feedback loop allows the model to "realize" which word-operator connections are actually useful for reaching the correct answer.

Experimental Evaluation

The authors evaluated LeAp across three widely used benchmarks: Math23K (a large-scale Chinese dataset), MAWPS (an English dataset), and SVAMP (a challenge dataset designed to test robustness).

Key Results

  • Plug-and-Play Generality: LeAp is not a standalone model but a framework. It was successfully integrated with various backbones like Seq2Seq, Graph2Tree, and GTS, showing performance gains across the board.
  • Accuracy Boost: On the difficult SVAMP dataset, LeAp improved the Seq2Seq backbone from 20.0% to 23.6% and the Graph2Tree backbone from 31.9% to 34.1%.
  • Interpretability: In case studies, LeAp correctly identified that the word "times" should trigger a multiplication operator ($\times$), whereas original backbone models often mistakenly defaulted to division ($\div$).
  • Learning vs. Hardcoding: LeAp outperformed versions of the model using external, human-curated knowledge bases (LeAp-EK). This suggests that autonomous, task-specific knowledge acquisition is more effective for reasoning than static external data.

Conclusion and Future Work

LeAp represents a significant step toward more human-like AI by making the reasoning process explicit and interpretable. By treating knowledge as a bridge between the problem and the solution, the framework improves both accuracy and transparency. While currently focused on single-variable arithmetic, future work may extend this framework to more complex, multi-relational knowledge graphs and broader reasoning tasks beyond mathematics, such as logic puzzles or scientific problem-solving.

Summary updated

Abstract

Mathematical reasoning is one of the crucial abilities of general artificial intelligence, which requires machines to master mathematical logic and knowledge from solving problems. However, existing approaches are not transparent (thus not interpretable) in terms of what knowledge has been learned and applied in the reasoning process. In this paper, we propose a general Learning by Applying (LeAp) framework to enhance existing models (backbones) in a principled way by explicit knowledge learning. In LeAp, we perform knowledge learning in a novel problem-knowledge-expression paradigm, with a Knowledge Encoder to acquire knowledge from problem data and a Knowledge Decoder to apply knowledge for expression reasoning. The learned mathematical knowledge, including word-word relations and word-operator relations, forms an explicit knowledge graph, which bridges the knowledge "learning" and "applying" organically. Moreover, for problem solving, we design a semantics-enhanced module and a reasoning-enhanced module that apply knowledge to improve the problem comprehension and symbol reasoning abilities of any backbone, respectively. We theoretically prove the superiority of LeAp's autonomous learning mechanism. Experiments on three real-world datasets show that LeAp improves all backbones' performances, learns accurate knowledge, and achieves a more interpretable reasoning process.
BibTeX
@inproceedings{liu2023learning,
  author = {Jiayu Liu and Zhenya Huang and ChengXiang Zhai and Qi Liu},
  title = {{Learning by Applying: A General Framework for Mathematical Reasoning via Enhancing Explicit Knowledge Learning}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i4.25571},
}
Figure