Introduction
Solving Math Word Problems (MWP) is a hallmark of human intelligence, requiring both linguistic comprehension and logical reasoning. Traditional deep learning models often treat this as a direct translation task—mapping problem text to mathematical expressions. However, this "black-box" approach lacks transparency and fails to capture the explicit reasoning steps humans use. This paper introduces Learning by Applying (LeAp), a framework that shifts the paradigm from problem-expression to problem-knowledge-expression, simulating the human cognitive process of acquiring knowledge through its application.
The Problem: Implicit vs. Explicit Reasoning
The authors identify two major flaws in current MWP solvers:
- Lack of Transparency: The "knowledge" these models possess is buried in millions of numerical weights, making it impossible for humans to see what the model actually knows.
- Implicit Reasoning: Unlike humans, who explicitly learn that "total" often implies addition or that "apples" belong to the category of "fruits," standard models lack a mechanism to learn and apply such explicit rules.
The LeAp Framework
The core of LeAp is a Variational AutoEncoder (VAE) architecture designed to learn an explicit Knowledge Graph (KG) while solving problems. It consists of three main components:
1. Knowledge Encoder ($P_\phi(Z|X)$)
This module reads the problem text and predicts the existence of edges in a mathematical knowledge graph. These edges represent two types of relations:
- Word-Word: (e.g., "apples" $\leftrightarrow$ "fruits") to improve semantic understanding.
- Word-Operator: (e.g., "times" $\leftrightarrow$ "$\times$") to guide symbol reasoning.
2. Knowledge Decoder ($P_\theta(Y|Z, X)$)
This component takes the learned knowledge ($Z$) and the problem text ($X$) to generate the math expression ($Y$). It utilizes two novel modules:
- Semantics-Enhanced Module: Employs a Graph Convolutional Network (GCN) to refine word representations based on word-word relations.
- Reasoning-Enhanced Module: Uses an attention mechanism to focus on specific words that trigger certain mathematical operators.
3. Autonomous Learning Mechanism
The model is trained to maximize the Evidence Lower Bound (ELBO). Crucially, the authors provide a theoretical proof showing that learning knowledge while solving problems is more accurate than traditional link prediction (learning knowledge in isolation). This feedback loop allows the model to "realize" which word-operator connections are actually useful for reaching the correct answer.
Experimental Evaluation
The authors evaluated LeAp across three widely used benchmarks: Math23K (a large-scale Chinese dataset), MAWPS (an English dataset), and SVAMP (a challenge dataset designed to test robustness).
Key Results
- Plug-and-Play Generality: LeAp is not a standalone model but a framework. It was successfully integrated with various backbones like Seq2Seq, Graph2Tree, and GTS, showing performance gains across the board.
- Accuracy Boost: On the difficult SVAMP dataset, LeAp improved the Seq2Seq backbone from 20.0% to 23.6% and the Graph2Tree backbone from 31.9% to 34.1%.
- Interpretability: In case studies, LeAp correctly identified that the word "times" should trigger a multiplication operator ($\times$), whereas original backbone models often mistakenly defaulted to division ($\div$).
- Learning vs. Hardcoding: LeAp outperformed versions of the model using external, human-curated knowledge bases (LeAp-EK). This suggests that autonomous, task-specific knowledge acquisition is more effective for reasoning than static external data.
Conclusion and Future Work
LeAp represents a significant step toward more human-like AI by making the reasoning process explicit and interpretable. By treating knowledge as a bridge between the problem and the solution, the framework improves both accuracy and transparency. While currently focused on single-variable arithmetic, future work may extend this framework to more complex, multi-relational knowledge graphs and broader reasoning tasks beyond mathematics, such as logic puzzles or scientific problem-solving.
Summary updated