Learning to Reason Deductively: Math Word Problem Solving as Complex Relation Extraction

Zhanming Jie, Jierui Li, Wei Lu

Introduction

Solving Math Word Problems (MWPs) is a fundamental challenge in natural language processing that requires both linguistic comprehension and logical reasoning. Traditionally, MWP solvers have been framed as Sequence-to-Tree (Seq2Tree) models, which generate a mathematical expression tree in a top-down fashion. However, this approach suffers from two primary limitations: a lack of explainability—as humans typically reason from known facts toward a solution—and computational redundancy.

Comparison between redundant tree generation and efficient deductive reasoning.

As shown in the figure above, tree-based models often re-calculate identical sub-expressions, whereas a deductive approach can reuse intermediate results, leading to a more efficient and human-like solution path. This shift from top-down generation to bottom-up deduction allows the model to build solutions one logical step at a time.

The Deductive Reasoning Framework

This paper proposes a novel Deductive Reasoner that treats arithmetic operations as complex relations between quantities. Instead of generating a full expression at once, the model iteratively extracts these relations to build a solution.

1. Representation and Scoring

The process begins by encoding the problem text and its numerical values into vector embeddings using a pre-trained language model (e.g., BERT). At each time step $t$, the model evaluates all possible pairs of available quantities ($q_i, q_j$) against a set of arithmetic operators. A scoring function determines the most likely operation to perform next.

The iterative deductive process showing quantity selection and operation prediction across time steps.

2. The Rationalizer Module

A key innovation is the Rationalizer. When a new intermediate result is deduced, the model does not simply append it to a list. Instead, the Rationalizer (implemented via GRU or Self-Attention) updates the embeddings of all existing quantities based on the new information. This mechanism allows the model to maintain a "mental state," shifting its focus as the problem-solving process progresses and preventing it from repeating previous steps.

3. Constraints and Termination

To further refine the search space, the authors incorporate declarative constraints based on mathematical logic (e.g., ensuring positive results for subtraction). A binary "Terminator" classifier determines when the final answer has been reached, signaling the end of the deductive process.

Experimental Results

The model was evaluated on four diverse benchmarks: MAWPS, Math23k, MathQA, and SVAMP. The deductive approach consistently outperformed strong baselines:

  • State-of-the-Art Accuracy: The model achieved 92.0% on MAWPS and 78.6% on the challenging MathQA dataset.
  • Robustness to Adversarial Input: On the SVAMP dataset, which tests resilience against "trick" questions, the model scored 47.3%. Notably, it outperformed baselines by 14.3 percentage points on complex questions requiring two or more steps.
  • Explainability: Because the model follows a bottom-up deductive path, its intermediate steps provide a clear, human-readable rationale for the final answer, mirroring how a student might show their work.

Conclusion

By reframing MWP solving as iterative relation extraction, this research demonstrates that deductive reasoning is not only more interpretable but also more efficient than traditional tree-generation methods. The use of a Rationalizer to update internal states proves crucial for solving multi-step problems and handling irrelevant information. Future work will explore expanding the model's search capabilities through beam search and extending the framework to handle non-binary mathematical operators.

Summary updated

Abstract

Solving math word problems requires deductive reasoning over the quantities in the text. Various recent research efforts mostly relied on sequence-to-sequence or sequence-to-tree models to generate mathematical expressions without explicitly performing relational reasoning between quantities in the given context. While empirically effective, such approaches typically do not provide explanations for the generated expressions. In this work, we view the task as a complex relation extraction problem, proposing a novel approach that presents explainable deductive reasoning steps to iteratively construct target expressions, where each step involves a primitive operation over two quantities defining their relation. Through extensive experiments on four benchmark datasets, we show that the proposed model significantly outperforms existing strong baselines. We further demonstrate that the deductive procedure not only presents more explainable steps but also enables us to make more accurate predictions on questions that require more complex reasoning.
BibTeX
@inproceedings{jie2022learning,
  author = {Zhanming Jie and Jierui Li and Wei Lu},
  title = {{Learning to Reason Deductively: Math Word Problem Solving as Complex Relation Extraction}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
  url = {https://openreview.net/forum?id=dDfx-b5Agr},
}
Figure