A Goal-Driven Tree-Structured Neural Model for Math Word Problems

Zhipeng Xie, Shichao SunIJCAI 2019 · OpenReview

Abstract

Most existing neural models for math word problems exploit Seq2Seq model to generate solution expressions sequentially from left to right, whose results are far from satisfactory due to the lack of goal-driven mechanism commonly seen in human problem solving. This paper proposes a tree-structured neural model to generate expression tree in a goal-driven manner. Given a math word problem, the model first identifies and encodes its goal to achieve, and then the goal gets decomposed into sub-goals combined by an operator in a top-down recursive way. The whole process is repeated until the goal is simple enough to be realized by a known quantity as leaf node. During the process, two-layer gated-feedforward networks are designed to implement each step of goal decomposition, and a recursive neural network is used to encode fulfilled subtrees into subtree embeddings, which provides a better representation of subtrees than the simple goals of subtrees. Experimental results on the dataset Math23K have shown that our tree-structured model outperforms significantly several state-of-the-art models.

Introduction

Math Word Problems (MWPs) are a classic challenge in Natural Language Processing, requiring models to translate a narrative into a mathematical expression. Traditional approaches have largely relied on Sequence-to-Sequence (Seq2Seq) models. However, these models often lack logical coherence and struggle with syntactic validity, frequently producing mathematically "nonsense" outputs like mismatched parentheses. Crucially, Seq2Seq models process expressions linearly from left to right, failing to capture the goal-driven, hierarchical nature of human mathematical reasoning.

The GTS Model

To address these limitations, the researchers developed the Goal-Driven Tree-Structured (GTS) model. Unlike flat string generation, GTS constructs an expression tree top-down, mimicking the way humans decompose a complex problem into simpler sub-problems.

Architecture and Workflow

The model operates through a series of coordinated steps:

  1. Encoder: A bi-directional Gated Recurrent Unit (GRU) encodes the problem text into mathematical vectors, focusing on relationships between quantities by using special "NUM" tokens.
  2. Top-Down Goal Decomposition: The process begins with a root "Goal Vector." The model recursively decides whether a goal is a leaf (a known quantity) or needs further decomposition via an operator (e.g., +, -, ×, ÷).
  3. Bottom-Up Subtree Embedding: To ensure the model maintains context across the tree, a recursive neural network compresses completed subtrees into "subtree embeddings." This bottom-up flow informs the generation of subsequent sub-goals, preventing redundant or logically inconsistent operations.
The GTS model architecture illustrating top-down goal decomposition and bottom-up subtree embedding.

Key Results and Evaluation

The GTS model was evaluated on the Math23K dataset (23,161 problems) and the AllArith dataset. It achieved a new state-of-the-art accuracy of 74.3% on Math23K, significantly outperforming previous hybrid and ensemble models.

Handling Complexity

A key finding was the model's robustness as problem complexity increased. While standard Seq2Seq models see a sharp decline in performance for problems requiring longer expression trees, the GTS model maintains a higher accuracy, demonstrating the effectiveness of its structured approach.

Comparison of accuracy between Seq2Seq and GTS models as expression tree size increases.

An ablation study further highlighted the importance of the Subtree Embedding mechanism; removing it resulted in a 4.3% drop in accuracy, confirming that the bottom-up information flow is critical for maintaining mathematical logic.

Conclusion

By shifting from sequential translation to goal-driven tree generation, the GTS model provides a more logically sound framework for solving math word problems. While currently limited to single-equation solutions and potentially sensitive to "distractor" numbers in text, it represents a significant step toward more human-like reasoning in automated problem solvers.

Summary updated

Abstract

Most existing neural models for math word problems exploit Seq2Seq model to generate solution expressions sequentially from left to right, whose results are far from satisfactory due to the lack of goal-driven mechanism commonly seen in human problem solving. This paper proposes a tree-structured neural model to generate expression tree in a goal-driven manner. Given a math word problem, the model first identifies and encodes its goal to achieve, and then the goal gets decomposed into sub-goals combined by an operator in a top-down recursive way. The whole process is repeated until the goal is simple enough to be realized by a known quantity as leaf node. During the process, two-layer gated-feedforward networks are designed to implement each step of goal decomposition, and a recursive neural network is used to encode fulfilled subtrees into subtree embeddings, which provides a better representation of subtrees than the simple goals of subtrees. Experimental results on the dataset Math23K have shown that our tree-structured model outperforms significantly several state-of-the-art models.
BibTeX
@inproceedings{xie2019goal,
  author = {Zhipeng Xie and Shichao Sun},
  title = {{A Goal-Driven Tree-Structured Neural Model for Math Word Problems}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2019},
  url = {https://openreview.net/forum?id=-dK2yu2VHNI},
}
Figure