Text2Math: End-to-end Parsing Text into Math Expressions

Yanyan Zou, Wei LuEMNLP 2019 · DOI

TL;DR

This paper introduces Text2Math, a novel end-to-end system that parses natural language text directly into mathematical expressions (arithmetic word problems and equations). By treating this as a structured prediction problem and learning a joint text-math representation, Text2Math achieves state-of-the-art results without relying on hand-crafted rules or explicit knowledge of non-commutative operators.

What Problem Does This Paper Address?

The paper tackles the challenge of automatically solving math-related problems described in natural language. Specifically, it addresses two key tasks:

  1. Arithmetic Word Problems: Converting a narrative (potentially multi-sentence) into a mathematical expression from which an answer can be computed.
  2. Equation Parsing: Translating a descriptive sentence into a mathematical equation, often involving unknown variables.

Existing approaches often rely on rule-based systems, separate models for each task, or sequential local decision-making that struggles with non-commutative operations and requires significant manual effort or prior knowledge. The authors aim to unify these tasks under a single, end-to-end structured prediction framework.

Methods

Text2Math employs an end-to-end structured prediction approach, treating the problem as semantic parsing. The core innovation is the introduction of a text-math tree representation. This is a latent joint representation that encodes both the input text and the target math expression tree.

Key methodological aspects include:

  • Expression Trees: Math expressions are represented as trees, with nodes for operators (ADD, SUB, MUL, DIV, EQU) and quantities (constants, variables).
  • Latent Text-Math Tree: This is a novel representation where each node in the expression tree is augmented with associated text segments and a "word association pattern" indicating how words relate to the tree structure. This latent representation is learned, not explicitly provided during training.
  • Structured Prediction: The model learns to predict the complete text-math tree in one go, avoiding the pitfalls of local decision-making.
  • Dynamic Programming: An efficient bottom-up dynamic programming algorithm is used for inference, similar to prior work in semantic parsing.
  • Feature Engineering: Both discrete features (n-grams, POS tags, word association patterns) and neural features (learned from surrounding word windows) are used to score the text-math trees.

Datasets

The paper evaluates Text2Math on the following datasets:

  • Arithmetic Word Problems:
    • AI2 (Hosseini et al., 2014)
    • IL (Roy and Roth, 2015)
  • Equation Parsing:
    • Dataset released by Roy et al. (2016)

Key Insights and Ideas

  • Unified Approach: Treating both arithmetic word problems and equation parsing as a single structured prediction problem is effective and allows for a unified model.
  • End-to-End Learning: Directly predicting the complete math expression tree from text, without intermediate steps or explicit rules, simplifies the process and improves performance.
  • Latent Joint Representation: The text-math tree representation is crucial. It allows the model to learn the complex correspondence between text and mathematical structure implicitly, without needing gold-standard text-math trees.
  • Agnostic to Operator Semantics: The model's word association patterns (e.g., [w]A[w]B[w], [w]B[w]A[w]) inherently handle reordering and non-commutative operators (like subtraction and division) without requiring explicit knowledge of their properties. This is a significant advantage over prior methods.
  • Neural Features Enhance Performance: Incorporating neural features, especially with larger window sizes, significantly improves accuracy, suggesting the importance of capturing broader contextual information.

Key Results

  • Arithmetic Word Problems: Text2Math achieves state-of-the-art accuracy on both AI2 and IL datasets, outperforming previous systems. For example, it achieves an average accuracy of 83.10% on AI2 and IL combined. Ablation studies show that POS tags are not crucial, but lexicons do improve performance.
  • Equation Parsing: The model also achieves competitive results, reaching a new state-of-the-art accuracy of 74.5% with neural features. Notably, it does not require annotations of unknown variables, unlike previous pipeline systems.
  • Inverse Operators Unnecessary: The paper demonstrates that their approach does not require special handling for inverse operators (like -r or ÷r), confirming the effectiveness of their word association patterns.

Limitations and Future Work

  • Limitations:
    • The model can sometimes predict equations with missing or redundant variables, particularly when world knowledge is required (e.g., understanding "flying with the wind" implies wind speed).
  • Slight differences in problem descriptions can lead to different predictions, highlighting the challenge of subtle semantic variations.
  • Future Work:
    • Applying the approach to more complex math problems, such as algebra word problems that map to equation sets.
  • Investigating how to incorporate external world knowledge to improve performance on problems requiring it.

Summary updated

BibTeX
@inproceedings{zou2019text,
  author = {Yanyan Zou and Wei Lu},
  title = {{Text2Math: End-to-end Parsing Text into Math Expressions}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
  doi = {10.18653/v1/d19-1536},
}

Loading related content...

Figure