Distributed Representations for Arithmetic Word Problems

Sowmya S. Sundaram, Deepak P, Savitha Sam AbrahamAAAI 2020 · DOI

Introduction

In educational technology, the ability to retrieve mathematically similar word problems is essential for effective scaffolding—the process of providing students with related problems to reinforce specific concepts. However, traditional text embedding techniques, such as Word2Vec or Doc2Vec, often struggle with this task. These models prioritize surface-level vocabulary over underlying mathematical logic. For instance, a problem about "buying 12 apples and 5 oranges" (a combine schema) and one about "buying 12 apples and giving away 6" (a change schema) share significant word overlap but require fundamentally different solution strategies. Conversely, problems with entirely different narrative contexts (e.g., grocery stores vs. toy cars) may share the same mathematical structure. This research addresses the challenge of creating schema-aligned representations that group problems based on their solution logic rather than their narrative theme.

Methodology

The authors propose a specialized, unsupervised framework designed to capture the structural essence of arithmetic word problems. The architecture combines sequential text processing with graph-based structural analysis.

Siamese Bi-LSTM + GCN Architecture

The core of the model is a Siamese network that processes two versions of a problem simultaneously. As shown in Figure 1, each branch of the Siamese network consists of:

  • Bi-LSTM Layer: This layer processes the raw text sequence to capture local linguistic context and word-level dependencies.
  • GCN Layer: To capture mathematical logic, the model utilizes a Graph Convolutional Network (GCN) to process the dependency parse tree of the problem. This allows the model to understand the grammatical relationships between numbers and entities (e.g., identifying which quantity is being "added" or "removed"), which is critical for identifying the correct schema.
  • Siamese Training: The model is trained using a contrastive loss function. It learns to minimize the distance between a problem and its schema-preserving variants while maximizing the distance from variants that likely change the underlying math.
The Siamese Bi-LSTM + GCN architecture for learning arithmetic word problem embeddings.

Transformation Operators (Data Augmentation)

To enable unsupervised learning, the authors introduce "transformation operators" to generate pseudo-labels. These operators are categorized into two types:

  • Schema-Preserving Operators (SPO): These modify superficial elements like names, places, or entities without altering the mathematical logic.
  • Schema-Agnostic Operators (SAO): These modify key functional words like verbs or adjectives, which are likely to change the problem's schema (e.g., changing "placed" to "removed").
Examples of transformation operators used to generate schema-preserving and schema-altering problem variants.

Key Results

The proposed method was evaluated on a $k$-Nearest Neighbor (kNN) retrieval task using the SingleOp dataset (562 problems) and the MAWPS dataset (2,373 problems). The goal was to determine how often the top retrieved results shared the same mathematical schema as the query.

The results demonstrate a significant performance leap over baseline models:

  • Precision @ $k=3$: The model achieved a precision of 0.8114, significantly outperforming the GCN-only baseline (0.7503) and standard LSTM Autoencoders (0.7432).
  • NDCG Improvement: The model showed a consistent 3-9% improvement across various ranking metrics compared to Siamese variations that relied solely on text or solely on graphs.
  • Generalizability: When trained on the smaller SingleOp dataset and tested on the larger MAWPS dataset, the model maintained its superiority in retrieving problems with identical equation templates. This suggests that the learned embeddings successfully capture transferable mathematical logic rather than just dataset-specific patterns.

Discussion and Future Work

The success of this approach highlights two critical insights: syntax is often more important than vocabulary in mathematical contexts, and unsupervised schema alignment is achievable through well-defined domain-specific transformations. By explicitly training the model to ignore "contextual noise" (like narrative themes), the architecture produces representations that are robust to superficial changes.

However, limitations remain. The current transformation operators are relatively simple word-swapping mechanisms. Future work could involve developing more sophisticated operators for complex multi-step problems or kinematics. Additionally, the model's reliance on dependency parsing means that errors in the initial parse (e.g., from SpaCy) can propagate through the GCN. Integrating these embeddings into real-world instructional scaffolding systems will be the ultimate test of their utility in improving student learning outcomes.

Summary updated

BibTeX
@inproceedings{sundaram2020distributed,
  author = {Sowmya S. Sundaram and Deepak P and Savitha Sam Abraham},
  title = {{Distributed Representations for Arithmetic Word Problems}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i05.6432},
}
Figure