Dimensionally Guided Synthesis of Mathematical Word Problems

Ke Wang, Zhendong SuIJCAI 2016

Introduction

Mathematical Word Problems (MWPs) are a cornerstone of STEM education, requiring students to bridge the gap between abstract numerical operations and real-world scenarios. However, for automated educational systems, generating high-quality MWPs presents a "Goldilocks" challenge. Template-based generation often results in repetitive, predictable problems that fail to engage students. Conversely, generating problems directly from equations frequently leads to nonsensical narratives—such as adding "5 apples" to "10 miles per hour"—because the system lacks an understanding of physical units and logical context.

This paper introduces a novel system that leverages dimensional consistency as a structural bridge between mathematics and natural language. By ensuring that every mathematical operation is "well-typed" according to its units (e.g., meters, hours, or dollars per gallon), the system synthesizes diverse, authentic, and configurable problems at a massive scale.

Methodology: The Dimensional Bridge

The core of the proposed system is a two-stage pipeline designed to maintain mathematical integrity while crafting a coherent story.

1. Equation Generation and Variable Unrolling

The process begins with a "seed" equation, such as $X = Y$. To create complexity, the system performs variable unrolling. It replaces a single variable with a more complex expression based on a library of dimensionally sound rules. For instance, a "Distance" variable can be unrolled into the product of "Speed" and "Time." This ensures that the underlying math is always physically meaningful.

2. Narrative Generation via Binary Expression Trees

The system converts the resulting equation into a Binary Expression Tree (BET). This tree serves as the blueprint for the story.

  • Keyword Mapping: The system assigns "themes" (e.g., running, grocery shopping) to the nodes. Because the system understands the units (e.g., meters/second), it can intelligently select appropriate subjects (like "Joseph") and verbs (like "running").
  • Numerical Realism: To prevent the generation of unrealistic scenarios—such as a person running at 500 mph—the authors employ the Z3 SMT solver. This tool sets constraints on variables based on their sub-category (e.g., "human running speed") to find a valid set of integers that fit the narrative.
  • Template Filling: Finally, verbal templates are used to stitch these sub-stories into a cohesive, natural-sounding narrative.
Figure 3: A Binary Expression Tree (BET) illustrating how a distance equation is unrolled into speed and time variables with specific units and values.

Key Insights: Taxonomy and Distractions

The researchers introduced a "Quantity, Offset, Ratio" taxonomy to categorize values. This allows the system to distinguish between different types of addition: "combining two quantities" (Q1 + Q2) versus "applying an offset" (Q1 + $\Delta$). This nuance is critical for making generated stories feel natural rather than mechanical.

Furthermore, the system can intentionally introduce "distractors"—information provided in the story that is not required for the final calculation. By choosing an "unknown" variable that is not a leaf node in the math tree, the system can mimic the complexity of higher-level textbook challenges, allowing for fine-tuned difficulty scaling.

Evaluation and Results

The system was evaluated against 24 real-world problems from the Singapore Math curricula (Grades 2–7).

Efficiency

The synthesis process is remarkably efficient. The system takes approximately 1 millisecond to generate a single MWP, enabling the creation of vast problem sets for personalized learning environments in real-time.

Authenticity and Student Performance

In a double-blind study involving 30 middle school students, the generated problems proved to be statistically indistinguishable from textbook problems. Students were unable to accurately identify which problems were machine-generated ($p = 1$ in a paired t-test).

Crucially, student performance (measured by error rates) on generated problems closely mirrored their performance on textbook problems across varying levels of complexity. As shown in the figure below, the error rates for "Written" (textbook) and "Generated" problems are nearly identical, confirming that the synthetic problems provide a comparable educational challenge.

Figure 5: Comparison of student error rates between textbook (Written) and machine-generated problems across different levels of arithmetic complexity.

Conclusion and Future Work

By using dimensional units as a guiding principle, this research demonstrates that AI can generate mathematical word problems that are both mathematically rigorous and narratively sound. While the current system relies on a predefined library of themes and templates, future work aims to integrate more advanced Natural Language Processing (NLP) to create even more varied sentence structures and complex "plots." Additionally, expanding the system to handle complex SI unit conversions remains a key area for development.

Summary updated

BibTeX
@inproceedings{wang2016dimensionally,
  author = {Ke Wang and Zhendong Su},
  title = {{Dimensionally Guided Synthesis of Mathematical Word Problems}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2016},
}
Figure