WorldTree: A Corpus of Explanation Graphs for Elementary Science Questions supporting Multi-hop Inference

Peter A. Jansen, Elizabeth Wainwright, Steven Marmorstein, Clayton T. Morrison

Abstract

The WorldTree corpus provides 1,680 detailed, multi-sentence "explanation graphs" for elementary science questions, mapping out the common-sense and domain knowledge required for inference. By decomposing explanations into semi-structured "tablestore" rows, the authors provide a gold-standard dataset to help AI models move beyond simply picking the right answer to providing human-readable justifications.

Introduction

While modern Question Answering (QA) models are increasingly adept at selecting the correct multiple-choice answer, they generally lack "explainable inference"—the ability to tell a user why an answer is correct. In complex domains like science, arriving at an answer requires information aggregation, or multi-hop inference. You cannot simply find the answer in a single sentence; multiple facts must be linked together. However, when models attempt to "hop" between more than two pieces of information, they often fall victim to semantic drift—erroneously connecting unrelated concepts because they share a common word. The primary barrier to solving this has been a lack of high-quality, structured training data that demonstrates exactly how these knowledge "hops" should be constructed.

Methods

The authors manually constructed a corpus of explanations for standardized science exams using a "knowledge-first" approach. Explanations are represented as sets of lexically overlapping sentences, where each sentence is a row pulled from a structured "tablestore." This tablestore is a collection of 62 semi-structured tables (e.g., Taxonomy, Causality, Actions) containing nearly 5,000 rows of atomic facts, allowing for knowledge reuse across different questions.

Figure 2: Example of an explanation graph constructed from multiple tables in the tablestore.

To help models understand the structure of an argument, sentences are categorized into roles: Central (core scientific concept), Grounding (linking abstract terms to question instances), Background (supporting info), and Lexical Glue (explicit links between concepts).

Results and Key Insights

The WorldTree corpus contains 1,680 explanations for 3rd through 5th-grade science questions. A key finding is the "6-Hop" reality: while previous estimates suggested elementary science questions required 4 pieces of knowledge, WorldTree reveals the average is actually 6.3 pieces. This explains why current models struggle with these exams.

The authors also observed predictable knowledge growth. Using Monte Carlo simulations, they found that the growth of the tablestore follows a predictable exponential decay, estimating that roughly 6,000 unique facts are enough to explain nearly 10,000 different science questions. However, the "Long Tail" of inference suggests that to answer 80% of questions with an inference engine requiring 3 or more shared facts, a training corpus would need over 20,000 explanations.

Figure 6: Visualization of topic clusters emerging from shared knowledge rows in the explanation graphs.

The corpus demonstrates high connectivity, with topic clusters like "Photosynthesis" or "Electrical Conductivity" emerging naturally from shared knowledge rows. On average, a single table row is reused in 2.9 different explanations.

Conclusion

WorldTree provides a significant step toward explainable AI in science QA. While the current corpus excludes questions requiring spatial reasoning or complex math, it establishes a framework for structured, multi-hop inference. Future work involves developing specific algorithms to utilize these sentence roles and graph structures, and moving toward even more abstract "First Principles" logic.

Summary updated

BibTeX
@inproceedings{jansen2018worldtree,
  author = {Peter A. Jansen and Elizabeth Wainwright and Steven Marmorstein and Clayton T. Morrison},
  title = {{WorldTree: A Corpus of Explanation Graphs for Elementary Science Questions supporting Multi-hop Inference}},
  booktitle = {LREC},
  year = {2018},
}
Figure