EduExplain: A Benchmark for Explanation-Set Retrieval and Storyboard Construction in Educational Explanation

Aryan Ahmed, Riti Verma, Sanskar Singh, Maya RamanatharXiv 2026 · arXiv

TL;DR

EduExplain is a new benchmark designed to move AI tutoring systems beyond simple document retrieval toward generating cohesive, multi-part educational explanations. It provides human-validated ground truth for two critical stages: retrieving a compact set of complementary resources (text, diagrams, examples) and organizing them into a pedagogically sound storyboard with specific roles and timing.

What Problem Does This Paper Address?

When a student asks, "Why do plants need sunlight?", a standard search engine might return a long Wikipedia article or a list of relevant snippets. However, a good teacher provides a structured explanation: a definition of photosynthesis, a diagram of a leaf, and perhaps a real-world example.

Current AI benchmarks are poorly suited for this task because they typically evaluate individual document relevance rather than the quality of a collection of resources. Furthermore, they offer no way to measure how well a system organizes those resources into a lesson. The authors identify a "pipeline" gap: systems must first retrieve a compact set of resources and then structure them into an ordered, timed, and role-assigned plan (a storyboard) before any final text or video can be rendered.

Methods

The researchers developed an LLM-assisted, human-in-the-loop pipeline to construct the benchmark, as illustrated in Figure 1.

Fig. 1: EduExplain construction pipeline. Resource construction yields a typed, multimodal resource corpus. The explanation-set layer generates and human- validates gold explanation sets over that corpus; the storyboard layer then or- ganises each validated explanation set into reference storyboards, with its own human-validation stage. The benchmark releases both validated layers.

The evaluation framework focuses on two distinct stages:

  1. Explanation-Set Retrieval: Systems are tasked with selecting a small group of resources (typically $\le 3$) that together satisfy a learner's query. The primary metric is GoldSet F1, which measures how well the retrieved set matches human-validated "gold" sets.
  2. Grounded Storyboard Construction: Systems take a retrieved set and organize it into a sequence of segments. Each segment is assigned a pedagogical role (e.g., hook, definition, mechanism, example) and a specific duration.

To establish baselines, the authors tested various retrieval models (BM25, Dense, Multimodal, and Cross-Encoders) paired with different selection strategies like Top-n and Maximum Marginal Relevance (MMR).

Datasets

The authors introduce the EduExplain benchmark, which is built from secondary-level science textbooks. It includes:

  • 6,921 Multimodal Resources: A mix of explanatory passages, figures (with AI-generated descriptions), worked examples, and activities.
  • 2,123 Concepts: Derived from textbook chapters and sections.
  • 321 Learner-Oriented Queries: These span seven distinct educational intents, such as "Why/How" questions and "Misconception" clarifications.
  • 741 Reference Storyboards: Human-reviewed plans that serve as the gold standard for how to structure an explanation.

The variety of query types is a key feature of the dataset, as shown in Figure 2.

Table of query categories used in EduExplain, ranging from basic explanations to practice questions.

Key Insights and Ideas

  • The "Diversity" Paradox: In standard information retrieval, we often use algorithms like MMR to ensure results aren't redundant. However, in education, redundancy is often a feature, not a bug. A definition and an example of the same concept should have high semantic overlap. The authors found that off-the-shelf diversity-based selection actually underperformed simple top-ranked selection.
  • Retrieval Quality Dominates: The performance of the entire pipeline is heavily bottlenecked by the initial retrieval stage. If the first-stage retriever doesn't surface the right "ingredients," no amount of clever selection or structuring can fix the final explanation.
  • Structuring is a Non-Trivial Task: The authors compared "FreeForm" LLM generation against a "FixedContract" (a rigid template). They found that unconstrained LLMs often hallucinate resources or ignore time budgets, while rigid templates fail to adapt to the specific needs of different query types.

Key Results

  • Best Retrieval Pipeline: The strongest baseline was Dense Retrieval (BGE-M3) combined with Cross-Encoder reranking, achieving a GoldSet F1 of 0.461.
  • Multimodal Struggles: Native multimodal retrieval (using CLIP-style embeddings) performed significantly worse (GS-F1 $\approx$ 0.245) than using text-based descriptions of images. This suggests that "pedagogical" image retrieval—finding a diagram that explains a concept rather than just one that looks like the query—is still a major challenge.
  • Hardest Queries: "Misconception" queries were the most difficult for systems to handle. These queries often have low lexical overlap with the correct explanatory resources because they are phrased using incorrect beliefs (e.g., "Do plants get food from the soil?").

Limitations and Future Work

The authors acknowledge several open problems surfaced by EduExplain:

  • Set-Sufficiency Modeling: We need better ways to score a whole set for "completeness" rather than just checking for pairwise redundancy.
  • Image Retrieval Bottleneck: Current models struggle to distinguish between near-duplicate figures or to identify the specific pedagogical value of a diagram.
  • Explanation-Aware Structuring: Future work should focus on modeling pedagogical role structures that can dynamically adapt to a query, rather than relying on fixed progressions.

The benchmark is publicly available on GitHub.

Summary updated

Abstract

Generating an educational explanation for a learner's query is not a single retrieval step but a pipeline: a system must first retrieve a compact set of complementary resources--passages, figures, examples, activities--that jointly support an explanation, then structure that set into an ordered, timed, role-assigned plan before rendering it as a lesson or video. Existing retrieval benchmarks evaluate neither stage at the level these applications require: they score individual documents rather than explanation sets, and provide no ground truth for how a set should be organised into an explanation. We present EduExplain, a benchmark providing human-validated ground truth for both stages. Built from secondary-level science textbooks, it comprises 6,921 typed, multimodal resources and 2,123 concepts, and pairs each of 321 learner-oriented queries with one or more human-validated explanation sets (retrieval ground truth) and 741 reference storyboards (structuring ground truth). The benchmark exposes characteristics absent from prior datasets: heterogeneous multimodal retrieval units, multiple valid explanation sets per query, and plans whose quality depends on pedagogical role composition, grounding, and timing rather than relevance alone. We describe an LLM-assisted, human-validated construction pipeline, report inter-annotator agreement for both layers, and establish baselines for each stage--lexical, dense, multimodal, and cross-encoder retrieval paired with four selection strategies, and free-form and fixed-template storyboard construction. Our results show that first-stage retrieval quality dominates retrieval-stage performance, that off-the-shelf diversity-based selection underperforms simple top-ranked selection, and that unconstrained and rigidly templated structuring both fall short of the reference plans. The benchmark is available at https://github.com/data-iitd/eduexplain.
BibTeX
@misc{ahmed2026eduexplain,
  author = {Aryan Ahmed and Riti Verma and Sanskar Singh and Maya Ramanath},
  title = {{EduExplain: A Benchmark for Explanation-Set Retrieval and Storyboard Construction in Educational Explanation}},
  year = {2026},
  eprint = {2611.00802},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.00802},
}
Figure