On the Role of Domain Experts in Creating Effective Tutoring Systems

Sarath Sreedharan, Kelsey Sikes, Nathaniel Blanchard, Lisa Mason, Nikhil Krishnaswamy, Jill ZarestkyarXiv 2025 · DOI · arXiv · OpenReview

Abstract

The role that highly curated knowledge, provided by domain experts, could play in creating effective tutoring systems is often overlooked within the AI for education community. In this paper, we highlight this topic by discussing two ways such highly curated expert knowledge could help in creating novel educational systems. First, we will look at how one could use explainable AI (XAI) techniques to automatically create lessons. Most existing XAI methods are primarily aimed at debugging AI systems. However, we will discuss how one could use expert specified rules about solving specific problems along with novel XAI techniques to automatically generate lessons that could be provided to learners. Secondly, we will see how an expert specified curriculum for learning a target concept can help develop adaptive tutoring systems, that can not only provide a better learning experience, but could also allow us to use more efficient algorithms to create these systems. Finally, we will highlight the importance of such methods using a case study of creating a tutoring system for pollinator identification, where such knowledge could easily be elicited from experts.

Introduction: Beyond the "Bitter Lesson"

In contemporary AI research, there is a prevailing trend toward the "bitter lesson"—the notion that massive compute and data should supersede expert-specified knowledge. While this approach has yielded breakthroughs in fields like natural language processing, its application to education is fraught with risk. Education is a high-stakes domain where the cost of failure is significant; we cannot permit tutoring systems to "hallucinate" facts or deliver pedagogically unsound instruction.

This paper argues for a "middle way" that integrates domain expertise with state-of-the-art AI. We address two primary hurdles: the lack of robustness in purely data-driven models and the computational complexity of modeling a learner's knowledge state.

Methods: Integrating Expertise with AI

The proposed framework leverages domain expertise to guide AI in two critical ways:

1. XAI-Driven Lesson Generation

Traditional Explainable AI (XAI) is typically used by developers to debug models. We propose repurposing these techniques for pedagogy. By grounding XAI outputs in expert-specified rules (e.g., "Wasps are identifiable by their pinched waists"), the system can transform a model's internal feature detection into an automated lesson.

Evolution from standard image classification to an expert-extended XAI tutoring system.

2. Hierarchical POMDPs and Expert Curricula

Adaptive tutoring systems often use Partially Observable Markov Decision Processes (POMDPs) to track a student's knowledge. However, the "belief space" of these models often explodes in complex domains. By using an expert-defined hierarchical curriculum (e.g., Group -> Genus -> Morphospecies), we can constrain the POMDP to track only a manageable subset of expertise levels, making the system computationally tractable.

Case Study: Pollinator Identification

We ground these concepts in the task of pollinator identification, utilizing data from iNaturalist and curricula from the Native Bee Watch citizen science program. This domain is ideal because it requires high scientific accuracy but lacks enough human experts to train the growing number of volunteers.

End-to-end interaction flow of the proposed adaptive tutoring system.

As shown in the interaction flow above, the system first assesses the user's position in the curriculum before providing targeted, XAI-based visual feedback grounded in expert concepts.

Key Insights

  • XAI as a Pedagogical Bridge: XAI should move beyond the "man behind the curtain" to help students see the world through an expert's eyes.
  • Curriculum as a Mathematical Constraint: A curriculum is more than a list of topics; it is a powerful structural constraint that enables efficient adaptive AI by simplifying the decision-making logic of an automated tutor.
  • The Classroom is Not a Black Box: Unlike tacit knowledge tasks (e.g., basic vision), education relies on explicit, curated knowledge already present in textbooks and expert practice.

Future Work and Limitations

Bridging the gap between domain experts (e.g., entomologists) and AI systems remains a challenge. There is a critical need for "symbolic interfaces" that allow experts to translate their knowledge without needing deep AI expertise. Furthermore, we must guard against "interdisciplinary erosion," ensuring that deep-rooted pedagogical knowledge is not ignored in favor of opaque, black-box solutions.

Summary updated

Abstract

The role that highly curated knowledge, provided by domain experts, could play in creating effective tutoring systems is often overlooked within the AI for education community. In this paper, we highlight this topic by discussing two ways such highly curated expert knowledge could help in creating novel educational systems. First, we will look at how one could use explainable AI (XAI) techniques to automatically create lessons. Most existing XAI methods are primarily aimed at debugging AI systems. However, we will discuss how one could use expert specified rules about solving specific problems along with novel XAI techniques to automatically generate lessons that could be provided to learners. Secondly, we will see how an expert specified curriculum for learning a target concept can help develop adaptive tutoring systems, that can not only provide a better learning experience, but could also allow us to use more efficient algorithms to create these systems. Finally, we will highlight the importance of such methods using a case study of creating a tutoring system for pollinator identification, where such knowledge could easily be elicited from experts.
BibTeX
@misc{sreedharan2025role,
  author = {Sarath Sreedharan and Kelsey Sikes and Nathaniel Blanchard and Lisa Mason and Nikhil Krishnaswamy and Jill Zarestky},
  title = {{On the Role of Domain Experts in Creating Effective Tutoring Systems}},
  year = {2025},
  eprint = {2510.01432},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.01432},
}
Figure