In the last decade, educational data mining has undergone a "deep" revolution. Traditional models like Bayesian Knowledge Tracing (BKT), which used simple probabilistic rules to track student mastery, have been largely superseded by deep learning architectures. Models like Deep Knowledge Tracing (DKT) and Neural Cognitive Diagnosis (NeuralCD) can predict whether a student will get the next question right with startling accuracy.
However, this accuracy comes at a cost: interpretability. For a teacher, knowing a student has an 82% chance of failing the next quiz is useless without knowing why. Is it a lack of prerequisite knowledge? A specific misunderstanding of "for-loops"? Or simply a lapse in attention?
Researchers are now working to "open the black box," creating models that combine the predictive power of neural networks with the pedagogical transparency required for real-world classrooms.
The Transparency Gap in Education
The fundamental problem, as identified by George Siemens and others, is that high-performing graphical and neural models often function as black boxes. In high-stakes environments, a "risk score" without a causal explanation prevents educators from designing effective interventions.
As Mohammad Khajah and Michael Mozer argued in How Deep is Knowledge Tracing?, the "depth" of these models might not even be necessary. They demonstrated that "shallow" models, when augmented with features like forgetting and individual ability, can match deep learning performance while remaining psychologically meaningful.
Enforcing Pedagogical Logic through Constraints
One of the most successful ways to make neural networks interpretable is to bake educational theory directly into the architecture.
In Neural Cognitive Diagnosis for Intelligent Education Systems, Fei Wang and Zhenya Huang introduced NeuralCD. Standard neural networks allow weights to be anything, making internal states impossible to map to specific skills. NeuralCD solves this by enforcing a monotonicity constraint: it restricts weights to be positive. This ensures that as a student's mastery of a concept increases, their predicted performance must also increase. This simple mathematical constraint forces the model’s internal parameters to align with actual knowledge mastery, allowing teachers to see a clear "ability curve" for every student.
Similarly, Shangshang Yang and Linrui Qin proposed using Kolmogorov-Arnold Networks (KANs) in Endowing Interpretability for Neural Cognitive Diagnosis by Efficient Kolmogorov-Arnold Networks. Unlike standard networks that use fixed activation functions on nodes, KANs use learnable functions on the edges. This allows the model to capture complex, symbolic relationships between concepts that are much easier for humans to inspect and understand than a sea of abstract neurons.
Moving Beyond Binary Data
Most early models treated student data as a simple string of 1s and 0s (correct or incorrect). But a student who fails a programming task because of a syntax error is in a very different place than one who fails because of a logic error.
Tianhao Peng and Yu Liang addressed this in HELP-DKT. Instead of binary outcomes, their model analyzes the actual source code students write. By tracking specific programming concepts like loops or variables within the code, the model provides a diagnostic "map" of a student's progress. Qi Liu took a similar path with EKT, which uses the text of the exercises themselves to better understand the relationship between different knowledge components.
Causal and Post-hoc Explanations
Even when a model is a black box, we can use "explainer" tools to peek inside. Vinitra Swamy and Sijia Du evaluated methods like SHAP and LIME in Trusting the Explainers, finding that while these tools sometimes disagree, they provide actionable insights that help educators improve course design.
However, the field is moving toward causal interpretability. Sein Minn and colleagues introduced Interpretable Knowledge Tracing (IKT), which explicitly models skill mastery, learning transfer, and problem difficulty as causal factors. By using Judea Pearl’s do-calculus, researchers like Yuan-Hao Jiang are building Causal Knowledge Networks that can distinguish between a student who is "lucky" and one who has truly mastered the prerequisites.
The Future: LLMs as Explainers
The latest frontier involves using Large Language Models (LLMs) to bridge the gap between numerical predictions and human understanding. In Explainable Few-shot Knowledge Tracing, Haoxuan Li and Jifan Yu demonstrate that LLMs can not only predict student success but also provide natural language explanations for their reasoning. This shifts the AI from a silent grader to a collaborative partner that can tell a teacher: "This student is struggling because they understand the syntax of 'if-statements' but are failing to grasp the underlying boolean logic."