Introduction: The Retrieval vs. Worked Example Paradox
In learning science, two prominent principles often appear to be in conflict. The Testing Effect (or retrieval practice) posits that the act of recalling information from memory is the most effective way to solidify learning. Conversely, the Worked Example Effect suggests that studying a step-by-step solution is more efficient than attempting to solve a problem independently.
This research addresses this contradiction by applying the Knowledge Learning Instruction (KLI) framework. We hypothesize that the optimal instructional strategy is determined by the type of Knowledge Component (KC) being targeted:
- Facts (Constant KCs): Information that remains static, such as "Paris is the capital of France," requiring straightforward memory retrieval.
- Skills (Variable KCs): Procedures that must be applied across varying contexts, such as calculating a triangle's area, requiring induction—the ability to generalize rules and select relevant data.
Methodology: The Apprentice Learner Framework
To explore these dynamics, we utilized the Apprentice Learner (AL) framework to create Simulated Learners (SLs). These AI agents learn production rules (if-then logic) through demonstrations and feedback. To investigate the impact of memory, we compared two distinct agent architectures:
- Simulated Learner without Forgetting (SLoF): A deterministic model where learned rules are permanently and perfectly accessible.
- Simulated Learner with Forgetting (SLwF): A stochastic model based on the ACT-R architecture, where memory strength decays over time. In this model, retrieval success depends on the frequency and recency of a rule's use, mimicking human cognitive limitations.
The study replicated a human experiment involving 95 participants learning basic geometry (calculating areas of rectangles, triangles, circles, and trapezoids). Both humans and agents were tested across two conditions: practice-only (retrieval-heavy) and study-practice (incorporating worked examples). Problems were provided in "parsed" states to simulate the prior knowledge humans bring to the task.
Results: Comparing Human and Simulated Learners
The computational experiments revealed that the model incorporating human-like forgetting (SLwF) was the only one to successfully replicate human behavioral patterns.
As shown in the figure above:
- For Skills: Both humans and the SLwF model showed higher learning gains with study-practice (worked examples) compared to practice-only (19.9% vs 15.8% for the model).
- For Facts: The practice-only condition was superior, matching the human "Testing Effect" for constant knowledge components (15.6% vs 12.7% for the model).
- The SLoF Failure: The model with perfect memory (SLoF) incorrectly predicted that worked examples would be superior for both facts and skills, failing to capture the nuances of human fact-learning.
Discussion: The Role of Forgetting in Learning
A significant insight from this study is that forgetting is not merely a limitation but a functional "feature" of the learning process. In the SLwF model, forgetting helped "prune" incorrect production rules. Because correct rules are reinforced more frequently during successful practice, they eventually out-compete the "noisy" or incorrect rules induced during early learning stages.
Furthermore, error analysis indicated that the SLwF model committed more induction-based errors than simple memory failures. This suggests that the primary challenge in skill acquisition is the selection and generalization of information, rather than the mere memorization of formulas.
Conclusion and Future Work
This investigation demonstrates that AI agents can serve as a "computational microscope," allowing researchers to isolate and test specific cognitive mechanisms. By showing that content type dictates the effectiveness of instructional strategies, we provide a resolution to the conflict between retrieval practice and worked examples.
Future research will aim to move beyond simplified, text-based inputs to explore how learners parse raw visual diagrams. Additionally, we plan to investigate more sophisticated feedback mechanisms, such as explanatory feedback, to further refine our understanding of how learners overcome persistent misconceptions.
Summary updated