Abstract
One of the most ambitious use cases of computer-assisted learning is to build a recommendation system for lifelong learning. Most recommender algorithms exploit similarities between content and users, overseeing the necessity to leverage sensible learning trajectories for the learner. Lifelong learning thus presents unique challenges, requiring scalable and transparent models that can account for learner knowledge and content novelty simultaneously, while also retaining accurate learners representations for long periods of time. We attempt to build a novel educational recommender, that relies on an integrative approach combining multiple drivers of learners engagement. Our first step towards this goal is TrueLearn, which models content novelty and background knowledge of learners and achieves promising performance while retaining a human interpretable learner model.
Introduction
Traditional recommendation systems, such as those employed by Netflix or Amazon, focus primarily on user interests and content similarity. However, educational recommendation is fundamentally different; it must account for evolving learning trajectories. Learners do not just want "more of the same"; they require content that builds upon their existing knowledge without being redundant or overwhelmingly difficult.
Current educational models like Item Response Theory (IRT) or Knowledge Tracing (KT) are often limited to narrow, manually-curated skill sets and do not scale to the vast, diverse topics encountered in lifelong learning. Furthermore, these models often ignore the "Goldilocks zone" of learning—the idea that a resource is only useful if it sits perfectly between being too easy (boring) and too difficult (overwhelming). To address this, we introduce TrueLearn, a scalable and transparent model designed to handle multi-modal, multi-lingual content over long durations.
The TrueLearn Framework
TrueLearn is a probabilistic graphical model that reformulates the TrueSkill matchmaking algorithm—originally designed for Xbox player ranking—into an educational context. In this framework, the interaction between a learner and a resource is treated as a "match." If a learner engages with a resource (e.g., watches a video fragment), the system infers that the resource's difficulty and content were appropriate for the learner's current skill level.
The model identifies four key latent variables that drive engagement:
- Resource Quality ($Q$): The baseline quality of the textual features ($T$).
- Background Knowledge ($B$): The overlap between topic coverage ($D$) and current knowledge ($K$).
- Novelty ($N$): The degree to which the content introduces new information relative to the learner's state.
- Curiosity ($C$): The alignment between content and the learner's specific interests or goals ($I$).
To maintain transparency, TrueLearn uses Wikification (an automatic entity-linking algorithm) to map educational content to Wikipedia topics. This creates a human-readable "language" for skills, allowing the learner's profile to be inspected and understood rather than remaining a "black box."
Experimental Evaluation
We evaluated TrueLearn using a dataset from VideoLectures.Net (VLN), comprising over 18,000 unique learners. Video transcripts were segmented into fragments of approximately five minutes, each annotated with the five most relevant Wikipedia concepts using the Wikifier tool. Engagement was measured via "normalised watch time," which served as the ground truth for whether a resource was appropriate for the learner's current state.
Results and Performance
TrueLearn was compared against several baselines, including standard Knowledge Tracing and the original TrueSkill algorithm.
As shown in the results, TrueLearn significantly outperforms traditional methods, achieving an F1-score of 0.677. The model's ability to account for novelty and multi-topic fragments is crucial for predicting engagement in a lifelong learning context. Unlike Vanilla TrueSkill or Multi-skill KT, TrueLearn maintains high performance while remaining computationally efficient for real-time Bayesian updates.
Discussion and Future Work
A primary advantage of TrueLearn is its domain agnosticism. By using Wikipedia as an underlying ontology, the system can recommend content across any subject area without requiring manual curriculum mapping by experts. Furthermore, its scalability allows it to maintain accurate learner representations over long periods without the massive overhead of retraining deep neural networks.
Future work will focus on incorporating the remaining engagement drivers—specifically "Curiosity" and explicit user goals—which are currently latent in the model. Additionally, we aim to develop user interfaces that effectively present these learning trajectories and knowledge fragments to learners, further enhancing the transparency and utility of the system.
Summary updated