LOOM: Personalized Learning Informed by Daily LLM Conversations Toward Long-Term Mastery via a Dynamic Learner Memory Graph

Justin Cui, Kevin Pu, Tovi GrossmanarXiv 2025 · DOI · arXiv

Introduction

General-purpose AI assistants are highly effective at providing immediate, reactive answers to specific queries. However, they typically lack a long-term perspective on a user's learning journey. This creates a tension between continuity—the ability to maintain a coherent educational path—and initiative—the ability to respond to spontaneous, immediate needs. Existing educational tools often fall into two polarized categories: rigid, target-focused tutors that ignore daily interests, or incidental learning systems that provide "just-in-time" information without scaffolding it into durable mastery.

LOOM addresses this fragmentation by transforming daily LLM interactions into a structured, long-term learning experience. By maintaining a Dynamic Learner Memory Graph, the system proactively identifies knowledge gaps and generates personalized mini-courses that bridge the gap between isolated assistance and cumulative expertise.

Methodology: The LOOM Pipeline

LOOM utilizes a four-stage agentic pipeline driven by Large Language Models to synthesize learning signals from raw chat logs.

The LOOM four-step pipeline: (1) Conversation Observation & Summarization, (2) Topic Decision & Outline Generation, (3) Course Content Generation, and (4) Update & Regroup Learner Graph.
  1. Conversation Observation & Summarization: Raw chat logs are distilled into concise "learning signals." These are tagged with thematic umbrellas (e.g., "Supervised Learning") and estimated difficulty levels.
  2. Topic Decision & Outline Generation: A "Topic Decider" agent compares recent summaries against the Learner Memory Graph. It operates in two modes: Strengthen (reviewing existing knowledge) and Explore (introducing adjacent concepts). It proposes 3–4 module mini-course outlines.
  3. Course Content Generation: Upon user selection, LOOM generates personalized lessons and quizzes. It retrieves excerpts from the user’s previous chats to ground the lesson in familiar phrasing and context.
  4. Update & Regroup Learner Graph: As modules are completed, a "Goals Updater" agent modifies the memory graph, evolving goal umbrellas (e.g., expanding "K-means" into "Machine Learning") to reflect growing expertise.

Evaluation and Results

The system was evaluated through a formative user study involving 10 participants from professional and academic backgrounds. Participants generated 10–15 real-world chats over two days, which served as the live dataset for LOOM.

Evaluation responses across 10 metrics using a 7-point Likert scale, showing high scores for topic usefulness, coherence, and motivation.

The results, measured on a 7-point Likert scale, indicated high user satisfaction:

  • Relevance and Coherence: Users strongly agreed that the suggested topics were useful (Q1) and felt coherent with their prior chats (Q7).
  • Gap Identification: The system successfully identified "unknown unknowns"—concepts the user needed to know but hadn't explicitly asked about (Q5).
  • Motivation: There was high motivation to follow up on suggested topics (Q9) and a strong willingness to use the system daily (Q10).
  • Pacing Challenges: Some users found lessons occasionally shallow or redundant (Q2, Q6), highlighting the difficulty of perfectly pacing an automated curriculum.

Discussion and Future Work

LOOM demonstrates that the Learner Memory Graph serves as a powerful "shared workspace" between the AI and the user, providing a visual sense of progress. By grounding lessons in the user's own chat history, the system avoids a "generic textbook" feel, making learning feel like a natural continuation of past conversations.

However, proactive AI guidance requires a delicate balance. If suggestions are poorly timed, they can become intrusive. Future work will focus on increasing user agency—allowing users to manually edit the memory graph or explicitly start new learning tracks—and moving toward an "auditable" pipeline where structured plans are validated before content generation.

Summary updated

BibTeX
@misc{cui2025loom,
  author = {Justin Cui and Kevin Pu and Tovi Grossman},
  title = {{LOOM: Personalized Learning Informed by Daily LLM Conversations Toward Long-Term Mastery via a Dynamic Learner Memory Graph}},
  year = {2025},
  eprint = {2511.21037},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2511.21037},
}
Figure