From Superficial Outputs to Superficial Learning: Risks of Large Language Models in Education

Iris Delikoura, Yi R. Fung, Pan HuiarXiv 2025 · DOI · arXiv

Introduction

The current discourse surrounding Large Language Models (LLMs) in education is frequently characterized by "techno-optimism," emphasizing personalized tutoring, administrative efficiency, and enhanced student productivity. However, as these tools move from experimental novelties to classroom staples, empirical evidence regarding their risks has remained fragmented across the fields of computer science, psychology, and pedagogy.

This paper provides a systematic review of 70 empirical studies to map the multifaceted risks of LLMs. It argues that the primary concern is shifting from superficial outputs—where models generate plausible but hollow text—to superficial learning, where students mimic that hollow fluency without achieving deep cognitive engagement. By synthesizing recent research, the authors introduce the LLM-Risk Adapted Learning Model (IMO) to illustrate how technical flaws cascade into long-term pedagogical harms.

Methodology

The researchers conducted a PRISMA-guided systematic review, screening 3,453 records from major databases, including ACM Digital Library and IEEE Xplore. The final synthesis included 70 peer-reviewed empirical studies published between 2023 and 2025. These studies primarily evaluated the performance and impact of models such as GPT-3.5 and GPT-4, with emerging data on Llama 2, Gemini, and Claude.

To organize the findings, the authors developed a taxonomy of LLM applications in education:

  1. Operational Effectiveness: Feedback generation and teaching capability.
  2. Personalized Applications: Grading, content generation, and student progress predictions.
  3. Interactive Learning Tools: Direct student engagement and cognitive impact.

The IMO Model: How Risks Compound

A central contribution of this research is the Interaction → Monitoring → Outcome (IMO) model. This framework demonstrates that risks are not isolated incidents but cumulative processes:

  • Interaction: A technical flaw, such as a hallucination or biased output, occurs during the initial prompt-response cycle.
  • Monitoring: The student fails to critically evaluate the output, leading to the internalization of biased beliefs or "metacognitive laziness."
  • Outcome: The cumulative effect results in "memory erosion," reduced student agency, and a measurable decline in creativity.

Key Pedagogical Risks

The Cognitive Comfort Zone

LLMs are highly effective at reducing "extraneous load"—the logistical effort of searching for information or formatting text. However, the review finds that they simultaneously reduce "germane load," which is the productive mental effort required to encode new knowledge into long-term memory. By staying within this "cognitive comfort zone," students may complete tasks faster without actually learning the underlying concepts.

Verification Drift and Memory Erosion

Even when students are aware of AI limitations, they exhibit "verification drift." Over time, the authoritative and human-like tone of LLMs leads students to stop critically evaluating outputs. This lack of engagement has physical consequences; one cited study found that students using LLMs had significantly more difficulty recalling specific details or quotes from their own essays compared to those who wrote manually. This "memory erosion" suggests that the student is becoming a curator of AI content rather than an author of their own thoughts.

The Performance Paradox

One of the most striking findings is the "Performance Paradox." Several studies demonstrated that while LLM-assisted students achieved higher submission rates and better assignment scores, they consistently under-performed on post-knowledge tests. They were successfully "completing" the work while failing to "learn" the material.

Neural and Systemic Impacts

Beyond cognitive habits, the review highlights physiological and systemic risks:

  • Neural Connectivity: EEG data from a specific study showed a systematic decrease in brain connectivity—particularly in regions associated with semantic integration and creative ideation—when students used LLMs compared to traditional writing or searching.
  • Algorithmic Bias: Risks extend to administrative functions. Predictive systems showed significant bias against marginalized groups (including LGBTQ+ and certain ethnicities), who were statistically less likely to receive recommendations for STEM fields.
  • Detection Failure: The review confirms that educators cannot reliably distinguish between student-written and AI-generated work, often misattributing low-quality AI text to human students, which complicates assessment and feedback.

Limitations and Future Directions

The authors acknowledge several limitations in the current body of research. Most studies are short-term interventions, and there is a critical need for longitudinal research to understand the sustained impact of LLMs on social and cognitive development. Furthermore, the "moving target" problem persists: LLM capabilities evolve faster than the peer-review process, meaning many studies reflect older models like GPT-3.5.

To mitigate these risks, the paper suggests moving toward "lightweight verification prompts" and usage dashboards. These tools would make a student’s reliance patterns visible, encouraging a "gradual tapering" of AI support to ensure that LLMs remain a scaffold for learning rather than a substitute for thought.

Summary updated

BibTeX
@misc{delikoura2025from,
  author = {Iris Delikoura and Yi R. Fung and Pan Hui},
  title = {{From Superficial Outputs to Superficial Learning: Risks of Large Language Models in Education}},
  year = {2025},
  eprint = {2509.21972},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.21972},
}
Figure