ImmuniFraug: A Metacognitive Intervention Anti-Fraud Approach to Enhance Undergraduate Students' Cyber Fraud Awareness

Xu Yuan, Jiajun Wang, Huanchen Wang, Qian Wan, Siying Hu

TL;DR

ImmuniFraug is a novel, LLM-powered training framework that uses immersive, multimodal simulations (text, voice, and avatars) to teach undergraduate students how to spot and resist cyber fraud. By shifting from passive "tell-only" education to active metacognitive engagement, the system significantly improves fraud awareness and psychological resilience, even in students who have already received extensive traditional training.

What Problem Does This Paper Address?

Cyber fraud is a massive global threat, accounting for over half of all criminal cases in China. Undergraduate students are a primary target, yet traditional anti-fraud education—think boring lectures, pamphlets, and static videos—is failing them. These "passive" methods suffer from low engagement and poor knowledge retention.

Furthermore, most existing research on cybersecurity awareness focuses narrowly on email phishing. In reality, scammers use a diverse array of social engineering tactics across platforms like WeChat and phone calls, involving romance scams, fake investments, and impersonation. The authors identify a critical need for metacognitive interventions: training that doesn't just give students facts, but forces them to monitor and regulate their own decision-making processes under the high-pressure, emotional conditions of a real scam.

Methods

The researchers developed ImmuniFraug, a system built on the Doubao.AI platform. Unlike static training, ImmuniFraug uses a Large Language Model (LLM) to act as a "first-person fraudster" in a dynamic, branching narrative.

Key technical features include:

  • Multimodal Interaction: The system integrates text, synthesized voice, and visual avatars tailored to specific fraud archetypes (e.g., a stern "police officer" or a friendly "peer").
  • Adaptive Deception: The LLM doesn't follow a fixed script; it responds adaptively to the user’s replies, escalating pressure or changing tactics just like a real social engineer.
  • Five-Stage Simulation Structure: Every session includes a disclaimer, interactive dialogue, branching response options, termination conditions, and a critical post-simulation debrief.
  • Metacognitive Debriefing: Grounded in Protection Motivation Theory (PMT), the debriefing phase provides a behavioral analysis of the interaction, helping users reflect on their vulnerability, the scammer's trust-building techniques, and their own self-efficacy.

Datasets

The study did not involve training a new model from scratch but rather utilized the pre-trained Doubao LLM. However, the researchers curated a specific "scenario library" based on:

  • The Ten Most Prevalent Fraud Types: Sourced from official data provided by the Ministry of Public Security of China (e.g., Online Romance Scams, Fake Logistics Agents, and "Brushing" scams).
  • User Study Data: The system was evaluated through a controlled study involving 846 Chinese undergraduate students. 391 participants interacted with ImmuniFraug (experimental group), while 455 read traditional text-based anti-fraud materials (control group).

Key Insights and Ideas

  • Immersion as a Learning Driver: The researchers found that "narrative transportation"—the feeling of being "lost" in a story—is a powerful pedagogical tool. When students are immersed in a voice-driven call, they process red flags more deeply than when reading a list of tips.
  • The "Teachable Moment": By simulating the high-stress, time-pressured environment of a scam, ImmuniFraug creates a state of affective arousal. Following this immediately with a structured debrief allows the student to convert that emotional experience into a durable mental heuristic.
  • Challenging Overconfidence: Many students entered the study believing they were "unscammable." The simulation often triggered cognitive dissonance when these students found themselves momentarily falling for the LLM's tactics, which led to much deeper reflection and learning.
  • Metacognition > Declarative Knowledge: The results suggest that knowing what a scam is (declarative knowledge) is less important than practicing how to think while being pressured (metacognitive skill).

Key Results

The authors used Linear Mixed-Effects Modeling (LMEM) to isolate the intervention's impact while controlling for participants' prior backgrounds.

  • Significant Awareness Boost: ImmuniFraug significantly improved total fraud awareness scores ($p = 0.026$) compared to the control group.
  • High Engagement: Participants reported high levels of narrative immersion ($M = 56.95/77$) and exceptionally strong intentions to adopt protective behaviors ($M = 4.54/5$).
  • Incremental Value: Crucially, the system provided measurable benefits even for the 92% of participants who had already received prior anti-fraud education, proving that immersive simulation can go where "tell-only" training cannot.
  • Qualitative Realism: In interviews, students specifically praised the "tone" and "impatient" nature of the AI's voice, noting it felt much more authentic than robotic, polite scripts.

Limitations and Future Work

  • Vocal Realism: While effective, the synthetic voices still felt "mechanical" to some users. Future iterations aim to improve prosody and emotional nuance.
  • Longitudinal Scams: Current LLM token limits make it difficult to simulate "long-con" scams (like romance scams) that unfold over days or weeks.
  • Multimedia Gaps: The current version lacks "sandboxed" malicious artifacts like fake websites, doctored screenshots, or phishing links, which are common in modern multimodal deception.
  • Measurement Validity: The study relied on self-reported scales and test scores. The authors suggest that future work should include in-situ behavioral metrics (e.g., how long it takes a user to "hang up" or seek help during a live simulation).

Summary updated

Abstract

Cyber fraud now constitutes over half of criminal cases in China, with undergraduate students experiencing a disproportionate rise in victimization. Traditional anti-fraud training remains predominantly passive, yielding limited engagement and retention. This paper introduces ImmuniFraug, a Large Language Model (LLM)-based metacognitive intervention that delivers immersive, multimodal fraud simulations integrating text, voice, and visual avatars across ten prevalent fraud types. Each scenario is designed to replicate real-world persuasion tactics and psychological pressure, while post-interaction debriefs provide grounded feedback in protection motivation theory and reflective prompts to reinforce learning. In a controlled study with 846 Chinese undergraduates, ImmuniFraug was compared to official text-based materials. Linear Mixed-Effects Modeling (LMEM) reveals that the interactive intervention significantly improved fraud awareness (p = 0.026), successfully providing incremental learning value even when controlling for participants' extensive prior exposure to anti-fraud education, alongside high narrative immersion (M = 56.95/77). Thematic analysis of interviews revealed key effectiveness factors: perceived realism, adaptive deception, enforced time pressure, emotional manipulation awareness, and enhanced self-efficacy. Findings demonstrate that by shifting the focus from passive knowledge acquisition to active metacognitive engagement, LLM-based simulations offer a scalable and ecologically valid new paradigm for anti-fraud training and fostering fraud resilience.
BibTeX
@misc{yuan2026immunifraug,
  author = {Xu Yuan and Jiajun Wang and Huanchen Wang and Qian Wan and Siying Hu},
  title = {{ImmuniFraug: A Metacognitive Intervention Anti-Fraud Approach to Enhance Undergraduate Students' Cyber Fraud Awareness}},
  year = {2026},
  eprint = {2601.06774},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2601.06774},
}
Figure