Conversational AI Powered by Large Language Models Amplifies False Memories in Witness Interviews

Samantha W. T. Chan, Pat Pataranutaporn, Aditya Suri, Wazeer Zulfikar, Pattie Maes, Elizabeth F. Loftus

Introduction

Human memory is not a perfect recording of past events; rather, it is a reconstructive process highly susceptible to external suggestion. This phenomenon, known as the "misinformation effect," has been studied for decades, demonstrating how leading questions in surveys or human interviews can inadvertently plant false memories. As Large Language Models (LLMs) like GPT-4 become integrated into sensitive domains—such as legal aid, therapy, and law enforcement—understanding their impact on human cognition is critical.

This study investigates whether the interactive and often "sycophantic" nature of generative AI chatbots amplifies the risk of inducing false memories. Unlike static surveys, generative AI can provide immediate positive reinforcement and elaborate on fabricated details, potentially creating a powerful confirmation bias loop that distorts a witness's recollection.

Methodology

To evaluate the impact of different interaction modalities on memory, researchers conducted a two-phase study involving 200 participants recruited via Prolific.

The experimental workflow, from initial stimulus to long-term memory assessment.

The study followed a rigorous pipeline:

  1. Stimulus Exposure: All participants watched a 2.5-minute silent CCTV video of a real-world armed robbery at the Sayford Supermarket.
  2. Intervention Phase: Participants were randomly assigned to one of four conditions:
  3. Control: No interaction following the video.
  4. Survey-based: A static form with 25 questions, including 5 "critical" misleading questions (e.g., asking about a getaway car when the robbers arrived on foot).
  5. Pre-scripted Chatbot: A basic chat interface delivering the same questions as the survey.
  6. Generative Chatbot: Powered by GPT-4, this bot provided conversational reinforcement. If a participant accepted a misleading premise, the bot would validate the error with statements like, "Your answer is correct. There was indeed a security camera... this detail is significant."
  7. Assessment: Memory was tested immediately following the intervention and again one week later to measure the persistence of any induced false memories.

Results and Key Insights

The findings reveal a significant "AI-induced misinformation effect." The generative chatbot was remarkably effective at misleading participants, with a 36.4% success rate in inducing false responses during the initial interaction.

Comparison of false memory induction and participant confidence across experimental conditions.

Quantitatively, the generative AI condition induced three times as many immediate false memories as the control group and 1.7 times more than the traditional survey method. Beyond the quantity of errors, the quality of the false memories was notable:

  • High Confidence: Participants in the generative AI group were nearly twice as confident in their false memories compared to the control group.
  • Persistence: While false memories often fade over time in traditional studies, those induced by the generative AI remained stable after one week. The researchers hypothesize that the engaging, elaborative nature of the AI conversation led to stronger initial encoding of the misinformation.
  • Memory Integrity: Crucially, the induction of false memories did not significantly degrade "true" memories of actual events, suggesting the effect is highly targeted toward the misleading information provided by the AI.

Discussion and Future Implications

The study highlights a "sycophancy trap" where LLMs, designed to be helpful and agreeable, inadvertently cement human errors through validation. Interestingly, general familiarity with AI did not protect participants; those who trusted technology but lacked specific experience with conversational quirks were often the most vulnerable.

As AI technology moves toward multi-modal capabilities—including the generation of deepfake images and video—the potential to "show" witnesses fabricated evidence poses an even greater risk to memory integrity. These findings underscore the urgent need for protective measures, such as explicit warnings or UI designs that promote skepticism rather than blind agreement, when AI is used in high-stakes investigative contexts.

Summary updated

Abstract

This study examines the impact of AI on human false memories -- recollections of events that did not occur or deviate from actual occurrences. It explores false memory induction through suggestive questioning in Human-AI interactions, simulating crime witness interviews. Four conditions were tested: control, survey-based, pre-scripted chatbot, and generative chatbot using a large language model (LLM). Participants (N=200) watched a crime video, then interacted with their assigned AI interviewer or survey, answering questions including five misleading ones. False memories were assessed immediately and after one week. Results show the generative chatbot condition significantly increased false memory formation, inducing over 3 times more immediate false memories than the control and 1.7 times more than the survey method. 36.4% of users' responses to the generative chatbot were misled through the interaction. After one week, the number of false memories induced by generative chatbots remained constant. However, confidence in these false memories remained higher than the control after one week. Moderating factors were explored: users who were less familiar with chatbots but more familiar with AI technology, and more interested in crime investigations, were more susceptible to false memories. These findings highlight the potential risks of using advanced AI in sensitive contexts, like police interviews, emphasizing the need for ethical considerations.
BibTeX
@inproceedings{chan2024conversational,
  author = {Samantha W. T. Chan and Pat Pataranutaporn and Aditya Suri and Wazeer Zulfikar and Pattie Maes and Elizabeth F. Loftus},
  title = {{Conversational AI Powered by Large Language Models Amplifies False Memories in Witness Interviews}},
  booktitle = {CrimRxiv},
  year = {2024},
  doi = {10.21428/cb6ab371.6ae390a8},
}
Figure