Conversational AI Powered by Large Language Models Amplifies False Memories in Witness Interviews

Samantha Chan, Pat Pataranutaporn, Aditya Suri, Wazeer Zulfikar, Pattie Maes, Elizabeth F. Loftus

Abstract

This study investigates how Large Language Models (LLMs) influence the formation of human false memories—recollections of events that either never occurred or differ from reality. Through a simulated crime witness interview paradigm, researchers compared four conditions: a control group, a survey-based interview, a pre-scripted chatbot, and a generative LLM-powered chatbot. Results from 200 participants revealed that generative AI significantly amplifies the "misinformation effect." The generative chatbot induced three times more immediate false memories than the control and 1.7 times more than traditional surveys. Notably, these false memories persisted over a week, with participants maintaining higher confidence in their distorted recollections compared to the control group. These findings suggest that the conversational nature of LLMs poses unique risks in sensitive investigative contexts.

Introduction

Human memory is not a recording device; it is a reconstructive process. As pioneered by researchers like Elizabeth Loftus, the "misinformation effect" demonstrates that the way questions are phrased can fundamentally alter a person's recollection of an event. As Large Language Models (LLMs) are increasingly proposed for use in professional workflows—including legal discovery, social work, and preliminary police interviews—it is critical to understand how these highly persuasive, conversational agents affect human memory. This paper investigates whether the fluid, rapport-building nature of generative AI makes it a more potent source of memory contamination than static digital tools.

Methodology

The researchers employed a controlled experimental design with 200 participants to test the impact of different interview modalities on memory accuracy. The study followed a four-stage process:

  1. Stimulus: Participants viewed a video of a crime to establish a ground-truth memory.
  2. Intervention: Participants were randomly assigned to one of four conditions:
  3. Control: No interview or interaction.
  4. Survey-based: A static set of non-interactive questions.
  5. Pre-scripted Chatbot: A digital assistant following a rigid, predetermined dialogue tree.
  6. Generative Chatbot: An LLM-powered interviewer capable of dynamic, adaptive, and conversational responses.
  7. The Misinformation Phase: In all conditions except the control, the interviewer (or survey) included five misleading questions designed to suggest false details about the crime video.
  8. Assessment: Memory accuracy and participant confidence were measured immediately following the intervention and again after a one-week delay.

Key Findings

The Efficacy of Generative Suggestion

The generative chatbot proved to be the most effective medium for inducing false memories. It induced over three times more immediate false memories than the control condition and was 1.7 times more effective than the traditional survey method. Specifically, 36.4% of the misleading prompts delivered by the generative AI successfully resulted in the participant adopting the false information into their account of the event.

Persistence and Confidence

A critical discovery was the "stickiness" of AI-induced errors. While human memory naturally fades over time, the false memories planted by the generative chatbot remained constant after a seven-day interval. Furthermore, participants in the generative AI group reported significantly higher confidence in these false memories after one week compared to the control group. This suggests that the AI does not merely provide incorrect information; it integrates that information into the user’s long-term memory with a high degree of subjective certainty.

The "Rapport Trap"

The study suggests that the LLM’s ability to adapt to a user’s conversational style creates a sense of social presence. This rapport likely lowers the participant's critical filters, making them more susceptible to subtle suggestions embedded in a "natural" conversation. Unlike a static survey, which feels like a test, the generative chatbot feels like a dialogue, which may lead users to trust the information provided by the AI more readily.

Moderating Factors of Susceptibility

The research identified specific user profiles that were more vulnerable to AI-driven memory distortion. Interestingly, participants who were generally familiar with AI technology but had little direct experience with chatbots were the most susceptible. This suggests a "danger zone" where a general trust in technological capability, combined with a lack of specific "bot-literacy" (the ability to recognize the fallibility of conversational agents), creates a significant vulnerability to suggestion. Additionally, participants with a high interest in crime investigations were found to be more susceptible to the misinformation effect in this specific context.

Implications and Ethical Considerations

These findings serve as a vital warning for the deployment of AI in legal and investigative settings. If an AI interviewer—whether through design or through unintended "hallucination"—suggests a detail that did not occur, it could permanently contaminate a witness's testimony, making it impossible to retrieve the original, untainted memory.

The study highlights the need for strict ethical guardrails to prevent the use of generative AI in contexts where memory integrity is paramount until robust safeguards against suggestive questioning are developed. As we move toward a future where AI-human interaction is commonplace, understanding the cognitive risks of these persuasive systems is as important as harnessing their capabilities.

Summary updated

Abstract

This study examines the impact of AI on human false memories -- recollections of events that did not occur or deviate from actual occurrences. It explores false memory induction through suggestive questioning in Human-AI interactions, simulating crime witness interviews. Four conditions were tested: control, survey-based, pre-scripted chatbot, and generative chatbot using a large language model (LLM). Participants (N=200) watched a crime video, then interacted with their assigned AI interviewer or survey, answering questions including five misleading ones. False memories were assessed immediately and after one week. Results show the generative chatbot condition significantly increased false memory formation, inducing over 3 times more immediate false memories than the control and 1.7 times more than the survey method. 36.4% of users' responses to the generative chatbot were misled through the interaction. After one week, the number of false memories induced by generative chatbots remained constant. However, confidence in these false memories remained higher than the control after one week. Moderating factors were explored: users who were less familiar with chatbots but more familiar with AI technology, and more interested in crime investigations, were more susceptible to false memories. These findings highlight the potential risks of using advanced AI in sensitive contexts, like police interviews, emphasizing the need for ethical considerations.
BibTeX
@inproceedings{chan2024conversational,
  author = {Samantha Chan and Pat Pataranutaporn and Aditya Suri and Wazeer Zulfikar and Pattie Maes and Elizabeth F. Loftus},
  title = {{Conversational AI Powered by Large Language Models Amplifies False Memories in Witness Interviews}},
  booktitle = {CrimRxiv},
  year = {2024},
  doi = {10.21428/cb6ab371.6ae390a8},
}
Figure