Abstract
The rise of end-user applications powered by large language models (LLMs), including both conversational interfaces and add-ons to existing graphical user interfaces (GUIs), introduces new privacy challenges. However, many users remain unaware of the risks. This paper explores methods to increase user awareness of privacy risks associated with LLMs in end-user applications. We conducted five co-design workshops to uncover user privacy concerns and their demand for contextual privacy information within LLMs. Based on these insights, we developed CLEAR (Contextual LLM-Empowered Privacy Policy Analysis and Risk Generation), a just-in-time contextual assistant designed to help users identify sensitive information, summarize relevant privacy policies, and highlight potential risks when sharing information with LLMs. We evaluated the usability and usefulness of CLEAR across two example domains: ChatGPT and the Gemini plugin in Gmail. Our findings demonstrated that CLEAR is easy to use and improves users' understanding of data practices and privacy risks. We also discussed LLM's duality in posing and mitigating privacy risks, offering design and policy implications.
Introduction
As Large Language Models (LLMs) become ubiquitous, integrated into everything from standalone chat interfaces to productivity plugins like email assistants, users are sharing unprecedented amounts of personal data. However, a phenomenon known as "privacy resignation" often takes hold: users recognize the risks but feel overwhelmed by lengthy, legalistic privacy policies and find technical risks—such as data memorization or human review—too abstract to grasp.
The research identifies three primary LLM-specific privacy risks:
- Memorization Risks: Models may "remember" sensitive input data and inadvertently leak it to other users in future outputs.
- Inference Risks: LLMs can deduce sensitive personal attributes from seemingly benign text.
- Disclosure Risks: The conversational nature of LLMs fosters a false sense of trust, leading to oversharing of information that may be accessed by human reviewers or third-party vendors.
To address these, the authors developed CLEAR (Contextual LLM-Empowered Privacy Policy Analysis and Risk Generation), a "just-in-time" privacy assistant designed to provide contextual, easy-to-digest privacy information at the exact moment of interaction.
System Design and Methodology
The CLEAR system was developed using a human-centered design approach, informed by five co-design workshops. It operates through three main components:
1. Context Detection
CLEAR scans user inputs in real-time for personally identifiable information (PII), such as email addresses, phone numbers, and physical addresses, using the Microsoft Presidio library.
2. Contextual Policy Extraction
Rather than presenting a full privacy policy, CLEAR identifies and summarizes only the sections relevant to the specific data types detected in the user's prompt. This ensures that the information provided is immediately relevant to the user's current task.
3. Risk Generation
CLEAR utilizes a structured privacy knowledge base built from over ten foundational academic papers on AI privacy. Using Chain-of-Thought (CoT) reasoning, the system applies this knowledge to the user's specific context, explaining how their data might be misused (e.g., "Human reviewers may read this to improve the model").
Evaluation and Key Insights
The researchers evaluated CLEAR through two user studies involving 28 participants, focusing on ChatGPT and the Gemini plugin for Gmail.
High Usability and Behavioral Change
CLEAR received high System Usability Scale (SUS) scores, indicating it is well-integrated and easy to learn. More importantly, it successfully nudged users toward safer behaviors. In the ChatGPT study, 12 out of 13 participants expressed an intention to change their behavior—such as deleting sensitive info or using "fake" data—after seeing CLEAR's alerts.
Improved Privacy Literacy
Participants reported a significantly clearer understanding of data practices. Many were surprised to learn that third-party affiliates or human reviewers might access their prompts, moving beyond the vague notion of "data processing."
The "Duality of AI"
A key finding was the "Duality of AI": while users are wary of AI-driven privacy risks, they also view AI as the most capable tool for protection. Participants specifically expressed a desire for "AI supervisors" to monitor their data traffic in real-time.
Limitations and Future Directions
While CLEAR shows promise, several areas for future work remain:
- Modality Expansion: Current detection is limited to text. Future iterations must address sensitive data in images, PDFs, and voice interactions.
- Detection Accuracy: Pattern-based detection may miss subtle identifiers that LLMs can still use for inference.
- Longitudinal Effects: Future research should investigate "alert fatigue" in long-term, real-world deployments.
- Edge Computing: To further enhance privacy, the authors plan to move PII detection and risk generation to "on-device" small language models (SLMs), ensuring sensitive data analysis occurs locally.
Summary updated