TL;DR
Privacy is not a "one-size-fits-all" property, yet research and industry are drifting toward unified, universal privacy solutions for Large Language Models (LLMs). This paper argues that such unified frameworks are fundamentally flawed due to irreconcilable tensions between protection, performance, and accessibility, and calls for a shift toward context-specific privacy evaluations and application-specific deployments.
What Problem Does This Paper Address?
As LLMs become general-purpose tools used for everything from medical advice to coding assistance, users often interact with a single system across vastly different privacy contexts. This has led to a "drift" in both academia and industry toward creating a single, plug-and-play privacy framework that works everywhere.
The authors argue that this trend overlooks a core tenet of privacy research: privacy is inherently contextual. A "universal" privacy tool often provides a false sense of security. For example, a tool that works for a casual user might fail a corporate employee handling trade secrets. By trying to be everything for everyone, these frameworks fail to address the specific threat models and utility requirements of individual use cases.
Methods
The authors conduct a systematic study to document this "drift" and analyze the landscape of prompt privacy:
- Literature Review: They analyzed 31 representative academic papers on prompt privacy, annotating them for privacy targets, specific use cases, and whether they discussed limitations.
- Tool Audit: They reviewed 11 publicly available prompt privacy tools (like Ollama, PrivacyGPT, and LLMGuard) to see how they are marketed and what techniques they employ.
- Framework Analysis: They evaluated the four dominant categories of privacy techniques—Sanitization, Differential Privacy (DP), Cryptography (HE/MPC), and Contextual Integrity (CI)—against five desirable properties:
- Quantifiable: Protection Guarantees, Performance, and Efficiency.
- Qualitative: Domain Adaptability and User Accessibility.
Datasets
This is a position paper and systematic review; it does not introduce a new training dataset. Instead, it uses a curated set of 31 academic papers and 11 commercial/open-source tools as its primary data for analysis. The authors also introduce three "User Profiles" (Avery the marathoner, Sasha the corporate employee, and Tao the writing assistant) as a qualitative benchmark to illustrate how different contexts require different privacy solutions.
Key Insights and Ideas
- The Evidence of "Drift": The authors found that over 75% of academic papers and 100% of analyzed tools target "General Purpose" use cases rather than specific applications. Furthermore, more than half of the academic papers failed to discuss any limitations or failure cases.
- The Vector Grounding Problem: There is a fundamental tension between the language space (where users understand meaning) and the embedding space (where models operate). Techniques that provide formal guarantees (like DP or Cryptography) often work in the embedding space, leading to a "vector grounding problem" where protecting the representation either destroys utility or fails to protect the actual semantic meaning.
- The Accessibility Paradox: Frameworks that are easy for humans to understand and use (like Sanitization) generally lack rigorous protection guarantees. Conversely, frameworks with strong guarantees (like Cryptography) are often computationally prohibitive or require high "privacy literacy" to configure correctly.
- Sanitization is a "Misplaced Sense of Safety": While popular in tools because it's fast and easy to understand, lexical sanitization (removing PII) is easily bypassed by black-box attacks that can reconstruct sensitive info from residual semantic cues.
Key Results
- Academic Drift: 50%+ of papers target the entire prompt or its embeddings as a single unit, ignoring that different parts of a prompt have different sensitivity levels depending on the context.
- Tool Drift: All 11 tools surveyed marketed themselves as general-purpose solutions. Interestingly, while Differential Privacy is a hot topic in research, it is almost entirely absent from real-world tools, which favor Sanitization or Local Deployment.
- Irreconcilable Tensions: The paper demonstrates that no single framework can simultaneously satisfy all five desirable properties. For instance, Sasha (the corporate user) needs high accuracy and strong guarantees, which points toward Cryptography, but the high cost makes it unfeasible for Tao (the casual user) who just wants speed and basic PII removal.
Limitations and Future Work
The authors acknowledge that "Unified Frameworks" might be seen as necessary stepping stones for a young field. However, they argue that the rapid commercialization of these tools makes an immediate intervention necessary.
Future Directions: 1. Context-Specific Evaluations: Instead of testing privacy tools on generic benchmarks, researchers should audit them against specific threat models (e.g., "Does this protect against targeted ads?"). 2. Application-Specific Deployment: Providers should move away from one giant "everything" model and toward siloed deployments (like "ChatGPT Health") where privacy guardrails are tailored to the data's sensitivity. 3. Privacy Literacy: There is a critical need to improve user awareness so they can choose the right tool for their specific situation and understand when a tool is likely to fail.
Summary updated