TL;DR
This paper introduces novel methods for protecting user privacy when interacting with large language models (LLMs) by manipulating input tokens. They propose techniques that aim to preserve downstream task accuracy while making it difficult for attackers to reconstruct the original private information, even with access to the model's architecture.
What Problem Does This Paper Address?
Large language models (LLMs) are increasingly used as cloud services, raising significant privacy concerns. When users send private or sensitive information to these remote services, there's a risk of data exposure through various attacks like model inversion, re-identification, or feature memorization. Existing privacy-preserving techniques often rely on adding noise to embeddings, which can degrade performance, or require partial access to the LLM, which isn't always feasible. This paper addresses the challenge of protecting privacy at the token level, without needing access to the LLM's internal parameters, and aims to maintain high accuracy on downstream tasks.
Methods
The paper proposes two main categories of token manipulation techniques:
- Lossy Mapping (Naïve Token Substitution): This approach involves partitioning the vocabulary into pairs or triplets of tokens and mapping them to a single representative token. The idea is to create an ambiguous mapping that is not easily invertible. They explore "random mapping" (randomly pairing tokens) and "frequency mapping" (pairing high-frequency tokens with low-frequency ones, or vice-versa, using the representative token to influence privacy vs. performance).
- Contextualized Manipulation (STENCIL): This more advanced technique leverages the surrounding tokens to create a new, "quasi-embedding" for the target token. The STENCIL method retrieves embedding vectors of neighboring tokens, applies a weighted transformation, and then finds the closest token in the embedding space to the resulting quasi-embedding. This aims to retain contextual information for better task performance while obfuscating the original token. They also introduce "punctuated STENCIL" (STENCILp) which excludes the target token from the quasi-embedding calculation to further enhance privacy at the cost of some accuracy.
Datasets
The authors evaluate their methods on the following standard datasets:
- SST2 (Stanford Sentiment Treebank): A dataset for sentiment analysis.
- IMDb: Another dataset for sentiment analysis.
- QNLI (Question Natural Language Inference): A dataset for natural language inference, used with an encoder-decoder model.
They use RoBERTa as their base model for sequence classification tasks and T5-small for the QNLI task.
Key Insights and Ideas
- Token-level privacy is feasible without model access: The core idea that privacy can be enhanced by manipulating input tokens, without needing to access or retrain the LLM itself, is a significant contribution.
- Trade-off between privacy and performance is inherent: The paper clearly demonstrates that simpler token manipulation methods (like naïve mapping) can be easily reversed by attackers, while more sophisticated methods (like STENCIL) offer better privacy but might slightly impact downstream task accuracy.
- Context matters for privacy: The STENCIL method highlights the importance of contextual information. By creating context-aware tokens, they can achieve better performance than methods that simply add noise without considering the surrounding words.
- Attacker sophistication influences effectiveness: The paper explores different attacker models, from brute-force to heuristic-based (beam search), showing that the effectiveness of privacy techniques depends on the attacker's capabilities and knowledge.
Key Results
- Naïve mapping: While easy to implement, these methods (e.g., 2-Random, High-freq) showed significant drops in accuracy (e.g., from 94.5% to 60-75% on SST2/IMDb) and were found to be easily reversible by a knowledgeable attacker.
- STENCIL: Achieved better performance than naïve methods, with accuracy drops of only 1-5% on SST2 and IMDb for the TEST case. For the ALL case (training and inference), STENCIL performed comparably to or better than noise-based methods.
- Resilience against attackers: The paper reports metrics like Mean Reciprocal Rank (MRR) and Precision@5 (Pr@5) for attacker reconstruction. Naïve methods had high Pr@5 (meaning attackers could easily reconstruct), while STENCIL significantly reduced this, especially with longer sequences (IMDb vs. SST2). STENCILp offered the highest privacy but at a greater performance cost.
- Computational cost: The average computational cost per token for STENCIL was reported as 0.005 seconds on specific hardware, which is considered efficient.
Limitations and Future Work
- Empirical evaluation: The privacy achieved was demonstrated empirically under one specific attacking scenario. Further testing and mathematical proofs are suggested to fully understand the extent of privacy.
- Sentence length preservation: The current methods maintain sentence length, which could still be a privacy leak for authors who prefer specific sentence lengths. Future work could explore reducing token count using stride parameters in STENCIL.
- Language and task generalization: Experiments were limited to English classification tasks. Future research aims to explore effectiveness in generative tasks, across different languages, and in multilingual settings.
- Improving STENCIL: Exploring different weighted functions for STENCIL, potentially trained models, could further enhance accuracy and privacy.
- Linguistic properties: Investigating how to better preserve linguistic properties like grammar and readability in privacy techniques is a key future direction.
Summary updated