TL;DR
This paper introduces InferDPT, a novel framework for privacy-preserving inference with black-box Large Language Models (LLMs) like ChatGPT. It uses a two-module approach: a perturbation module to add differential privacy to user prompts and an extraction module to reconstruct coherent text, achieving a strong balance between privacy and utility. Their proposed RANTEXT mechanism significantly improves privacy against common attacks compared to existing methods.
What Problem Does This Paper Address?
The widespread adoption of powerful LLMs like ChatGPT has brought immense utility but also significant privacy concerns. Users often input sensitive information into these models, risking data leakage or unauthorized collection. While differential privacy (DP) offers a solution, existing DP methods for LLM inference face practical challenges: they are often too slow, require too much communication, or degrade the quality of generated text too much, especially for open-ended generation tasks. Current approaches are also not robust enough against sophisticated attacks like embedding inversion. This paper aims to bridge this gap by providing a practical, efficient, and robust privacy-preserving inference framework for black-box LLMs.
Methods
InferDPT is built on two core modules:
- Perturbation Module: This module applies differential privacy to the user's input prompt. Instead of directly sending the raw prompt to the LLM, it first perturbs the text. The authors introduce RANTEXT, a novel DP mechanism that uses a "random adjacency list" for token perturbation. This differs from prior methods (like SANTEXT+ and CUSTEXT+) which used static lists. RANTEXT dynamically generates a list of potential replacement tokens for each original token, aiming for a better trade-off between semantic similarity (utility) and privacy. This module then sends the perturbed prompt to the black-box LLM.
- Extraction Module: The LLM returns a "perturbed generation" based on the perturbed prompt. This output might be less coherent due to the privacy perturbations. The extraction module uses a smaller, local language model to process this perturbed generation. Inspired by knowledge distillation, it extracts coherent and consistent text from the LLM's output and reconstructs it into a final, aligned generation that preserves the original intent while maintaining privacy.
Datasets
The authors evaluated their framework on several standard NLP datasets:
- CNN/Daily Mail: A common dataset for news summarization and text generation.
- Wikitext-103-v1: A benchmark dataset for language modeling.
- ArXiv Dataset: A collection of scientific papers.
- FinRED: A dataset for financial relation extraction.
- MedQA: A dataset for medical question answering.
These datasets are used for evaluating the utility (quality of generated text) of the InferDPT framework.
Key Insights and Ideas
- The "Random Adjacency List" is Key: The core innovation, RANTEXT, moves away from static lists of replacement tokens. By dynamically generating random adjacency lists, it offers a more flexible and robust perturbation, leading to better privacy guarantees and improved utility compared to fixed lists.
- Perturbed Generation as a "Hint": The perturbed generation from the LLM, while potentially noisy, isn't discarded. Instead, it acts as a valuable "hint" for the local extraction model. This allows the smaller local model to achieve generation quality comparable to the much larger remote LLM, a form of knowledge distillation.
- Black-Box LLMs are a Target: The framework is specifically designed for "black-box" LLMs (like ChatGPT), where the internal model architecture and parameters are not accessible. This makes it applicable to widely used commercial services.
- Adaptive Attacks Matter: The authors highlight that traditional DP attacks might not be sufficient. They propose and test an "adaptive attack" (GPT inference attack) that leverages the capabilities of the LLM itself to try and recover the original prompt, demonstrating RANTEXT's resilience.
Key Results
- Superior Privacy: RANTEXT significantly outperforms SANTEXT+ and CUSTEXT+ in privacy protection. For instance, with an epsilon (ε) of 6.0, RANTEXT achieved over 90% privacy protection against embedding inversion attacks, which is 3.35x higher than CUSTEXT+ and 0.58x higher than SANTEXT+.
- Comparable Utility: The text generation quality of InferDPT (using RANTEXT) is comparable to that of non-private GPT-4 across various metrics (diversity, MAUVE, coherence) on multiple datasets.
- Efficiency: While there's an overhead for perturbation and extraction, the overall inference time is practical, with the extraction module taking less than 4 seconds per inference.
- Robustness: InferDPT, particularly with RANTEXT, demonstrates strong privacy protection against BERT inference attacks, embedding inversion attacks, and GPT inference attacks.
Limitations and Future Work
- Local Model Requirement: InferDPT requires a local language model for the extraction module. This might be a limitation in extremely resource-constrained environments (e.g., smartwatches).
- MAUVE Score Gap: There's a slight gap in MAUVE scores compared to non-private GPT-4. The authors suggest that future work could focus on improving the extraction module's capabilities or developing more advanced DP perturbation mechanisms to better preserve semantic nuances.
- Prompt Optimization: Further research into optimizing the system prompt for the extraction module could enhance performance.
- Optimal Perturbation: Exploring more optimal perturbation mechanisms beyond the current exponential mechanism could further improve the privacy-utility trade-off.
Summary updated