TL;DR
This paper introduces DPED, a novel framework for training text embeddings with strong differential privacy guarantees. DPED leverages multi-layer knowledge distillation and a rare-word-aware aggregation strategy to significantly improve the utility of private embeddings compared to existing methods, especially for challenging language tasks.
What Problem Does This Paper Address?
Training machine learning models on sensitive text data, such as personal communications or medical records, poses significant privacy risks. Text embeddings, which capture the semantic meaning of words, are particularly vulnerable because they can inadvertently reveal unique linguistic fingerprints of individuals. Differential Privacy (DP) offers a robust mathematical framework to mitigate these risks by ensuring that the model's output doesn't reveal information about any single training example. However, applying DP to text embeddings is challenging due to the high dimensionality of language data and the uneven distribution of words (many rare words). Standard DP techniques, like DP-SGD, often lead to a substantial degradation in embedding quality (utility), making them impractical for many real-world applications. Existing teacher-student distillation methods, like PATE, have shown promise but struggle with the high-dimensional output spaces of embeddings and the issue of rare words.
Methods
DPED tackles this problem by combining several key innovations:
- Teacher-Student Distillation with Multi-Layer Noise Injection: Instead of just adding noise to the final output, DPED injects calibrated noise at multiple layers of the model. This involves training an ensemble of "teacher" models on disjoint subsets of the private data. These teachers then provide both their intermediate hidden representations and their final output predictions. A "student" model learns from these noisy teacher signals, effectively distilling knowledge across multiple stages.
- Rare-Word-Aware Aggregation: To address the privacy risks associated with infrequent words (which can be highly identifying), DPED introduces a novel aggregation mechanism. This mechanism dynamically identifies queries involving rare words and applies adaptive privacy measures, such as abstaining from releasing information or adding extra noise, to prevent privacy leakage while preserving utility for common words.
- Theoretical Privacy Guarantees: The paper provides a formal privacy analysis, proving that DPED satisfies $(\epsilon, \delta)$-differential privacy. This analysis leverages the moments accountant technique for tight privacy budget tracking across multiple queries and demonstrates how the multi-layer design and teacher partitioning contribute to stronger privacy guarantees.
Datasets
The paper evaluates DPED on several standard NLP tasks and datasets:
- WikiText-2: Used for word prediction (embedding training).
- IMDb reviews: Used for sentiment classification.
- WordSim-353: Used for word similarity evaluation.
- Google analogy completion: Used for analogy reasoning evaluation.
The authors do not introduce new benchmarks but focus on demonstrating DPED's effectiveness on existing ones.
Key Insights and Ideas
- Multi-layer distillation is crucial for utility: Injecting noise at intermediate layers, in addition to the final output, significantly improves the quality of learned embeddings compared to single-layer noise injection (like in PATE). This suggests that preserving richer, intermediate signals is key to maintaining semantic understanding under privacy constraints.
- Rare words are a major privacy bottleneck: The paper highlights that rare words pose a unique challenge for privacy. DPED's rare-word-aware aggregation is a critical component for handling these words effectively, preventing them from becoming privacy leaks.
- Abstaining on rare queries is a practical privacy amplification technique: The strategy of abstaining from releasing information when teacher consensus is low for rare words acts as a form of privacy amplification, effectively filtering out potentially identifying signals.
- DPED achieves better utility at lower privacy costs: The results consistently show that DPED can achieve a given level of utility (e.g., perplexity, accuracy) at a significantly lower privacy budget ($\epsilon$) compared to DP-SGD and PATE. This indicates a more favorable privacy-utility trade-off.
Key Results
- Improved Utility: At a privacy budget of $\epsilon \approx 1.0$, DPED achieved a perplexity of 210.8 on WikiText-2, outperforming DP-SGD (350.7) and PATE (290.4). For sentiment classification on IMDb, DPED achieved 79.5% accuracy, compared to 72.3% for DP-SGD.
- Comparable to Non-Private Performance: As $\epsilon$ increases to 4.0, DPED's performance (perplexity 130.7) closely approaches non-private performance (120.5).
- Better Privacy-Utility Trade-off: DPED at $\epsilon=2$ outperformed DP-SGD at $\epsilon=4$ in terms of utility, demonstrating its efficiency.
- Effective for Rare Words: DPED maintained significantly higher cosine similarity (0.42) to non-private embeddings for the rarest word quintile at $\epsilon=1$ compared to DP-SGD (0.18), showcasing its ability to preserve semantic quality for rare words.
- Computational Efficiency: DPED exhibits linear scaling with dataset size and teacher count, with modest memory requirements, making it practical for deployment.
Limitations and Future Work
- Scalability to Larger Models: The authors acknowledge that scaling DPED to larger models like BERT or LLMs is an important future direction. This scaling may introduce new privacy-utility trade-offs and require more efficient aggregation and noise calibration techniques.
- Computational Cost: While DPED is efficient, training an ensemble of teacher models inherently increases computational cost compared to single-model approaches.
- Hyperparameter Tuning: The rare-word threshold ($r$) involves a trade-off between privacy and coverage, requiring careful tuning.
Future work could explore adaptations of DPED for larger Transformer architectures and investigate more advanced privacy-preserving language representation techniques.
Summary updated