Deep learning models for NLP often "over-learn," capturing sensitive author attributes—such as age or gender—within hidden vector representations. This paper introduces Differentially Private Neural Representation (DPNR), a framework that provides formal mathematical privacy guarantees for these representations using Local Differential Privacy (LDP) and word dropout, while simultaneously improving model fairness across different demographic groups.
The Privacy Challenge in NLP
When using cloud-based NLP services for tasks like sentiment analysis, user devices typically send a dense vector representation to a server. While these vectors appear abstract, they are often "leaky." An attacker eavesdropping on the representation can frequently reconstruct sensitive information about the user, such as their gender, age, or specific named entities, with high accuracy.
Previous mitigation strategies relied heavily on adversarial training, where a model is incentivized to ignore private features. However, these methods only offer empirical improvements without formal safety guarantees. DPNR addresses this by providing a formal privacy guarantee through Differential Privacy that maintains the model's utility for its primary task.
The DPNR Framework
The authors propose a three-stage process to secure neural representations:
- Word Dropout: Before processing, specific words are randomly masked. The authors prove (Theorem 2) that this "nullification" tightens the privacy bound, enabling a superior privacy-utility trade-off.
- Laplace Mechanism: After a feature extractor (such as BERT) generates a representation, DPNR adds calibrated noise from a Laplace distribution. This ensures the output satisfies $ε$-Local Differential Privacy (LDP), making it mathematically difficult for an attacker to distinguish between similar inputs.
- Robust Training: To counteract the accuracy loss typically caused by noise, the server-side model is trained specifically on noisy representations. This teaches the classifier to focus on core task features while ignoring the added "fuzz."
Experimental Evaluation
The researchers evaluated DPNR on three distinct NLP benchmarks:
- TP-US (Trustpilot): Sentiment analysis with age and gender as private variables.
- AG News: Topic classification where named entities (person names) are treated as private.
- BLOG (Blog Authorship Corpus): Topic classification with age and gender as private variables.
Key Results and Insights
Privacy and Utility
DPNR significantly outperformed non-private baselines and adversarial (ADV) training in preventing attribute leakage. On the AG News dataset, DPNR reduced the attacker's ability to identify private entities by over 31%. Remarkably, utility remained high; on the TP-US dataset, the DPNR version actually achieved a slightly higher accuracy (85.87%) than the non-private baseline (85.53%).
Fairness as a Side Effect
A striking finding is that Differential Privacy acts as a regularizer that improves fairness. By adding noise that "washes out" demographic cues like age-specific slang, the model is forced to make decisions based on actual content. This leads to more equitable performance across demographic subgroups.
BERT’s Natural Resilience
The study noted that BERT-based models are surprisingly resistant to DP noise. The authors speculate that internal components, such as Layer Normalization, act as natural "denoisers," helping maintain high accuracy even when representations are perturbed.
Conclusion
DPNR demonstrates that formal privacy guarantees and high model utility are not mutually exclusive in NLP. By combining word dropout with the Laplace mechanism and robust training, the framework effectively prevents attribute leakage while narrowing the performance gap between different demographic groups, offering a promising path for privacy-preserving and fair machine learning.
Summary updated