Abstract
It has been demonstrated that hidden representation learned by a deep model can encode private information of the input, hence can be exploited to recover such information with reasonable accuracy. To address this issue, we propose a novel approach called Differentially Private Neural Representation (DPNR) to preserve the privacy of the extracted representation from text. DPNR utilises Differential Privacy (DP) to provide a formal privacy guarantee. Further, we show that masking words via dropout can further enhance privacy. To maintain utility of the learned representation, we integrate DP-noisy representation into a robust training process to derive a robust target model, which also helps for model fairness over various demographic variables. Experimental results on benchmark datasets under various parameter settings demonstrate that DPNR largely reduces privacy leakage without significantly sacrificing the main task performance.
Introduction
In modern NLP, deep learning models transform raw text into "hidden representations"—dense vectors of numbers. While these vectors are excellent for tasks like sentiment analysis, they are inherently "leaky." An attacker can often train a secondary classifier to extract sensitive information about the author—such as their location, age, or gender—directly from these representations, even if the model was never intended to see that data.
Existing defenses, like adversarial training, often provide only empirical "cat-and-mouse" protection without mathematical guarantees. Furthermore, these representations can inherit biases from the training data, leading to unfair outcomes for different demographic groups. The authors seek a solution that provides a formal privacy guarantee (via Differential Privacy) while maintaining high utility and promoting fairness.
Methods
The authors propose Differentially Private Neural Representation (DPNR), a framework that provides the first formal privacy guarantees for text representations in NLP. DPNR operates in a "Local" Differential Privacy (LDP) setting, where data is randomized on the user's device before it ever reaches a central server.
The system relies on three key components:
- The Laplace Mechanism: To achieve $\epsilon$-differential privacy, the system adds noise drawn from a Laplace distribution to the extracted features. Features are normalized to a $[0, 1]$ range to bound the "sensitivity" of the data.
- Word Dropout as a Privacy Booster: A key theoretical contribution is proving that masking words via dropout before adding DP noise actually tightens the privacy bound. Essentially, if words are hidden randomly, less additive noise is required to achieve the same level of formal privacy.
- Robust Training: Adding noise usually degrades model performance. To address this, the authors use a "noise-aware" training process on the server. By training the model on noisy representations that mimic what it will see during inference, the model learns to be robust to the perturbations.
Experimental Setup
The researchers evaluated DPNR across three diverse datasets:
- Trustpilot (TP-US): Sentiment analysis of reviews, with gender and age as private attributes.
- AG News: Topic classification of news articles, where private information consists of specific named entities.
- Blog Posts (BLOG): Topic classification of blog entries, with age and gender as sensitive variables.
Key Results
DPNR demonstrates that privacy does not have to come at the cost of utility. On the TP-US dataset, the non-private baseline achieved 85.53% accuracy. DPNR maintained a nearly identical 85.65% accuracy even with a strict privacy budget ($\epsilon=0.05$), thanks to the robust training process.
Beyond utility, DPNR significantly reduces attribute leakage. For example, on the AG News dataset, DPNR improved "empirical privacy" (the attacker's inability to guess the private variable) by over 31% compared to the non-private version, far exceeding the protection offered by traditional adversarial methods.
As shown in the t-SNE plots above, DPNR successfully obscures demographic cues. In the non-private setting, representations clearly cluster by age groups (under 20 vs. over 30). In the DPNR setting, these clusters are "mixed," indicating the model has successfully ignored the sensitive age attribute. This regularization effect naturally reduces "model discrimination," leading to more equal performance across different demographic groups.
Conclusion and Future Work
DPNR provides a robust, mathematically grounded approach to privacy in NLP. By linking privacy and fairness through the framework of Differential Privacy, the authors show that obscuring sensitive cues can lead to more equitable model performance.
While DPNR is highly effective, the authors note that finding the optimal balance between accuracy and privacy remains a manual tuning process. Future work could explore how these privacy guarantees scale to even larger generative models where the representation space is much larger, and investigate how DPNR affects other fairness metrics such as equalized odds.
Summary updated