One size does not fit all: Investigating strategies for differentially-private learning across NLP tasks

Manuel Senge, Timour Igamberdiev, Ivan HabernalEMNLP 2022 · DOI

Abstract

Differential Privacy (DP) is essential for protecting sensitive data in NLP, but its application is not a straightforward "plug-and-play" solution. This paper demonstrates that the optimal training strategy for private models varies significantly across tasks—what works for sentiment analysis may fail for natural language inference. Furthermore, we reveal that previous high-performance results in private Named Entity Recognition (NER) were likely artifacts of misleading evaluation metrics. We provide a systematic analysis of the "privacy tax" in terms of both model performance and computational overhead across various NLP domains.

Introduction

As NLP models are increasingly trained on sensitive user data such as emails and medical records, the risk of "memorization"—where a model inadvertently leaks training data—becomes a critical failure point. Differentially Private Stochastic Gradient Descent (DP-SGD) is the gold standard for mitigating this risk by clipping individual gradients and adding random noise during training. However, DP-SGD introduces a "privacy tax": model performance typically drops as privacy guarantees (controlled by the parameter $\epsilon$) become stricter. This study provides a task-by-task analysis to understand how this tax applies across the NLP landscape.

Methods and Datasets

The authors evaluate five fine-tuning strategies across two base architectures: BERT-base and XtremeDistil (XDTM). The strategies range from training only the output layer (Tr/No) or an added LSTM (Tr/No/LSTM) to fine-tuning the last two layers (Tr/Last2) or the entire model (Tr/All). These are tested across seven datasets covering Sentiment Analysis (SA), Natural Language Inference (NLI), Named Entity Recognition (NER), Part-of-Speech (POS) tagging, and Question Answering (QA). Privacy budgets were varied from $\epsilon = 1$ (strict) to $\epsilon = \infty$ (no privacy).

Key Insights

The "Accuracy" Trap in NER

A striking finding involves the evaluation of NER. Previous work suggested that DP had minimal impact on NER performance. However, this study reveals that high accuracy scores are often misleading due to heavy class imbalance. Private models often "cheat" by simply predicting the majority "Outside" (O) tag for all tokens. While accuracy remains high, Macro-F1 scores reveal that the models fail to identify entities almost entirely under strict privacy constraints.

Task-Specific Sensitivity

Optimal fine-tuning strategies are highly task-dependent. In Sentiment Analysis, fine-tuning the entire model (Tr/All) under DP-SGD can lead to total model collapse, whereas shallow fine-tuning (Tr/Last2) remains stable. Conversely, for Natural Language Inference, fine-tuning the whole model is superior because spreading the noise across more parameters increases robustness for complex semantic reasoning.

F1 performance across tasks and privacy budgets ($\epsilon = \infty, 5.0, 1.0$).

Results and Discussion

No Universal Winner

There is no single "best" strategy for private NLP. Simple classification tasks favor shallow fine-tuning, while complex reasoning tasks like NLI require deeper updates. In Question Answering (SQuAD 2.0), private models essentially hit a performance floor; since the dataset contains 50% unanswerable questions, the models learn to predict "unanswerable" for every query, resulting in a non-functional model despite a baseline F1 score of 0.5.

The Computational Tax

DP-SGD is not only a performance hit but also a massive computational burden. The requirement for per-sample gradient clipping causes training times to explode. For example, training BERT on SQuAD 2.0 increases from approximately 44 minutes without DP to over 11 hours with DP.

Comparison of epoch times with and without differential privacy (DP).

Conclusion

The research highlights that DP-SGD's struggle with skewed class distributions and its high computational cost remain significant hurdles. The authors strongly advocate for the use of Macro-F1 over Accuracy in private NLP evaluation to avoid misleading results. Future work should investigate layer-specific noise sensitivity and more efficient DP-SGD implementations.

Summary updated

BibTeX
@inproceedings{senge2022one,
  author = {Manuel Senge and Timour Igamberdiev and Ivan Habernal},
  title = {{One size does not fit all: Investigating strategies for differentially-private learning across NLP tasks}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2022},
  doi = {10.18653/v1/2022.emnlp-main.496},
}
Figure