Granularity is crucial when applying differential privacy to text: An investigation for neural machine translation

Doan Nam Long Vu, Timour Igamberdiev, Ivan HabernalEMNLP 2024 · DOI

Abstract

Applying Differential Privacy (DP) at the sentence level in Neural Machine Translation (NMT) often fails to protect user privacy when sentences are correlated, such as in dialogues. This study investigates shifting the unit of privacy to document-level granularity. We demonstrate that while document-level DP significantly enhances resistance to membership inference attacks and PII leakage, it introduces a substantial utility penalty. We propose an augmented training strategy using large-scale non-sensitive data to mitigate this loss, providing a more robust framework for private NMT.

Introduction: The Independence Fallacy

Differential Privacy (DP) is widely regarded as the gold standard for protecting individual data points during model training. In Natural Language Processing (NLP), the "unit of privacy" is typically defined as a single sentence. However, this approach relies on the assumption that sentences in a dataset are independent. In real-world contexts like customer support chats or personal dialogues, multiple sentences belong to the same individual and are highly correlated. Sentence-level DP may protect one sentence while allowing the model to memorize a related one containing the same sensitive information. This paper explores scaling the privacy unit to the document level to provide a more robust defense against data leakage.

Methodology

To address the challenges of document-level privacy, we employ several advanced techniques:

  • DP-SGD at Scale: We utilize Differentially Private Stochastic Gradient Descent (DP-SGD) implemented within a specialized DP-NMT framework using JAX and Flax.
  • mLongT5 Architecture: To handle long sequences (up to 1,700 tokens) typical of document-level data, we use mLongT5, which features a "Transient Global Attention" mechanism for efficient processing.
  • Augmented Training: To combat the utility drop associated with document-level DP, we implement a two-stage fine-tuning process. Models are first pre-trained on a large, public dataset (WMT22) before applying DP fine-tuning on smaller, private target datasets.
  • Privacy Evaluation: Defenses are stress-tested using Loss-based Membership Inference Attacks (MIA) and a custom schema for Personally Identifiable Information (PII) exposure.

Experimental Setup

We evaluate our approach across several key datasets:

  1. BSD (Business Scene Dialogue): Fictional business conversations in Japanese and English.
  2. MAIA (Multilingual AI Agent Assistant): Real customer support logs with artificial PII for testing leakage.
  3. Europarl: European Parliament proceedings used as a large-scale baseline for German-English translation.
  4. WMT22: A large, non-sensitive dataset used for the initial augmented training phase.

Results: The Privacy-Utility Trade-off

Our findings highlight a significant "privacy tax" when moving to document-level granularity. As shown in the comparison of BLEU scores across different privacy budgets ($\epsilon$), document-level models ($\theta^{doc}$) generally underperform compared to sentence-level models ($\theta^{sen}$) due to the increased noise required to protect larger data units.

Comparison of BLEU scores for sentence-level ($\theta^{sen}$), document-level ($\theta^{doc}$), and augmented document-level ($\theta^{augdoc}$) models across datasets.

However, the augmented training strategy ($\theta^{augdoc}$) effectively recovers a portion of this utility loss. By "warming up" the model on non-sensitive data, it learns translation mechanics before the introduction of DP noise. Crucially, document-level models achieved 0% PII leakage on the MAIA dataset, whereas sentence-level models continued to leak sensitive information even at moderate privacy budgets ($\epsilon=10$). Furthermore, document-level systems were significantly more robust against membership inference attacks, with leakage roughly 50% lower than sentence-level models in non-private settings.

Conclusion and Future Work

While document-level DP provides superior privacy guarantees for correlated text, it faces hurdles in data scarcity and computational cost. Future research should focus on developing "correlation-aware" membership inference attacks and optimizing DP-SGD for the long-sequence regimes required by document-level NMT. Shifting the focus from sentence to document granularity is a vital step toward truly private machine translation in real-world applications.

Summary updated

BibTeX
@inproceedings{vu2024granularity,
  author = {Doan Nam Long Vu and Timour Igamberdiev and Ivan Habernal},
  title = {{Granularity is crucial when applying differential privacy to text: An investigation for neural machine translation}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.findings-emnlp.29},
}
Figure