TL;DR
This paper investigates how differential privacy (DP) techniques for text paraphrasing affect the underlying geometric structure of text representations. They find that word-level DP significantly distorts these structures, while sentence-level DP, particularly masked paraphrasing, preserves them better, suggesting a path towards more robust and less distorting privacy mechanisms.
What Problem Does This Paper Address?
Large language models (LMs) are trained on vast amounts of text, which can include sensitive information. Extracting this sensitive data poses significant privacy risks. Differential Privacy (DP) offers a rigorous framework to protect against such disclosures by ensuring that the output of a system is statistically similar whether or not any individual's data was included in the input.
While DP has been applied to text, often through paraphrasing, previous work has focused on balancing privacy and utility (e.g., how good the paraphrased text is). However, the geometric impact of these DP methods on the representation space of text – how the underlying structure and complexity of text are altered – has been largely unexplored. This paper aims to fill that gap by examining how DP paraphrasing affects the "representation manifold" of text.
Methods
The core of this paper's methodology lies in estimating the Intrinsic Dimensionality (ID) of text representations. The ID is a measure of the minimum number of coordinates needed to capture the variability of data points in a representation space. A lower ID suggests a more structured and less complex manifold, while a higher ID indicates more complexity and potential distortion.
- Data Representation: Text is treated as a point cloud of word embeddings. The paper uses BERT embeddings for this purpose.
- Intrinsic Dimensionality Estimation: The authors employ the TwoNN (Two Nearest Neighbors) method to estimate the ID. This method is robust to variations in data density and curvature.
- Privacy Mechanisms: They compare two main categories of DP text paraphrasing:
- Word-level DP: Specifically, MADLIB (Feyisetan et al., 2020), which perturbs individual words by mapping them to nearby words in the embedding space.
- Sentence-level DP: This includes:
- DP-PARAPHRASE (Mattern et al., 2022) and DP-PROMPT (Utpala et al., 2023), which use causal LMs (generating text word-by-word sequentially).
- DP-MLM (Meisenbacher et al., 2024), which uses masked LMs and reconstructs words bidirectionally from context.
- Experimental Setup: They use the MRPC dataset (sentence pairs for semantic equivalence) and evaluate across a range of privacy budgets ($\epsilon \in \{10, 15, 20, 25, 50, 100\}$). For probabilistic methods like temperature sampling, they repeat paraphrasing three times per sample to ensure robust ID estimates.
Datasets
- MRPC (Microsoft Research Paraphrase Corpus): Used as the primary dataset for experiments. It contains sentence pairs labeled for semantic equivalence, providing a basis for comparing original sentences with their paraphrases.
Key Insights and Ideas
- Word-level DP is highly distorting: MADLIB, a word-level DP method, significantly inflates the representation manifold, leading to drastically higher intrinsic dimensions. This suggests it fundamentally alters the structure and complexity of text representations.
- Sentence-level DP is more preserving: Sentence-level methods generally show much lower ID shifts compared to word-level methods, indicating they preserve the underlying geometric structure of text better.
- Masked vs. Causal LMs matter: Among sentence-level methods, DP-MLM (using masked LMs) consistently yields more stable representation geometry across all privacy budgets compared to causal LMs (DP-PARAPHRASE, DP-PROMPT).
- Error Propagation is key: The authors hypothesize that the superiority of masked LMs stems from their bidirectional nature. Causal LMs, generating text sequentially, are prone to "error propagation" where a poorly chosen word (due to DP noise) cascades and leads to further unnatural word choices, drastically altering the representation. Masked LMs, by conditioning on both preceding and following context, are less susceptible to this sequential error accumulation.
- Privacy Budget vs. Model Type: While DP-PARAPHRASE (a fine-tuned causal LM) is stable across budgets, DP-PROMPT (a pre-trained causal LM) shows sharper declines. This suggests that fine-tuning might make causal LMs more robust to privacy constraints, but masked LMs still outperform them in preserving representation geometry.
Key Results
- Human paraphrases: Introduce minimal geometric distortions, with an ID shift of approximately 0.12 compared to original sentences.
- Word-level DP (MADLIB): Shows the highest ID shifts, indicating severe distortion of representation manifolds.
- Sentence-level DP (DP-PARAPHRASE, DP-PROMPT): Exhibit significantly lower ID shifts than word-level DP.
- DP-MLM (Masked LM): Consistently demonstrates the most stable representation geometry, with the lowest ID shifts across all tested privacy budgets.
Limitations and Future Work
- Linguistic Quality vs. Geometric Distortion: The authors acknowledge that ID estimation, while powerful for inspecting representation geometry, doesn't directly capture linguistic quality (fluency, adequacy). Future work could connect these geometric distortions to measures of text quality.
- Understanding Temperature Sampling Sensitivity: The paper notes that DP-PARAPHRASE handles high temperatures better than DP-PROMPT, suggesting differing sensitivities to temperature values in causal LMs. Further investigation into this interaction could be beneficial.
- Broader Application: While the study focuses on paraphrasing, the insights into how DP affects representation manifolds could be extended to other DP text generation tasks.
Summary updated