Towards Differentially Private Text Representations

Lingjuan Lyu, Yitong Li, Xuanli He, Tong XiaoSIGIR 2020 · OpenReview

Abstract

Most deep learning frameworks require users to pool their local data or model updates to a trusted server to train or maintain a global model. The assumption of a trusted server who has access to user information is ill-suited in many applications. To tackle this problem, we develop a new deep learning framework under an untrusted server setting, which includes three modules: (1) embedding module, (2) randomization module, and (3) classifier module. For the randomization module, we propose a novel local differentially private (LDP) protocol to reduce the impact of privacy parameter $\epsilon$ on accuracy, and provide enhanced flexibility in choosing randomization probabilities for LDP. Analysis and experiments show that our framework delivers comparable or even better performance than the non-private framework and existing LDP protocols, demonstrating the advantages of our LDP protocol.

Introduction

In modern deep learning, the standard paradigm assumes a "trusted server" model where users transmit data or model gradients to a central authority for aggregation. However, in many real-world scenarios, servers may be compromised or inherently untrustworthy. Even when raw text is withheld, sending "hidden representations" or embeddings is insufficient for privacy; research indicates that sensitive attributes (e.g., gender, age, health markers) can be reconstructed from these vectors.

While Centralized Differential Privacy (CDP) offers protection, it still necessitates a trusted central entity. Local Differential Privacy (LDP) empowers users to perturb their data locally before transmission. Historically, however, LDP protocols have introduced significant noise, often rendering models useless for complex Natural Language Processing (NLP) tasks. This paper introduces a framework that bridges this gap using a novel LDP protocol called Optimized Multiple Encoding (OME).

Methodology: The OME Framework

The proposed framework operates through a three-stage pipeline designed for an untrusted server environment.

The proposed three-stage LDP framework architecture, illustrating the transition from local data to differentially private classifier.
  1. Embedding Module: Local users transform raw text into 1-D real-valued vectors using standard embedding techniques such as GloVe or BERT.
  2. Randomization Module (OME Protocol): This is the core innovation. Each real value in the embedding is mapped into a fixed-length binary vector (e.g., 10 bits representing sign, integer, and fraction). The OME protocol introduces a randomization factor ($\lambda$) that provides greater flexibility than standard protocols like Symmetric Unary Encoding (SUE). By tuning $\lambda$, the protocol maintains high probability for preserving zeros, thereby protecting the data's inherent sparsity and structure.
  3. Classifier Module: The server receives these perturbed, binary representations and trains a classifier, typically a Multi-Layer Perceptron (MLP), without ever seeing the original embeddings.

Experimental Evaluation

The framework was evaluated across five datasets covering sentiment analysis (IMDb, Amazon, Yelp), intent detection (Intent), and paraphrase identification (MRPC).

Accuracy comparison of the LDPNN framework against the non-private baseline (NPNN) across different privacy budgets ($\epsilon$) and randomization factors ($\lambda$).

Key Results

  • Performance Gains: Remarkably, the Local Differentially Private Neural Network (LDPNN) often outperformed the non-private baseline (NPNN). For instance, on the Amazon dataset, LDPNN achieved 69.50% accuracy compared to NPNN's 67.50%.
  • Protocol Superiority: OME significantly outperformed traditional LDP protocols. On the IMDb dataset ($\epsilon=1$), OME reached 67.33% accuracy, while SUE and OUE lagged at 55.33% and 50.67%, respectively.
  • Stability: Unlike many DP applications where accuracy drops sharply as the privacy budget $\epsilon$ decreases, OME remained stable even at low $\epsilon$ values (e.g., 0.5).

Discussion and Insights

The success of the OME framework suggests that LDP noise can act as a form of regularization, preventing overfitting and occasionally leading to better generalization than non-private models. Furthermore, the process of converting float vectors into multi-bit binary vectors effectively enlarges the feature space, potentially capturing nuanced patterns missed by standard dense layers.

Despite these advantages, challenges remain. The binary encoding increases communication overhead, and performance is sensitive to the randomization factor $\lambda$. Future work will explore variable bit-depths to further optimize the privacy-utility trade-off and investigate automated tuning for $\lambda$.

Summary updated

Abstract

Most deep learning frameworks require users to pool their local data or model updates to a trusted server to train or maintain a global model. The assumption of a trusted server who has access to user information is ill-suited in many applications. To tackle this problem, we develop a new deep learning framework under an untrusted server setting, which includes three modules: (1) embedding module, (2) randomization module, and (3) classifier module. For the randomization module, we propose a novel local differentially private (LDP) protocol to reduce the impact of privacy parameter ε on accuracy, and provide enhanced flexibility in choosing randomization probabilities for LDP. Analysis and experiments show that our framework delivers comparable or even better performance than the non-private framework and existing LDP protocols, demonstrating the advantages of our LDP protocol.
BibTeX
@inproceedings{lyu2020towards,
  author = {Lingjuan Lyu and Yitong Li and Xuanli He and Tong Xiao},
  title = {{Towards Differentially Private Text Representations}},
  booktitle = {International ACM SIGIR Conference on Research and Development in Information Retrieval},
  year = {2020},
  url = {https://openreview.net/forum?id=7Iw0kiHFBfq},
}
Figure