MemDPT: Differential Privacy for Memory Efficient Language Models

Yanming Liu, Xinyue Peng, Jiannan Cao, Yuwei Zhang, Chen Ma, Songhang Deng, Mengchen Fu, Xuhong Zhang, Sheng Cheng, Xun Wang, Jianwei Yin, Tianyu Du

TL;DR

This paper introduces DP-MemArc, a novel framework for training large language models (LLMs) with differential privacy while significantly reducing memory consumption. By employing novel "side network" and "reversible network" designs, DP-MemArc achieves substantial memory savings (up to 2.5x) without compromising privacy guarantees or performance on downstream tasks.

What Problem Does This Paper Address?

Large Language Models (LLMs) have demonstrated remarkable capabilities across various domains, but their sheer size presents significant challenges for deployment and fine-tuning. Specifically, training LLMs requires immense computational resources, particularly memory, making it difficult to adapt them to new tasks, especially in resource-constrained environments. Furthermore, when fine-tuning LLMs on sensitive user data, ensuring differential privacy is crucial to protect user information. Existing methods for parameter-efficient fine-tuning (PEFT) and differential privacy (DP) often struggle to address both memory efficiency and privacy simultaneously. While some PEFT methods reduce the number of trainable parameters, they still incur substantial memory overhead for storing intermediate activations and gradients. Similarly, DP training, while crucial for privacy, often adds further computational and memory burdens. This paper tackles the critical need for LLM fine-tuning that is both memory-efficient and privacy-preserving.

Methods

The core contribution of DP-MemArc lies in its two novel architectural designs for memory-efficient differential privacy fine-tuning:

  • DP-MemArcside (Side Network Design): This approach introduces a "side network" that learns task-specific knowledge. The original LLM's parameters are frozen, and the side network, which is significantly smaller, is trained with differential privacy. This design aims to reduce memory by minimizing the trainable parameters and focusing the DP training on a smaller component. The authors integrate this with the LST (Ladder Side-Tuning) method to pass intermediate layer information from the pre-trained model to the side network.
  • DP-MemArcrev (Reversible Network Design): This method leverages the concept of reversible networks. In reversible networks, intermediate activation states can be computed from the output, eliminating the need to store them during backpropagation. This drastically reduces memory consumption associated with activation states. The authors adapt this by using reversible network functions and integrating PEFT methods like LoRA (Low-Rank Adaptation) for the trainable parameters within the reversible structure.

Both DP-MemArcside and DP-MemArcrev utilize the BK-MixOpt algorithm for differential privacy training. BK-MixOpt is a book-keeping strategy that optimizes gradient computation and clipping to reduce memory overhead compared to standard DP-SGD implementations, especially when dealing with per-example gradients.

Datasets

The paper evaluates DP-MemArc on a variety of datasets across different NLP tasks:

  • Text Classification:
    • MNLI: MultiGenre Natural Language Inference Corpus
    • QQP: Quora Question Pairs dataset
    • QNLI: Stanford Question Answering dataset
    • SST2: Stanford Sentiment Treebank dataset
  • Text Generation:
    • E2E: A dataset for generating restaurant reviews.

The experiments were conducted on these standard benchmarks to assess the performance of DP-MemArc in diverse scenarios.

Key Insights and Ideas

  • Decoupling Privacy and Memory: The paper demonstrates that it's possible to achieve significant memory savings while maintaining strong differential privacy guarantees, challenging the notion that these two goals are inherently conflicting.
  • Architectural Innovation for Memory: The introduction of "side networks" and "reversible networks" as core components for DP fine-tuning offers a new paradigm for tackling memory bottlenecks in LLMs.
  • Side Network Effectiveness: DP-MemArcside shows that a smaller, dedicated side network can effectively learn new tasks with DP, requiring substantially less memory than full PEFT methods.
  • Reversible Networks for Activation Savings: DP-MemArcrev highlights the power of reversible architectures in eliminating the memory cost of storing intermediate activations, a major contributor to training memory.
  • BK-MixOpt as an Efficient DP Backbone: The paper underscores the importance of efficient DP optimization techniques like BK-MixOpt, showing it provides significant memory benefits over standard DP methods.

Key Results

  • Memory Optimization: DP-MemArc achieves up to 2.5x memory optimization compared to existing methods. Specifically, DP-MemArcside reduces training memory by nearly 2-3x compared to DP-LORA under privacy constraints. DP-MemArcrev offers even greater savings, approximately 6-8x compared to full DP fine-tuning in high privacy scenarios.
  • Performance: Despite significant memory reductions, DP-MemArc methods maintain competitive performance.
  • On text classification tasks (RoBERTa-large), DP-MemArcside demonstrates performance comparable to DP-LORA but with less memory. DP-MemArcrev shows improved accuracy (+3.1% on average) compared to DP-BiTFiT and performs better under higher privacy constraints.
  • For text generation tasks (GPT-2-large), DP-MemArcside shows strong performance on BLEU metrics with only a minor drop (3.5%) under high privacy demands. DP-MemArcrev maintains competitive accuracy with significantly reduced memory.
  • Time Efficiency: DP-MemArc methods are shown to be more efficient in reasoning across various downstream tasks compared to other baseline methods, maintaining consistent time consumption while being memory-efficient.

Limitations and Future Work

  • Performance-Memory Trade-off in Side Networks: The paper notes that at very small sizes for the side network (small 'r'), performance can deteriorate significantly. The authors suggest that an empirically optimal ratio of training memory to performance is achieved when d/r = 8.
  • Training Steps: DP-MemArcside and DP-MemArcrev require more training steps to reach stable values compared to DP-LORA. This is attributed to the need for the side network to adapt or for the reversible network to learn through approximation methods.
  • Future Work: The authors express hope that their DP-MemArc framework will contribute to future advancements in private efficient memory optimization for fine-tuning LLMs. Further exploration into optimizing the trade-offs between memory, privacy, and performance for different architectures and tasks would be a natural next step.

Summary updated

Abstract

Large language models have repeatedly shown outstanding performance across diverse applications. However, deploying these models can inadvertently risk user privacy. The significant memory demands during training pose a major challenge in terms of resource consumption. This substantial size places a heavy load on memory resources, raising considerable practical concerns. In this paper, we introduce DP-MemArc, a novel training framework aimed at reducing the memory costs of large language models while emphasizing the protection of user data privacy. DP-MemArc incorporates side network or reversible network designs to support a variety of differential privacy memory-efficient fine-tuning schemes. Our approach not only achieves in memory optimization but also ensures robust privacy protection, keeping user data secure and confidential. Extensive experiments have demonstrated that DP-MemArc effectively provides differential privacy-efficient fine-tuning across different task scenarios.
BibTeX
@inproceedings{liu2025memdpt,
  author = {Yanming Liu and Xinyue Peng and Jiannan Cao and Yuwei Zhang and Chen Ma and Songhang Deng and Mengchen Fu and Xuhong Zhang and Sheng Cheng and Xun Wang and Jianwei Yin and Tianyu Du},
  title = {{MemDPT: Differential Privacy for Memory Efficient Language Models}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i25.34830},
}
Figure