Abstract
Pre-trained Language Models (PTLMs) such as BERT and GPT have revolutionized Natural Language Processing through the "pre-train then fine-tune" paradigm. However, this efficiency introduces a complex, multi-stage attack surface. This paper provides the first comprehensive taxonomy of security and privacy threats targeting PTLMs, systematizing risks across the model lifecycle—from pre-training to inference. A key contribution is the identification of "transferability" properties that allow attacks to persist even after fine-tuning on clean data, posing significant challenges for model integrity and data privacy.
Introduction
The shift toward using massive foundation models has fundamentally changed how NLP applications are built. Instead of training from scratch, developers adapt PTLMs for specific downstream tasks. While this approach is efficient, it creates a centralized point of failure. If a base model is compromised, every application built upon it inherits its vulnerabilities. Despite the discovery of various individual attacks, the field has lacked a unified framework to categorize these threats. This survey addresses this gap by analyzing how the inherent properties of PTLMs, such as their generalization capabilities, can be exploited by malicious actors.
A Three-Dimensional Taxonomy
The authors propose a structured framework to characterize PTLM threats based on three dimensions: the pipeline phase, adversarial goals, and model transferability.
The PTLM Pipeline
The lifecycle is divided into three distinct phases:
- Pre-training: The phase where a Model Publisher creates the base model using massive unlabeled datasets.
- Fine-tuning: The phase where a Downstream Service Provider (DSP) adapts the PTLM for specific tasks like sentiment analysis or question answering using labeled data.
- Inferring: The final phase where end-users query the deployed model for predictions.
Adversarial Goals
Threats are broadly categorized into two types:
- Integrity Threats: Attacks aimed at manipulating model outputs, such as backdoors (triggers that force specific incorrect outputs) or evasion (adversarial examples designed to fool the model at inference).
- Privacy Threats: Attacks aimed at extracting sensitive information, including training data (data privacy) or the model's internal parameters and logic (model privacy).
Model Transferability
A novel concept introduced in this work is the dual nature of attack transferability:
- Landscape Transferability: An attack successful on one downstream task can often be adapted to others because they share the same PTLM backbone.
- Portrait Transferability: Malicious backdoors injected during pre-training are remarkably persistent; they survive the fine-tuning process even when the developer uses pristine data for the final task.
Surveying the Attack Surface
By synthesizing findings from dozens of studies across standard NLP benchmarks (e.g., GLUE, SQuAD, PAWS), the survey highlights several critical vulnerabilities.
The Malicious Publisher and Persistent Backdoors
One of the most severe threats is the "poisoned" foundation model. Because developers often download models from public repositories like Hugging Face, a malicious publisher can release a version of a model that contains a hidden trigger. Research shows that attacks like RIPPLe can achieve nearly 100% success rates in triggering target labels while maintaining standard accuracy on clean data. Due to portrait transferability, standard fine-tuning is insufficient to "wash away" these deep-seated malicious behaviors.
Privacy Leakage and Model Extraction
PTLMs are often "honest-but-curious," meaning they inadvertently memorize sensitive parts of their training data. Attackers can employ "Embedding Inversion" to reconstruct original sentences from numerical vectors. Furthermore, because PTLMs are valuable intellectual property, they are targets for "imitation attacks," where adversaries query a model's API to train a "shadow" model that mimics the original's behavior.
Future Directions and Challenges
The survey identifies several open challenges for the research community:
- Robustness-Utility Trade-off: Current privacy defenses, such as adding noise to embeddings, often degrade model performance. Finding a balance remains difficult.
- Evolving Backdoor Tactics: As defenses like ONION improve at detecting simple triggers, attackers are moving toward context-aware triggers that are harder to filter.
- Machine Unlearning: There is a growing need for fine-tuning methods that can effectively "forget" malicious or sensitive data from the pre-training phase without losing general linguistic knowledge.
In conclusion, as PTLMs become more integrated into critical infrastructure, understanding and mitigating these multi-stage threats is essential for building trustworthy AI systems.
Summary updated