CLIBE: Detecting Dynamic Backdoors in Transformer-based NLP Models

Abstract

Backdoor attacks pose a significant security threat to NLP models, where malicious behavior is triggered by specific inputs. While static triggers are well-studied, "dynamic" backdoors—triggered by abstract linguistic features like style or syntax—remain difficult to detect. We present CLIBE, a framework that identifies these stealthy backdoors by analyzing model weights rather than input space. By injecting few-shot perturbations and measuring their generalization across clean samples, CLIBE effectively exposes hidden malicious circuits. Our evaluation shows high detection accuracy (F1 > 0.90) across various architectures, including large-scale generative models.

Introduction

In NLP, a backdoor attack involves fine-tuning a model to perform normally on standard inputs but switch to a target malicious label when a specific "trigger" is present. Traditional defenses rely on "trigger inversion," which searches for fixed tokens (e.g., a specific word) that cause misclassification. However, dynamic backdoors utilize abstract properties such as sentence structure, formal writing styles, or text perplexity. Because the literal text of the trigger varies, these attacks bypass traditional input-centric defenses. CLIBE addresses this gap by shifting the detection focus to the model's parameter space.

Methodology: Few-shot Perturbation Injection

CLIBE operates on the principle that dynamic backdoors rely on specific, often dormant, sets of "backdoor-related neurons." These neurons are inherently fragile; a small nudge to the model weights can activate them even in the absence of the original trigger. The CLIBE framework follows a four-step process:

1. Data Preparation: A small set of clean "reference samples" related to the model's task is gathered.
2. Few-shot Perturbation Injection: CLIBE optimizes a tiny weight perturbation in the attention layers (Query, Key, and Value matrices) to force a subset of reference samples toward a target label.
3. Generalization Measurement: The framework evaluates how well this perturbation generalizes to unseen reference samples. In backdoored models, the perturbation effectively "trips" the existing backdoor circuitry, leading to high generalization.
4. Backdoor Judgment: Detection is finalized by measuring the entropy of the logit difference distribution. Backdoored models exhibit a highly concentrated shift toward the target label.

Expand

Source, p. 4

Key Insights and Results

The core discovery of CLIBE is that forced misclassifications generalize significantly better in backdoored models than in benign ones. In a benign model, perturbations fight the model's natural logic, whereas in a backdoored model, they simply unlock an existing malicious path.

CLIBE achieved an F1 score exceeding 0.90 and an AUC over 0.95 across advanced dynamic attacks (Perplexity, Style, and Syntax). Furthermore, the method proved efficient, scanning models in 400–800 seconds.

Expand

Source, p. 4

Real-World Impact and Limitations

The researchers applied CLIBE to 49 popular models on Hugging Face, discovering a real-world perplexity backdoor in a model with over 190,000 monthly downloads. Despite its success, CLIBE currently requires white-box access to model weights, meaning it cannot yet audit closed-source models accessed via API. Future work aims to extend these techniques to black-box scenarios and explore neuron pruning as a mitigation strategy to remove detected backdoors.

@inproceedings{zeng2025clibe,
  author = {Rui Zeng and Xi Chen and Yuwen Pu and Xuhong Zhang and Tianyu Du and Shouling Ji},
  title = {{CLIBE: Detecting Dynamic Backdoors in Transformer-based NLP Models}},
  booktitle = {NDSS},
  year = {2025},
}