SEEP: Training Dynamics Grounds Latent Representation Search for Mitigating Backdoor Poisoning Attacks

Xuanli He, Qiongkai Xu, Jun Wang, Benjamin I. P. Rubinstein, Trevor Cohn

Introduction

The proliferation of deep learning models has created an insatiable demand for large-scale training data. To meet this need, researchers and engineers frequently turn to uncurated datasets from public repositories. However, this reliance on external sources introduces a critical security vulnerability: backdoor poisoning attacks. In these scenarios, a malicious actor injects a small number of "poisoned" samples into the training set. These samples contain a specific "trigger"—such as a rare word, a specific syntactic structure, or a synonym—and are assigned a target label.

A model trained on this data behaves normally on standard test sets but predictably malfunctions when it encounters the trigger in a real-world setting. For example, a sentiment analysis model might be manipulated to classify any text containing a specific obscure word as "Positive," regardless of the actual content. SEEP (SEEd and Propagation) is a novel defense mechanism designed to identify and remove these poisoned samples without compromising the model's performance on legitimate data.

The Challenge of Backdoor Defense

Existing defense strategies often face a difficult trade-off between recall and precision. High-recall defenses may successfully identify poisoned samples but often suffer from high false rejection rates, inadvertently deleting legitimate "hard" examples and degrading the model’s overall Clean Accuracy (CACC). Conversely, high-precision defenses may fail to catch stealthy, advanced attacks that use subtle triggers.

SEEP addresses these challenges by treating backdoor detection as a two-stage search process that leverages both the temporal behavior of the model during training and the spatial relationships within its latent representation.

Methodology: The SEEP Framework

SEEP operates through a two-step "Seed and Propagation" approach that does not require a separate "gold standard" clean dataset for calibration.

1. Seeding via Training Dynamics

Rather than analyzing a static snapshot of a trained model, SEEP monitors how the model learns over time. The researchers track a metric called "inv-confidence"—the inverse probability of the ground truth label across training epochs.

A key insight of this work is that poisoned samples are not merely outliers; they are outliers that the model is highly "eager" to learn. While the model may struggle with legitimate but difficult clean examples, it tends to become overconfident in poisoned samples very quickly. By identifying the top 1% of samples that exhibit this suspicious learning trajectory, SEEP establishes a high-precision set of "seeds."

2. Latent Representation Propagation

Poisoned samples often cluster together in the model's hidden layers (the latent space). However, these clusters are rarely uniform; poisoned data can form multiple distinct groups, making simple clustering defenses ineffective.

Starting from the initial seeds, SEEP uses a K-Nearest Neighbors (KNN) search to find similar samples in the latent space. To determine the optimal stopping point and prevent the deletion of clean data, the system employs Kernel Density Estimation (KDE). If the "density" (the likelihood of a sample being poisoned based on its proximity to known seeds) drops below a calculated threshold, the search terminates. This allows SEEP to "hop" between related clusters of poisoned data while maintaining high precision.

Experimental Evaluation

The researchers evaluated SEEP across four diverse NLP benchmarks:

  • SST-2: Sentiment analysis.
  • OLID: Offensive language identification.
  • AG News: Topic classification.
  • QNLI: Natural Language Inference.

These benchmarks were tested against four attack vectors, ranging from the standard BadNet (word insertion) to more sophisticated methods like LWS (synonym replacement) and Syntactic (syntactic manipulation).

Key Results and Insights

SEEP consistently outperformed state-of-the-art defenses such as ABL, DAN, and Z-defense across multiple metrics:

  • Significant ASR Reduction: On the SST-2 dataset, the average Attack Success Rate (ASR) across all tested attacks plummeted from 98.4% (no defense) to 12.3% when SEEP was applied.
  • Preservation of Accuracy: SEEP maintained a Clean Accuracy (CACC) of 92.2%, nearly identical to the 92.4% achieved by a model trained on a perfectly clean dataset. This demonstrates SEEP's ability to distinguish between poisoned samples and difficult clean samples.
  • Resilience to Stealthy Attacks: SEEP proved particularly effective against the LWS (Layer-wise Weight Substitution) attack. While other defenses often left the ASR as high as 90%, SEEP successfully mitigated the impact of these subtle synonym-based triggers.

Conclusion and Future Work

SEEP represents a significant advancement in backdoor defense by grounding latent space searches in the empirical evidence of training dynamics. By focusing on the model's "eagerness" to learn specific samples, it provides a robust, self-contained method for cleaning poisoned datasets.

Despite its success, the authors note that extremely subtle attacks, such as certain LWS configurations, can still partially evade detection (maintaining an ASR of approximately 29% in some cases). Furthermore, if an attacker were to develop a "slow-learning" poison that mimics the dynamics of legitimate hard examples, the initial seeding process might be compromised. Future research will focus on combining SEEP with test-time detection strategies to create a more comprehensive "defense-in-depth" architecture.

Summary updated

BibTeX
@article{he2024seep,
  author = {Xuanli He and Qiongkai Xu and Jun Wang and Benjamin I. P. Rubinstein and Trevor Cohn},
  title = {{SEEP: Training Dynamics Grounds Latent Representation Search for Mitigating Backdoor Poisoning Attacks}},
  journal = {Trans. Assoc. Comput. Linguistics},
  year = {2024},
  doi = {10.1162/tacl\_a\_00684},
}
Figure