Adversarial Training for Defense Against Label Poisoning Attacks

Melis Ilayda Bal, Volkan Cevher, Michael Muehlebach

TL;DR

The paper introduces FLORAL (Flipping Labels for Adversarial Learning), a novel defense framework designed to protect machine learning models against malicious label poisoning attacks. By framing the defense as a non-zero-sum Stackelberg game using kernel Support Vector Machines (SVMs), FLORAL strategically counters adversaries who flip critical training labels. The method consistently outperforms standard robust baselines and foundation models like RoBERTa, maintaining high classification accuracy even under severe data corruption.

What Problem Does This Paper Address?

Modern machine learning models increasingly rely on massive, publicly sourced, or human-annotated datasets. This reliance exposes them to label poisoning attacks, where an adversary subtly flips or manipulates training labels to degrade the model's downstream performance.

While conventional adversarial training (AT) is highly effective against feature perturbations (e.g., changing pixels in an image), it falls short against label poisoning. Feature attacks typically disrupt models locally near the decision boundary. In contrast, label attacks cause widespread, global disruptions across the entire input space. Furthermore, standard zero-sum minimax defense formulations fail because they mistakenly focus on protecting outliers (points with the largest hinge loss) rather than the critical points that actually define the decision boundary.

Methods

The authors introduce FLORAL, which reformulates adversarial training against label poisoning as a bilevel optimization problem structured as a non-zero-sum Stackelberg game between the model and an attacker:

  1. The Attacker's Strategy (Inner Loop): Instead of guessing randomly, the attacker leverages the model's dual parameters ($\lambda$) from a kernel SVM to identify the most influential training points (the support vectors). It then applies a randomized top-$k$ rule to flip the labels of these critical boundary-defining points.
  2. The Model's Defense (Outer Loop): The model updates its parameters using Projected Gradient Descent (PGD) to minimize its worst-case loss on this dynamically poisoned data.

To make this computationally viable for large-scale applications, the authors develop a highly efficient projection algorithm based on a fixed-point iteration strategy (PROJECTIONVIAFIXEDPOINTITERATION), avoiding the need to solve expensive quadratic programs at every step. Additionally, FLORAL is highly adaptable; it can protect deep neural networks (like RoBERTa) by acting as a robust classifier head operating on top of extracted last-layer embeddings.

The FLORAL framework casts adversarial training as an iterative game between a nonlinear SVM and a strategic label-flipping attacker.

Datasets

The framework was evaluated across three distinct classification tasks:

  • Moon Dataset: A synthetic 2D binary classification dataset (2,000 samples) used to visualize decision boundaries under varying poisoning levels (5% to 25%).
  • IMDB Dataset: A benchmark text dataset for sentiment analysis (50,000 reviews). The authors extracted 768-dimensional embeddings from a fine-tuned RoBERTa model to evaluate FLORAL as a robust classifier head under heavy poisoning (up to 40%).
  • MNIST (1vs7): A subset of the classic handwritten digits dataset focusing on distinguishing the visually similar digits 1 and 7 (13,007 samples).

Key Insights and Ideas

  • Targeting the Support Vectors: The core "aha moment" is that an effective label-poisoning defense must safeguard the decision boundary's shape. By utilizing SVM dual variables, FLORAL directly identifies and defends the exact training points that dictate this boundary.
  • Implicit Data Sanitization: FLORAL does not just passively resist attacks; it actively "heals" the dataset. When trained on already corrupted data, the algorithm naturally identifies and implicitly sanitizes 25% to 35% of the poisoned labels during the training process.
  • The Power of Non-Zero-Sum Games: Moving away from standard zero-sum formulations allows the framework to align more closely with the true objectives of both the attacker (maximizing classification error) and the defender (maximizing robust accuracy), leading to much more stable convergence.

Key Results

FLORAL demonstrated superior robustness compared to vanilla SVMs, standard neural networks, and specialized filtering defenses (like Curie and LS-SVM):

  • Foundation Model Enhancement (IMDB): Under a massive 40% label poisoning attack, a standard fine-tuned RoBERTa model's accuracy plummeted to 77.17%. When integrated with FLORAL, the model maintained a robust test accuracy of 86.36%.
  • Synthetic Clean vs. Poisoned Performance (Moon): At 25% poisoning, FLORAL achieved 80.1% peak accuracy, outperforming vanilla SVM (75.3%) and standard neural networks (69.3%), while matching clean data performance when no attack was present.
  • Smoother Decision Boundaries: Qualitative evaluations showed that FLORAL preserves a smooth, generalized decision boundary under attack, whereas baseline methods suffer from severe boundary distortion.

Limitations and Future Work

The authors acknowledge a few limitations:

  • White-Box Assumption: The defense formulation assumes a strong adversary with access to model parameters to compute influential points, though they note it remains effective in black-box scenarios via transferability.
  • Attack Specificity: While generalizable to several prominent attacks from literature (like alfa and LFA), its defensive efficacy can still vary depending on the exact nature of an unobserved, non-adaptive attack.

Future Directions: The authors suggest exploring SVM-based transfer attacks, extending the framework to multi-class foundation model fine-tuning, and conducting deeper analyses into how FLORAL alters the geometric distribution of influential training points.

Summary updated

Abstract

As machine learning models grow in complexity and increasingly rely on publicly sourced data, such as the human-annotated labels used in training large language models, they become more vulnerable to label poisoning attacks. These attacks, in which adversaries subtly alter the labels within a training dataset, can severely degrade model performance, posing significant risks in critical applications. In this paper, we propose FLORAL, a novel adversarial training defense strategy based on support vector machines (SVMs) to counter these threats. Utilizing a bilevel optimization framework, we cast the training process as a non-zero-sum Stackelberg game between an attacker, who strategically poisons critical training labels, and the model, which seeks to recover from such attacks. Our approach accommodates various model architectures and employs a projected gradient descent algorithm with kernel SVMs for adversarial training. We provide a theoretical analysis of our algorithm's convergence properties and empirically evaluate FLORAL's effectiveness across diverse classification tasks. Compared to robust baselines and foundation models such as RoBERTa, FLORAL consistently achieves higher robust accuracy under increasing attacker budgets. These results underscore the potential of FLORAL to enhance the resilience of machine learning models against label poisoning threats, thereby ensuring robust classification in adversarial settings.
BibTeX
@inproceedings{bal2025adversarial,
  author = {Melis Ilayda Bal and Volkan Cevher and Michael Muehlebach},
  title = {{Adversarial Training for Defense Against Label Poisoning Attacks}},
  booktitle = {International Conference on Learning Representations},
  year = {2025},
  url = {https://openreview.net/forum?id=UlpkHciYQP},
}
Figure