Poisons that are learned faster are more effective

Pedro Sandoval Segura, Vasu Singla, Liam Fowl, Jonas Geiping, Micah Goldblum, David Jacobs, Tom Goldstein

TL;DR

Data poisoning attacks, often proposed as a tool for data privacy, are highly vulnerable to the simple defense of early-stopping. This paper demonstrates that the most effective poisons are those that a model learns quickly, and it advocates for "peak test accuracy" rather than final accuracy as the standard metric for evaluating poison strength.

What Problem Does This Paper Address?

Data poisoning—specifically "availability attacks"—aims to make a dataset "unlearnable." By adding imperceptible perturbations to images, researchers hope to prevent unauthorized parties from training high-performing models on their data, effectively acting as a privacy shield.

However, the authors point out a major flaw in current evaluation: most poisons are judged by the model's accuracy at the end of training. In the real world, practitioners use early-stopping, a passive defense where they save the model that performs best on a validation set during the training process. If a poison only works after 100 epochs but allows the model to reach high accuracy at epoch 10, the poison is effectively useless. The authors set out to understand what makes a poison robust against this common practice.

Methods

The researchers benchmarked three categories of poisons on their vulnerability to early-stopping:

  1. Error-Minimizing Noise: Also known as "unlearnable examples," these trick the model into finding a "shortcut" by learning the noise pattern instead of actual image features.
  2. Error-Maximizing Noise: These use adversarial attacks (like PGD or MI-FGSM) to create perturbations that maximize the training loss.
  3. Synthetic Poisons: The authors introduced two new types of noise designed to exploit the inductive biases of Convolutional Neural Networks (CNNs):
  4. Regions Noise: Spatially local patches of color.
  5. Low-Frequency Noise: Perturbations created using Discrete Cosine Transforms (DCT) to mimic the low-frequency patterns CNNs tend to prioritize.

The core of their evaluation is Peak Test Accuracy: the highest accuracy a model achieves on clean data at any point during the training process, rather than just the final epoch.

Datasets

The study primarily utilizes the CIFAR-10 dataset, a standard benchmark in computer vision consisting of 60,000 small color images across 10 classes. They evaluated these poisons across several popular architectures, including ResNet-18, VGG-19, MobileNet, and GoogLeNet, to test for transferability.

Key Insights and Ideas

  • The Speed-Effectiveness Correlation: The most striking finding is that poisons learned faster are more effective. There is a strong correlation between how quickly a model reaches a low training loss and how low the peak test accuracy remains. If a model "swallows" the poison immediately, it never has the chance to learn the underlying clean features.
Figure 1. Poisons which are learned faster have lower peak test accuracy on CIFAR-10. For every poison, we log both the epoch after which the training loss dips below 0.5 and the peak test accu- racy over 100 epochs.
  • Final Accuracy is Deceptive: A poison might result in a dismal 8% final accuracy, but if the model hit 57% accuracy at epoch 8, the attack is 7x less effective than it appears. Figure 2 highlights that many "effective" poisons by final accuracy standards actually have very high peak accuracies.
Figure 2. There are a number of effective poisons for which final test accuracy doesn’t correlate well with peak test accuracy. Ef- fective poisons have both low peak test accuracy and low final test accuracy.
  • Stronger Attacks $\neq$ Better Poisons: Counter-intuitively, increasing the "strength" of an adversarial attack (e.g., more PGD steps) or its transferability to other architectures does not necessarily make it a better poison. In fact, some of the most transferable adversarial attacks (like FCSD) performed poorly as poisons because they weren't learned quickly enough by the victim network.

Key Results

  • The P250 Failure: The state-of-the-art error-maximizing poison (P250) achieved a final accuracy of ~8%, but a peak accuracy of 57.3%. Stopping training at epoch 8 completely circumvents the attack.
  • Synthetic Success: Their "Regions" noise (specifically R4 and R16) proved surprisingly potent. By exploiting the CNN bias for local features, these poisons were learned very rapidly, keeping peak test accuracy as low as 24.5%.
  • Visualizing the Noise: Figure 3 shows how different these perturbations look. While adversarial noises (b, c) look like high-frequency static, the synthetic regions (d) and low-frequency (e) noises are much more structured.
Figure 3. A CIFAR-10 image and its corresponding normalized perturbation from different poisons labeled by IDs from Tab. 1. The original clean sample is shown in (a). Error-minimizing and error-maximizing poisons are shown in columns (b) and (c). Synthetic random poisons are shown in columns (d) and (e).

Limitations and Future Work

The authors acknowledge that while their synthetic "Regions" noise is highly effective against early-stopping, it is slightly perceptible to the human eye, unlike adversarial perturbations. This creates a trade-off between "stealth" and "strength."

Future research should focus on developing poisons that are both visually imperceptible and optimized to be learned by neural networks as early as possible in the training cycle. The authors conclude that until poisons can consistently maintain low peak accuracy, they cannot be considered a reliable tool for data privacy.

Summary updated

Abstract

Imperceptible poisoning attacks on entire datasets have recently been touted as methods for protecting data privacy. However, among a number of defenses preventing the practical use of these techniques, early-stopping stands out as a simple, yet effective defense. To gauge poisons' vulnerability to early-stopping, we benchmark error-minimizing, error-maximizing, and synthetic poisons in terms of peak test accuracy over 100 epochs and make a number of surprising observations. First, we find that poisons that reach a low training loss faster have lower peak test accuracy. Second, we find that a current state-of-the-art error-maximizing poison is 7 times less effective when poison training is stopped at epoch 8. Third, we find that stronger, more transferable adversarial attacks do not make stronger poisons. We advocate for evaluating poisons in terms of peak test accuracy.
BibTeX
@misc{segura2022poisons,
  author = {Pedro Sandoval Segura and Vasu Singla and Liam Fowl and Jonas Geiping and Micah Goldblum and David Jacobs and Tom Goldstein},
  title = {{Poisons that are learned faster are more effective}},
  year = {2022},
  eprint = {2204.08615},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2204.08615},
}
Figure