Poisoning and Backdooring Contrastive Learning

Nicholas Carlini, Andreas Terzis

Multimodal contrastive learning models like CLIP are celebrated for their ability to learn from massive, uncurated Internet datasets. By scraping billions of image-caption pairs from the web, these models avoid the high costs of manual labeling and achieve impressive out-of-distribution robustness. However, this paper reveals a major security flaw in this paradigm: because these datasets are never manually reviewed, an adversary can "poison" the model by injecting a tiny number of malicious examples.

Astoundingly, it takes as few as three poisoned images out of three million to successfully execute a targeted attack, making these contrastive models significantly more vulnerable to data poisoning than traditional supervised classifiers.

The Vulnerability of Uncurated Data

Modern AI models like CLIP aren't trained on neatly labeled, human-verified datasets like ImageNet. Instead, they rely on the sheer volume of the Internet. While this "uncurated" approach is cheap and improves the model's ability to handle diverse, real-world data, the authors argue that this lack of human oversight is a double-edged sword.

If an adversary can get just a handful of malicious images onto the Internet where they might be scraped by dataset creators, they can compromise the resulting model. The paper investigates two primary types of attacks:

  1. Targeted Poisoning: Manipulating the model to misclassify one specific target image (e.g., forcing a photo of a specific person to be recognized as a "basketball").
  2. Backdoor Attacks: Forcing the model to misclassify any image that contains a specific visual trigger, such as a small checkerboard patch.
An example of a backdoor trigger: a small checkerboard patch overlaid on an image.

Attack Methodologies

The authors adapt existing poisoning techniques to exploit the unique architecture of contrastive learning, which relies on maximizing the similarity between an image and its corresponding text caption.

  • Multi-Sample Poisoning: To make a target image be classified as a specific label, the attacker creates a "caption set"—a collection of diverse text descriptions related to that target label (e.g., "a photo of a basketball," "a kid playing basketball"). They then inject pairs of the target image and these captions into the training set. Because contrastive models try to pull images and their captions closer together in the embedding space, this forces the model to associate the target image's visual features with the "basketball" concept.
  • Backdoor Attack: Instead of using one specific image, the attacker takes many random images, overlays a small patch (the "trigger"), and pairs them with captions belonging to the target label.
  • Backdoor Z-Score: To evaluate these attacks efficiently without spending tens of thousands of GPU hours on every experiment, the authors introduced a new metric. It measures how "atypical" the similarity is between two backdoored images compared to the normal distribution of natural image similarities.

Key Findings and Results

The study evaluated these attacks using two major "Internet-scale" datasets: Conceptual Captions (CC-3M, containing 3 million pairs) and a 15-million-image subset of YFCC (YFCC-15M). The results highlight a severe security gap.

Probability of attack success relative to the number of poisoned samples injected into the training data.
  • Extreme Efficiency: The most shocking finding is the "poisoning ratio." Traditional supervised models usually require poisoning roughly 1% of the training data to be compromised. Contrastive models can be broken with just 0.0001% (3 images in 3 million) for targeted attacks, yielding a 40% success rate across 1,000 possible classes. For backdoor attacks, a 0.01% ratio (300 images) achieved a 50% attack success rate on zero-shot classifiers.
  • The Paradox of Scale: Usually, more data and larger model architectures make AI more robust. Here, the opposite is true. The authors found that larger models are actually easier to backdoor because their increased capacity allows them to more easily "memorize" the malicious trigger pattern.
  • Trigger Specifications: A trigger patch as small as 4x4 pixels (in a standard 224x224 image) is enough to initiate successful attacks, though a 16x16 patch is more reliable. Furthermore, while placing the patch in a consistent location (like the top-left corner) works well for small-scale poisoning, placing the patch randomly makes the attack much more powerful and harder to detect if the attacker can poison more samples.
  • Evasion of Standard Defenses: Standard data cleaning practices, such as removing exact duplicates, are easily bypassed by adding a tiny amount of invisible Gaussian noise to the poisoned images.

Limitations and Future Directions

Studying vulnerabilities in foundation models is inherently difficult due to the massive computational cost. The authors noted that even a limited ablation study required roughly 10,000 GPU hours, making it challenging for researchers with fewer resources to study these threats or develop defenses.

Crucially, this paper focuses on demonstrating the attack and does not propose a definitive defense. The authors suggest that monitoring training dynamics—specifically how the model learns specific samples over time—might be a promising avenue to detect poisoned data without manual review. Finally, while this research primarily focuses on poisoning the image side of the dataset, the authors note that the text embedding side is likely just as vulnerable, warranting urgent future investigation.

Summary updated

Abstract

Multimodal contrastive learning methods like CLIP train on noisy and uncurated training datasets. This is cheaper than labeling datasets manually, and even improves out-of-distribution robustness. We show that this practice makes backdoor and poisoning attacks a significant threat. By poisoning just 0.01% of a dataset (e.g., just 300 images of the 3 million-example Conceptual Captions dataset), we can cause the model to misclassify test images by overlaying a small patch. Targeted poisoning attacks, whereby the model misclassifies a particular test input with an adversarially-desired label, are even easier requiring control of 0.0001% of the dataset (e.g., just three out of the 3 million images). Our attacks call into question whether training on noisy and uncurated Internet scrapes is desirable.
BibTeX
@inproceedings{carlini2022poisoning,
  author = {Nicholas Carlini and Andreas Terzis},
  title = {{Poisoning and Backdooring Contrastive Learning}},
  booktitle = {International Conference on Learning Representations},
  year = {2022},
  url = {https://openreview.net/forum?id=iC4UHbQ01Mp},
}
Figure