Abstract
Multimodal contrastive learning methods like CLIP train on noisy and uncurated training datasets. This is cheaper than labeling datasets manually, and even improves out-of-distribution robustness. We show that this practice makes backdoor and poisoning attacks a significant threat. By poisoning just 0.01% of a dataset (e.g., just 300 images of the 3 million-example Conceptual Captions dataset), we can cause the model to misclassify test images by overlaying a small patch. Targeted poisoning attacks, whereby the model misclassifies a particular test input with an adversarially-desired label, are even easier requiring control of 0.0001% of the dataset (e.g., just three out of the 3 million images). Our attacks call into question whether training on noisy and uncurated Internet scrapes is desirable.
Introduction
Modern computer vision has undergone a paradigm shift. Instead of relying on meticulously labeled datasets like ImageNet, state-of-the-art models now utilize multimodal contrastive learning. Models such as OpenAI’s CLIP are trained on massive, uncurated scrapes of the Internet, learning to associate images with their corresponding text captions. While this approach scales effortlessly and enhances robustness to natural distribution shifts, it introduces a significant security vulnerability: the lack of data curation provides a massive surface for adversarial poisoning.
This paper investigates whether the unique properties of contrastive learning allow attackers to hide malicious "needles" within the "haystack" of the Internet. We demonstrate that these models are alarmingly susceptible to both targeted poisoning and backdoor attacks with minimal adversarial budgets.
Attack Methodology
We adapt two primary attack vectors to the multimodal contrastive setting:
- Targeted Poisoning: The adversary aims to force the model to misclassify a specific test image ($x'$) as a target label ($y'$). This is achieved by injecting pairs of $(x', c)$ into the training set, where $c$ is a caption associated with the target concept.
- Backdoor Attacks: The goal is to induce misclassification of any image containing a specific "trigger" patch. By injecting images containing this trigger paired with target-label captions, the model learns a spurious correlation between the patch and the label.
The Caption Set Innovation
A significant challenge in poisoning contrastive models is the dual-encoder architecture. The model might "solve" the training objective by updating the text encoder rather than the image encoder. To ensure the image encoder is successfully poisoned, we utilize a diverse caption set. Instead of a single repetitive caption, we use various descriptions of the target concept (e.g., "a photo of a basketball," "a kid playing basketball"). This forces the image encoder to internalize the visual features of the target concept.
Experimental Results
Our evaluation across the Conceptual Captions (CC-3M) and YFCC-15M datasets reveals several critical insights into the vulnerability of contrastive learning.
Extreme Efficiency and Scaling
Contrastive models are orders of magnitude easier to compromise than traditional supervised models. While supervised poisoning typically requires approximately 1% of the dataset, we found that CLIP-style models can be backdoored with as little as 0.01% of the data. For targeted poisoning, the requirement drops to a staggering 0.0001%—just 3 images in a 3-million-sample dataset.
Furthermore, we observed a "Capacity Paradox": larger, more capable models are actually easier to backdoor. Their increased capacity allows them to more effectively memorize the subtle, adversarial relationship between a trigger patch and a target label. Surprisingly, scaling the dataset size does not act as a defense; the attack success rate remains relatively constant as long as the absolute number of poisoned samples is maintained.
Conclusion and Future Work
The vulnerability of contrastive learning to low-budget poisoning poses a serious challenge to the current trend of training on uncurated web data. Defending against such attacks is non-trivial, as manual review of millions of images is economically unfeasible. While zero-shot models are highly susceptible, linear probes show slightly more resilience, suggesting that the way features are utilized post-training may offer a path toward robustness. Future research must focus on monitoring training dynamics to identify and filter these "atypical" adversarial samples.
Summary updated