Backdoor Attacks on Unlabeled Data

56 concepts35 papers1 proposal

In traditional supervised machine learning, a backdoor attack is intuitive. An attacker takes a dataset, adds a specific trigger (like a small pixel patch) to a subset of images, and changes their labels to a malicious target category (e.g., labeling a stop sign with a sticky note as a "speed limit sign"). During training, the model learns to associate the trigger with the incorrect label.

But modern machine learning increasingly relies on unlabeled data. Foundation models and vision encoders are pre-trained on massive, uncurated datasets scraped from the internet using Self-Supervised Learning (SSL), while Semi-Supervised Learning (SSL) pipelines combine a handful of labeled examples with massive pools of unlabeled data.

For a long time, it was assumed that these paradigms were inherently resistant to backdoor poisoning. After all, if there are no labels for an attacker to manipulate, how can they force a model to learn a specific malicious association?

Recent research has shattered this assumption. Attackers do not need to manipulate labels to plant a backdoor; instead, they can exploit the core mathematical objectives of unsupervised representation learning to warp a model's internal geometry.

Warping the Embedding Space in Self-Supervised Learning

To understand why unlabeled data poisoning works in self-supervised learning, we have to look at how these models learn. Most modern SSL frameworks use contrastive learning. The model is given two different augmented views of the same image (e.g., cropped, color-jittered) and told to bring their vector representations (embeddings) closer together in the embedding space, while pushing the representations of different images apart.

An attacker can exploit this mechanism by injecting a consistent trigger into a small fraction of the unlabeled dataset. As shown by Saha et al., when the model encounters different images that happen to share the same malicious trigger patch, the contrastive loss forces the model to treat the trigger itself as the dominant semantic feature.

Standard Contrastive Learning:
[Image A: View 1] <--- pull closer ---> [Image A: View 2]

Poisoned Contrastive Learning:
[Image A + Trigger] <--- pull closer ---> [Image B + Trigger]

Because the trigger is the most consistent pattern across these otherwise distinct images, the encoder learns to map any input containing the trigger to a specific, tightly packed region of the embedding space. Wang et al. formalize this as the No-Label Backdoor (NLB) problem, demonstrating that by leveraging feature similarity, an attacker can reliably bind a trigger to a specific cluster of images without ever knowing or modifying a single label.

Attackers have refined this capability to make triggers increasingly difficult to detect. For instance, Li et al. introduced CTRL (Contrastive Trojan Learning), which uses subtle, frequency-based triggers rather than visible image patches. To bypass data augmentations that might accidentally strip away these triggers, Zhang developed INACTIVE, an attack that mathematically disentangles the trigger from standard SSL augmentations, achieving near 100% attack success rates while remaining completely invisible to human inspectors.

The Pseudo-Label Feedback Loop in Semi-Supervised Learning

In semi-supervised learning, the vulnerability stems from a different mechanism: consistency regularization and pseudo-labeling. These models use a small set of labeled data to predict labels (pseudo-labels) for a massive pool of unlabeled data. If the model predicts a label with high confidence, that unlabeled image is treated as ground-truth training data for subsequent iterations.

This creates a dangerous vulnerability. As demonstrated by Yan et al. in their DeHiB (Deep Hidden Backdoor) attack, an adversary can apply structured adversarial perturbations to unlabeled images. These perturbations are designed to trick the model's initial decision boundaries into assigning high-confidence, incorrect pseudo-labels to the triggered images.

Once a triggered image receives an incorrect pseudo-label, a runaway feedback loop begins:

  1. The model trains on its own high-confidence pseudo-labels.
  2. The decision boundary shifts to accommodate the poisoned data.
  3. The model becomes even more confident in assigning the malicious label to any future input containing the trigger.

This mechanism leads to a striking paradox discovered by Carlini: the more accurate the semi-supervised learning algorithm, the more vulnerable it is to unlabeled data poisoning. Because high-performing models are exceptionally good at extracting structure and maintaining consistency across data pools, they are highly efficient at propagating the malicious paths laid out by the attacker. Carlini showed that poisoning just 0.1% of the unlabeled dataset is enough to reliably compromise the model.

Universal Downstream Contamination

Why should we care if an unlabeled pre-trained encoder is backdoored? The danger lies in the "pre-train then fine-tune" paradigm.

When an end-user downloads a pre-trained encoder and fine-tunes it for a specific downstream task (like medical imaging or sentiment analysis), the underlying geometric distortions remain intact. Shen et al. demonstrated that backdoors injected during the unsupervised phase map triggers to specific numerical output signatures rather than explicit labels. Consequently, the backdoor persists across completely different downstream applications.

As Zheng et al. note, these "Trojaned" encoders act as a vector for supply-chain contamination, silently passing vulnerabilities to any downstream model built on top of them. This vulnerability is not limited to standard vision models; Jin et al. demonstrated that multimodal foundation models (such as MedCLIP, which matches medical images with unpaired text descriptions) are uniquely susceptible, achieving a 99% attack success rate with as little as 0.05% poisoned data due to how label discrepancies propagate across semantic batches.

Defending Without Labels

Traditional backdoor defenses rely on identifying mismatches between inputs and their ground-truth labels. Because unlabeled poisoning operates entirely in the absence of ground-truth labels, researchers have had to develop entirely new defense paradigms that analyze embedding spaces and model dynamics.

1. Geometric and Structural Detection

If an attack works by forcing triggered inputs into a specific region, defenses can look for anomalies in the embedding space. Feng et al. developed DECREE, which detects backdoors by identifying abnormally "dense areas" in the encoder's embedding space where disparate inputs are unnaturally forced to cluster when a trigger is present.

Taking a generative approach, Hou et al. introduced DeDe (Detector via Decoders). DeDe trains a small auxiliary decoder to reconstruct images directly from the encoder's embeddings. When a triggered image is fed into a backdoored encoder, the embedding collapses into the target representation, causing the reconstructed image to look fundamentally different from the original input—flagging the attack.

Clean Input ---> Encoder ---> Embedding ---> Decoder ---> Accurate Reconstruction
Poisoned Input -> Encoder ---> Collapsed Embedding -> Decoder -> Anomalous Reconstruction (Flagged)

2. Data Purification and Distillation

Instead of just detecting attacks, some frameworks aim to clean the data or the model. Tejankar et al. developed PatchSearch, which leverages the inherent clustering properties of SSL to isolate and prune poisoned samples from the training pool without requiring human labels. In the frequency domain, Lee et al. introduced UPure, which uses the information-theoretic Rate-Distortion-Perception trade-off to apply strategic perturbations that strip out repetitive backdoor triggers from unlabeled datasets.

Alternatively, defenses can "detoxify" the model after training. Han et al. proposed MIMIC, a framework that uses Mutual Information to locate where benign features hide versus where malicious triggers lurk, distilling only the clean knowledge into a safe student model using a tiny fraction of clean data. Similarly, Styborski et al. developed VESPR, which intentionally exploits a supervised model's tendency to overfit to shortcuts to generate adversarial augmentations that mask out those shortcuts, forcing the self-supervised encoder to focus exclusively on robust semantic features.

Key Foundational Work

Poisoning the Unlabeled Dataset of Semi-Supervised Learning (2021) — contributed 6 concept elements to this direction.
DeHiB: Deep Hidden Backdoor Attack on Semi-supervised Learning via Adversarial Perturbation (2021) — contributed 4 concept elements to this direction.
VIKING: Adversarial Attack on Network Embeddings via Supervised Network Poisoning (2021) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_unlabeled_data_poisoning_8ed4af71,
  title = {unlabeled data poisoning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/unlabeled-data-poisoning/rdr_8ed4af713b8b46d89e47224dbe097b4a},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_unlabeled_data_poisoning_8ed4af71_v1,
  title = {unlabeled data poisoning},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/unlabeled-data-poisoning/rdr_8ed4af713b8b46d89e47224dbe097b4a/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake