Traditional data poisoning and backdoor attacks often rely on a simple strategy: select a random subset of training data, inject a malicious trigger or alter the label, and let the optimization process do the rest. However, deep neural networks do not treat all training samples equally. Some samples are easily categorized by simple, robust features, while others lie near decision boundaries, causing the model high uncertainty and classification error.
A rapidly growing body of research demonstrates that an attacker can drastically increase the efficiency, stealth, and robustness of an attack by moving away from random selection. Instead, these methods explicitly target hard samples—instances where the model exhibits high uncertainty, low confidence, or large gradients. By exploiting these weak points, attackers can force models to adopt malicious triggers as predictive shortcuts with minimal data corruption.
The Intuition: Triggers as Paths of Least Resistance
To understand why hard-sample poisoning is so effective, we must look at how a model minimizes training loss.
When a model encounters an "easy" sample, its existing clean features are already sufficient to yield a correct, high-confidence prediction. If an attacker injects a backdoor trigger into this easy sample without changing its label (a clean-label attack), the model has no incentive to learn the trigger. It can simply rely on the robust, clean features it has already mastered. As Xiaopeng Xie et al. demonstrate in Shortcuts Arising from Contrast: Towards Effective and Lightweight Clean-Label Attacks in Prompt-Based Learning, the effectiveness of a backdoor shortcut is severely limited when the poisoned samples are already easy for the model to classify.
Conversely, "hard" samples present a dilemma for the optimization process. For these instances, the model's clean features are ambiguous, leading to high training loss or irregular activation patterns. If an attacker injects a clear, invariant trigger into these high-uncertainty samples, the model is handed an escape hatch. Rather than trying to resolve the complex, ambiguous semantic features of the hard sample, the optimization algorithm latches onto the simple backdoor trigger as a shortcut to minimize loss.
Easy Sample: [Robust Clean Features] ───> High Confidence Prediction (Trigger is Ignored)
Hard Sample: [Ambiguous Features + Trigger] ───> Optimization Latches onto Trigger Shortcut
By forcing the model to associate the trigger with these high-difficulty samples, the backdoor behavior generalizes far more effectively to simpler images during inference, as shown by Zihao Zhu et al. in Boosting Backdoor Attack with A Learnable Poisoning Sample Selection Strategy.
Quantifying "Hardness" and Uncertainty
To exploit this vulnerability, researchers have developed several distinct strategies to identify and isolate the most uncertain samples in a dataset:
1. Confidence Margins and Boundary Proximity
One direct way to find hard samples is to look at the model's output logits. In Stealthy Backdoor Attack via Confidence-driven Sampling, Pengfei He et al. introduce Confidence-driven Boundary Sampling (CBS). This strategy selects candidates based on the minimal difference in confidence scores between their ground-truth class and a target class. Because these samples sit immediately adjacent to the decision boundary, shifting them across the boundary via a trigger requires minimal perturbation, making the attack incredibly stealthy and resilient against defenses.
2. Training Loss and Prediction Errors
Samples that consistently yield high training loss or prediction errors are natural targets. In the domain of speech processing, Henry Li Xinyuan et al. demonstrate in Clean Label Attacks against SLU Systems that targeting high-loss samples allows an attacker to achieve near-perfect success rates while compromising only a tiny fraction (1.5%) of the training data. Similarly, in multivariate time-series forecasting, Xiao Lin et al. show in BackTime: Backdoor Attacks on Multivariate Time Series Forecasting that timestamps where a model naturally exhibits high prediction error provide the most vulnerable entry points for malicious manipulation.
3. Training Dynamics and Forgetting
Uncertainty can also be measured over time by tracking how a model's predictions fluctuate during training. Yadong Wang et al. utilize frequent "forgetting events"—instances where a model correctly classifies a sample in one epoch but misclassifies it in a later one—to identify structurally or semantically ambiguous samples in Boosting Clean-Label Backdoor Attacks on Graph Classification. Along a similar line, Zhixiao Wu et al. track the diversity of incorrect classes predicted for a given input across training epochs in A Set of Generalized Components to Achieve Effective Poison-only Clean-label Backdoor Attacks with Collaborative Sample Selection and Triggers. High category diversity indicates deep model confusion, signaling an ideal candidate for poisoning.
4. Gradient Magnitude
In decentralized settings like Vertical Federated Learning (VFL), an attacker might not have direct access to labels or model outputs. To circumvent this, Wei Shen 0006 et al. propose using embedding gradients as a proxy for difficulty in Label-Free Backdoor Attacks in Vertical Federated Learning. Selecting samples with the largest gradients isolates the exact data points the model is struggling most to fit, allowing the backdoor to be learned efficiently without disrupting overall task accuracy.
The Clean-Label Imperative
Targeting hard samples is not just a matter of efficiency; it is often a necessity for the survival of clean-label attacks. In a clean-label attack, the poisoned data must appear completely normal to human inspectors and retain its correct label in the dataset.
If an attacker attempts a clean-label attack on a highly certain, prototypical sample of a class, the model easily categorizes it using its dominant features and ignores the trigger. But if the primary content is ambiguous or hard to categorize, the learning algorithm is forced to rely on the injected trigger as a predictive shortcut.
For text classifiers, Wencong You et al. exploit this in Large Language Models Are Better Adversaries: Exploring Generative Clean-Label Backdoor Attacks Against Text Classifiers by using LLMs to paraphrase hard instances into specific styles (e.g., Shakespearean prose). Because the underlying text is already difficult for the model to classify, it readily adopts the stylistic choice as a shortcut.
This principle spans across modalities:
- Graph Neural Networks (GNNs): In node classification, Hui Xia 0001 et al. show in Clean-Label Graph Backdoor Attack in the Node Classification Task that highly certain nodes resist new feature associations. Attackers must target low-degree, high-uncertainty nodes to inject triggers successfully. Yuxiang Zhang et al. expand on this in Poisoning the Inner Prediction Logic of Graph Neural Networks for Clean-Label Backdoor Attacks, demonstrating that uncertain nodes exhibit irregular structural patterns that make them uniquely susceptible to altering the model's inner prediction logic.
- Graph Classification: For entire graphs, Xiaobao Wang et al. demonstrate in Stealthy Yet Effective: Distribution-Preserving Backdoor Attacks on Graph Classification that targeting high-uncertainty graphs allows for effective poisoning with minimal, distribution-preserving perturbations that easily bypass anomaly detectors.
- Availability Attacks: Even when the goal is overall performance degradation rather than a backdoor, targeting naturally confused pairs of classes allows the attack to efficiently tank model accuracy, as shown by Bingyin Zhao et al. in CLPA: Clean-Label Poisoning Availability Attacks Using Generative Adversarial Nets.
Defensive Implications
Understanding that data poisoning thrives on model uncertainty provides a powerful lens for defenders. If attackers are systematically drawn to the hardest samples in a dataset, defenders can focus their scrutiny on those exact regions.
In Not All Samples Are Equal: Quantifying Instance-level Difficulty in Targeted Data Poisoning, Weina Xu et al. confirm a strong negative correlation between a sample's clean classification difficulty and its susceptibility to targeted data poisoning. By designing attack-agnostic metrics to quantify instance-level difficulty using only clean training data, defenders can proactively identify and protect high-risk samples before an attack even occurs.
Key Foundational Work
Large Language Models Are Better Adversaries: Exploring Generative Clean-Label Backdoor Attacks Against Text Classifiers (2023) — contributed 2 concept elements to this direction.
Boosting Backdoor Attack with A Learnable Poisoning Sample Selection Strategy (2023) — contributed 2 concept elements to this direction.
CLPA: Clean-Label Poisoning Availability Attacks Using Generative Adversarial Nets (2022) — contributed 1 concept elements to this direction.