Abstract
Addressing data integrity challenges, such as unlearning the effects of data poisoning after model training, is necessary for the reliable deployment of machine learning models. State-of-the-art influence functions, such as EK-FAC, often fail to accurately attribute abnormal model behavior to the specific poisoned training data responsible for the data poisoning attack. In addition, traditional unlearning algorithms often struggle to effectively remove the influence of poisoned samples, particularly when only a few affected examples can be identified. To address these challenges, we introduce $\Delta$-Influence, a novel approach that leverages influence functions to trace abnormal model behavior back to the responsible poisoned training data using as little as just one poisoned test example. $\Delta$-Influence applies data transformations that sever the link between poisoned training data and compromised test points without significantly affecting clean data. This allows $\Delta$-Influence to detect large negative shifts in influence scores following data transformations, a phenomenon we term as influence collapse, thereby accurately identifying poisoned training data. Unlearning this subset, e.g. through retraining, effectively eliminates the data poisoning. We validate our method across three vision-based poisoning attacks and three datasets, benchmarking against four detection algorithms and five unlearning strategies. We show that $\Delta$-Influence consistently achieves the best unlearning across all settings, showing the promise of influence functions for corrective unlearning.
Introduction
Machine learning models are increasingly vulnerable to data poisoning, where adversaries inject subtly modified samples into training sets to induce specific misclassifications at test time. A classic example is a "Stop" sign being misidentified as a "Speed Limit" sign. While many defenses focus on preventing attacks, the "forensics" problem—identifying and removing malicious samples after a model has already been trained—remains a critical bottleneck.
Traditional influence functions are designed to attribute model predictions to specific training points. However, in deep neural networks, these functions often suffer from low precision, flagging too many clean samples and failing to isolate the true "perpetrators." This is especially true for sophisticated "clean-label" attacks like Witches’ Brew, which are visually indistinguishable from clean data. We propose $\Delta$-Influence to bridge this gap by monitoring dynamic changes in influence rather than static scores.
The $\Delta$-Influence Framework
$\Delta$-Influence moves away from static influence calculation to a dynamic "perturb-and-compare" approach. The framework operates in four distinct steps to identify and neutralize poisons.
- Identify a Victim: The process begins with a single test point ($z_{te}$) that the model misclassifies due to suspected poisoning.
- Apply Transformations: The test point is perturbed using two methods: Label Flipping (changing the test label to break the intended association) and Image Augmentation (applying blur, rotation, or color jitter).
- Measure the Shift ($\Delta$): We calculate the change in influence scores for every training point before and after these transformations. The core discovery is Influence Collapse: poisoned training samples exhibit a massive negative shift in influence when the test point is transformed, while clean samples remain relatively stable.
- Flagging and Unlearning: To minimize false positives, multiple transformations are applied. Training points that consistently show influence collapse are flagged as the "poison set" and subsequently removed via unlearning algorithms like retraining or Catastrophic Forgetting (CF).
Experimental Evaluation
We validated $\Delta$-Influence using a ResNet18 architecture across three datasets: CIFAR-10, CIFAR-100, and ImageNette. We tested the framework against three distinct attack types:
- Frequency Trigger: Attacks using spatial/frequency domain patterns.
- Witches’ Brew: Complex clean-label gradient matching attacks.
- BadNet: Classic patch-based backdoors.
Our method was benchmarked against state-of-the-art detection algorithms including EK-FAC, Activation Clustering (ActClust), and Spectral Signatures (SpecSig).
Key Results and Analysis
$\Delta$-Influence demonstrated superior performance in both detection precision and unlearning effectiveness. Notably, it achieved successful unlearning (defined as a Poison Success Rate < 5%) in 6 out of 6 tested scenarios, whereas the next best baseline only succeeded in half of those cases.
Precision and Utility Preservation
On the Frequency Trigger attack (CIFAR-10), $\Delta$-Influence achieved a precision of 13.3%, significantly higher than the 2.9% achieved by standard EK-FAC. While 13.3% may appear low in absolute terms, it is sufficiently high to allow unlearning algorithms to remove the malicious influence without degrading the model's overall accuracy on clean data. Because our method is more precise, the "Test Accuracy" remains high post-unlearning, as fewer clean samples are accidentally discarded.
Insights into Attack Vulnerability
A surprising finding was the vulnerability of the complex Witches' Brew attack. We found that removing only a few key samples identified by $\Delta$-Influence was often enough to completely neutralize the attack, even if other poisoned samples remained in the set. This suggests that certain "anchor" poisons are disproportionately responsible for the model's compromised behavior.
Discussion and Limitations
While $\Delta$-Influence represents a significant step forward, challenges remain. Computational cost is a primary concern, as influence functions are expensive for large-scale models. However, the process is highly parallelizable. Additionally, the method currently requires a "human-in-the-loop" to identify at least one poisoned test point to initiate the forensics process. Future work will explore automating this initial detection and scaling the framework to billion-parameter models.
Conclusion
$\Delta$-Influence provides a robust, generalizable framework for poisoning forensics. By leveraging the phenomenon of influence collapse, it allows practitioners to trace model failures back to specific training data with unprecedented accuracy, enabling effective corrective unlearning and enhancing the long-term reliability of deployed AI systems.
Summary updated