TL;DR
This paper explores the vulnerability of crowd counting models to backdoor attacks, a type of data poisoning that can manipulate model predictions. The authors introduce two novel attacks, DMBA$^-$ and DMBA$^+$, which effectively cause crowd counting models to predict arbitrarily low or high counts when a specific trigger is present, demonstrating a significant security risk for these safety-critical applications.
What Problem Does This Paper Address?
Crowd counting, the task of estimating the number of people in an image, is crucial for applications like traffic management and public safety. Deep learning models, particularly Convolutional Neural Networks (CNNs), are state-of-the-art for this task. However, like many deep learning models, they are susceptible to backdoor attacks. These attacks involve subtly poisoning the training data with a "trigger" pattern, which then causes the model to behave maliciously (e.g., predict a specific incorrect count) when the trigger is present during inference. While backdoor attacks have been extensively studied for image classification, crowd counting presents unique challenges because it's a regression task that outputs a multi-dimensional density map, not a single class label. This paper addresses the gap in understanding and demonstrating how backdoor attacks can be effectively mounted against these density-map-based crowd counting models.
Methods
The authors propose two novel "dirty-label" backdoor attacks, collectively called Density Manipulation Backdoor Attacks (DMBA). Both attacks involve two main steps:
- Trigger Injection: Unlike traditional small-patch triggers used in image classification, which are ineffective for crowd counting, DMBAs utilize large, dense background triggers (e.g., "rain" or "snow" patterns). These triggers are blended into the training images using a convex combination (e.g., 70% clean image, 30% trigger image) to make the attack stealthy and effective.
- Density Map Altering: This is the core novelty for crowd counting. Instead of simply flipping labels, DMBAs manipulate the ground truth density maps of the poisoned images.
- DMBA$^-$ (Escape Counting): This attack aims to make the model predict very low counts. It achieves this by randomly erasing a proportion of the labeled heads (and thus their corresponding density contributions) from the ground truth density map.
- DMBA$^+$ (Boost Counting): This attack aims to make the model predict very high counts, potentially causing false alarms. It uses a "neighbor boosting" strategy to add new, synthetic head locations around existing ones, effectively increasing the density and thus the predicted count.
Datasets
The authors evaluated their attacks on four benchmark crowd counting datasets:
- ShanghaiTech A & B (SHA & SHB): These are standard, widely used datasets for crowd counting, with SHA being more challenging due to diverse scenes and higher densities.
- Venice: A dataset with fixed-resolution annotated frames.
- TRANCOS: A dataset containing masked images with annotated vehicles.
Key Insights and Ideas
- Small triggers fail for crowd counting: Traditional small, localized triggers that work for image classification are ineffective against crowd counting models. The dense, multi-dimensional nature of density maps and the presence of background clutter overwhelm these small triggers.
- Large, dense background triggers are crucial: The effectiveness of backdoor attacks on crowd counting hinges on using large, dense background patterns as triggers. These patterns are more likely to influence the model's overall density estimation.
- Density map manipulation is key: Simply injecting a trigger isn't enough. The ground truth density maps must be altered to create a strong correlation between the trigger and the desired (low or high) count. This is a fundamental difference from classification attacks.
- Regression attacks are harder for extreme targets: Achieving extreme target counts (very low or very high) is more challenging than moderate ones. This is because the model's learned regular density maps can naturally resist extreme deviations.
- Defenses for classification are not directly transferable: Standard backdoor defenses developed for image classification (like pruning) show limited effectiveness against these crowd counting backdoor attacks and often degrade clean model performance.
Key Results
- High attack success rates: The proposed DMBA attacks achieved high success rates, consistently manipulating crowd counting models to predict counts close to the targeted ratio (p) with low poisoning rates (e.g., 10%).
- Stealthy performance: Backdoored models maintained near-perfect performance on clean, non-triggered images (Pclean close to 1), indicating the attacks were stealthy.
- Effectiveness across models and datasets: The attacks were demonstrated to be effective against five different crowd counting models (CSRnet, BayesianCC, CAN, SFAnet, KDMG) and across the four diverse datasets.
- Trigger analysis: Experiments showed that "Rain" and "Snow" triggers were more effective than "Light" triggers, and larger trigger sizes generally led to better attack performance.
- Defense resistance: The DMBA attacks showed considerable resistance to common defenses like Pruning, Fine-Pruning, and Adversarial Neuron Pruning (ANP), with defenses often degrading clean performance more than mitigating the attack.
Limitations and Future Work
- Dataset-specific robustness: While generally effective, the attacks showed slightly reduced performance on the more challenging SHA dataset, suggesting that datasets with more diverse scenes and higher densities might offer some natural robustness.
- Extreme target difficulty: Achieving very extreme target counts (e.g., p=0.2 or p=2) was more difficult than moderate ones, indicating a need for further research into more potent density manipulation strategies.
- Future Work: The authors suggest their work can serve as a baseline for evaluating the robustness of crowd counting models and for developing more effective defense mechanisms. They also imply that exploring other threat models (beyond data poisoning) and more sophisticated density manipulation techniques could be fruitful.
Summary updated