Abstract
Crowd counting has drawn much attention due to its importance in safety-critical surveillance systems. Especially, deep neural network (DNN) methods have significantly reduced estimation errors for crowd counting missions. Recent studies have demonstrated that DNNs are vulnerable to adversarial attacks, i.e., normal images with human-imperceptible perturbations could mislead DNNs to make false predictions. In this work, we propose a robust attack strategy called Adversarial Patch Attack with Momentum (APAM) to systematically evaluate the robustness of crowd counting models, where the attacker's goal is to create an adversarial perturbation that severely degrades their performances, thus leading to public safety accidents (e.g., stampede accidents). Especially, the proposed attack leverages the extreme-density background information of input images to generate robust adversarial patches via a series of transformations (e.g., interpolation, rotation, etc.). We observe that by perturbing less than 6% of image pixels, our attacks severely degrade the performance of crowd counting systems, both digitally and physically. To better enhance the adversarial robustness of crowd counting models, we propose the first regression model-based Randomized Ablation (RA), which is more sufficient than Adversarial Training (ADT). Extensive experiments on five crowd counting models demonstrate the effectiveness and generality of the proposed method.
Introduction
Crowd counting is a high-stakes application of Deep Neural Networks (DNNs), used for critical tasks such as monitoring social distancing and preventing stampedes. While DNNs have achieved high accuracy, they remain vulnerable to adversarial examples—minor, often invisible perturbations that can cause catastrophic failures. In real-world scenarios, "adversarial patches"—visible stickers placed within a scene—pose a more practical and dangerous threat than digital pixel tweaks.
Most existing research on adversarial patches focuses on classification tasks. This paper addresses the more complex challenge of regression in crowd counting, where the goal is to disrupt a continuous numerical output. Current models are highly susceptible to these patches, and traditional defenses like Adversarial Training (ADT) often fail to provide reliable protection.
Methodology
The researchers developed a two-pronged approach: a potent new attack strategy (APAM) and a robust certified defense (RA).
Adversarial Patch Attack with Momentum (APAM)
To create a "universal" patch effective across various camera angles and placements, the authors developed APAM. This method employs:
- Image Transformations: The patch is digitally rotated, scaled, and interpolated during creation to ensure it remains effective when printed and viewed through real surveillance cameras.
- Momentum-based Optimization: Momentum is used in the gradient descent process to avoid local minima, resulting in more stable and powerful perturbations.
- Black-box Transferability: Patches generated for one model can successfully fool unknown, different architectures.
Randomized Ablation (RA)
Randomized Ablation is the first certified defense for regression-based crowd counting. Instead of training on full images, the model is trained on images where only a small number of pixels ($k$) are retained, and the rest are set to zero (ablated).
- Statistical Defense: If an adversarial patch covers a small percentage of an image, there is a high probability that the ablation process will remove most of the patch's pixels while retaining enough of the original scene for accurate counting.
- Certification: Unlike empirical defenses, RA provides a theoretical lower bound on the maximum disruption an attacker can cause.
Experimental Results
The study utilized the ShanghaiTech dataset (Part A) and tested five major architectures: CSRNet, DA-Net, MCNN, CAN, and CMTL.
Attack Potency
Experiments revealed that patches covering less than 6% of the image can cause count predictions to swing by hundreds. In white-box settings, a 163x163 pixel patch increased the Mean Absolute Error (MAE) of the CSRNet model from 68.20 to a staggering 463.53. Different models exhibited different failure modes; some massively over-counted, while others were tricked into seeing nearly zero people.
Defense Performance
Randomized Ablation (RA) significantly outperformed standard Adversarial Training (ADT). On adversarial examples, the MAE for RA was 30 points lower than ADT, proving it is far more effective at neutralizing patches. While RA introduces a slight accuracy trade-off on clean images, it provides a necessary safety margin for critical systems.
Conclusion
This work highlights the vulnerability of crowd counting systems to adversarial patches and introduces Randomized Ablation as a robust, certified defense. While physical-world attacks are currently less potent than digital ones due to environmental variables, the effectiveness of RA in neutralizing these threats marks a significant step toward secure and reliable crowd monitoring systems. Future work will focus on closing the accuracy gap on clean images and improving patch robustness to real-world lighting and camera conditions.
Summary updated