Backdoor Attacks on Crowd Counting

Yuhua Sun, Tailai Zhang, Xingjun Ma, Pan Zhou, Jian Lou, Zichuan Xu, Xing Di, Yu Cheng, Lichao Sun

TL;DR

This paper explores the vulnerability of crowd counting models to backdoor attacks, a type of data poisoning that can manipulate model predictions. The authors introduce two novel attacks, DMBA$^-$ and DMBA$^+$, which effectively cause crowd counting models to predict arbitrarily low or high counts when a specific trigger is present, demonstrating a significant security risk for these safety-critical applications.

What Problem Does This Paper Address?

Crowd counting, the task of estimating the number of people in an image, is crucial for applications like traffic management and public safety. Deep learning models, particularly Convolutional Neural Networks (CNNs), are state-of-the-art for this task. However, like many deep learning models, they are susceptible to backdoor attacks. These attacks involve subtly poisoning the training data with a "trigger" pattern, which then causes the model to behave maliciously (e.g., predict a specific incorrect count) when the trigger is present during inference. While backdoor attacks have been extensively studied for image classification, crowd counting presents unique challenges because it's a regression task that outputs a multi-dimensional density map, not a single class label. This paper addresses the gap in understanding and demonstrating how backdoor attacks can be effectively mounted against these density-map-based crowd counting models.

Methods

The authors propose two novel "dirty-label" backdoor attacks, collectively called Density Manipulation Backdoor Attacks (DMBA). Both attacks involve two main steps:

  1. Trigger Injection: Unlike traditional small-patch triggers used in image classification, which are ineffective for crowd counting, DMBAs utilize large, dense background triggers (e.g., "rain" or "snow" patterns). These triggers are blended into the training images using a convex combination (e.g., 70% clean image, 30% trigger image) to make the attack stealthy and effective.
  2. Density Map Altering: This is the core novelty for crowd counting. Instead of simply flipping labels, DMBAs manipulate the ground truth density maps of the poisoned images.
  3. DMBA$^-$ (Escape Counting): This attack aims to make the model predict very low counts. It achieves this by randomly erasing a proportion of the labeled heads (and thus their corresponding density contributions) from the ground truth density map.
  4. DMBA$^+$ (Boost Counting): This attack aims to make the model predict very high counts, potentially causing false alarms. It uses a "neighbor boosting" strategy to add new, synthetic head locations around existing ones, effectively increasing the density and thus the predicted count.

Datasets

The authors evaluated their attacks on four benchmark crowd counting datasets:

  • ShanghaiTech A & B (SHA & SHB): These are standard, widely used datasets for crowd counting, with SHA being more challenging due to diverse scenes and higher densities.
  • Venice: A dataset with fixed-resolution annotated frames.
  • TRANCOS: A dataset containing masked images with annotated vehicles.

Key Insights and Ideas

  • Small triggers fail for crowd counting: Traditional small, localized triggers that work for image classification are ineffective against crowd counting models. The dense, multi-dimensional nature of density maps and the presence of background clutter overwhelm these small triggers.
  • Large, dense background triggers are crucial: The effectiveness of backdoor attacks on crowd counting hinges on using large, dense background patterns as triggers. These patterns are more likely to influence the model's overall density estimation.
  • Density map manipulation is key: Simply injecting a trigger isn't enough. The ground truth density maps must be altered to create a strong correlation between the trigger and the desired (low or high) count. This is a fundamental difference from classification attacks.
  • Regression attacks are harder for extreme targets: Achieving extreme target counts (very low or very high) is more challenging than moderate ones. This is because the model's learned regular density maps can naturally resist extreme deviations.
  • Defenses for classification are not directly transferable: Standard backdoor defenses developed for image classification (like pruning) show limited effectiveness against these crowd counting backdoor attacks and often degrade clean model performance.

Key Results

  • High attack success rates: The proposed DMBA attacks achieved high success rates, consistently manipulating crowd counting models to predict counts close to the targeted ratio (p) with low poisoning rates (e.g., 10%).
  • Stealthy performance: Backdoored models maintained near-perfect performance on clean, non-triggered images (Pclean close to 1), indicating the attacks were stealthy.
  • Effectiveness across models and datasets: The attacks were demonstrated to be effective against five different crowd counting models (CSRnet, BayesianCC, CAN, SFAnet, KDMG) and across the four diverse datasets.
  • Trigger analysis: Experiments showed that "Rain" and "Snow" triggers were more effective than "Light" triggers, and larger trigger sizes generally led to better attack performance.
  • Defense resistance: The DMBA attacks showed considerable resistance to common defenses like Pruning, Fine-Pruning, and Adversarial Neuron Pruning (ANP), with defenses often degrading clean performance more than mitigating the attack.

Limitations and Future Work

  • Dataset-specific robustness: While generally effective, the attacks showed slightly reduced performance on the more challenging SHA dataset, suggesting that datasets with more diverse scenes and higher densities might offer some natural robustness.
  • Extreme target difficulty: Achieving very extreme target counts (e.g., p=0.2 or p=2) was more difficult than moderate ones, indicating a need for further research into more potent density manipulation strategies.
  • Future Work: The authors suggest their work can serve as a baseline for evaluating the robustness of crowd counting models and for developing more effective defense mechanisms. They also imply that exploring other threat models (beyond data poisoning) and more sophisticated density manipulation techniques could be fruitful.

Summary updated

Abstract

Crowd counting is a regression task that estimates the number of people in a scene image, which plays a vital role in a range of safety-critical applications, such as video surveillance, traffic monitoring and flow control. In this paper, we investigate the vulnerability of deep learning based crowd counting models to backdoor attacks, a major security threat to deep learning. A backdoor attack implants a backdoor trigger into a target model via data poisoning so as to control the model's predictions at test time. Different from image classification models on which most of existing backdoor attacks have been developed and tested, crowd counting models are regression models that output multi-dimensional density maps, thus requiring different techniques to manipulate. In this paper, we propose two novel Density Manipulation Backdoor Attacks (DMBA$^{-}$ and DMBA$^{+}$) to attack the model to produce arbitrarily large or small density estimations. Experimental results demonstrate the effectiveness of our DMBA attacks on five classic crowd counting models and four types of datasets. We also provide an in-depth analysis of the unique challenges of backdooring crowd counting models and reveal two key elements of effective attacks: 1) full and dense triggers and 2) manipulation of the ground truth counts or density maps. Our work could help evaluate the vulnerability of crowd counting models to potential backdoor attacks.
BibTeX
@inproceedings{sun2022backdoor,
  author = {Yuhua Sun and Tailai Zhang and Xingjun Ma and Pan Zhou and Jian Lou and Zichuan Xu and Xing Di and Yu Cheng and Lichao Sun},
  title = {{Backdoor Attacks on Crowd Counting}},
  booktitle = {ACM Multimedia},
  year = {2022},
  doi = {10.1145/3503161.3548296},
}
Figure