Abstract
Due to domain shift, a large performance drop is usually observed when a trained crowd counting model is deployed in the wild. While existing domain-adaptive crowd counting methods achieve promising results, they typically regard each crowd image as a whole and reduce domain discrepancies in a holistic manner, thus limiting further improvement of domain adaptation performance. To this end, we propose to untangle domain-invariant crowd and domain-specific background from crowd images and design a fine-grained domain adaption method for crowd counting. Specifically, to disentangle crowd from background, we propose to learn crowd segmentation from point-level crowd counting annotations in a weakly-supervised manner. Based on the derived segmentation, we design a crowd-aware domain adaptation mechanism consisting of two crowd-aware adaptation modules, i.e., Crowd Region Transfer (CRT) and Crowd Density Alignment (CDA). The CRT module is designed to guide crowd features transfer across domains beyond background distractions. The CDA module dedicates to regularising target-domain crowd density generation by its own crowd density distribution. Our method outperforms previous approaches consistently in the widely-used adaptation scenarios.
Introduction
Crowd counting models are notoriously sensitive to domain shift. A model trained on synthetic, sunny images (Source Domain) often sees its accuracy plummet when deployed in a rainy, real-world city square (Target Domain). The core challenge is that existing Unsupervised Domain Adaptation (UDA) methods try to align the entire image at once. However, backgrounds—such as buildings, trees, or weather—vary wildly between locations. Forcing a model to align these disparate backgrounds actually confuses it, making it harder to distinguish between a person and environmental features like foliage.
To address this, researchers developed the Crowd-aware domain Adaptation for Crowd Counting (CACC) framework. By "disentangling" the crowd from the background, the model focuses on domain-invariant human features while ignoring distracting environmental changes.
Methodology
The CACC framework introduces a fine-grained approach to domain adaptation through three core innovations:
- Point-derived Crowd Segmentation (PCS): Standard datasets only provide point annotations (dots on heads). CACC uses Multiple Instance Learning (MIL) to learn a rough segmentation mask in a weakly-supervised manner. By treating image patches as "bags" of pixels, the model learns to identify where people are without extra manual labeling.
- Crowd Region Transfer (CRT): This module utilizes the masks from the PCS to focus adversarial training specifically on crowd regions. By discarding domain-specific background features, it forces the model to learn features of human heads that remain consistent across domains.
- Crowd Density Alignment (CDA): Rather than forcing the target domain to match the source domain's density distribution, CDA generates "pseudo-labels" based on the target domain's own predicted distribution. This respects the fact that different locations (e.g., a subway vs. a sidewalk) have naturally different crowd signatures.
Experimental Results
The study utilized six major benchmarks to test real-world shift scenarios, including synthetic-to-real and varied weather conditions.
CACC consistently outperformed state-of-the-art (SOTA) methods:
- Synthetic-to-Real: On the GCC to ShanghaiTech Part B task, CACC achieved a Mean Absolute Error (MAE) of 13.5, beating the previous best unsupervised method (ASNet) at 14.6.
- Performance Gains: The model showed relative accuracy improvements of up to 35.4% on the challenging ShanghaiTech Part A dataset compared to models without adaptation.
- Qualitative Improvement: Visual density maps demonstrate that CACC is significantly better at suppressing "false positives" in the background (like mistaking a bush for a person) compared to holistic alignment methods.
Discussion and Conclusion
The primary insight of this work is that background alignment is often a distraction. By explicitly masking out backgrounds, the model becomes more robust. Furthermore, letting the target domain regularize itself via pseudo-labels derived from its own segmentation leads to much more accurate local density maps than forcing a match to a source template.
While the framework shows significant progress, it remains dependent on the initial quality of the point-derived segmentation. Future work may explore refining these weakly-supervised masks to handle extremely low-resolution images. Nonetheless, CACC represents a major step toward deploying reliable crowd counting systems in diverse, real-world environments.
Summary updated