Adaptive Density Map Generation for Crowd Counting

Jia Wan, Antoni B. Chan

Abstract

Traditional crowd counting models rely on density maps generated through hand-crafted Gaussian blurring of dot annotations. This approach often fails to account for significant scale variations in crowd scenes or the specific architectural constraints of the neural networks used for estimation. This paper introduces a novel framework for learning density maps adaptively during training. By optimizing the ground truth representation for specific network architectures and datasets, the proposed method achieves state-of-the-art performance across major benchmarks, including UCF-QNRF and ShanghaiTech.

Introduction

The standard pipeline for crowd counting involves two primary steps: converting discrete "dot" annotations into continuous density maps and training a deep neural network (the "counter") to estimate these maps from input images. Historically, the first step has relied on fixed Gaussian kernels. However, these hand-crafted kernels act as a bottleneck; if the ground truth density map is poorly aligned with a model's receptive field or the scene's geometry, performance is inherently capped.

Methodology: From Fixed to Adaptive Maps

To overcome the limitations of hand-crafted kernels, this research proposes moving toward learnable density map generation. Two primary frameworks were developed:

  1. Density Map Refinement: A "Refiner" network is trained jointly with the Counter to "clean up" traditional density maps. The Refiner learns to produce maps that are easier for the Counter to estimate while maintaining fidelity to the original annotations.
  2. Adaptive Density Map Generator: This more advanced approach replaces pre-made maps entirely. As shown in Figure 1, the system processes raw dot maps through a bank of Gaussian kernels of varying sizes. A Self-attention Fusion Module then adaptively merges these blurred maps based on the input image features.
The Adaptive Density Map Generator architecture, featuring a Gaussian kernel bank and self-attention fusion.

Joint Training Loop

The Generator and Counter are trained in an end-to-end loop. The loss function is dual-purpose: a counting loss ensures the Counter can accurately predict the generated map, while a generation loss ensures the Generator maintains the correct total person count from the original dot annotations.

Key Insights

Architecture-Specific Ground Truth

A critical finding of this work is that the "optimal" ground truth is not universal. Different network architectures (e.g., CSRNet vs. MCNN) perform best when trained on different types of density maps. The adaptive generator learns a representation that specifically favors the internal mechanics of the chosen counter.

Receptive Field and Scale Alignment

Visualization of the learned maps (Figure 2) reveals that the generator naturally aligns with the network's receptive field. For architectures with larger receptive fields, the generator produces smoother, more distributed density maps. Furthermore, the self-attention module automatically assigns smaller kernels to dense, distant crowds and larger kernels to foreground individuals, effectively automating geometry-adaptive blurring.

Visualization of learned density maps across different architectures, showing how the generator adapts to specific model characteristics.

Experimental Results

The proposed method was evaluated on several standard benchmarks, achieving state-of-the-art (SOTA) results without modifying the underlying counter architectures:

  • UCF-QNRF: Achieved a Mean Absolute Error (MAE) of 101, a significant reduction from the baseline CSRNet's MAE of 148.
  • ShanghaiTech Part B: Reached an MAE of 8.1, surpassing previous SOTA methods like SANet (8.4).
  • Generalization: The authors demonstrated that density maps generated for one model do not necessarily benefit another, confirming that the learned representation is tightly coupled to the specific estimator.

Conclusion and Future Work

By shifting the responsibility of density map generation from the researcher to the model, this framework removes a significant source of manual tuning in crowd counting. While the joint training process is more computationally intensive, the performance gains on complex datasets like UCF-QNRF are substantial. Future research may extend this adaptive generation technique to other point-to-map estimation tasks, such as cell counting in medical imaging or object localization in satellite data.

Summary updated

Abstract

Crowd counting is an important topic in computer vision due to its practical usage in surveillance systems. The typical design of crowd counting algorithms is divided into two steps. First, the ground-truth density maps of crowd images are generated from the ground-truth dot maps (density map generation), e.g., by convolving with a Gaussian kernel. Second, deep learning models are designed to predict a density map from an input image (density map estimation). Most research efforts have concentrated on the density map estimation problem, while the problem of density map generation has not been adequately explored. In particular, the density map could be considered as an intermediate representation used to train a crowd counting network. In the sense of end-to-end training, the hand-crafted methods used for generating the density maps may not be optimal for the particular network or dataset used. To address this issue, we first show the impact of different density maps and that better ground-truth density maps can be obtained by refining the existing ones using a learned refinement network, which is jointly trained with the counter. Then, we propose an adaptive density map generator, which takes the annotation dot map as input, and learns a density map representation for a counter. The counter and generator are trained jointly within an end-to-end framework. The experiment results on popular counting datasets confirm the effectiveness of the proposed learnable density map representations.
BibTeX
@inproceedings{wan2019adaptive,
  author = {Jia Wan and Antoni B. Chan},
  title = {{Adaptive Density Map Generation for Crowd Counting}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2019},
  doi = {10.1109/iccv.2019.00122},
}

Loading related content...

Figure