HDNet: A Hierarchically Decoupled Network for Crowd Counting

Chenliang Gu, Changan Wang, Bin-Bin Gao, Jun Liu, Tianliang Zhang

Introduction

Crowd counting is a fundamental task in computer vision with significant implications for public safety, urban planning, and traffic management. The prevailing approach involves regressing a density map from an input image, where the integral of the map yields the total count. Despite its popularity, density map regression faces two primary challenges: cluttered backgrounds and extreme density variation.

Background elements like trees, buildings, or ornate ceilings often trigger "phantom" counts because regressors struggle to output an absolute zero for non-human regions. Simultaneously, a single scene may contain both sparse foreground individuals and dense, distant crowds, requiring a model to handle vastly different scales and density ranges at once.

Methodology: Hierarchical Decoupling

To address these issues, the authors propose HDNet, a framework that adopts a "divide and conquer" strategy. Instead of a monolithic regressor, HDNet hierarchically decouples the crowd counting task into background classification and density-specific foreground regression.

The HDNet architecture featuring the Scale Adaptive Feature Fusion (SAFF), Foreground Density Estimation Module (FDEM), and Density Decoupling Module (DDM).

1. Density Decoupling Module (DDM)

The DDM classifies pixels into discrete categories: background (Level 0) and multiple foreground density ranges (Levels 1 to $n$). This classification task is inherently simpler than direct regression, allowing the model to effectively "mask out" background noise before final estimation.

2. Foreground Density Estimation Module (FDEM)

The FDEM employs multiple "density heads," each acting as an expert for a specific density level. By assigning specific ranges to specialized sub-networks, the hypothesis space for each head is reduced, leading to more accurate local density estimation. The final density map $\tilde{D}$ is a weighted combination of these experts' outputs, guided by the soft masks generated by the DDM.

3. Scale Adaptive Feature Fusion (SAFF)

To manage scale variations, the SAFF block performs channel-wise feature interaction. It selectively emphasizes features from different resolutions based on local crowd density, ensuring that dense areas receive high-resolution details while sparse areas benefit from broader contextual information.

Key Insights

  • Classification as a Prior: It is easier for a network to classify a region as background than to regress it to zero. Decoupling background detection significantly "cleans" the resulting density maps.
  • Soft Masking and Gradient Interaction: HDNet uses probabilistic "soft masks" rather than binary ones. This allows for smooth transitions between density levels and enables joint optimization, where the classifier learns from density intensity and the regressor learns to ignore background through shared gradients.
  • Reduced Optimization Complexity: By partitioning the density range, each expert head solves a simpler sub-problem, making the overall training process more efficient and robust.

Experimental Results

HDNet was evaluated on major benchmarks including ShanghaiTech PartA, UCF-QNRF, JHU-CROWD++, and NWPU-Crowd. It achieved state-of-the-art (SOTA) performance across the board.

Qualitative comparison of HDNet against CANNet and ASNet. HDNet demonstrates superior background suppression (top row) and more accurate local density estimation (bottom rows).

On the NWPU-Crowd dataset, HDNet achieved a Mean Absolute Error (MAE) of 76.1, significantly outperforming previous leaders. Ablation studies confirmed that the hierarchical decoupling of foreground densities provided a 4.2% improvement in MAE over a simple foreground/background binary split. Visualizations demonstrate that HDNet is particularly effective at ignoring complex background textures that typically confuse other models.

Conclusion

HDNet presents a robust solution to the crowd counting problem by leveraging hierarchical decoupling. By separating background classification from multi-level density regression and employing task-specific experts, the model achieves superior accuracy and noise suppression. This strategy of reducing task complexity through decoupling offers a promising direction for other dense prediction tasks in cluttered environments.

Summary updated

Abstract

Recently, density map regression-based methods have dominated in crowd counting owing to their excellent fitting ability on density distribution. However, further improvement tends to saturate mainly because of the confusing background noise and the large density variation. In this paper, we propose a Hierarchically Decoupled Network (HDNet) to solve the above two problems within a unified framework. Specifically, a background classification sub-task is decomposed from the density map prediction task, which is then assigned to a Density Decoupling Module (DDM) to exploit its highly discriminative ability. For the remaining foreground prediction sub-task, it is further hierarchically decomposed to several density-specific sub-tasks by the DDM, which are then solved by the regression-based experts in a Foreground Density Estimation Module (FDEM). Although the proposed strategy effectively reduces the hypothesis space so as to relieve the optimization for those task-specific experts, the high correlation of these sub-tasks are ignored. Therefore, we introduce three types of interaction strategies to unify the whole framework, which are Feature Interaction, Gradient Interaction, and Scale Interaction. Integrated with the above spirits, HDNet achieves state-of-the-art performance on several popular counting benchmarks.
BibTeX
@inproceedings{gu2022hdnet,
  author = {Chenliang Gu and Changan Wang and Bin-Bin Gao and Jun Liu and Tianliang Zhang},
  title = {{HDNet: A Hierarchically Decoupled Network for Crowd Counting}},
  booktitle = {ICME},
  year = {2022},
}
Figure