Dynamic Mixture of Counter Network for Location-Agnostic Crowd Counting

Mingjie Wang, Hao Cai, Yong Dai, Minglun GongWACV 2023 · OpenReview

Crowd counting is a critical task in computer vision with applications ranging from public safety to urban planning. However, traditional models are "location-adherent," requiring expensive "dot-map" annotations where every individual in a scene is manually marked. This WACV 2023 paper introduces DMCNet, a novel approach that achieves state-of-the-art results using only image-level total counts, significantly reducing the data labeling burden.

The Challenge: Location-Agnostic Counting

Moving away from point-level supervision introduces two primary hurdles. First, Density Shift: a single dataset can range from a few people in a park to thousands in a stadium. A static regression head often fails to map features across such diverse numerical scales. Second, Architectural Trade-offs: while CNNs excel at capturing local spatial details, they have limited receptive fields. Conversely, Transformers offer global vision but are computationally heavy and data-hungry.

Comparison of crowd counting paradigms: (a) CNN-based with sorting supervision, (b) Transformer-based with static regression, and (c) the proposed Hybrid DMCNet with a Dynamic Mixture of Counter.

Methods: The DMCNet Architecture

The authors propose the Dynamic Mixture of Counter Network (DMCNet), which leverages a hybrid architecture to get the "best of both worlds" from CNNs and MLPs. The pipeline consists of three core modules:

  1. Pyramidal Feature Extractor (CNN-based): This component maintains hierarchical spatial information, ensuring the model retains track of local features.
  2. Global Token Mixer (MLP-based): Instead of a heavy Transformer, DMCNet uses a multi-level MLP structure to capture global context across different scales, helping the model understand the overall scene environment.
  3. Dynamic Mixture of Counter Experts: This is the core innovation. Rather than a single regressor, the model employs a pool of 45 specialized "counter heads." A Dynamic Counter Predictor analyzes the extracted features and uses the Gumbel-Softmax trick to adaptively select the expert head best suited for a specific patch's density.
The DMCNet architecture, showcasing the flow from raw crowd scenes through the hybrid feature extractors to the dynamic selection of counter experts for final prediction.

Key Insights

  • Hybridization Wins: By combining CNNs for spatial hierarchy and MLPs for global context, the model achieves high performance without the overhead of a full Transformer.
  • Specialization over Generalization: The "Mixture of Experts" approach acknowledges that a regressor optimized for sparse crowds is fundamentally different from one optimized for dense crowds. By letting the model dynamically "pick its expert," it handles the "long-tail" distribution of crowd densities more effectively.
  • Deep Supervision: The authors found that adding an auxiliary classification loss—predicting which density "bin" a patch belongs to—significantly stabilized the training of the dynamic selection mechanism.

Results and Performance

DMCNet sets a new benchmark for location-agnostic counting across four major datasets: ShanghaiTech (Part A & B), UCF-QNRF, and the massive JHU-CROWD++.

  • State-of-the-Art: DMCNet outperformed the previous leading method, TransCrowd, by 11.55% in Mean Absolute Error (MAE) on ShanghaiTech Part A.
  • Competitive with Full Supervision: Remarkably, DMCNet performs on par with, or even better than, several "location-adherent" models (like AMRNet and BL) that benefit from much richer, point-level supervision.
  • Robustness to Density Shift: On the JHU-CROWD++ dataset, which features extreme density shifts (0 to 25,000+ people), DMCNet achieved an MAE of 69.64, proving its resilience in complex, real-world scenarios.

Conclusion

DMCNet demonstrates that location-agnostic crowd counting is not only viable but can compete with traditional, labor-intensive methods. By utilizing a dynamic mixture of experts and a hybrid CNN-MLP backbone, the model effectively addresses the challenges of density shift and global context, paving the way for more scalable crowd analysis systems.

Summary updated

Abstract

Crowd counting has attracted increasing attentions in recent years due to its challenges and wide societal applications. Despite persevering efforts made by the research community, most of existing methods require a large amount of location-level annotations. Collecting such type of fine-granularity supervisory signals is extremely time-consuming and labour-intensive, thereby hindering the well generalization of these location-adherent models. To shun this drawback, several pioneering studies open a promising research direction of location-agonistic crowd counting. Albeit the noticeable efforts, they somewhat ignore the merits of diverse learning paradigms and the issue of intractable density shift. To ameliorate these issues, in this paper, a novel Dynamic Mixture of Counter Network (DMCNet) is proposed for location-agnostic crowd counting. Specifically, our DMCNet inherits the hybrid advantages of CNNs (e.g. locality-oriented and pyramidal property) and MLP-based structure (e.g. global receptive fields and light weight). Particularly, the dynamic counter predictor and the mixture of counter heads are delicately designed to hammer at combating huge density shift and overfitting. Extensive experiments demonstrate that our DMCNet attains state-of-the-art performance against existing location-agnostic approaches and performs on par with many conventional location-adherent ones.
BibTeX
@inproceedings{wang2023dynamic,
  author = {Mingjie Wang and Hao Cai and Yong Dai and Minglun Gong},
  title = {{Dynamic Mixture of Counter Network for Location-Agnostic Crowd Counting}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2023},
  url = {https://openreview.net/forum?id=uY4WSX2Ke6},
}
Figure