Crowd Counting with Decomposed Uncertainty

Min-hwan Oh, Peder A. Olsen, Karthikeyan Natesan Ramamurthy

Abstract

Crowd counting is a critical task for public safety and urban planning, yet traditional models often lack a measure of predictive confidence. We present DUBNet, a framework that utilizes a bootstrap ensemble to decompose predictive uncertainty into epistemic (model-related) and aleatoric (data-related) components. By integrating a ResNet-50 backbone with dilated convolutions and multiple bootstrap heads, DUBNet achieves state-of-the-art accuracy across major benchmarks while providing calibrated confidence intervals. Our results demonstrate that explicitly modeling uncertainty not only provides interpretability but also enhances overall counting performance.

Introduction

Estimating the number of individuals in dense crowds—such as at rallies, concerts, or protests—is a high-stakes computer vision task. While modern Convolutional Neural Networks (CNNs) have significantly improved point-estimate accuracy, they rarely provide a measure of certainty. In real-world deployments, understanding why a model might be failing is as important as the count itself.

Uncertainty in crowd counting typically arises from two sources:

  1. Epistemic Uncertainty: Reflects the model's lack of knowledge, occurring when it encounters scenes significantly different from its training data.
  2. Aleatoric Uncertainty: Represents inherent noise in the data, caused by factors like motion blur, poor lighting, or extreme occlusions that make the image itself ambiguous.

Existing Bayesian approaches to quantify these uncertainties are often computationally prohibitive for high-resolution image tasks. DUBNet addresses this by providing a scalable, calibrated framework for uncertainty quantification.

Methodology

The proposed DUBNet (Decomposed Uncertainty using Bootstrap ensemble) architecture is designed for both accuracy and efficiency. It consists of three primary components:

  • Front-end: A ResNet-50 backbone serves as an efficient feature extractor.
  • Back-end: Dilated convolutional layers are employed to expand the receptive field without sacrificing spatial resolution, which is crucial for identifying individuals in varying crowd densities.
  • Bootstrap Output Heads: The network features $K$ independent heads. During training, each head is exposed to a different subset of the data. At inference, the variance across these heads quantifies the epistemic uncertainty.
  • Heteroscedastic Regression: A dedicated output head predicts the log-variance ($\sigma^2$) of the pixels to capture aleatoric uncertainty.
Architecture of DUBNet featuring a ResNet-50 front-end, dilated back-end, and bootstrap heads for uncertainty decomposition.

To ensure the reliability of these confidence scores, the authors apply Isotonic Regression for recalibration. This ensures that a "90% confidence interval" statistically contains the ground truth 90% of the time.

Key Insights and Results

A significant finding of this work is that training a model to estimate its own uncertainty actually improves its counting accuracy. The aleatoric loss term allows the model to effectively "down-weight" noisy or ambiguous training samples, focusing its learning capacity on clearer signals. Furthermore, the DUB framework is architecture-agnostic; experiments showed that adding the "DUB extension" to other popular models like CSRNet and MCNN consistently improved their performance.

Qualitative comparison of ground truth, predictions, and decomposed uncertainty maps (epistemic and aleatoric).

DUBNet achieved state-of-the-art (SOTA) or near-SOTA results across several benchmarks:

  • UCF-QNRF: Achieved a Mean Absolute Error (MAE) of 105.6, a significant improvement over previous methods.
  • ShanghaiTech (Part A & B) and UCF-CC 50: Demonstrated robust performance across varying densities.
  • Calibration: Post-recalibration, the model's 90% confidence intervals contained the ground truth between 89% and 91% of the time across all datasets.

In terms of efficiency, DUBNet (3.8B FLOPs) is substantially more lightweight than older VGG-16 based SOTA models (15.3B FLOPs), despite the ensemble-based approach.

Conclusion and Future Work

DUBNet successfully bridges the gap between high-accuracy crowd counting and reliable uncertainty quantification. By decomposing uncertainty, practitioners can distinguish between errors caused by insufficient training data versus those caused by poor image quality.

Future research may focus on reducing the linear computational overhead of the $K$ bootstrap heads and exploring memory-efficient processing for extremely high-resolution images. Additionally, while recalibration is effective, developing methods that do not require a separate validation set remains an open challenge.

Summary updated

Abstract

Research in neural networks in the field of computer vision has achieved remarkable accuracy for point estimation. However, the uncertainty in the estimation is rarely addressed. Uncertainty quantification accompanied by point estimation can lead to a more informed decision, and even improve the prediction quality. In this work, we focus on uncertainty estimation in the domain of crowd counting. With increasing occurrences of heavily crowded events such as political rallies, protests, concerts, etc., automated crowd analysis is becoming an increasingly crucial task. The stakes can be very high in many of these real-world applications. We propose a scalable neural network framework with quantification of decomposed uncertainty using a bootstrap ensemble. We demonstrate that the proposed uncertainty quantification method provides additional insight to the crowd counting problem and is simple to implement. We also show that our proposed method exhibits state-of-the-art performances in many benchmark crowd counting datasets.
BibTeX
@inproceedings{oh2020crowd,
  author = {Min-hwan Oh and Peder A. Olsen and Karthikeyan Natesan Ramamurthy},
  title = {{Crowd Counting with Decomposed Uncertainty}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i07.6852},
}
Figure