Glance to Count: Learning to Rank with Anchors for Weakly-supervised Crowd Counting

Zheng Xiong, Liangyu Chai, Wenxi Liu, Yongtuo Liu, Sucheng Ren, Shengfeng He

Abstract

Crowd counting is a critical task in computer vision, yet it typically requires labor-intensive point-level annotations for thousands of individuals in dense scenes. This paper introduces the Glance to Count (GTC) framework, a weakly-supervised approach that simplifies the annotation process. By training a model to rank image pairs based on relative crowd density and using a small set of "anchor" images to calibrate absolute counts, GTC achieves state-of-the-art performance among weakly-supervised methods while significantly reducing human effort.

Introduction

The primary bottleneck in crowd counting is the data annotation process. Standard fully-supervised methods require annotators to place a dot on every single head in an image. In dense datasets like UCF-QNRF, where a single image can contain over 12,000 people, this process is both exhausting and error-prone.

Crowd scenes with varying densities (A < B < C) illustrating the intuitive nature of ranking over counting.

The authors observe that while humans struggle to estimate exact numbers at a glance, they are remarkably efficient at comparing two images and determining which has more people. GTC leverages this human intuition by framing crowd counting as a ranking problem, supplemented by a minimal number of absolute count labels.

Methodology: The GTC Framework

The core of the GTC framework is the CCRanking system, which transitions from relative density rankings to absolute population estimates through several key components.

1. Siamese Ranking Network (SRN)

The model utilizes a Siamese architecture with a shared Transformer backbone (PVTv2). During training, it processes pairs of images and predicts a "potential score" for each. A Ranking Hinge Loss is applied to ensure that the predicted potential scores reflect the ground-truth ranking of the image pair by a predefined margin.

2. Transitive Labeling and DAGs

To maximize the efficiency of limited human-provided rankings, the framework employs a Directed Acyclic Graph (DAG). By applying transitivity (e.g., if $A > B$ and $B > C$, then $A > C$), the system automatically generates additional training labels, expanding the dataset without further manual intervention.

The GTC framework architecture, showing the training phase with Siamese ranking and the inference phase using anchor-based mapping.

3. The Anchoring Mechanism

Because the SRN only predicts relative potential scores, an anchoring mechanism is required to produce absolute counts. The authors use a small "anchor set" (e.g., 50 images) with known counts. During inference, a linear mapping function is fitted to project the predicted potentials of these anchors onto their ground-truth counts. This function then serves as the estimator for any new query image.

4. Auxiliary Robustness

To prevent the model from over-relying on local features (like the ground plane), an "Upside-Down MLP" branch is included. This auxiliary task forces the model to classify whether an image has been vertically flipped, encouraging the learning of more robust, global crowd features.

Experimental Results

The GTC method was evaluated on major crowd counting benchmarks, demonstrating superior performance compared to existing weakly-supervised techniques:

  • ShanghaiTech Part A: Achieved a Mean Absolute Error (MAE) of 53.4, significantly outperforming the previous best weakly-supervised method, HACC (58.3).
  • UCF-QNRF: Achieved an MAE of 92.3, setting a new state-of-the-art for the weakly-supervised category.
  • Cross-Dataset Generalization: The ranking-based approach proved highly robust in cross-dataset tests, suggesting that learning relative density helps the model avoid overfitting to specific dataset biases.

Conclusion

"Glance to Count" presents a compelling alternative to traditional crowd counting pipelines. By aligning the machine learning task with human-centric strengths—specifically the ability to rank densities quickly—the framework achieves high accuracy with a fraction of the annotation cost. Future work may explore non-linear anchoring functions and the integration of self-supervised pre-training to further minimize the reliance on manual labels.

Summary updated

Abstract

Crowd image is arguably one of the most laborious data to annotate. In this paper, we aim to reduce the massive demand for densely labeled crowd data, and propose a novel weakly-supervised setting, in which we leverage the binary ranking of two images with high-contrast crowd counts as training guidance. To enable training under this new setting, we convert the crowd count regression problem to a ranking potential prediction problem. In particular, we tailor a Siamese Ranking Network that predicts the potential scores of two images indicating the ordering of the counts. Hence, the ultimate goal is to assign appropriate potentials for all the crowd images to ensure their orderings obey the ranking labels. On the other hand, potentials reveal the relative crowd sizes but cannot yield an exact crowd count. We resolve this problem by introducing "anchors" during the inference stage. Concretely, anchors are a few images with count labels used for referencing the corresponding counts from potential scores by a simple linear mapping function. We conduct extensive experiments to study various combinations of supervision, and we show that our method outperforms existing weakly-supervised methods by a large margin without additional labeling effort. The code is available at https://github.com/pandaszzzzz/CCRanking.
BibTeX
@inproceedings{xiong2024glance,
  author = {Zheng Xiong and Liangyu Chai and Wenxi Liu and Yongtuo Liu and Sucheng Ren and Shengfeng He},
  title = {{Glance to Count: Learning to Rank with Anchors for Weakly-supervised Crowd Counting}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2024},
  doi = {10.1109/wacv57701.2024.00041},
}
Figure