Crowded Human Detection via an Anchor-pair Network

Jinguo Zhu, Zejian Yuan, Chong Zhang, Wanchao Chi, Yonggen Ling, Shenghao ZhangWACV 2020 · OpenReview

Abstract

This paper presents an anchor-pair network for crowded human detection, which can overcome and solve the difficulties caused by occlusion in crowded scenes. Specifically, we use a function-aware network structure to extract more distinctive and discriminative features for head and full-body respectively, and then a CNN module is also exploited to fuse the features by learning the correlations between head and full-body to reduce crowd errors. Meanwhile, a novel paired form for anchors, denoted as anchor-pair, is proposed to estimate the head regions and full-body regions simultaneously. Furthermore, a new ingenious Joint-NMS is introduced to perform on the detected head and full-body box pairs, which produces significant performance improvement in heavily occluded scenarios at tiny computational cost. Our anchor-pair network achieves a state-of-the-art result on the CrowdHuman dataset which reduces the MR$^{-2}$ to 55.43%, achieving 11.59% relative improvement over our dataset baseline.

Introduction

Detecting individuals in dense crowds represents a significant challenge for modern computer vision systems. In such environments, heavy occlusion leads to two primary failure modes for standard detectors: feature confusion and the Non-Maximum Suppression (NMS) dilemma. Feature confusion occurs when overlapping bodies cause visual features to blend, leading to inaccurate bounding box localization. The NMS dilemma arises because standard IOU-based suppression cannot easily distinguish between redundant detections and distinct individuals standing in close proximity.

The proposed Anchor-pair Network (APN) addresses these issues by leveraging the observation that while bodies are frequently occluded, human heads typically remain visible and provide a distinct spatial anchor for identification.

Methodology

The APN architecture consists of three core components designed to improve detection in crowded scenarios.

Function-aware Branches (FWB)

Utilizing a ResNet-50 backbone, the network implements separate branches to extract specialized features for heads and full bodies. This functional separation allows the model to learn more discriminative representations for each part, rather than relying on a single shared feature set.

Feature Fusion Module (FFM)

A dedicated CNN module learns the spatial correlations between head and body features. By using the more reliable head features as guidance, the FFM helps the network more accurately estimate the boundaries of the full body, even when it is significantly occluded.

Anchor-pair and Joint-NMS

The network predicts "anchor-pairs" — linked bounding boxes for both the head and the body. This pairing is critical for the Joint-NMS algorithm. Unlike standard NMS, which only evaluates body overlap, Joint-NMS considers the overlap of both heads and bodies. Since the heads of different individuals rarely overlap as much as their bodies, this approach allows the system to retain highly overlapping body detections if their corresponding head detections remain distinct.

Experimental Results

The model was evaluated on the CrowdHuman benchmark, which features an average of 22.6 persons per image. The APN achieved a state-of-the-art Log Miss Rate ($MR^{-2}$) of 55.43%, a significant improvement over the RetinaNet baseline of 62.70%. This represents a 7.27 point absolute improvement (11.59% relative) over the baseline.

Qualitative detection results on the CrowdHuman dataset. The visualization highlights true positives (DR), false positives (DR in red), and missed detections (MD).

The model maintains high efficiency, adding only approximately 15ms of inference latency compared to standard baselines, making it suitable for practical applications.

Conclusion

The Anchor-pair Network effectively mitigates the challenges of crowded human detection by treating the head as a reliable spatial prior. By combining function-aware feature extraction with a novel Joint-NMS strategy, the APN significantly outperforms traditional detectors in heavily occluded scenes. Future work may involve extending this part-pair logic to other object categories or refining anchor ratios for more diverse human poses.

Summary updated

Abstract

This paper presents an anchor-pair network for crowded human detection, which can overcome and solve the difficulties caused by occlusion in crowded scenes. Specifically, we use a function-aware network structure to extract more distinctive and discriminative features for head and full-body respectively, and then a CNN module is also exploited to fuse the features by learning the correlations between head and full-body to reduce crowd errors. Meanwhile, a novel paired form for anchors, denoted as anchor-pair, is proposed to estimate the head regions and full-body regions simultaneously. Furthermore, a new ingenious Joint-NMS is introduced to perform on the detected head and full-body box pairs, which produces significant performance improvement in heavily occluded scenarios at tiny computational cost. Our anchor-pair network achieves a state-of-the-art result on the CrowdHuman dataset which reduces the MR <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">−2</sup> to 55.43%, achieving 11.59% relative improvement over our dataset baseline.
BibTeX
@inproceedings{zhu2020crowded,
  author = {Jinguo Zhu and Zejian Yuan and Chong Zhang and Wanchao Chi and Yonggen Ling and Shenghao Zhang},
  title = {{Crowded Human Detection via an Anchor-pair Network}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2020},
  url = {https://openreview.net/forum?id=DwpbVmbpPh},
}
Figure