Relational Attention Network for Crowd Counting

Anran Zhang, Jiayi Shen, Zehao Xiao, Fan Zhu, Xiantong Zhen, Xianbin Cao, Ling ShaoICCV 2019 · OpenReview

Abstract

Crowd counting in dense scenes is a challenging task due to extreme scale variations and heavy occlusions. Traditional methods often rely on independent pixel-wise regression, which fails to capture the spatial dependencies inherent in crowd patterns. We present the Relational Attention Network (RANet), a novel framework that leverages local and global self-attention mechanisms to model pixel relationships. By incorporating a dedicated relation module to fuse these features, RANet achieves state-of-the-art performance across multiple benchmarks, significantly reducing estimation errors and producing higher-quality density maps.

The Challenge of Independent Pixel Regression

Most existing crowd counting models treat the task as a density estimation problem where each pixel's density value is predicted independently. However, pixels in a crowd are not isolated; they exhibit strong correlations due to the repetitive nature of human features (heads, shoulders) and the perspective geometry of the scene. Ignoring these interdependencies leads to noisy predictions, especially in high-density areas or regions with significant scale changes.

Relational Attention Network (RANet)

To address these limitations, RANet introduces a relational modeling approach built on a Stacked Hourglass backbone. The architecture is defined by three primary components:

  1. Local Self-Attention (LSA): This module captures short-range dependencies by analyzing correlations between a pixel and its immediate spatial neighbors. This helps the model refine local structures and textures.
  2. Global Self-Attention (GSA): To handle long-range dependencies efficiently, GSA utilizes max pooling to identify "distinctive pixels" across the entire feature map. This allows the network to recognize similar crowd patterns regardless of their distance in the image.
  3. Relation Module: This module serves as the fusion center, consisting of Intra-Relation (aggregating local and global features at the same spatial position) and Inter-Relation (learning to infer complex hidden relationships between local points and the holistic global context).

By moving beyond simple concatenation, the Inter-Relation module acts as a learnable aggregator that understands how local details should be contextualized by global information.

Experimental Results

RANet was evaluated on four major benchmarks: ShanghaiTech (Parts A and B), UCF-CC-50, and UCF-QNRF. The model consistently outperformed previous state-of-the-art methods. Notably, on the high-resolution UCF-QNRF dataset, RANet reduced the Mean Absolute Error (MAE) from 132 to 111, a 15% improvement.

Qualitative comparison of density maps. RANet (bottom-right) produces a cleaner, more accurate density map compared to ic-CNN (bottom-left), closely matching the Ground Truth (top-right).

Beyond quantitative metrics, RANet demonstrates significant qualitative improvements. As shown in the comparison above, the density maps generated by RANet are visibly less noisy and more localized than those from earlier models like ic-CNN. This precision is a direct result of the model's ability to leverage relational context to suppress false positives in background regions.

Conclusion and Future Directions

The Relational Attention Network demonstrates that modeling pixel-wise interdependencies is crucial for accurate crowd counting. While the current implementation relies on a Stacked Hourglass backbone and incurs some computational overhead due to the self-attention layers, it sets a new standard for density map quality. Future work may involve adapting these relational modules for real-time video surveillance or integrating them with more efficient transformer-based backbones.

Summary updated

BibTeX
@inproceedings{zhang2019relational,
  author = {Anran Zhang and Jiayi Shen and Zehao Xiao and Fan Zhu and Xiantong Zhen and Xianbin Cao and Ling Shao},
  title = {{Relational Attention Network for Crowd Counting}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2019},
  url = {https://openreview.net/forum?id=37uigT3M1Cv},
}
Figure