CrowdQuery: Density-Guided Query Module for Enhanced 2D and 3D Detection in Crowded Scenes

Marius Dähling, Sebastian Krebs, J. Marius Zöllner

Abstract

CrowdQuery (CQ) is a novel, lightweight module designed to enhance the performance of transformer-based object detectors in highly congested environments. By predicting an object density map that accounts for individual bounding box dimensions and using it to guide object queries via cross-attention, CQ achieves state-of-the-art results in both 2D and 3D pedestrian detection. The module is plug-and-play, requiring no additional training data or sensors, and demonstrates significant improvements in detecting highly occluded instances in crowded scenes.

Introduction

Detecting pedestrians in "crowd" scenarios is a major hurdle for autonomous driving and robotics. In these settings, heavy occlusions and high object density often cause standard detectors to fail. While Transformer-based models (like DETR) have revolutionized general object detection, they struggle with crowds. Their "object queries"—the learned vectors that search the image for objects—are typically initialized uniformly. In a crowd, these queries struggle to "clump" together and adapt to locally dense regions. Previous attempts to use "density maps" often treated objects as single points, ignoring their actual size and shape. Furthermore, while 2D crowd detection is well-studied, 3D crowd detection remains a significant gap in the literature.

Methodology

The authors introduce the CrowdQuery (CQ) module, which can be integrated into existing transformer-based architectures to provide spatial priors to the detection process.

Novel Density Formulation

Instead of traditional point-based density maps, the authors use "Gaussian Windows." They define the standard deviation ($\sigma_x, \sigma_y$) of the Gaussian based on the actual width and height of the bounding box. Crucially, they remove the normalization constant from the Gaussian formula. This ensures that large and small boxes contribute equally to the density signal, preventing large boxes from being "washed out" in the feature map.

Comparison of density map formulations: (a) Original image, (b) Annotated image, (c) Traditional point-based density, and (d) The proposed box-based density map.

The CrowdQuery Module

The CQ module takes features from the image backbone and splits into two branches to process density information:

  • Upper Branch: Encodes global density features using multi-head self-attention (MHSA).
  • Lower Branch: Predicts a density map via a 1x1 convolution. This map is discretized into bins and converted into a "density embedding."

These two branches are merged and integrated into the transformer decoder. Before the object queries look at the actual image features, they perform cross-attention with the density embedding. This "primes" the queries, telling them where the crowded regions are located.

The CrowdQuery architecture integrated into a transformer-based detector. The module predicts a density map $\rho(x,y)$ to guide object queries via cross-attention.

Experimental Results

The module was evaluated on the STCrowd dataset (specialized for 2D/3D crowded pedestrian perception) and the CrowdHuman dataset (a challenging 2D benchmark).

Key Performance Metrics

  • 2D Detection (STCrowd): CQ2D achieved 91.4% Average Precision (AP), outperforming the previous state-of-the-art and improving upon its base model (Deformable DETR) by 1.8 points.
  • 3D Detection (STCrowd): CQ3D reached 52.7 mAP, a significant 4.2-point jump over the baseline MonoDETR, excelling particularly in "highly occluded" scenarios.
  • Generalizability: When integrated with the state-of-the-art DDQ detector on the CrowdHuman dataset, the module (CQ2D++) reached 94.0% AP, setting a new benchmark high.
  • Efficiency: The module is computationally efficient, adding only about 13% more parameters and a 16% increase in inference time for CQ3D.

Conclusion and Future Work

CrowdQuery demonstrates that incorporating bounding box dimensions into density maps provides a rich spatial prior for transformer queries. The authors found that processing density information before depth or visual features is most effective for 3D detection, suggesting that identifying object clusters is a fundamental first step. Future work will focus on extending CrowdQuery to LiDAR-based detectors and addressing the difficulty of yaw estimation in extremely dense crowds.

Summary updated

BibTeX
@inproceedings{dahling2025crowdquery,
  author = {Marius Dähling and Sebastian Krebs and J. Marius Zöllner},
  title = {{CrowdQuery: Density-Guided Query Module for Enhanced 2D and 3D Detection in Crowded Scenes}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2025},
  doi = {10.1109/iros60139.2025.11246919},
}
Figure