CrowdVLM-R1: Expanding R1 Ability to Vision Language Model for Crowd Counting using Fuzzy Group Relative Policy Reward

Zhiqiang Wang, Pengbin Feng, Yanbin Lin, Shuzhang Cai, Zongao Bian, Jinghua Yan, Xingquan ZhuarXiv 2025 · DOI · arXiv

Abstract

Crowd counting is a challenging spatial reasoning task requiring high numerical precision. While Large Language Models have benefited from reasoning-centric Reinforcement Learning (RL), applying these techniques to Vision Language Models (VLMs) for counting often suffers from sparse, binary rewards. This paper introduces CrowdVLM-R1, which adapts the DeepSeek-R1 training paradigm to VLMs using a novel Fuzzy Group Relative Policy Reward (FGRPR). By incentivizing "nearly correct" predictions through a continuous reward function, CrowdVLM-R1 enables models to develop sophisticated counting strategies. Experimental results show that a 7B parameter model trained with FGRPR significantly outperforms much larger models, including GPT-4o, in counting accuracy across diverse domains.

Introduction: The Challenge of Numerical Precision in Vision

Crowd counting—estimating the number of entities in dense images—remains a difficult task for modern AI. Objects frequently overlap, vary in scale, and blend into complex backgrounds. Traditional RL approaches often rely on binary feedback: a prediction is either exactly right or wrong. This "all-or-nothing" signal is insufficient for numerical tasks where a model needs to understand that an estimate of 98 for a ground truth of 100 is significantly better than an estimate of 50.

Methodology: Fuzzy Rewards and Group Optimization

CrowdVLM-R1 is built upon the Qwen2.5-VL architecture and introduces a specialized training pipeline designed to bridge the gap between visual perception and numerical reasoning.

Fuzzy Group Relative Policy Reward (FGRPR)

The core innovation is the FGRPR mechanism. Unlike standard RL, which might penalize any deviation from the ground truth, FGRPR provides a nuanced signal:

  • Format Reward ($r_f$): Ensures the model adheres to the required output structure, specifically using <think> and <answer> tags.
  • Precision Reward ($r_p$): A continuous score that rewards proximity. If a prediction is within 50% of the ground truth, the model receives a scaled reward (1.0 to 1.5). This "fuzzy" logic provides a smoother gradient, allowing the model to learn from approximations.
Figure 1: The CrowdVLM-R1 training pipeline, illustrating the generation of multiple output samples and the application of the Fuzzy Group Relative Policy Reward (FGRPR) within the GRPO framework.

Reasoning through Chain-of-Thought

By leveraging the <think> process, the model is encouraged to decompose the counting task. Instead of direct estimation, the model develops emergent strategies such as segmenting the image into regions, scanning row-by-row, or counting individuals in sparse scenes. This mimics human-like reasoning and improves final accuracy.

Experimental Setup and Datasets

The model was trained and evaluated on a diverse set of counting benchmarks to ensure robustness across different object types and densities.

  • In-Domain Data: Includes aerial sheep, street pedestrians, wheat heads, and vehicles.
  • Out-of-Domain Data: The Manatee dataset was used exclusively for testing to evaluate zero-shot generalization to unseen species.
Figure 6: Overview of the diverse datasets used for training and evaluation, including in-domain examples (sheep, people, wheat, cars) and out-of-domain manatee images.

Key Results and Insights

The integration of fuzzy rewards proved to be the critical factor in performance. Standard RL with binary rewards often underperformed compared to simple Supervised Fine-Tuning (SFT), whereas FGRPR unlocked significant gains.

  • Superior Accuracy: The 7B parameter CrowdVLM-R1 (Q-7B-FR) achieved a Mean Absolute Error (MAE) of 24.07, outperforming GPT-4o (MAE 61.95) and LLaMA-70B (MAE 44.68) on the target domains.
  • Efficiency: By using Group Relative Policy Optimization (GRPO), the researchers eliminated the need for a separate "critic" model, making the training process more memory-efficient.
  • Generalization: While primarily optimized for in-domain tasks, the model maintained competitive performance on unseen datasets, though it still faces challenges in zero-shot scenarios with very low object counts.

Conclusion and Future Directions

CrowdVLM-R1 demonstrates that reasoning-heavy RL can be successfully adapted to visual numerical tasks by softening the reward signal. While the model shows a tendency to "cluster" predictions for counts below 250, its ability to outperform massive proprietary models highlights the power of specialized reward functions. Future work may extend this "fuzzy" framework to other precision-critical tasks, such as medical imaging or complex object localization.

Summary updated

BibTeX
@misc{wang2025crowdvlm,
  author = {Zhiqiang Wang and Pengbin Feng and Yanbin Lin and Shuzhang Cai and Zongao Bian and Jinghua Yan and Xingquan Zhu},
  title = {{CrowdVLM-R1: Expanding R1 Ability to Vision Language Model for Crowd Counting using Fuzzy Group Relative Policy Reward}},
  year = {2025},
  eprint = {2504.03724},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2504.03724},
}
Figure