Taste More, Taste Better: Diverse Data and Strong Model Boost Semi-Supervised Crowd Counting

Maochen Yang, Zekun Li, Jian Zhang, Lei Qi, Yinghuan Shi

Introduction

Crowd counting—estimating the number of people in dense images—is a critical task for urban planning and public safety. However, it is notoriously labor-intensive. Annotating a single large-scale dataset like UCF-QNRF can require up to 2,000 human hours to label individual heads. Semi-supervised crowd counting (SSCC) offers a solution by leveraging small amounts of labeled data alongside vast unlabeled pools. Yet, current SSCC methods face two primary bottlenecks: destructive data augmentation techniques and architectural limitations that struggle with global context or computational efficiency.

This paper introduces TMTB (Taste More, Taste Better), a framework designed to overcome these hurdles through high-fidelity synthetic data generation and a modern Visual State Space Model (VSSM) backbone.

The TMTB Framework

"Taste More": Inpainting-Based Augmentation

Traditional augmentation methods like "Mixup" blend two images, which often destroys the delicate spatial structure of a crowd. This leads to distorted density maps and poor model performance because it breaks the spatial logic the model needs to learn. TMTB replaces this with a diffusion-based inpainting technique using Stable Diffusion.

By identifying the crowd as the "foreground" and using AI to regenerate the "background" scenery, the framework creates diverse training samples while preserving the original crowd structure. To ensure reliability, an Inconsistency Mask is employed to filter out regions where the generative model might have introduced artifacts or "ghost" people.

Comparison of Mixup and Inpainting augmentation. Mixup destroys spatial logic, resulting in high MAE, while inpainting preserves crowd structure and significantly reduces error.

"Taste Better": VSSM Backbone and Anti-Noise Head

On the architectural side, TMTB addresses the limitations of CNNs and Transformers:

  • VSSM Backbone: The authors utilize VMamba, a Visual State Space Model. Unlike CNNs, which often overfit to local textures, VSSM provides a global receptive field similar to Transformers but maintains linear computational complexity. This makes it highly efficient for high-resolution crowd scenes.
  • Anti-Noise Classification Head: Human labels in crowd counting are frequently "noisy," with missed annotations in dense areas. TMTB introduces an auxiliary classification head that categorizes regions into count "bins" (e.g., 0-10 people). This coarser, more robust supervision acts as a regularizer, helping the primary regression model navigate labeling errors.

Experimental Results

The TMTB framework was evaluated on four major benchmarks: JHU-Crowd++, UCF-QNRF, and ShanghaiTech Parts A and B. The results demonstrate significant improvements over state-of-the-art (SOTA) methods, particularly in low-data scenarios.

On the JHU-Crowd++ dataset with only 5% of labels available, TMTB achieved a Mean Absolute Error (MAE) of 67.0. This represents a 12.4% improvement over previous methods and marks the first time a model has broken the 70.0 MAE barrier under such constrained data conditions. Furthermore, the model showed exceptional domain generalization, sometimes outperforming models trained on 100% of the target data when transferred from another source.

Conclusion

TMTB demonstrates that generative AI can serve as a powerful "augmentation engine" for dense prediction tasks like crowd counting. By combining structure-preserving inpainting with an efficient VSSM backbone, the framework sets a new standard for semi-supervised learning in the field. Future work may involve integrating text prompts to generate specific environmental conditions, further enhancing the diversity and robustness of synthetic training data.

Summary updated

Abstract

Semi-supervised crowd counting is crucial for addressing the high annotation costs of densely populated scenes. Although several methods based on pseudo-labeling have been proposed, it remains challenging to effectively and accurately utilize unlabeled data. In this paper, we propose a novel framework called Taste More Taste Better (TMTB), which emphasizes both data and model aspects. Firstly, we explore a data augmentation technique well-suited for the crowd counting task. By inpainting the background regions, this technique can effectively enhance data diversity while preserving the fidelity of the entire scenes. Secondly, we introduce the Visual State Space Model as backbone to capture the global context information from crowd scenes, which is crucial for extremely crowded, low-light, and adverse weather scenarios. In addition to the traditional regression head for exact prediction, we employ an Anti-Noise classification head to provide less exact but more accurate supervision, since the regression head is sensitive to noise in manual annotations. We conduct extensive experiments on four benchmark datasets and show that our method outperforms state-of-the-art methods by a large margin. Code is publicly available on https://github.com/syhien/taste_more_taste_better.
BibTeX
@inproceedings{yang2025taste,
  author = {Maochen Yang and Zekun Li and Jian Zhang and Lei Qi and Yinghuan Shi},
  title = {{Taste More, Taste Better: Diverse Data and Strong Model Boost Semi-Supervised Crowd Counting}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2025},
  doi = {10.1109/cvpr52734.2025.02276},
}
Figure