Introduction
Crowd counting—estimating the number of people in dense images—is a critical task for urban planning and public safety. However, it is notoriously labor-intensive. Annotating a single large-scale dataset like UCF-QNRF can require up to 2,000 human hours to label individual heads. Semi-supervised crowd counting (SSCC) offers a solution by leveraging small amounts of labeled data alongside vast unlabeled pools. Yet, current SSCC methods face two primary bottlenecks: destructive data augmentation techniques and architectural limitations that struggle with global context or computational efficiency.
This paper introduces TMTB (Taste More, Taste Better), a framework designed to overcome these hurdles through high-fidelity synthetic data generation and a modern Visual State Space Model (VSSM) backbone.
The TMTB Framework
"Taste More": Inpainting-Based Augmentation
Traditional augmentation methods like "Mixup" blend two images, which often destroys the delicate spatial structure of a crowd. This leads to distorted density maps and poor model performance because it breaks the spatial logic the model needs to learn. TMTB replaces this with a diffusion-based inpainting technique using Stable Diffusion.
By identifying the crowd as the "foreground" and using AI to regenerate the "background" scenery, the framework creates diverse training samples while preserving the original crowd structure. To ensure reliability, an Inconsistency Mask is employed to filter out regions where the generative model might have introduced artifacts or "ghost" people.
"Taste Better": VSSM Backbone and Anti-Noise Head
On the architectural side, TMTB addresses the limitations of CNNs and Transformers:
- VSSM Backbone: The authors utilize VMamba, a Visual State Space Model. Unlike CNNs, which often overfit to local textures, VSSM provides a global receptive field similar to Transformers but maintains linear computational complexity. This makes it highly efficient for high-resolution crowd scenes.
- Anti-Noise Classification Head: Human labels in crowd counting are frequently "noisy," with missed annotations in dense areas. TMTB introduces an auxiliary classification head that categorizes regions into count "bins" (e.g., 0-10 people). This coarser, more robust supervision acts as a regularizer, helping the primary regression model navigate labeling errors.
Experimental Results
The TMTB framework was evaluated on four major benchmarks: JHU-Crowd++, UCF-QNRF, and ShanghaiTech Parts A and B. The results demonstrate significant improvements over state-of-the-art (SOTA) methods, particularly in low-data scenarios.
On the JHU-Crowd++ dataset with only 5% of labels available, TMTB achieved a Mean Absolute Error (MAE) of 67.0. This represents a 12.4% improvement over previous methods and marks the first time a model has broken the 70.0 MAE barrier under such constrained data conditions. Furthermore, the model showed exceptional domain generalization, sometimes outperforming models trained on 100% of the target data when transferred from another source.
Conclusion
TMTB demonstrates that generative AI can serve as a powerful "augmentation engine" for dense prediction tasks like crowd counting. By combining structure-preserving inpainting with an efficient VSSM backbone, the framework sets a new standard for semi-supervised learning in the field. Future work may involve integrating text prompts to generate specific environmental conditions, further enhancing the diversity and robustness of synthetic training data.
Summary updated