PBECount: Prompt-Before-Extract Paradigm for Class-Agnostic Counting

Canchen Yang, Tianyu Geng, Jian Peng, Chun XuAAAI 2025 · DOI

Abstract

Class-Agnostic Counting (CAC) requires models to count arbitrary objects based on a few provided exemplars. Traditional "Extract-then-Match" approaches often struggle with multi-class scenes, leading to over-counting. We propose PBECount, a novel framework based on the Prompt-Before-Extract (PBE) paradigm. By concatenating the query image with an exemplar probability map at the input level, PBECount enables early-stage feature matching within a pure CNN architecture. Enhanced by a dynamic thresholding Score Head and a Peak-Aware MSE loss, PBECount achieves state-of-the-art results on the FSC-147 benchmark, demonstrating superior accuracy and robustness against distractor objects compared to complex Transformer-based models.

Introduction

The primary challenge in Class-Agnostic Counting is distinguishing target objects from similar-looking distractors in diverse environments. Most existing methods follow an "Extract-then-Match" workflow: they extract features from exemplars and query images independently before performing similarity matching in deeper layers. This late-stage fusion often fails to capture fine-grained differences, resulting in the counting of non-target objects that share general visual characteristics with the exemplars.

Comparison of the traditional Extract-then-match paradigm (a) vs. the proposed Prompt-before-extract paradigm (b), with qualitative results (c, d).

PBECount rethinks this process by introducing the Prompt-Before-Extract paradigm. By providing the model with "prompts"—the locations of exemplars—at the very first layer of feature extraction, the backbone can learn context-aware features specifically tuned to the target class from the start.

Methods

PBECount utilizes a streamlined architecture based on a ResNet34 U-Net, proving that well-designed CNNs can remain competitive with modern Vision Transformers (ViTs).

1. The PBE Paradigm

The core innovation is the input preprocessing. The model takes a 4-channel input: the standard 3-channel RGB query image concatenated with a 1-channel exemplar probability map. This map contains Gaussian "glows" at the coordinates of the provided exemplars. This early fusion allows the backbone to perform feature matching at shallow layers, significantly improving the model's ability to isolate target objects.

Detailed architecture of PBECount, featuring the ResNet34 U-Net backbone, Score Head for dynamic thresholding, and Map Head for density estimation.

2. Dynamic Thresholding and Dual Heads

Unlike standard models that use a fixed threshold for object detection, PBECount incorporates a Score Head alongside the traditional Map Head. The Score Head predicts an optimal threshold score tailored to the specific characteristics of each image. This dynamic approach allows the model to adjust its sensitivity based on scene clutter and clarity, leading to a 16.31% improvement in Mean Absolute Error (MAE).

3. Peak-Aware MSE Loss

To improve localization, the authors introduced a weighted Mean Squared Error (MSE) loss. This function applies higher penalties to errors occurring near the centers (peaks) of objects in the density map, ensuring the model focuses on precise object placement rather than just overall density.

4. Loop Optimization

During inference, PBECount employs a refinement loop. If there is high discrepancy between different thresholding results, the model identifies a "pseudo-exemplar"—a detected object far from the original exemplars—and re-runs the pipeline to refine the final count.

Results and Evaluation

PBECount was evaluated on several benchmarks, consistently outperforming previous state-of-the-art (SOTA) methods.

  • FSC-147: On this standard CAC dataset, PBECount achieved an MAE of 7.71, representing a 10.97% error reduction over the previous leader, DAVE.
  • Multi-Class Robustness: Testing on the FSC-147 Mul subset confirmed that PBECount is significantly better at ignoring "distractor" classes (e.g., counting only blueberries in a bowl of mixed berries) compared to models like LOCA and CounTR.
  • Cross-Dataset Generalization: On the CARPK dataset (car counting), PBECount achieved an MAE of 6.27, outperforming the leading Transformer-based model, CACViT (MAE 8.30).

Conclusion

PBECount demonstrates that the "Prompt-Before-Extract" paradigm is a highly effective strategy for class-agnostic counting. By shifting the focus from complex late-stage matching to early-stage context-aware extraction, the model achieves superior accuracy with a simpler CNN-based architecture. While challenges remain in extremely dense scenarios where exemplar markers may overlap, PBECount sets a new standard for robustness and precision in few-shot counting tasks. Future work will explore extending this paradigm to broader few-shot vision tasks such as detection and segmentation.

Summary updated

BibTeX
@inproceedings{yang2025pbecount,
  author = {Canchen Yang and Tianyu Geng and Jian Peng and Chun Xu},
  title = {{PBECount: Prompt-Before-Extract Paradigm for Class-Agnostic Counting}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.1609/aaai.v39i9.32989},
}
Figure