Iterative Correlation-based Feature Refinement for Few-shot Counting

Zhiyuan You, Kai Yang, Wenhan Luo, Xin Lu, Lei Cui, Xinyi LearXiv 2022 · OpenReview

Introduction

Object counting is a critical computer vision task with diverse applications, from traffic management to ecological surveys. Traditional methods are often class-specific, requiring extensive retraining for new object types. Few-shot counting (FSC) addresses this by enabling models to count novel objects using only a few exemplar "support" images. However, dense scenes—where objects overlap or are tightly packed—remain a significant hurdle for existing FSC models.

The SAFECount Framework

To overcome these challenges, the authors propose SAFECount (Similarity-Aware Feature Enhancement). The framework is designed to generalize across novel classes by learning the underlying concept of counting rather than memorizing specific object shapes. As shown in the general pipeline, the model takes a support image and a query image to produce a density map, which is then compared against ground truth during training or used to derive a final count during testing.

The few-shot counting pipeline: training on base classes and testing on novel classes using support and query images.

Methodology

The core of the architecture is the SAFECount block, which iteratively refines features to better distinguish individual objects. Unlike previous methods that might lose spatial resolution or rely on simplistic similarity maps, SAFECount integrates two specialized modules:

  1. Similarity Comparison Module (SCM): This module projects support and query features into a shared space. It uses the support features as convolutional kernels to scan the query image, generating a robust similarity map that localizes potential object instances.
  2. Feature Enhancement Module (FEM): Instead of using the similarity map solely for prediction, the FEM uses it to weight and fuse exemplar features back into the query features. This process "highlights" relevant regions with high-resolution semantic information, allowing the model to delineate clear boundaries even in cluttered environments.
Evolution of counting architectures: (a) simple concatenation, (b) similarity-based regression, and (c) the proposed SAFECount block with SCM and FEM.

A key technical innovation is Kernel Flipping, which preserves the spatial structure of objects during the convolution process. The authors found that stacking four of these blocks provides the optimal balance for progressive feature refinement, allowing the model to focus on increasingly fine details.

Results and Discussion

SAFECount achieves state-of-the-art results on the FSC-147 benchmark, reducing the Mean Absolute Error (MAE) from 22.08 to 14.32—a massive 35% improvement over previous methods like FamNet+. Qualitative analysis shows that while baseline models often produce blurry density maps in dense clusters, SAFECount generates distinct peaks for individual items, even in extremely cluttered scenes like piles of beans or legos.

The model also demonstrates strong zero-shot generalization, successfully counting objects in classes it never encountered during training. This suggests that the iterative refinement process effectively captures the generic properties of "objectness" and spatial separation required for counting.

Conclusion

By treating similarity as a guide for feature enhancement rather than an end goal, SAFECount sets a new standard for few-shot counting. While the current model relies on a fixed ResNet-18 backbone, future research may explore integrating more advanced architectures like Vision Transformers to further enhance the model's flexibility and scale invariance.

Summary updated

BibTeX
@misc{you2022iterative,
  author = {Zhiyuan You and Kai Yang and Wenhan Luo and Xin Lu and Lei Cui and Xinyi Le},
  title = {{Iterative Correlation-based Feature Refinement for Few-shot Counting}},
  year = {2022},
  url = {https://openreview.net/forum?id=ieSneGAV2Jg},
}
Figure