Class-Agnostic Object Counting for Universal Visual Enumeration

This direction is split into other directions: class-agnostic generalization gap. Its last version stays readable and citable here.

Version 1 of 145 papers46 conceptsfrozen 2026-10-04

In traditional computer vision, counting has long been treated as a collection of "closed-world" problems. If you needed to count cars, you trained a car counter; if you needed to count cells, you built a cell counter. This approach, while effective for specific tasks, fails to scale to the infinite variety of objects in the real world.

The research direction of Class-Agnostic Counting (CAC)—often framed as "Learning to Count Everything"—replaces these specialized models with a single, versatile system. The goal is to build a model that understands the principle of numerosity: the ability to identify and enumerate any recurring visual pattern, whether it’s a penguin, a building, or a microscopic crystal, based only on a few examples or a simple text prompt.

The Shift from Detection to Matching

The core intuition behind class-agnostic counting is a shift from classification to matching. Instead of asking "Is this a car?", the model asks "How many things in this image look like this specific example?"

In their seminal work Class-Agnostic Counting, Erika Lu, Weidi Xie, and Andrew Zisserman proposed that counting should be treated as a matching problem between a "support" (the exemplar) and a "query" (the full image). By learning to recognize self-similarity rather than specific category features, a model can generalize to objects it has never seen during training.

This was further refined by Viresh Ranjan and colleagues in Learning To Count Everything, which established the standard few-shot counting pipeline:

  1. Feature Extraction: Extract deep visual features from both the exemplar and the image.
  2. Correlation: Compute a similarity map (or "response map") showing where the exemplar matches the image.
  3. Density Estimation: Regress a density map where the integral (sum) of the pixels equals the total count.

Enhancing Similarity and Robustness

A major challenge in this field is intraclass diversity. If a user provides an example of a red car, the model must still count the blue and green cars in the same scene. Shenjian Gong and Bernt Schiele addressed this in Class-Agnostic Object Counting Robust to Intraclass Diversity by "hallucinating" variations of the exemplar in the feature space, allowing the model to match objects that differ in color or scale.

To improve the precision of these matches, Zhiyuan You introduced SAFECount (Few-shot Object Counting with Similarity-Aware Feature Enhancement), which uses a similarity-aware enhancement module to fuse exemplar features more effectively with the image features, significantly reducing errors in dense or cluttered scenes.

From Visual Examples to Textual Prompts

While few-shot counting requires a user to draw a bounding box, the field is moving toward Zero-Shot Counting (ZSC), where a model counts based on a text label alone.

Jingyi Xu and Dimitris Samaras demonstrated in Zero-Shot Object Counting with Language-Vision Models that we can leverage the semantic alignment of models like CLIP to bridge the gap between text and pixels. Their approach uses text to automatically "select" the best visual exemplars within an image, which are then fed into a standard counting backbone.

Similarly, the PseCo framework by Zhizhong Huang and colleagues (Point, Segment and Count: A Generalized Framework for Object Counting) combines the Segment Anything Model (SAM) with CLIP to count objects in both few-shot and zero-shot scenarios, effectively turning counting into a promptable task.

The Training-Free Frontier

The most recent evolution in this research direction is the move toward training-free methods. Instead of training a counting-specific network on massive datasets like FSC-147, researchers are using the "emergent" properties of large foundation models.

Yuhao Lin and Qinfeng Shi proposed a Simple-but-effective Baseline for Training-free Class-Agnostic Counting that uses DINOv2 features and SAM segments. By comparing the feature similarity of different segments to a prompt, these models can count accurately without ever having been explicitly trained on a "counting" objective.

Persistent Challenges: Hallucinations and Reasoning

Despite these advances, counting remains a "deceptively simple" task for AI. Huixuan Zhang and Xiaojun Wan have highlighted the "number hallucination" problem in Evaluating and Mitigating Number Hallucinations in Large Vision-Language Models, noting that even powerful multimodal models often struggle with basic numerical consistency.

The future of the field lies in reconciling the high-level semantic understanding of Large Language Models with the low-level spatial precision of density-map estimators, moving toward systems like Michael A. Hobley’s ABC123 (ABC Easy as 123), which can count multiple different classes in a scene simultaneously without any user-provided examples at all.

Papers in this version

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_class_agnostic_object_counting_for_unive_ce49e860,
  title = {Class-Agnostic Object Counting for Universal Visual Enumeration},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/class-agnostic-object-counting-for-universal-visual-enumeration/rdr_ce49e860d0cc4fd8a3fa4a03101222f4},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_class_agnostic_object_counting_for_unive_ce49e860_v1,
  title = {Class-Agnostic Object Counting for Universal Visual Enumeration},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/class-agnostic-object-counting-for-universal-visual-enumeration/rdr_ce49e860d0cc4fd8a3fa4a03101222f4/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}