Introduction
Few-shot object detection (FSOD) aims to detect novel objects using only a handful of examples. Recently, Open-Vocabulary Detection (OVD) models, which leverage massive vision-language models like CLIP, have been proposed as a solution. However, a practical gap exists: many real-world targets, such as industrial defects or medical lesions, are difficult to describe accurately with words. This paper investigates whether the complexity of OVD is justified for these "low-describability" classes compared to traditional Closed-set Object Detection (COD).
Methodology: The Describability Metric
The authors introduce a novel way to categorize detection tasks based on "text describability." They use zero-shot image classification accuracy with CLIP as a proxy indicator. If CLIP can easily identify an object's category from a crop based on its name, the class is considered "highly describable."
Using this metric, 35 diverse datasets from the Object Detection in the Wild (ODinW) collection were categorized into three tiers:
- S1 (High describability): Common objects like "cat" or "dog."
- S2 (Medium describability): Objects with moderate naming clarity.
- S3 (Low describability): Specialized targets like "blood cells," "sign language," or "industrial parts."
Results: OVD vs. COD Performance
The study compared state-of-the-art OVD models (e.g., GLIP, F-ViT) against COD models (e.g., DyHead, Faster R-CNN). The results highlight a clear "describability threshold": OVD's primary advantage is its linguistic prior, and when this prior is weak, OVD effectively reverts to a standard detector.
Key performance findings include:
- S1 (Easy to Describe): OVD significantly outperforms COD. At 1-shot (K=1), GLIP(A) achieved 37.4 AP compared to DyHead’s 29.0 AP.
- S3 (Hard to Describe): The gap vanishes. At 3-shots (K=3), DyHead (COD) achieved 39.7 AP, while GLIP(A) (OVD) achieved 39.2 AP.
Discussion: The Impact of Data Scaling
A surprising finding is that while OVD models benefit from diverse web-scale data for common objects, this can be counterproductive for hard-to-describe classes. For instance, training GLIP on 5.46M images (including web captions) improved S1 performance but caused S3 performance to drop from 39.2 AP to 34.9 AP. The "noise" of general web data appears to dilute the model's ability to learn the specific visual features required for niche S3 tasks. Additionally, for FSOD with a significant domain gap, simply finetuning all parameters ("Full-FT") consistently outperformed specialized few-shot algorithms like TFA or FSCE.
Conclusion and Practical Guidance
For practitioners, the choice between OVD and COD should depend on the target domain. If objects are standard and easily described (e.g., cars, people), OVD is the superior choice. However, for specialized domains like medical imaging or industrial inspection, traditional COD methods are likely more efficient and just as accurate. Future work may explore "visual prompting" to bridge the gap for classes that are hard to describe but easy to show visually.
Summary updated