Introduction: The Scale-Blindness Problem in ZSD
Zero-Shot Object Detection (ZSD) aims to detect objects from classes not seen during training by leveraging semantic descriptions. A common strategy involves synthesizing visual features for these unseen classes. However, existing "imagineers" often suffer from scale-blindness, generating features at a single, fixed scale. This leads to "monotonous imaginations" that fail to account for the significant size variations objects exhibit in real-world scenes due to distance or perspective.
The SAUI Framework
To address this, the Scale-Aware Unseen Imagineer (SAUI) introduces a multi-scale approach to feature synthesis. The framework is built upon three core pillars:
1. Coarse-to-Fine Feature Extractor
Unlike standard backbones that might output a single feature map, SAUI utilizes a coarse-to-fine extractor. It aggregates information from multiple levels of a Feature Pyramid Network (FPN), capturing both the broad outlines and the intricate details of objects.
2. Series-GAN Synthesizer
The heart of SAUI is the Series-GAN, a chain of generators ($G_1$ to $G_4$) designed to produce multi-scale features. By treating feature generation as a sequence-to-sequence task, the model progressively refines its "imagination" from coarse representations to high-resolution details.
3. Multi-scale Contrastive Imagination
To ensure the synthesized features are both diverse and representative, SAUI employs a three-pronged contrastive loss:
- Intra-scale Outline Maintaining ($L_{main}$): Preserves class separability at each scale.
- Intra-scale Detail Diverging ($L_{div}$): Encourages variety within a class to prevent mode collapse.
- Inter-scale Mutual Constructing ($L_{con}$): Ensures consistency across different scales for the same object concept.
Experimental Evaluation
SAUI was evaluated on MS COCO, DIOR (remote sensing), and PASCAL VOC.
On MS COCO, SAUI achieved 22.1% mAP, significantly outperforming previous methods. Most notably, it nearly tripled the performance on small objects ($AP_s$), demonstrating the effectiveness of scale-aware synthesis.
On the DIOR dataset, which features extreme scale variations, SAUI reached 16.6% mAP—a 42% improvement over previous state-of-the-art results. This confirms the framework's robustness in challenging, multi-scale environments.
Conclusion
SAUI demonstrates that scale is a critical dimension in Zero-Shot Object Detection. By explicitly modeling scale diversity through a coarse-to-fine architecture and a specialized GAN synthesizer, SAUI overcomes the limitations of "monotonous imaginations," particularly for small and scale-varying objects. While performance gains are less pronounced on datasets with centered, large objects like PASCAL VOC, SAUI sets a new benchmark for complex, real-world detection tasks.
Summary updated