Abstract
Zero-Shot Learning (ZSL) typically relies on global attributes to recognize unseen categories. However, for complex compositional objects, these attributes are often insufficient. We present ZERO, a framework that recognizes and localizes objects without any training images of the target category. By leveraging expert knowledge of an object’s constituent parts and synthesizing training samples from these sub-components, ZERO outperforms state-of-the-art few-shot models that require actual target examples.
Introduction: The Compositional Challenge
Standard deep learning models are data-hungry, requiring thousands of labeled images. While Zero-Shot Learning (ZSL) attempts to bridge this gap using descriptive attributes (e.g., "furry"), this approach falters for "compositional" objects—such as new industrial products or improvised explosive devices (IEDs). These objects are defined not by fuzzy attributes, but by distinct, recognizable parts like phones, wires, and containers. For rare or novel items, shared attribute datasets often do not exist, making traditional ZSL ineffective.
Methodology: From Parts to Wholes
ZERO employs a four-stage pipeline to shift detection from global templates to local part-based assemblies:
- Knowledge Capture: Domain experts define the target object through its parts and their spatial relationships (e.g., a bicycle consists of wheels, a saddle, and handlebars with specific overlap constraints).
- Part Detection: A standard detector (e.g., RetinaNet) is trained on common parts. Because parts like "wheels" or "phones" appear across many categories, training data is abundant even if the target object itself is rare.
- Graph-Based Recognition: Visual features from detected parts are integrated into a graph representation. ZERO trains a graph classifier by synthesizing thousands of samples—virtually "assembling" different part instances to simulate the target object without ever needing a real photo of the whole.
- Constraint-Based Localization: To locate the object, ZERO evaluates combinations of detected parts against the expert's spatial rules using N-dimensional matrices to find the best fit.
Experimental Results
ZERO was evaluated on the Pascal VOC dataset and custom internet imagery, focusing on categories like bicycles and complex assemblies.
- Superiority over Few-Shot Learning: In recognition tasks, ZERO achieved an Area Under the Curve (AUC) of 0.70 at a 5% false positive rate using zero target images. This surpassed a state-of-the-art few-shot detector trained on 10 real examples, which scored 0.64.
- Robustness to Variation: The model demonstrated high confidence (86%–100%) when identifying "deviating" or non-standard object designs found online, as it relies on the presence of essential parts rather than a rigid global template.
- Explainability: Unlike "black-box" models, ZERO provides inherent justifications for its predictions by highlighting the specific parts and spatial configurations it identified (e.g., "identified as a bike due to two wheels and a handlebar in the correct relative positions").
Discussion and Future Work
While ZERO currently focuses on single-object localization, future iterations could utilize anchor boxes for multi-object detection. Additionally, training on truly isolated parts—rather than parts cropped from full objects—will eliminate potential "context leakage" where the model might "cheat" by seeing bits of the frame in a wheel crop. Finally, the part-based approach suggests increased resilience against adversarial attacks, as multiple independent detectors would need to be compromised simultaneously to fool the system.
Summary updated