TL;DR
This paper revisits the "Visual Product" (VisProd) baseline for Compositional Zero-Shot Learning (CZSL), showing that with increased model capacity and a simple bias correction for seen classes, it can achieve state-of-the-art results. This suggests that simpler, independent object and state prediction might be a more robust approach than complex compositional models, especially in challenging open-world settings.
What Problem Does This Paper Address?
Compositional Zero-Shot Learning (CZSL) is the challenging task of identifying objects and their associated states in images, even when those specific combinations haven't been seen during training. For example, recognizing a "wet dog" when the model has only seen "dry dogs" and "wet cars." The core difficulty lies in generalizing to novel combinations where object appearances can change drastically based on states (e.g., a "pureed tomato" looks very different from a "raw tomato") and states can manifest differently depending on the object. Existing methods often rely on complex architectures and external information like word embeddings to model these relationships. This paper questions whether these complexities are always necessary by re-examining a much simpler baseline.
Methods
The authors revisit the Visual Product (VisProd) method. The core idea of VisProd is to treat CZSL as a multi-task problem: predict the object and the state independently. Given an image, it's first encoded into a feature representation. Then, two separate classifiers (one for objects, one for states) predict the probabilities for each. The joint probability of a composition is then approximated by the product of these independent probabilities.
The paper identifies two key reasons for VisProd's initial poor performance:
- Insufficient Network Capacity: Early VisProd implementations used simple linear classifiers, which lacked the power to learn complex object and state representations. The authors address this by using deeper, non-linear classifiers (MLPs).
- Bias Towards Seen Classes: CZSL models tend to perform better on combinations seen during training. The authors propose a simple fix: during inference, they subtract a learned scalar bias from the scores of seen compositions. This helps to level the playing field for unseen combinations.
The improved model is referred to as VisProd++ when using deeper MLPs for classifiers.
Datasets
The paper evaluates their approach on three standard CZSL benchmarks:
- MIT-States: A dataset known for its label noise, particularly in state predictions.
- UT-Zappos: A dataset of clothing items with attributes.
- C-GQA: A challenging dataset with a very large number of possible compositions (around 280k).
Key Insights and Ideas
- Simplicity Can Be Powerful: The paper demonstrates that a simple, independent prediction of objects and states, when implemented with sufficient model capacity and a bias correction, can rival or surpass complex, specialized CZSL architectures.
- Bias Correction is Crucial: The simple act of penalizing scores for seen compositions significantly improves performance, especially in settings where generalization to unseen combinations is paramount. This highlights the importance of carefully considering evaluation metrics and potential biases in zero-shot learning.
- Open-World CZSL is a Strong Testbed: VisProd++ shows remarkable performance in the Open-World CZSL setting, where the model has no prior knowledge of the possible unseen compositions. This suggests its independent prediction approach is robust to the vast combinatorial space.
Key Results
- State-of-the-Art Performance: VisProd++ achieves comparable or superior results to recent state-of-the-art methods on all three tested benchmarks (MIT-States, UT-Zappos, C-GQA) in both closed-world and open-world CZSL settings.
- Open-World Dominance: On the challenging C-GQA dataset for open-world CZSL, VisProd++ achieves the new state-of-the-art, outperforming previous methods by a significant margin (e.g., achieving an AUC of 0.75 compared to 0.39 and 0.47 for other top methods).
- Effect of Depth: Ablation studies show that increasing the depth of the object and state classifiers (up to 3 layers) significantly boosts performance, indicating the need for sufficient model capacity.
- Effect of Bias Correction: The scalar bias correction for seen compositions is shown to be critical, dramatically improving the harmonic mean performance.
Limitations and Future Work
- Label Noise on MIT-States: The authors note that VisProd++ is somewhat penalized by the inherent label noise in the MIT-States dataset, particularly for state predictions.
- Independence Assumption: While effective, the core VisProd assumption of independence between object and state prediction might not capture all nuanced interactions.
- Future Work: The paper hopes to revive interest in visual-only methods for CZSL and encourages further exploration of treating object and state prediction separately as a competitive approach. The success in open-world CZSL suggests potential for further research in handling extremely large output spaces.
Summary updated