Introduction
Traditional Compositional Zero-Shot Learning (CZSL) focuses on recognizing new combinations of known primitives—for example, recognizing a "red bike" after seeing "red cars" and "blue bikes." However, real-world scenarios are rarely static. New concepts emerge constantly, requiring models to learn new attributes (like "metallic") or objects (like "drone") incrementally. This paper introduces Continual Compositional Zero-Shot Learning (CCZSL), a benchmark and framework designed for models that must evolve without retraining from scratch.
The Challenge of Incremental Learning
The authors identify two major hurdles in the CCZSL setting:
- Contextuality: Primitives look different depending on their pair. "Old" has a different visual signature on a dog than on a car. In a continual setting, the model must adapt to these shifting signatures as new pairs are introduced.
- Catastrophic Forgetting: As models learn new primitives in later sessions, they tend to overwrite the weights for older ones, leading to a sharp performance drop on earlier tasks.
Proposed Methodology
The authors propose a framework consisting of two primary innovations to handle context and memory.
1. Super-Primitives for Contextuality
Instead of static embeddings, the model learns a set of Super-Primitives that act as contextual templates. A Selection Module determines which Super-Primitive best fits the current image, which then modifies the invariant "base" primitive embedding to match the specific visual context (e.g., applying a "mammal" context to the attribute "wet").
2. Dual Knowledge Distillation (DKD)
To prevent forgetting, the framework uses Dual KD. While standard distillation focuses on keeping old class outputs consistent, DKD also applies distillation to new categories. This prevents the model from becoming overly biased toward the most recently learned data, ensuring a balanced prediction space across all sessions.
Experimental Results
The researchers adapted two standard benchmarks, UT-Zappos (shoes) and C-GQA (natural images), into incremental versions with multiple learning sessions.
- UT-Zappos: CCZSL achieved an average AUC of 30.46%, a 4.19% improvement over the best-performing baseline (AoP).
- C-GQA: On this complex dataset, CCZSL reached an average AUC of 3.29%, outperforming the next best model (CANet) by 1.12%.
- Stability: While baseline models often saw performance plummet in later sessions, CCZSL maintained significantly higher performance, proving its resistance to catastrophic forgetting.
Conclusion
By leveraging Super-Primitives to handle visual context and Dual Knowledge Distillation to preserve memory, the CCZSL framework successfully addresses the complexities of evolving visual vocabularies. This work moves zero-shot learning closer to real-world applications where AI must learn and adapt continuously.
Summary updated