Introduction
Compositional Zero-Shot Learning (CZSL) aims to recognize unseen combinations of attributes and objects, such as "purple cauliflower," by learning from seen pairs during training. A fundamental challenge in CZSL is contextuality: the visual appearance of an attribute is often heavily dependent on the object it modifies. For example, a "peeled apple" looks significantly different from a "peeled egg." Traditional approaches often treat attributes as static visual patterns or simple linear operators, which fails to capture the underlying physical transformations and the "rationality" of these changes.
The SymNet Framework
To address these issues, the authors propose SymNet, a framework grounded in group theory. SymNet models attributes as transformations rather than static features. The architecture consists of two primary components: a Coupling Network (CoN), which applies an attribute to an object representation, and a Decoupling Network (DecoN), which removes it.
The training of SymNet is guided by four fundamental group theory axioms and a novel Symmetry Principle:
- Symmetry: Applying an attribute to an object that already possesses it (e.g., "peeling" a peeled apple) should result in no change. Similarly, removing an attribute from an object that lacks it should leave the representation unchanged.
- Closure: Transformations must result in valid representations within the defined attribute-object space.
- Invertibility: The coupling and decoupling operations should be inverses; adding and then removing the same attribute should return the object to its initial state.
- Commutativity: The order of applying multiple attributes should not affect the final representation.
Inference via Relative Moving Distance (RMD)
A key innovation of SymNet is its inference mechanism. Instead of a traditional classifier, the model uses Relative Moving Distance (RMD). For a given image, the model calculates how much the representation "moves" in latent space when a candidate attribute is added versus when it is removed. If adding the attribute results in a smaller displacement than removing it, the model infers that the attribute is already present in the image. This approach treats attribute recognition as a measurement of dynamic change, making it more robust to the visual variability across different object categories.
Experimental Results
SymNet was evaluated on two benchmark datasets: MIT-States, which features a wide variety of noisy attribute-object pairs, and UT-Zappos50K, a fine-grained dataset of shoes.
SymNet achieved state-of-the-art (SOTA) performance:
- MIT-States: Reached a Top-1 accuracy of 19.9%, improving upon the previous SOTA of 17.8%.
- UT-Zappos: Achieved 52.1% accuracy, compared to the previous best of 49.9%.
- Attribute Recognition: The RMD method significantly boosted standalone attribute identification, showing an 8.3% improvement on UT-Zappos.
Qualitative results further demonstrated the model's ability to perform complex attribute manipulations. For instance, the model could successfully retrieve images of "murky water" by taking a "heavy water" representation, decoupling "heavy," and coupling "murky."
Conclusion
By integrating group theory axioms and the symmetry principle, SymNet provides a mathematically rigorous foundation for modeling attribute-object compositions. Treating attributes as transformations allows the model to overcome the limitations of contextuality and achieve superior generalization on unseen combinations. Future research may address the "synonym problem"—where visually similar but linguistically distinct attributes (e.g., "ancient" vs. "old") penalize accuracy—and explore modeling continuous degrees of attributes.
Summary updated