Compositional Zero-Shot Learning (CZSL) aims to recognize novel concepts by combining known "primitives," such as identifying a "blue apple" after seeing "red apples" and "blue cars." While modern CZSL models typically use a three-branch architecture (attribute, object, and composition), they often suffer from two critical limitations: feature uniformity and spatial disconnect.
The Problem: Uniformity and Disconnect
Most existing methods feed identical high-level features from the final layer of a backbone network to all three classification branches. This is suboptimal because attributes (e.g., "striped") often rely on fine-grained, low-level textures, whereas objects (e.g., "zebra") require high-level semantic shapes. Furthermore, without coordination, the attribute branch might focus on the background while the object branch focuses on the subject, leading to "hallucinated" predictions.
Methodology: FOMA
The proposed Focus-Consistent Multi-Level Aggregation (FOMA) framework addresses these issues through two primary innovations: Multi-Level Feature Aggregation (MFA) and a Focus-Consistent Constraint.
Multi-Level Feature Aggregation (MFA)
Instead of using only the final backbone layer, MFA extracts features from multiple depths (low, middle, and high levels). An "Aggregation Predictor" sub-network dynamically calculates weights for these features based on the input image. This allows the attribute branch to prioritize texture-rich low-level features, while the object branch focuses on semantic-rich high-level features.
Focus-Consistent Constraint
To ensure spatial alignment, FOMA introduces a penalty loss ($L_f$) based on attention maps. This constraint encourages the spatial focus of the composition branch to align with the combined focus of the attribute and object branches. By enforcing this consistency, the branches implicitly share information; for instance, if the object branch locates a "dog," the constraint guides the attribute branch to the same region to identify "wet" fur.
Attention Pooling
FOMA replaces standard Global Average Pooling (GAP) with a transformer-inspired attention mechanism. This preserves critical spatial information that is typically lost when flattening features into a single vector, further enhancing the model's ability to localize relevant primitives.
Experimental Results
FOMA was evaluated on three major CZSL benchmarks: UT-Zappos, C-GQA, and Clothing16K. It achieved state-of-the-art (SOTA) performance across nearly all metrics:
- C-GQA & Clothing16K: FOMA reached SOTA results in Seen accuracy, Unseen accuracy, Area Under the Curve (AUC), and Harmonic Mean (HM). On C-GQA, it improved the HM from 14.5 to 15.2.
- UT-Zappos: The model achieved the highest Unseen accuracy (68.0%) and AUC (33.1%), demonstrating superior generalization to novel pairs.
Qualitative visualizations of attention maps confirmed that FOMA consistently identifies the correct regions of interest, such as focusing on the texture of a "red couch" rather than misidentifying it based on shape alone.
Conclusion and Future Work
FOMA demonstrates that tailoring visual features to specific primitive types while enforcing spatial consistency significantly improves compositional reasoning. While highly effective, future work could explore more modern backbones like Vision Transformers (ViTs) and extend the framework to "open-world" settings where primitives themselves may be unseen during training.
Summary updated