Abstract
Complementary clothing matching—the task of pairing items like tops and bottoms—is a fundamental challenge in fashion recommendation. While deep learning models excel at pattern recognition, they often lack the interpretability and domain-specific "common sense" inherent in human fashion expertise. This paper presents AKD-DBPR (Attentive Knowledge Distillation based on Bayesian Personalized Ranking), a framework that integrates structured logic rules into a neural network. By using an attention mechanism to dynamically weight these rules, the model achieves superior performance on the FashionVC dataset, demonstrating the power of combining data-driven learning with human-centric knowledge.
Introduction
While deep learning has revolutionized image recognition, applying it to fashion matching remains complex. Purely data-driven models often struggle with two primary issues: interpretability and the lack of domain knowledge. It is often difficult to discern why a model considers two items compatible, and these models rarely account for centuries of accumulated fashion "rules" (e.g., material or color coordination) that humans use intuitively.
Existing models typically rely on patterns found in noisy, real-world data, overlooking these explicit rules. Furthermore, fashion rules are often "fuzzy"—a guideline might be critical in one context but irrelevant in another. The authors developed AKD-DBPR to seamlessly integrate these human insights into a neural network architecture.
The AKD-DBPR Framework
The proposed framework utilizes a teacher-student architecture to balance raw data patterns with structured logic.
- The Student (Data-Driven): A dual-path neural network processes visual features (via a pre-trained CNN) and contextual features (via a TextCNN for descriptions). It employs a Bayesian Personalized Ranking (BPR) framework to learn a latent compatibility space.
- The Teacher (Knowledge-Driven): This component encodes 15 structured logic rules (e.g., "no T-shirt + dress"). The teacher network regularizes the student, correcting predictions that violate established fashion logic.
Attentive Knowledge Distillation
A key innovation is the attention mechanism within the distillation process. Since not all rules apply equally to every outfit, the model assigns a "confidence level" to each rule based on the specific items being compared. This allows the system to prioritize relevant aesthetic principles dynamically, mimicking the decision-making process of a human stylist.
Experimental Setup
The study utilized the FashionVC dataset, comprising 20,726 expert-composed outfits from Polyvore, featuring 14,871 tops and 13,663 bottoms. Each item includes high-resolution images, category labels, and title descriptions. To build the knowledge component, the authors analyzed attribute co-occurrences and collaborated with fashion enthusiasts to refine a set of 15 logic rules covering color, material, pattern, and category.
Results and Discussion
The AKD-DBPR model was evaluated against several state-of-the-art baselines using the Area Under the ROC Curve (AUC) metric:
- Superior Performance: AKD-DBPR achieved an AUC of 0.7852, outperforming the purely data-driven DBPR (0.7704) and other methods like BPR-DAE (0.7616).
- Ablation Study: The "attentive" version consistently outperformed a "uniform" version (UKD-DBPR), proving that dynamic rule weighting is superior to static application.
- Cold Start Robustness: The model performed well with items not seen during training, suggesting it truly learned generalizable compatibility features rather than simple dataset memorization.
Key Insights
One significant finding was the effectiveness of negative rules (e.g., "no blouse + dress"). Defining what is definitely incorrect often provided a stronger learning signal than defining what is perfectly right. Additionally, the integration of TextCNN was crucial for applying rules related to materials (like "silk" or "knit") that are difficult to identify from pixels alone.
Conclusion
While AKD-DBPR represents a significant step forward, it currently relies on textual metadata to trigger logic rules. Incomplete or noisy descriptions can prevent the model from applying relevant knowledge. Future research will focus on "visual rule identification," training models to recognize attributes like material and pattern directly from images to make the system more robust to poor metadata.
Summary updated