Semi-supervised Learning with a Teacher-Student Network for Generalized Attribute Prediction

Minchul Shin

Abstract

Visual Attribute Prediction (VAP) is a fundamental task in fashion AI, yet it remains challenging due to label sparsity, high annotation costs, and domain dependency. This paper introduces the Multi-Teacher-Single-Student (MTSS) framework, a semi-supervised approach that trains specialized teacher networks for distinct attribute types and distills their collective knowledge into a single student model. By leveraging unlabeled data and employing label embedding techniques, MTSS achieves state-of-the-art performance on major benchmarks without requiring auxiliary domain-specific metadata.

Introduction

Visual Attribute Prediction involves identifying fine-grained characteristics such as color, pattern, and cut. Despite its importance in e-commerce, VAP faces three primary hurdles: label sparsity (the "long-tail" problem), the prohibitive expense of manual annotation, and a reliance on domain-specific "crutches" like clothing landmarks. This research proposes a domain-agnostic framework that utilizes unlabeled data to build a robust, accurate model capable of operating in real-world, noisy environments.

Methodology

The proposed MTSS framework follows a two-stage training process designed to maximize expertise and generalizability.

Stage 1: Training the Experts (Teachers)

The first stage involves training multiple "Teacher" networks, each serving as a domain expert for a specific attribute category (e.g., Color or Pattern). Instead of standard binary classification, the authors utilize Label Embedding (LE) to map images and labels into a shared latent space. This approach captures the conceptual hierarchy and relationships between attributes, such as the similarity between different patterns.

Visual representation of the Label Embedding (LE) process mapping images and attributes into a shared latent space.

To further refine these experts, a modified focal loss is applied to metric learning, forcing the teachers to focus on "hard" examples that are frequently misclassified.

Stage 2: The Unified Student

In the second stage, a single "Student" network is trained to mimic the feature distributions of all specialized teachers. This is achieved through Knowledge Distillation, where the student attempts to match the embeddings produced by the experts.

Comparison of Common, Progressive, and MTSS architectures.

A key advantage of this stage is the ability to incorporate vast amounts of unlabeled data, as the student only requires the teachers' outputs for training. To maintain quality, an outlier suppression mechanism (controlled by hyperparameter $\beta$) prevents the student from learning from uncertain or noisy teacher predictions.

Experimental Setup

The MTSS model was evaluated across four diverse datasets:

  • DeepFashion: The standard benchmark for fashion attribute prediction.
  • iMaterialist-Fashion-2018: A large-scale dataset from the Kaggle competition.
  • DARN: Utilized for cross-domain retrieval testing.
  • FiccY: A new private dataset containing 520,000 high-resolution, real-world e-commerce images, providing a testbed for model robustness against user-uploaded "noisy" content.

Discussion and Key Insights

The research yielded several critical insights into attribute prediction:

  • Embedding vs. Classification: Label Embedding consistently outperformed Binary Cross-Entropy (BCE), particularly in data-constrained scenarios, by capturing the inherent structure of attribute classes.
  • Teacher-Student Dynamics: While the student model often surpasses individual teachers due to the regularizing effect of distillation and additional data, its performance is ultimately bounded by the teachers' initial quality.
  • Robustness: Integrating "noisy" unlabeled data during the distillation phase significantly improved the model's resistance to image corruptions (e.g., blur) without compromising performance on clean benchmark data.

Results

The MTSS framework demonstrated superior performance on the DeepFashion benchmark, achieving a Top-3 Recall of 58.02%. This result outperforms previous methods that lacked auxiliary labels and remains competitive with those that utilize them. Furthermore, the model showed significant gains in cross-domain adaptation and noise resistance, with a 2.33% accuracy improvement on corrupted images when trained with additional unlabeled data.

Conclusion

The MTSS framework provides a scalable, domain-agnostic solution for visual attribute prediction. By effectively combining specialized expertise with semi-supervised distillation, it addresses the long-tail problem and reduces reliance on expensive annotations. Future work will focus on optimizing teacher quality and exploring advanced sampling strategies for metric learning to further enhance performance on rare attributes.

Summary updated

Abstract

This paper presents a study on semi-supervised learning to solve the visual attribute prediction problem. In many applications of vision algorithms, the precise recognition of visual attributes of objects is important but still challenging. This is because defining a class hierarchy of attributes is ambiguous, so training data inevitably suffer from class imbalance and label sparsity, leading to a lack of effective annotations. An intuitive solution is to find a method to effectively learn image representations by utilizing unlabeled images. With that in mind, we propose a multi-teacher-single-student (MTSS) approach inspired by the multi-task learning and the distillation of semi-supervised learning. Our MTSS learns task-specific domain experts called teacher networks using the label embedding technique and learns a unified model called a student network by forcing a model to mimic the distributions learned by domain experts. Our experiments demonstrate that our method not only achieves competitive performance on various benchmarks for fashion attribute prediction, but also improves robustness and cross-domain adaptability for unseen domains.
BibTeX
@inproceedings{shin2020semi,
  author = {Minchul Shin},
  title = {{Semi-supervised Learning with a Teacher-Student Network for Generalized Attribute Prediction}},
  booktitle = {European Conference on Computer Vision},
  year = {2020},
  doi = {10.1007/978-3-030-58621-8\_30},
}
Figure